Project

General

Profile

Feature #8243

Support meek bridges

Added by intrigeri almost 5 years ago. Updated 15 days ago.

Status:
Confirmed
Priority:
Elevated
Assignee:
-
Category:
Tor configuration
Target version:
-
Start date:
11/08/2014
Due date:
% Done:

100%

Feature Branch:
feature/8243-meek feature/8243-meek+force-all-tests
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Related issues

Related to Tails - Feature #7980: Support obfs4 bridges Resolved 10/01/2014
Related to Tails - Feature #7909: Support ScrambleSuit bridges Rejected 09/17/2014
Related to Tails - Feature #8287: Explain what is required to have new pluggable transports Rejected 01/03/2015
Related to Tails - Bug #8775: Tor Launcher should be able to resolve DNS host names for the proxy configuration Resolved 01/22/2015
Related to Tails - Bug #14555: Migrate to Tor Launcher compatible with Firefox ESR60 Resolved 08/30/2017
Related to Tails - Feature #15144: Consider migrating from Tor Launcher to anon-connection-wizard Confirmed 01/03/2018
Related to Tails - Feature #8825: Provide default bridges Confirmed 01/30/2015
Related to Tails - Feature #15197: Upgrade to Tor Browser 7.5 Resolved 01/18/2018
Related to Tails - Feature #15331: Support automatic bridge retrieval in Tor Launcher (Moat) Confirmed 12/12/2017
Related to Tails - Feature #5461: Persistence preset: Tor configuration Confirmed
Duplicated by Tails - Bug #15836: Meek-lite Duplicate 08/24/2018
Duplicated by Tails - Feature #15972: Adding meek bridges to Tails Duplicate 09/23/2018

Associated revisions

Revision 4fc2cd47 (diff)
Added by anonym over 1 year ago

Tor: enable clearnet DNS resolution in bridge mode.

So now users can input a human-readable hostname for proxies and
bridges (and we can support meek_lite!).

Will-fix: #8775
Refs: #8243

Revision a993d3eb (diff)
Added by anonym over 1 year ago

Tor: add support for meek_lite.

Will-fix: #8243

Revision f13eb68e (diff)
Added by anonym over 1 year ago

Tor Launcher: provide Tor's default Meek bridges.

Refs: #8243

Revision 585b0907 (diff)
Added by anonym over 1 year ago

Tor Launcher: provide Tor's default Meek bridges.

It is not clear we want to do this, but for now we enable them as a
workaround:

In current Tor Launcher (whatever version shipped with Tor Browser
8.0a1), if no default bridges are configured the whole group of radio
buttons that provide the Moat option ("Request another bridge") is
hidden (with only the text box for manual entry available). So let's
provide some to enable Moat.

Refs: #8243

Revision bf317f15 (diff)
Added by bertagaz over 1 year ago

Ship systemd from stretch-backports.

Install systemd v236, required to get the meek_lite PT to work, and have
the unsafe browser and the Tor launcher applications do clearnet DNS
resolution. This is required to get systemd's `BindReadOnlyPaths`
directive introduced in commit 4fc2cd4.

Refs: #8243, #8775

Revision 4bd6d7b4
Added by bertagaz over 1 year ago

Merge remote-tracking branch 'origin/feature/8243-meek' into devel

Fix-committed: #8243

Revision 2b5f771b (diff)
Added by bertagaz over 1 year ago

Fix APT preferences file comment for real.

Refs: #8243

Revision 4d78082c
Added by bertagaz over 1 year ago

Merge remote-tracking branch 'origin/feature/8243-meek' into devel

Fix-committed: #8243

Revision e4115249 (diff)
Added by anonym about 1 month ago

Tor Launcher: provide Tor's default Meek bridges.

It is not clear we want to do this, but for now we enable them as a
workaround:

In current Tor Launcher (whatever version shipped with Tor Browser
8.0a1), if no default bridges are configured the whole group of radio
buttons that provide the Moat option ("Request another bridge") is
hidden (with only the text box for manual entry available). So let's
provide some to enable Moat.

Refs: #8243

History

#1 Updated by intrigeri almost 5 years ago

#2 Updated by intrigeri almost 5 years ago

#3 Updated by intrigeri over 4 years ago

  • Target version set to Tails_1.4

Given the feedback we got on tor-talk ("Which PTs shall we prioritize for inclusion in Tails?" thread), this should be our top-priority in this area, along with #7980 (obfs4).

Now, we haven't reached a practically usable conclusion in the "keeping up with pluggable transports by using TBB's Tor and tor-launcher" thread, so it's not clear how meek support should be implemented.

#4 Updated by BitingBird over 4 years ago

  • Target version changed from Tails_1.4 to Tails_1.5

#5 Updated by BitingBird about 4 years ago

  • Target version deleted (Tails_1.5)

#6 Updated by intrigeri about 4 years ago

Target version deleted (Tails_1.5)

(As per monthly meeting.)

#7 Updated by xiaolan over 3 years ago

intrigeri wrote:

Given the feedback we got on tor-talk ("Which PTs shall we prioritize for inclusion in Tails?" thread), this should be our top-priority in this area, along with #7980 (obfs4).

Now, we haven't reached a practically usable conclusion in the "keeping up with pluggable transports by using TBB's Tor and tor-launcher" thread, so it's not clear how meek support should be implemented.

meek in the Tor Browser works in China, but seems Tails doesn't support it ..
if Tails supports meek it will be really great and I can test it in China..

https://trac.torproject.org/projects/tor/wiki/doc/meek

Thank you :)

#8 Updated by emmapeel over 2 years ago

Another user has asked today in #tails for meek in Tails because is the only option in China currently.

#9 Updated by u about 2 years ago

Two years ago intrigeri wrote that this should be our next top priority in this area.

What are the next tasks?

Upstream Tor bug: https://trac.torproject.org/projects/tor/ticket/13160 where Ximin says that he wants https://trac.torproject.org/projects/tor/ticket/12716 fixed before uploading the meek packages to Debian. However, there seems to be some confusion, see https://trac.torproject.org/projects/tor/ticket/12716#comment:32

I can ask Ximin about the current state of things to see how realistic this is.

#10 Updated by u about 2 years ago

  • Type of work changed from Code to Wait

For now, we simply have to wait.

#11 Updated by u about 2 years ago

  • Related to Feature #8287: Explain what is required to have new pluggable transports added

#12 Updated by intrigeri about 2 years ago

intrigeri wrote:

Now, we haven't reached a practically usable conclusion in the "keeping up with pluggable transports by using TBB's Tor and tor-launcher" thread, so it's not clear how meek support should be implemented.

After having a look at the status of the packaging, taking a step back to look at the big picture, and learning about the fact that obfs4proxy can act as a simplistic meek client, I'm starting to lean towards not blocking on meek-client being in Debian, or at least not assuming we just have to wait and it'll happen (that sounded realistic a couple years ago, but nobody is actively working on it anymore). So our options are: either we do that meek packaging work ourselves (seems hard but perhaps focussing on meek-client and forgetting the reflector etc. makes it easier), or we use obfs4proxy's meek support, or we use the meek client shipped with Tor Browser.

I've tried obfs4proxy's meek client support on Tails 3.1. In the Greeter I told Tails I need bridges, in Tor Launcher I've pasted the Tor Browser's default meek bridges (replacing "meek" with "meek_lite" otherwise it won't work), it fails as expected so I've set ClientTransportPlugin obfs3,obfs4,meek_lite exec /usr/bin/obfs4proxy managed in /etc/tor/torrc, restarted tor, clicked "Reconfigure" in Tor Launcher, and I see tor is starting obfs4proxy and delegating to it the task of talking meek protocol. But there's a catch 22: connecting to the meek bridge requires working DNS… and we torify DNS resolution. This has been discussed already on #8775. Manually adding currently working IP addresses for the 2 default meek bridges fronting domains to /etc/hosts fixes that:

93.184.221.200 ajax.aspnetcdn.com
52.222.172.206 a0.awsstatic.com

… and I'm connected to Tor via meek, without installing any special software :) Now, two problems remain:

  1. I have no idea if that's enough to work in China because obfs4proxy's minimal meek client does not normalize TLS signatures.
  2. I don't know if hardcoding IP addresses in the ISO at Tails release time has any chance to work in the real world. If we're willing to give it a try, we should first check what pieces of Tails would break when the IP changes (software that does proper socks5h won't be affected, but things that go through libc/NSS will) and whether that's acceptable. Or it could be time to look into using a different DNS resolver for tor/obfs4proxy. See #8775 that's specifically about this problem.

#13 Updated by intrigeri about 2 years ago

  • Related to Bug #8775: Tor Launcher should be able to resolve DNS host names for the proxy configuration added

#14 Updated by intrigeri about 2 years ago

  • Type of work changed from Wait to Research

#15 Updated by iry about 2 years ago

Hi intigeri!

Thank you so much for testing these out! This will be a significant improvement on the censorship circumvention in Tails!

intrigeri wrote:

So our options are: either we do that meek packaging work ourselves (seems hard but perhaps focussing on meek-client and forgetting the reflector etc. makes it easier), or we use obfs4proxy's meek support, or we use the meek client shipped with Tor Browser.

Just a reminder: it seems the first two options do not have meek-reflector support on which new Tor launcher relies?

isis said in Tor ticket (https://trac.torproject.org/projects/tor/ticket/22871#comment:7):

I also just remembered that you actually can't do a POST /meek/* to BridgeDB unless you go through the meek reflector, because of the way the TLS termination is handled. Also FYI, this distributor relies on getting the client's IP address in an X-Forwarded-For header from the meek reflector. We could consider setting up the same moat API as its own separate distributor for clients which can't use meek, but that should be a new ticket. (Also, I'd prefer that they be separate distributors, since there's a possibility that we may need to allocate differently, or treat different automated bridge distribution clients differently, e.g. different rate limiting, in the future.)

I've tried obfs4proxy's meek client support on Tails 3.1. In the Greeter I told Tails I need bridges, in Tor Launcher I've pasted the Tor Browser's default meek bridges (replacing "meek" with "meek_lite" otherwise it won't work), it fails as expected so I've set ClientTransportPlugin obfs3,obfs4,meek_lite exec /usr/bin/obfs4proxy managed in /etc/tor/torrc, restarted tor, clicked "Reconfigure" in Tor Launcher, and I see tor is starting obfs4proxy and delegating to it the task of talking meek protocol. But there's a catch 22: connecting to the meek bridge requires working DNS… and we torify DNS resolution. This has been discussed already on #8775. Manually adding currently working IP addresses for the 2 default meek bridges fronting domains to /etc/hosts fixes that:

[...]

That is awesome! I tested in Debian8 and it worked well, too: https://forums.whonix.org/t/censorship-circumvention-tor-pluggable-transports/2601/3

… and I'm connected to Tor via meek, without installing any special software :) Now, two problems remain:

  1. I have no idea if that's enough to work in China because obfs4proxy's minimal meek client does not normalize TLS signatures.

Good news: I tested using meek bridges via obfs4proxy under different ISPs in China, and this approach worked pretty well in those environment. However, we need more feedback from users in China to know how well this approach work.

This is also described in the link above, btw.

  1. I don't know if hardcoding IP addresses in the ISO at Tails release time has any chance to work in the real world. If we're willing to give it a try, we should first check what pieces of Tails would break when the IP changes (software that does proper socks5h won't be affected, but things that go through libc/NSS will) and whether that's acceptable. Or it could be time to look into using a different DNS resolver for tor/obfs4proxy. See #8775 that's specifically about this problem.

That is an interesting problem, too. I am looking forward to seeing how Tails is going to solve it.

#16 Updated by yawning almost 2 years ago

intrigeri wrote:

So our options are: either we do that meek packaging work ourselves (seems hard but perhaps focussing on meek-client and forgetting the reflector etc. makes it easier), or we use obfs4proxy's meek support, or we use the meek client shipped with Tor Browser.

Unless dcf changed something when I wasn't paying attention, both helper-less meek-client and obfs4proxy's meek_lite act near identically, because I cribbed off the former to implement the latter.

  1. I have no idea if that's enough to work in China because obfs4proxy's minimal meek client does not normalize TLS signatures.

Currently this doesn't appear to matter much, or at least, if it does, then Nathan hasn't told me (Orbot uses meek_lite).

  1. I don't know if hardcoding IP addresses in the ISO at Tails release time has any chance to work in the real world. If we're willing to give it a try, we should first check what pieces of Tails would break when the IP changes (software that does proper socks5h won't be affected, but things that go through libc/NSS will) and whether that's acceptable. Or it could be time to look into using a different DNS resolver for tor/obfs4proxy. See #8775 that's specifically about this problem.

It's not immediately obvious to me if Azure does any sort of DNS based load balancing. AWS appears to be doing so.

#17 Updated by intrigeri almost 2 years ago

iry wrote:

  1. I don't know if hardcoding IP addresses in the ISO at Tails release time has any chance to work in the real world. If we're willing to give it a try, we should first check what pieces of Tails would break when the IP changes (software that does proper socks5h won't be affected, but things that go through libc/NSS will) and whether that's acceptable. Or it could be time to look into using a different DNS resolver for tor/obfs4proxy. See #8775 that's specifically about this problem.

That is an interesting problem, too. I am looking forward to seeing how Tails is going to solve it.

I've updated #8775 that now is ready to be implemented by anyone interested. iry, do you know how Whonix solved it?

#18 Updated by iry almost 2 years ago

intrigeri wrote:

I've updated #8775 that now is ready to be implemented by anyone interested. iry, do you know how Whonix solved it?

That is an exciting news to hear! Thank you so much for your work, intrigeri!

Whonix solved the problem by "allow[ing] Tor do resolve DNS using clearnet with your usual DNS settings that any clearnet VM would be using". In Whonix-Gateway, Tor is running under debian-tor user which is the only user that is granted clearnet DNS access. I am not sure if this also satisfies the adversary mode of Tails.

The code of implementation by Patrick and other development discussion can be found here: https://forums.whonix.org/t/censorship-circumvention-tor-pluggable-transports/2601/14

Please let me know if there is anything I can help with!

#19 Updated by intrigeri almost 2 years ago

  • Related to Bug #14555: Migrate to Tor Launcher compatible with Firefox ESR60 added

#20 Updated by anonym over 1 year ago

iry suggested a "not so elegant" (his words!) solution: we ship a hosts file in Tails that contain the resolutions of the needed domain names. :)

#21 Updated by intrigeri over 1 year ago

iry suggested a "not so elegant" (his words!) solution: we ship a hosts file in Tails that contain the resolutions of the needed domain names. :)

See the earlier discussion about this idea on this very ticket: it might work for Azure but probably not for AWS.

#22 Updated by iry over 1 year ago

intrigeri wrote:

iry suggested a "not so elegant" (his words!) solution: we ship a hosts file in Tails that contain the resolutions of the needed domain names. :)

See the earlier discussion about this idea on this very ticket: it might work for Azure but probably not for AWS.

AWS is using DNS based load balancing, which means the IP address of the domain name we get may change all the time; however, I doubt this does not necessarily mean the older IP addresses we got does not work at all. (Maybe the performance is not the best, but it may still work.)

Hardcoding /etc/hosts is only a temporary approach which helps to mitigate the severe problem that Tails hardly works in China.

Anonym said he would keep trying this approach: https://labs.riseup.net/code/issues/8775#note-9 which is the elegant implementation.

#23 Updated by iry over 1 year ago

  • Related to Feature #15144: Consider migrating from Tor Launcher to anon-connection-wizard added

#24 Updated by anonym over 1 year ago

  • Status changed from Confirmed to In Progress
  • Assignee set to intrigeri
  • Target version set to Tails_3.6
  • % Done changed from 0 to 40
  • QA Check set to Ready for QA
  • Feature Branch set to feature/8243-meek

The feature branch implements this ticket and its blocker, #8775. Since you've already had an initial look I don't think I have to say much about it, and I think you'll be pretty happy with its current state, except:

  • there's no design doc changes or automated test suite updates yet. I'll wait until #15197 is merged in order to save me some work.
  • commit:f5cbdd2d9008b79901ce5114d613a499b609697b enables Tor's default Meek bridges in our Tor Launcher. My understanding is that it is the stealthiest pluggable transport (in practice, among the ones we support at least) despite requiring no user input, so it feels sane to ship them since users otherwise will be manually inputting the same text any way (and probably screwing up the "meek*_lite*" part). In fact, AFAIK there is no user-friendly way to learn about meek bridges. OTOH, my worry is that Tails might overload the meek bridges, since it is the only type of bridge we support. Thoughts?

So, I'm not asking for a merge, just for input on the above question, and a review and manual testing!

#25 Updated by intrigeri over 1 year ago

anonym wrote:

  • commit:f5cbdd2d9008b79901ce5114d613a499b609697b enables Tor's default Meek bridges in our Tor Launcher. My understanding is that it is the stealthiest pluggable transport (in practice, among the ones we support at least) despite requiring no user input, so it feels sane to ship them since users otherwise will be manually inputting the same text any way (and probably screwing up the "meek*_lite*" part). In fact, AFAIK there is no user-friendly way to learn about meek bridges. OTOH, my worry is that Tails might overload the meek bridges, since it is the only type of bridge we support. Thoughts?

I have no idea how Tor Launcher looks like when only 1 type of default bridges is enabled => in order to understand if and how this is related to our reasoning and decision on #8825, I'll need to see this thing in action => I'll build an ISO and will try it out.

a review

Looks great to me modulo:

  • Let's please not introduce new non-extended regexps. Sorry for being like a broken record.
  • The "Remove TBB's default bridges. If we want them, we want it in Tor Launcher's own profile" comment is confusing: it seems to conflict with the fact we're enabling the default meek bridges in /etc/xul-ext/tor-launcher.js. Or I misunderstood, which proves it's confusing.
  • If you don't know about grep's --line-regexp, I find it useful to move complexity out of regexps, which can be interesting given the definition of ^ and $ varies accross the regexp languages (beginning/end of string vs. beginning/end of line).
  • I suspect one of your commits fixes #7903 (untested) => if I'm guessing right, please mark that ticket as RfQA to and consider rewriting history to encode this in the corresponding commit.

and manual testing!

Not there yet.

#26 Updated by intrigeri over 1 year ago

#27 Updated by intrigeri over 1 year ago

And also: since this is being retargeted to 3.6, please merge devel into your branch so we can do QA with systemd 236 (that Tails 3.6 will get if this branch is merged) instead of systemd 234 (that current builds of this branch get). Better do that before you bother running the full test suite.

#28 Updated by intrigeri over 1 year ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Dev Needed

intrigeri wrote:

anonym wrote:

  • commit:f5cbdd2d9008b79901ce5114d613a499b609697b enables Tor's default Meek bridges in our Tor Launcher. My understanding is that it is the stealthiest pluggable transport (in practice, among the ones we support at least) despite requiring no user input, so it feels sane to ship them since users otherwise will be manually inputting the same text any way (and probably screwing up the "meek*_lite*" part). In fact, AFAIK there is no user-friendly way to learn about meek bridges. OTOH, my worry is that Tails might overload the meek bridges, since it is the only type of bridge we support. Thoughts?

I have no idea how Tor Launcher looks like when only 1 type of default bridges is enabled => in order to understand if and how this is related to our reasoning and decision on #8825, I'll need to see this thing in action => I'll build an ISO and will try it out.

I've tried it and ouch, that's a tough one. I think I agree with your description of the problem. I'm not sure about what conclusion we can/should draw and how. To make sure we're on the same page let me phrase how I see things using my own words:

  • On the one hand, the reasoning that lead us (#8825) to not propose default Tor bridges applies here even stronger: back then we were considering proposing all default bridges from TBB, which would have spread the load somewhat. So I don't think we (you and I) can dismiss that alone and implement something that's totally against something we've decided at the project scale.
  • On the other hand, your UX argument makes sense: it would feel stupid to say "we now support meek but to use them, you need to locate the corresponding bridge addresses in the Tor Browser Git repo or some hidden wiki page; and then every time you start Tails, you'll need to type them correctly and of course, don't forget to replace "meek" with "meek_lite"; what, wasn't that obvious?".

So I propose we do this:

  1. Once the other known issues are resolved, merge this without the default meek bridges. Don't announce this as a new feature except in changelog. Rationale: the code is basically ready, let's not let it bitrot. It might even be useful to have this code around, merged and tested for moat support, even without the default set of bridges; we'll see, I have no idea how moat support will look like.
  2. Wait for moat support in Tor Launcher to be ready for us to test it. Rationale: it might make this discussion moot (I wonder if the default bridges will still be a thing in TB once moat is there; and even if they're not a thing anymore, for Tails users the benefits of having default bridges available becomes almost null once there's a nice UI that lets them fetch their own bridges automatically).
  3. If moat integration in Tor Launcher does not magically make this problem disappear, go back to the drawing board (i.e. reopen the project-wide discussion on #8825 and/or seek advice from UX people).

I think this plan only makes sense if we are strongly committed to make step 2 real. It might be that we have to (and then it's FT core work), it might be that we don't have to (and then we need to do it as volunteers). Either way, count me in but not alone.

#29 Updated by u over 1 year ago

#30 Updated by anonym over 1 year ago

  • Type of work changed from Research to Code

intrigeri wrote:

anonym wrote:

[...] default Meek bridges [...]

Let's continue this discussion on #8825.

a review

Looks great to me modulo:

  • Let's please not introduce new non-extended regexps. Sorry for being like a broken record.

Fixed in feature/8825-default-meek-bridges.

  • The "Remove TBB's default bridges. If we want them, we want it in Tor Launcher's own profile" comment is confusing: it seems to conflict with the fact we're enabling the default meek bridges in /etc/xul-ext/tor-launcher.js. Or I misunderstood, which proves it's confusing.

53131b0a30cb09e8eaf1368cc9f7f02509312eb1 should make it clearer.

  • If you don't know about grep's --line-regexp, I find it useful to move complexity out of regexps, which can be interesting given the definition of ^ and $ varies accross the regexp languages (beginning/end of string vs. beginning/end of line).

I know about it, but personally prefer ^$ over some tool-specific option.

  • I suspect one of your commits fixes #7903 (untested) => if I'm guessing right, please mark that ticket as RfQA to and consider rewriting history to encode this in the corresponding commit.

Actually it doesn't: notice how /etc/resolv-over-clearnet.conf is copied into the chroot. A bind-mount should solve it, but might get tricky to get right given the mount namespace. I'll have a quick look.

#31 Updated by anonym over 1 year ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

anonym wrote:

  • I suspect one of your commits fixes #7903 (untested) => if I'm guessing right, please mark that ticket as RfQA to and consider rewriting history to encode this in the corresponding commit.

Actually it doesn't: notice how /etc/resolv-over-clearnet.conf is copied into the chroot. A bind-mount should solve it, but might get tricky to get right given the mount namespace. I'll have a quick look.

Never mind the part about "mount namespaces", that's for tor only. So the fix is a trivial bind-mount, with the "hard" thing being how to deal with that mount vs try_cleanup_browser_chroot(). findmnt to the rescue! Yup, the problem #7903 wants to document a workaround for is now solved!

I have rewritten the history of feature/8243-meek, but I've only rebased on devel, reordered commits and squashed commits via fixup. You have reviewed up to and including a993d3eb30f88bf0a7a085db7e63653979846502.

#32 Updated by intrigeri over 1 year ago

  • Assignee changed from intrigeri to bertagaz

#33 Updated by intrigeri over 1 year ago

  • Related to Feature #15331: Support automatic bridge retrieval in Tor Launcher (Moat) added

#34 Updated by intrigeri over 1 year ago

anonym, even though we're not going to announce this feature anywhere but in the changelog, as a preparatory step for #15331, may I suggest you call for testing of this half-baked feature (with suitable disclaimers) on -testers@?

#35 Updated by bertagaz over 1 year ago

  • Assignee changed from bertagaz to anonym
  • QA Check changed from Ready for QA to Info Needed

This branch automated test suite runs always end up with a failure of the "Starting the Unsafe Browser without a network connection results in a complaint about no DNS server being configured" scenario, in Jenkins and at home. It does not seem related to #8775 though. Maybe #7903? Did I miss something?

#36 Updated by bertagaz over 1 year ago

I've been trying to use meek_lite PT as explained in #8243#note-12, but no success so far. I had to do some iptables ACCEPT'ing + apparomoring but it still fails to contact the authorities. I'll go on testing a bit more. Meek newbie here.

#37 Updated by bertagaz over 1 year ago

I've done further testing and stumbled on something that's been raised here before:

intrigeri wrote:

And also: since this is being retargeted to 3.6, please merge devel into your branch so we can do QA with systemd 236 (that Tails 3.6 will get if this branch is merged) instead of systemd 234 (that current builds of this branch get). Better do that before you bother running the full test suite.

Actually it seems we ship systemd 232 at the moment, and that the BindReadOnlyPaths directive was introduced in systemd 233. So we need to upgrade it for this branch to work.

#38 Updated by bertagaz over 1 year ago

bertagaz wrote:

I've done further testing and stumbled on something that's been raised here before:

intrigeri wrote:

And also: since this is being retargeted to 3.6, please merge devel into your branch so we can do QA with systemd 236 (that Tails 3.6 will get if this branch is merged) instead of systemd 234 (that current builds of this branch get). Better do that before you bother running the full test suite.

Actually it seems we ship systemd 232 at the moment, and that the BindReadOnlyPaths directive was introduced in systemd 233. So we need to upgrade it for this branch to work.

Pushed a commit installing systemd from stretch-backports. Builds fine and manual test confirms it fixes the issue. I can use meek bridges! :)

Let see what Jenkins says about shipping this systemd version. Meanwhile I'll play a bit more with this system too.

#39 Updated by anonym over 1 year ago

  • Assignee changed from anonym to bertagaz
  • QA Check changed from Info Needed to Ready for QA

bertagaz wrote:

Pushed a commit installing systemd from stretch-backports.

Crap! The plan was for me to fix that before assigning it for review to you. I'm very sorry for the time wasted on debugging this! :/

Reassigning it back to you then! :)

#40 Updated by bertagaz over 1 year ago

  • Assignee changed from bertagaz to anonym
  • QA Check changed from Ready for QA to Info Needed

bertagaz wrote:

This branch automated test suite runs always end up with a failure of the "Starting the Unsafe Browser without a network connection results in a complaint about no DNS server being configured" scenario, in Jenkins and at home. It does not seem related to #8775 though. Maybe #7903? Did I miss something?

You did not reply about this though. What's the plan? I guess it's expected that this test now fails (should have think twice before commenting) and we should just remove it?

#41 Updated by anonym over 1 year ago

  • Assignee changed from anonym to bertagaz
  • % Done changed from 40 to 50
  • QA Check changed from Info Needed to Ready for QA

bertagaz wrote:

bertagaz wrote:

This branch automated test suite runs always end up with a failure of the "Starting the Unsafe Browser without a network connection results in a complaint about no DNS server being configured" scenario, in Jenkins and at home. It does not seem related to #8775 though. Maybe #7903? Did I miss something?

You did not reply about this though. What's the plan? I guess it's expected that this test now fails (should have think twice before commenting) and we should just remove it?

Gah! So it is expected because the error we look for has moved to after the Unsafe Browser verification. I updated the test for this, but cannot find the change (I even tried a bit using git reflog). I suspect I have done some work on this branch that has been lost... any way, now I think the branch should be actually ready for you to look at and possibly merge. Sorry for the messy review! :(

#42 Updated by bertagaz over 1 year ago

  • % Done changed from 50 to 90
  • QA Check changed from Ready for QA to Pass

anonym wrote:

Gah! So it is expected because the error we look for has moved to after the Unsafe Browser verification. I updated the test for this, but cannot find the change (I even tried a bit using git reflog). I suspect I have done some work on this branch that has been lost... any way, now I think the branch should be actually ready for you to look at and possibly merge. Sorry for the messy review! :(

That's fine. I've done more tests, and finished the code reviewing. I'm ok with it now, I think we're good to release that in 3.6 so I'll merge it, yay!

#43 Updated by bertagaz over 1 year ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 90 to 100

#44 Updated by bertagaz over 1 year ago

  • Assignee deleted (bertagaz)

And so is merged!

#45 Updated by intrigeri over 1 year ago

  • Status changed from Fix committed to In Progress
  • Assignee set to anonym
  • % Done changed from 100 to 90
  • QA Check changed from Pass to Dev Needed

Ouch, while reviewing ..origin/devel I just noticed that bf317f15289f3c3beb966a9967d58f96debd0cd4 is incomplete: it only pins a subset of the binary packages built from src:systemd. E.g. in the .packages file I see libudev1:amd64 232-25+deb9u2. I really don't think that mixing 2 different versions of systemd is a good idea.

Also, I believe the corresponding comment might be incorrect: we need systemd >= v233, not systemd > v233. Finally, commenting out that text with # is not supported. I'm surprised it seemingly works and I'd rather see us 1. be consistent with the rest of this file; 2. only use documented syntax instead of relying on luck.

#46 Updated by intrigeri over 1 year ago

  • Priority changed from Normal to Elevated

#47 Updated by anonym over 1 year ago

  • QA Check changed from Dev Needed to Ready for QA
  • Feature Branch changed from feature/8243-meek to feature/8243-meek feature/8243-meek+force-all-tests

intrigeri wrote:

Ouch, while reviewing ..origin/devel I just noticed that bf317f15289f3c3beb966a9967d58f96debd0cd4 is incomplete: it only pins a subset of the binary packages built from src:systemd.

Good catch!

E.g. in the .packages file I see libudev1:amd64 232-25+deb9u2. I really don't think that mixing 2 different versions of systemd is a good idea.

Agreed! Fixed! And I also pushed a +force-all-tests branch so we get to see the results of a full run given the udev bump.

Also, I believe the corresponding comment might be incorrect: we need systemd >= v233, not systemd > v233. Finally, commenting out that text with # is not supported. I'm surprised it seemingly works and I'd rather see us 1. be consistent with the rest of this file; 2. only use documented syntax instead of relying on luck.

Fixed!

#49 Updated by anonym over 1 year ago

  • QA Check changed from Ready for QA to Dev Needed

We have one run of each branch:

Notably, the only scenario that failed in the normal (only @fragile) run was Boot with a hardware clock set way in the past and make sure that Tails sets the clock to the source date, which also failed in the force-all-tests run. It seems like the udev bump might have caused issues with at least the RTC used by our test suite VM. I will retry locally to get some more clarity.

#50 Updated by bertagaz over 1 year ago

anonym wrote:

We have one run of each branch:

Notably, the only scenario that failed in the normal (only @fragile) run was Boot with a hardware clock set way in the past and make sure that Tails sets the clock to the source date, which also failed in the force-all-tests run. It seems like the udev bump might have caused issues with at least the RTC used by our test suite VM. I will retry locally to get some more clarity.

Indeed this test seems to always fail.

It seems systemd has its own way to set the clock for PID 1 when RTC is too old (see https://github.com/systemd/systemd/commit/021dd87bc055a5bfb2dcef83fc868fe24648b959).

It uses systemd' build time to set the clock. Having a look at what time is set in the failed scenario (Sun Jan 28 15:58:39 UTC 2018.), it seems to correspond to systemd's release/build date, so I think that's the explanation. Not sure what to do about that though.

#51 Updated by anonym over 1 year ago

bertagaz wrote:

anonym wrote:

We have one run of each branch:

Notably, the only scenario that failed in the normal (only @fragile) run was Boot with a hardware clock set way in the past and make sure that Tails sets the clock to the source date, which also failed in the force-all-tests run. It seems like the udev bump might have caused issues with at least the RTC used by our test suite VM. I will retry locally to get some more clarity.

Indeed this test seems to always fail.

It seems systemd has its own way to set the clock for PID 1 when RTC is too old (see https://github.com/systemd/systemd/commit/021dd87bc055a5bfb2dcef83fc868fe24648b959).

It uses systemd' build time to set the clock. Having a look at what time is set in the failed scenario (Sun Jan 28 15:58:39 UTC 2018.), it seems to correspond to systemd's release/build date, so I think that's the explanation. Not sure what to do about that though.

Cheers! What a perfect timing, I was just investigating this and had came to the conclusion that the systemd bump probably introduced something like this. I'll come up with something!

#52 Updated by intrigeri over 1 year ago

bertagaz wrote:

It uses systemd' build time to set the clock. Having a look at what time is set in the failed scenario (Sun Jan 28 15:58:39 UTC 2018.), it seems to correspond to systemd's release/build date, so I think that's the explanation. Not sure what to do about that though.

FTR I see this exact same failure, with that exact same time, on a test suite run for an ISO built from a branch built on top of current devel. So it seems this regression was already introduced there and is not caused by the follow-up fixes anonym is working on. That commit first appeared in systemd v229 so we're only been "lucky" not to be hit by that test suite bug earlier.

#53 Updated by anonym over 1 year ago

  • Assignee changed from anonym to bertagaz
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

bertagaz wrote:

It uses systemd' build time to set the clock. Having a look at what time is set in the failed scenario (Sun Jan 28 15:58:39 UTC 2018.), it seems to correspond to systemd's release/build date, so I think that's the explanation. Not sure what to do about that though.

FTR I see this exact same failure, with that exact same time, on a test suite run for an ISO built from a branch built on top of current devel. So it seems this regression was already introduced there and is not caused by the follow-up fixes anonym is working on. That commit first appeared in systemd v229 so we're only been "lucky" not to be hit by that test suite bug earlier.

Indeed! I sneaked the fix into this branch any way. :)

#54 Updated by bertagaz over 1 year ago

  • Status changed from In Progress to Fix committed
  • Assignee deleted (bertagaz)
  • % Done changed from 90 to 100
  • QA Check changed from Ready for QA to Pass

anonym wrote:

intrigeri wrote:

FTR I see this exact same failure, with that exact same time, on a test suite run for an ISO built from a branch built on top of current devel. So it seems this regression was already introduced there and is not caused by the follow-up fixes anonym is working on. That commit first appeared in systemd v229 so we're only been "lucky" not to be hit by that test suite bug earlier.

Indeed! I sneaked the fix into this branch any way. :)

Works fine so it's merged into devel. Let's time this time we're good. :)

#55 Updated by bertagaz over 1 year ago

  • Status changed from Fix committed to Resolved

#56 Updated by mercedes508 about 1 year ago

#57 Updated by mercedes508 11 months ago

#58 Updated by emmapeel 3 months ago

  • Status changed from Resolved to Confirmed
  • Target version deleted (Tails_3.6)

after some discussion on XMPP, we don't see this issue as solved. reopening thus.

#59 Updated by intrigeri about 1 month ago

  • Related to Feature #5461: Persistence preset: Tor configuration added

#60 Updated by anonym about 1 month ago

  • Status changed from Confirmed to In Progress

#61 Updated by intrigeri about 1 month ago

  • Status changed from In Progress to Confirmed

#62 Updated by intrigeri 15 days ago

It's worth noting that we should try to have our meek client have a close enough fingerprint to Tor Browser's: otherwise, censors can more easily block our meek connections; and Tails users that choose meek bridges can be singled out and tracked (especially true as it's currently so hard to use meek in Tails, I doubt you need more than one hand to count these users).

So far, we barely tried: we've been using a fully different client (obfs4proxy's meek_lite, while Tor Browser 8.5 used the full-blown meek client, which uses Firefox for its network communication and especially TLS). But the situation is changing — starting with Tor Browser 9, they also use obfs4proxy's meek_lite: https://trac.torproject.org/projects/tor/ticket/29430.

This is good because it's closer to what we do. But if I got it right, they could do it because the new version of obfs4proxy they're using uses its own, forked uTLS, that can emulate closely enough Firefox' fingerprint: https://trac.torproject.org/projects/tor/ticket/29077, https://trac.torproject.org/projects/tor/ticket/29627. There seems to be little chances Debian can match this any time soon, so if we want our meek client to have the same fingerprint as Tor Browser's, likely we'll need to use the binary of obfs4proxy bundled with Tor Browser instead of the one from Debian.

Also available in: Atom PDF