Project

General

Profile

Feature #16356

Upgrade to Tor Browser 9.0 (based on Firefox 68)

Added by intrigeri 7 months ago. Updated 10 days ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
01/14/2019
Due date:
% Done:

0%

Feature Branch:
feature/16356-tor-browser-9.0+force-all-tests
Type of work:
Code
Blueprint:
Starter:
Affected tool:
Browser

Description


Subtasks

Feature #16357: Deal with Torbutton being integrated into Tor BrowserConfirmed


Related issues

Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed 03/22/2019
Blocks Tails - Bug #16693: Upgrade to Tor Browser based on Firefox 68.2 Confirmed

Associated revisions

Revision b88bb369 (diff)
Added by anonym 3 months ago

Upgrade Tor Browser to 9.0a1.

Refs: #16356

Revision af719e9d (diff)
Added by anonym 3 months ago

Adapt Tor Launcher installation vs Tor Browser 9.0a1.

As of Tor Browser 9.0a1 the Tor Launcher extension is a "system"
add-on (like pdfjs) but we can still convert it to something that can
be run as a XUL standalone application by just moving things around a
bit.

Refs: #16356, #15709

Revision 7c96cb58 (diff)
Added by anonym 3 months ago

Tor Browser: refresh patch for apply_extension_code_signing_hacks().

Refs: #16356

Revision 1744b37b (diff)
Added by anonym 3 months ago

Unsafe/Tor Browser: disable the integrated Tor Launcher.

Refs: #16356, #15709

Revision f74cec4d (diff)
Added by anonym 3 months ago

onion-grater: retry connecting to the real control port.

This should not be necessary and is just placed here as a workaround
until I can figure out the real issue (bug in onion-grater? bug in
stem?).

When working on the Tor Browser 9.0 migration there were issues with
Tor Launcher. It starts and does its initial connection to the control
port where it successfully fetches CONFs etc. When clicking "Connect"
it successfully sends a few more things and disconnects, only to
reconnect immediately later (due to some implementation detail in Tor
Launcher). This reconnection fails:

[...]
/usr/local/lib/tor-browser/firefox-unconfined (pid: 8865, user: tor-launcher, port: 57472, filter: tor-launcher): > SAVECONF
/usr/local/lib/tor-browser/firefox-unconfined (pid: 8865, user: tor-launcher, port: 57472, filter: tor-launcher) disconnected: Client closed its socket
[ Here comes the reconnect: ]
/usr/local/lib/tor-browser/firefox-unconfined (pid: 8865, user: tor-launcher, port: 57476, filter: tor-launcher) connected: loaded filter: tor-launcher
Final rules: [...]
Unable to connect to tor. Maybe it's running without a ControlPort?
/usr/local/lib/tor-browser/firefox-unconfined (pid: 8865, user: tor-launcher, port: 57476, filter: tor-launcher) disconnected: client quit
---------------------------------------

Exception happened during processing of request from ('127.0.0.1', 57476)
Traceback (most recent call last):
File "/usr/lib/python3.5/socketserver.py", line 625, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python3.5/socketserver.py", line 354, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python3.5/socketserver.py", line 681, in init
self.handle()
File "/usr/local/lib/onion-grater", line 629, in handle
self.controller = self.connect_to_real_control_port()
File "/usr/local/lib/onion-grater", line 570, in connect_to_real_control_port
stem.connection.authenticate_cookie(controller, cookie_path=global_args.control_cookie_path)
File "/usr/lib/python3/dist-packages/stem/connection.py", line 803, in authenticate_cookie
auth_response = _msg(controller, msg)
File "/usr/lib/python3/dist-packages/stem/connection.py", line 1055, in _msg
return controller.msg(message)
AttributeError: 'NoneType' object has no attribute 'msg'
----------------------------------------

But if we just connect yet again, it works, hence the workaround.

Refs: #16356, #15709

Revision 3b6c5dbc (diff)
Added by anonym 2 months ago

Upgrade Tor Browser to 9.0a2 (refs: #16356).

History

#1 Updated by intrigeri 7 months ago

#2 Updated by intrigeri 5 months ago

#3 Updated by intrigeri 5 months ago

#4 Updated by intrigeri 4 months ago

  • Blocks Bug #16693: Upgrade to Tor Browser based on Firefox 68.2 added

#5 Updated by anonym 3 months ago

  • Status changed from Confirmed to In Progress
  • Assignee set to anonym

Gonna import 9.0a1 as a first step.

#6 Updated by anonym 3 months ago

  • Feature Branch set to feature/16356-tor-browser-9.0

Unless I messed up my rebase, the feature branch should build and result in a working Tor Browser and Unsafe Browser, with Tor Launcher disabled. I've tested them minimally without issue, so that's progress!

Unfortunately Tor Launcher doesn't start properly -- its window starts as just a few pixels but can at least be resized, but then you see it's just all black and uninteractable. I think my last commit will fix this. If not, the script below (from my experimenting on #15709) succeeds at what 10-tbb.sh apparently fails at and should be useful to debug this (e.g. compare the resulting dir with what 10-tbb.sh puts into the builds):

rm -rf /usr/local/lib/tor-launcher-standalone /tmp/browser-omni /home/tor-launcher/.tor-launcher/
7z x -o/tmp/browser-omni /usr/local/lib/tor-browser/browser/omni.ja
cp -a /tmp/browser-omni/chrome/torlauncher/ /usr/local/lib/tor-launcher-standalone
cp /lib/live/mount/rootfs/filesystem.squashfs/usr/local/lib/tor-launcher-standalone/application.ini /usr/local/lib/tor-launcher-standalone/application.ini
mkdir /usr/local/lib/tor-launcher-standalone/chrome
mv /usr/local/lib/tor-launcher-standalone/{content,locale,skin} /usr/local/lib/tor-launcher-standalone/chrome
mkdir -p /usr/local/lib/tor-launcher-standalone/defaults/preferences
cp /tmp/browser-omni/defaults/preferences/torlauncher-prefs.js /usr/local/lib/tor-launcher-standalone/defaults/preferences/prefs.js
grep torlauncher /tmp/browser-omni/chrome/chrome.manifest \
  | sed --regexp-extended \
    -e 's@^(content|locale|skin) (torlauncher.*) torlauncher/(.*)$@\1 \2 chrome/\3@' \
    -e 's@^(component) (\S+) torlauncher/(.+)$@\1 \2 \3@' \
    -e 's@^(resource torlauncher) .*$@\1 ./@' \
  > /usr/local/lib/tor-launcher-standalone/chrome.manifest
chmod -R a+rX /usr/local/lib/tor-launcher-standalone
tails-tor-launcher

#7 Updated by anonym 3 months ago

Note to self: I never actually successfully configured anything with Tor Launcher, I've just seen it started. Trying to configure anything actually just says there are problems with the control port. In the journal I see:

stem.ProtocolError: GETINFO response didn't have an OK status:
Command filtered

so I probably just have to fix something in the onion-grater filter.

#8 Updated by anonym 3 months ago

NOTE TO SELF: These results were from my test setup, not Tails, so this problem has never occurred in Tails.

Eh, from what I can tell, when I press "Connect" in Tor Launcher, onion-grater tries to connect to itself and (naturally) fails:

pgrep onion-grater
23458

[... normal stuff, where the tor-launcher filter is loaded and there's some back-and-forth ...]
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None) connected: loaded filter: None
Final rules:
commands: {}
events: {}
restrict-stream-events: false

/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): -> PROTOCOLINFO 1
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): <- 250-PROTOCOLINFO 1
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): <- 250-AUTH METHODS=NULL
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): <- 250-VERSION Tor="0.3.5.8" 
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): <- 250 OK
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): -> AUTHENTICATE
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): <- 250 OK
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): -> SETEVENTS
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): <- 250 OK
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): -> GETCONF __owningcontrollerprocess
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): command filtered: GETCONF __owningcontrollerprocess
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): <- 510 Command filtered
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): -> GETINFO version
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): command filtered: GETINFO version
/usr/bin/python3.5 (pid: 23458, user: root, port: 59818, filter: None): <- 510 Command filtered
/usr/bin/python3.5 (pid: 23458, user: root, port: 59812, filter: None) disconnected: client quit
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 59812)
Traceback (most recent call last):
  File "/usr/lib/python3.5/socketserver.py", line 625, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.5/socketserver.py", line 354, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.5/socketserver.py", line 681, in __init__
    self.handle()
  File "./onion-grater", line 629, in handle
    self.controller = self.connect_to_real_control_port()
  File "./onion-grater", line 569, in connect_to_real_control_port
    controller = stem.connection.connect(control_socket=global_args.control_socket_path)
  File "/usr/lib/python3/dist-packages/stem/connection.py", line 287, in connect
    return _connect_auth(control_connection, password, password_prompt, chroot_path, controller)
  File "/usr/lib/python3/dist-packages/stem/connection.py", line 371, in _connect_auth
    return controller(control_socket, is_authenticated = True)
  File "/usr/lib/python3/dist-packages/stem/control.py", line 1041, in __init__
    self.add_event_listener(_sighup_listener, EventType.SIGNAL)
  File "/usr/lib/python3/dist-packages/stem/control.py", line 2997, in add_event_listener
    if event_type and (self.get_version() < event_type._VERSION_ADDED):
  File "/usr/lib/python3/dist-packages/stem/control.py", line 454, in wrapped
    return func(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/stem/control.py", line 1235, in get_version
    version = stem.version.Version(self.get_info('version'))
  File "/usr/lib/python3/dist-packages/stem/control.py", line 454, in wrapped
    return func(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/stem/control.py", line 1163, in get_info
    stem.response.convert('GETINFO', response)
  File "/usr/lib/python3/dist-packages/stem/response/__init__.py", line 132, in convert
    message._parse_message(**kwargs)
  File "/usr/lib/python3/dist-packages/stem/response/getinfo.py", line 40, in _parse_message
    raise stem.ProtocolError("GETINFO response didn't have an OK status:\n%s" % self)
stem.ProtocolError: GETINFO response didn't have an OK status:
Command filtered

How/why does it connect to itself????

#9 Updated by anonym 3 months ago

I think I have stumbled upon a bug in stem (or onion-grater?) that I had to workaround with f74cec4d7fc44d9b9eec37090990e1868ba592db so we can get some test results. That commit should be reverted, or the code it adds should be made saner (e.g. don't retry indefinitely) if we really need something like that.

Any way, as of now, everything seems to work on the surface level! \o/ Let's see what Jenkins thinks of the non-fragile tests -- later I should repush with +force-all-tests@ and compare those tests too.

#10 Updated by anonym 3 months ago

  • Feature Branch changed from feature/16356-tor-browser-9.0 to feature/16356-tor-browser-9.0+force-all-tests

anonym wrote:

Let's see what Jenkins thinks of the non-@fragile tests

Ah, it turns out those already passes: https://jenkins.tails.boum.org/job/test_Tails_ISO_feature-16356-tor-browser-9.0/1/

later I should repush with +force-all-tests and compare those tests too.

So I'll repush now.

#11 Updated by anonym 3 months ago

The first full test suite run went pretty well:

Failing Scenarios:
cucumber features/additional_software_packages.feature:69 # Scenario: Recovering in offline mode after Additional Software previously failed to upgrade and then succeed to upgrade when online
cucumber features/electrum.feature:15 # Scenario: Using a persistent Electrum configuration
cucumber features/tor_stream_isolation.feature:9 # Scenario: tails-security-check is using the Tails-specific SocksPort
cucumber features/totem.feature:50 # Scenario: Watching a WebM video over HTTPS

219 scenarios (4 failed, 215 passed)
1658 steps (3 failed, 1655 passed)
346m16.087s

No failure is related to the browser, so we're pretty in great shape!

#12 Updated by anonym 2 months ago

Updated to Tor Browser 9.0a2. What do you think, Jenkins?

#13 Updated by intrigeri about 2 months ago

  • Description updated (diff)

#14 Updated by intrigeri 15 days ago

#15 Updated by segfault 10 days ago

  • Description updated (diff)

Also available in: Atom PDF