Project

General

Profile

Feature #15022

Electrum package update to version 3.0.6

Added by s7r almost 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
-
Target version:
Start date:
12/07/2017
Due date:
% Done:

100%

Feature Branch:
feature/15022-electrum-3.0.6
Type of work:
Code
Blueprint:
Starter:
Affected tool:
Electrum

Description

There was a major upgrade in Bitcoin (SegWit) that aims to increase network's capacity in terms of transactions per second. Big picture - it consists in discounting the bytes of ECDSA signatures and should have the effect to reduce transaction fees and fix some unwanted transaction mutation. This brings new address types (bench32) which are not compatible with legacy clients.

Taking these in consideration, we must upgrade to (at least) Electrum 3.0.2 as fast as possible. The current version still works, and will continue to do so for sufficient time in advance (without the SegWit feature) so setting this as normal priority.

The config file format did not change from 2.7.x so users with persistence enabled that have a config file on disk should not be affected.

Upstream migrated this project to python3 and at this moment we are waiting for python3-jsonrpclib-pelix to go though new, before we can have Electrum 3.0.2 in testing.

We will include 3.0.2 in the first Tails release after it lands in stretch-backports. I'll be sure to update this ticket when this happens.


Related issues

Related to Tails - Bug #15390: TREZOR hardware wallet support is gone since Electrum was updated to 3.x Resolved 03/09/2018
Related to Tails - Bug #15391: Stop shipping python-qt4 that is no longer required Resolved 03/09/2018
Related to Tails - Bug #15452: Check the status of Electrum qrcode support since we upgraded it to 3.x Rejected 03/23/2018
Duplicated by Tails - Bug #15151: Upgrade Electrum to version 3.0.4. Older versions have JSONRPC vulnerability to steal cryptocurrencies Duplicate 01/07/2018
Duplicated by Tails - Bug #15207: JSONRPC vulnerability in Electrum 2.6 to 3.0.4 Duplicate 01/21/2018 01/22/2018
Blocks Tails - Bug #12323: Electrum offline signing Resolved 03/12/2017
Blocks Tails - Bug #15201: Disable Memory Hole for outgoing emails in Tails Resolved 01/19/2018

Associated revisions

Revision de845c29 (diff)
Added by bertagaz over 1 year ago

Ship electrum from stretch-backports.

It's backward compatible with 2.x, is reported to work well when
migrating, and brings improvements.

Refs: #15022

Revision daf6318d (diff)
Added by anonym over 1 year ago

Install required deps for Electrum from stretch-backports.

Refs: #15022

Revision 2ed2f4d6 (diff)
Added by anonym over 1 year ago

Test suite: dogtailify electrum.feature.

Upgrading to 3.x requires non-trivial modifications, so I took
the (slight) extra time to make us less dependent on future
changes (and possibly solving robustness issues for free).

Refs: #15022, #10475, #11697

Revision 6bf98ac1 (diff)
Added by bertagaz over 1 year ago

Bump a bit the time waiting for electrum window to appear.

It takes a bit of time and without that waiting, the step fails.

Refs: #15022

Revision 795caa42
Added by anonym over 1 year ago

Merge remote-tracking branch 'origin/feature/15022-electrum-3.0.6' into devel

Fix-committed: #15022

History

#1 Updated by intrigeri almost 2 years ago

  • Affected tool changed from Additional Software Packages to Electrum

#2 Updated by intrigeri almost 2 years ago

  • Status changed from New to Confirmed
  • Assignee changed from anonym to s7r
  • Type of work changed from Code to Wait
  • Starter deleted (Yes)

(Apparently the next step — keeping us updated once upgrading is a workable option — is on your plate. Calling this "Wait" but if you're actually helping make this happen in Debian, please set Type of work = "Debian" instead.)

#3 Updated by intrigeri almost 2 years ago

  • Blocks Bug #12323: Electrum offline signing added

#4 Updated by intrigeri almost 2 years ago

  • Description updated (diff)

#5 Updated by s7r over 1 year ago

  • Subject changed from Electrum package update to version 3.0.2 to Electrum package update to version 3.0.5

Update: Electrum 3.0.5 just hit unstable, waiting for migration to testing and backport. Includes proper fix for the non protected JSONRPC interface, bug that appears not exploitable in Tails.

#6 Updated by s7r over 1 year ago

  • Assignee changed from s7r to anonym
  • Type of work changed from Wait to Test

Ok, we are good to go here.
Electrum 3.0.5-1~bpo9+1 is in `stretch-backports`, ported from `testing`. Taking the liberty to assign the ticket to anonym (if I get a punch at the next dev meeting I'll know why).

Notes:
- Python3 required;

- This must be included in the next Tails release, but it's not so critical in order to alter the release roadmap (the previous version works and the JSON RPC non-protected interface bug is not exploitable in Tails).

- The config file format located in ~/.electrum did not change, like it happened from 1.9.x to 2.7.x that affected users with persistence enabled that upgraded but maintained their old config in ~/.electrum and needed to delete the folder entirely and restore wallet from seed (as we recommended in documentation). So, at this upgrade we should not run into this problem.

#7 Updated by intrigeri over 1 year ago

  • Duplicated by Bug #15151: Upgrade Electrum to version 3.0.4. Older versions have JSONRPC vulnerability to steal cryptocurrencies added

#8 Updated by intrigeri over 1 year ago

  • Target version set to Tails_3.6

#9 Updated by intrigeri over 1 year ago

  • Assignee changed from anonym to s7r

Ok, we are good to go here.

Great!

FYI nobody is responsible for maintaining Electrum in Tails, so it would be nice if you could prepare a branch (based on devel) that pulls electrum from stretch-backports, build an ISO and confirm it works fine. You've got plenty of time given our next major release is scheduled for mid-March (expect a feature freeze around the end of February though). If you want to do that, starting points:

:)

#10 Updated by anonym over 1 year ago

  • Status changed from Confirmed to In Progress
  • Target version changed from Tails_3.6 to Tails_3.5
  • % Done changed from 0 to 20
  • QA Check changed from Dev Needed to Info Needed

s7r wrote:

Ok, we are good to go here.
Electrum 3.0.5-1~bpo9+1 is in `stretch-backports`, ported from `testing`.

Great news!

Taking the liberty to assign the ticket to anonym (if I get a punch at the next dev meeting I'll know why).

You did the right thing!

Notes:
- Python3 required;

One less Python2 (and Qt4!) dependency, so this is good news!

- This must be included in the next Tails release, but it's not so critical in order to alter the release roadmap (the previous version works and the JSON RPC non-protected interface bug is not exploitable in Tails).

If this is important enough, we'll include it in Tails 3.5 (planned on Jan 23) even though it is a "bugfix" release only and we only have a few more days to do the necessary work and testing.

However, I have trouble assessing this importance; I find the statements "must be included in the next Tails release, but it's not so critical" self-contradictory. What is the actual impact for our users if we wait for another two months? What are the expected common/worst bad user scenarios? E.g. is there a problem with receiving/sending BTC from/to a SegWit address? (Note: I need an answer to this question ASAP, everything else can wait!)

- The config file format located in ~/.electrum did not change, like it happened from 1.9.x to 2.7.x that affected users with persistence enabled that upgraded but maintained their old config in ~/.electrum and needed to delete the folder entirely and restore wallet from seed (as we recommended in documentation). So, at this upgrade we should not run into this problem.

Thanks for making this clear!

intrigeri wrote:

FYI nobody is responsible for maintaining Electrum in Tails [...]

I am committed to maintain the basic functionality. (Have I said anything different? I very well may have, and forgotten about it, but in that case I have changed my mind :)) This commitment completely depends on that s7r (or someone else) keeps me up-to-date about changes in Electrum that impact Tails, and does the needed Debian work (or make someone else do it), which seems to have worked brilliantly for this ticket. Yay! :)

[...] so it would be nice if you could prepare a branch [...]

So no need for this, I'll happily do the Tails bits.

That said, if you, s7r, really want to do the Tails bits as well, no one would be happier than me, and I'd be delighted to mentor you, if needed. Of course, time is short if this is going into Tails 3.5, so if that is our goal I'd suggest we'd postpone until a time then tings are more relaxed.

#11 Updated by u over 1 year ago

  • Target version changed from Tails_3.5 to Tails_3.6

I can't see this version of Electrum in stretch-backports https://tracker.debian.org/pkg/electrum, postponing to 3.6.

#12 Updated by s7r over 1 year ago

anonym wrote:

Notes:

If this is important enough, we'll include it in Tails 3.5 (planned on Jan 23) even though it is a "bugfix" release only and we only have a few more days to do the necessary work and testing.

However, I have trouble assessing this importance; I find the statements "must be included in the next Tails release, but it's not so critical" self-contradictory. What is the actual impact for our users if we wait for another two months? What are the expected common/worst bad user scenarios? E.g. is there a problem with receiving/sending BTC from/to a SegWit address? (Note: I need an answer to this question ASAP, everything else can wait!)

You are right - apologies for the wrong wording. What I wanted to say is to include it in the next release but don't have to do one release just for Electrum upgrade.
I don't see a bad scenario:
- The only security bug in the version currently in Tails is not exploitable in our setup;
- The version currently shipped is compatible with the server protocol;
- The version currently shipped is able to properly calculate dynamic fees for transaction dependent on the network overall transaction volume, so our users can make payments;
- There is no problem in receiving BTC (regardless if they come from native SegWit inputs, nested SegWit in P2SH inputs, legacy inputs or whatever mixture of these).

The only limitation is sending BTC to a native SegWit bech32 address (but the limitation does not apply for SegWit nested in P2SH) but this I am not considering this a problem because native bech32 SegWit addresses are new and not currently implemented. Exchanges don't allow withdraws to such addresses, refunds from BTC payment processors are also not supported in native SegWit addresses and so on. Native SegWit is not even a default in Electrum 3.0.5, it has to be explicitly opt-in by user at setup and I don't think everyone knows what it means currently so they will just select "Standard" wallet which is the same. Nothing prevents users from transacting native SegWit <-> native SegWit if both use and know about it and want to benefit of lower fees, so we shouldn't forbid this for too much time.

intrigeri wrote:

FYI nobody is responsible for maintaining Electrum in Tails [...]

I am committed to maintain the basic functionality. (Have I said anything different? I very well may have, and forgotten about it, but in that case I have changed my mind :)) This commitment completely depends on that s7r (or someone else) keeps me up-to-date about changes in Electrum that impact Tails, and does the needed Debian work (or make someone else do it), which seems to have worked brilliantly for this ticket. Yay! :)

Already committed to this. A promise is a promise. If something outside my control happens that prevents me from keeping up to this, I'll surely give a heads up in advance, but very low chances for this.

[...] so it would be nice if you could prepare a branch [...]

So no need for this, I'll happily do the Tails bits.

That said, if you, s7r, really want to do the Tails bits as well, no one would be happier than me, and I'd be delighted to mentor you, if needed. Of course, time is short if this is going into Tails 3.5, so if that is our goal I'd suggest we'd postpone until a time then tings are more relaxed.

Thanks for this. It would be my pleasure to learn some more and and maybe after I can work and help with more stuff, not just Electrum. I have infrastructure I can use for testing. I will need some basic guidance at first but nothing is entirely new to me so I'll catch up quick. However I'm not sure I can properly address this in time even for 3.6 (starting February I need to migrate the office entirely to a new building).

However, until then I'd suggest for you to take care of the Tails bits in this particular one and move on with the upgrade asap. Given the high ethics standards we follow here, I know you agree we should not postpone an upgrade to software that handles people's money. I'd say at most 3.6.
.

#13 Updated by s7r over 1 year ago

u wrote:

I can't see this version of Electrum in stretch-backports https://tracker.debian.org/pkg/electrum, postponing to 3.6.

https://ftp-master.debian.org/new/electrum_3.0.5-1~bpo9+1.html

The upload succeeded but I think it has to pass through NEW. Some of the dependencies were not in `stretch-backports` (like python3-jsonrpclib-pelix) and were uploaded at the same time.

@anonym given my answers to your questions, you think we are OK with leaving this for 3.6?

#14 Updated by intrigeri over 1 year ago

Also note that Electrum 2.x is going away from Stretch: https://bugs.debian.org/887412

#15 Updated by s7r over 1 year ago

intrigeri wrote:

Also note that Electrum 2.x is going away from Stretch: https://bugs.debian.org/887412

Right. 3.x was upgraded to server <-> client protocol 1.1.

Update for @anonym wrt to worst case scenarios / limitations:

This doesn't necessarily mean that 2.7.9 is currently not working at all, because there are still servers out there that run the legacy protocol on server side. However, we can't say exactly how many of them are out there and there is no guarantee or estimation of how long in the future these legacy servers will continue to run.

Maybe we have enough time till Jan 23 for 3.5? If yes, please update the ticket target accordingly.

#16 Updated by intrigeri over 1 year ago

FWIW I've pushed a few commits to feature/buster today that adjust stuff for Electrum 3.x. I guess they could be cherry-picked for this ticket.

#17 Updated by intrigeri over 1 year ago

  • Duplicated by Bug #15207: JSONRPC vulnerability in Electrum 2.6 to 3.0.4 added

#18 Updated by s7r over 1 year ago

  • % Done changed from 20 to 40
  • QA Check changed from Info Needed to Dev Needed
From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>
Date: Wed, 24 Jan 2018 at 12:00
Subject: electrum_3.0.5-1~bpo9+1_amd64.changes ACCEPTED into stretch-backports, stretch-backports

Now the future updates should take much less time to migrate from `unstable` to `stable-backports` because this package won't have to go through the NEW queue again.

#19 Updated by intrigeri over 1 year ago

Woohoo! I guess next step is you test it in Tails 3.5 and once happy, reassign to anonym?

#20 Updated by s7r over 1 year ago

  • Subject changed from Electrum package update to version 3.0.5 to Electrum package update to version 3.0.6

Electrum 3.0.6 was released and I would like to use this one since we are doing the work to upgrade -- as soon as it's in deb, I will do the tests and reassign to anonym. Time should be sufficient until the release.

3.0.6 is not a security update, but it includes a fix that is related to incoming transaction parsing (segwit transactions with more than 1 witness program). This is important because users might not have their correct (complete) balance shown, and we know at least 1 exchange is sending transactions to users that are not possible to show in balance by Electrum < 3.0.6.

#21 Updated by intrigeri over 1 year ago

Electrum 3.0.6 was released and I would like to use this one since we are doing the work to upgrade -- as soon as it's in deb, I will do the tests and reassign to anonym.

Great!

Time should be sufficient until the release.

Keep in mind that the freeze is scheduled on 2018-03-01 (the actual release date is not very relevant here).

#22 Updated by s7r over 1 year ago

  • Assignee changed from s7r to anonym
  • % Done changed from 40 to 70
  • Type of work changed from Test to Pass Test

I have tested Electrum 3.0.6 in last nightly build: tails-amd64-devel-3.6-20180224T0700Z-c02640b433 by installing it via apt-get from stretch-backports. It installed without any problems and I was able to generate a new native segwit wallet, perform functions, connect to multiple servers in the server pool (which is not currently possible with 2.7.9 because of the client-to-server protocol upgrade).

I also tested in an existing Tails 3.5 with persistence enabled as follows:
a) install Tails 3.5 and setup persistence for everything;
b) run electrum 2.7.9 (the one which is currently in Tails 3.5) and create a wallet (it was not able to connect to any servers);
c) did some customizations in regards to denomination and fees so that the config file is edited; took a snapshot of the config file in ~/.electrum/config and reboot
d) add stretch-backports to apt/sources.list in Tails 3.5 and upgrade to electrum 3.0.6 from stretch-backports. The installation went good with no errors.

Started again Electrum 3.0.6 and:
- gui started without any errors;
- it was able to connect to the servers and started downloading headers;
- the previous wallet was there, was opened without errors, addresses were indexed properly and preserved properly;
- compared the new config file with the snapshot took previously with 2.7.9 and everything that was previously there was preserved in the same form, and some new stuff specific to 3.0.6 was added (such as rpc password with strong entropy).

So, the config file format from 2.7.9 is ok when persistent users upgrade, there is no need to manually delete it or alter it in any way.
I am happy with all test results.

Tests 3 and 4 I did was with an used wallet that has historic transactions, and tests went good for both:
a) directly install Electrum 3.0.6 and restore wallet from seed - historic transactions recovered properly;
b) restored the old wallet seed to Electrum 2.7.9 and then upgraded to 3.0.6 with persistence enabled - I couldn't connect to any server with 2.7.9, but after upgrading to 3.0.6 history was recovered without having to restore from seed again;

So I think all tests went good and Electrum 3.0.6 can be included in Tails 3.6. Assigning to @anonym.

#23 Updated by bertagaz over 1 year ago

  • Feature Branch set to feature/15022-electrum-3.0.6

(I've taken the liberty to push a branch installing electrum from stretch-backports to have CI results ASAP.)

#24 Updated by intrigeri over 1 year ago

  • Type of work changed from Pass Test to Code

#25 Updated by intrigeri over 1 year ago

bertagaz wrote:

(I've taken the liberty to push a branch installing electrum from stretch-backports to have CI results ASAP.)

Thanks! Unfortunately it FTBFS (electrum : Depends: python3-electrum (= 3.0.6-1~bpo9+1) but it is not going to be installed).

#26 Updated by anonym over 1 year ago

  • Priority changed from Normal to Elevated

#27 Updated by anonym over 1 year ago

  • Feature Branch changed from feature/15022-electrum-3.0.6 to feature/15022-electrum-3.0.6feature/15022-electrum-3.0.6

intrigeri wrote:

bertagaz wrote:

(I've taken the liberty to push a branch installing electrum from stretch-backports to have CI results ASAP.)

Thanks! Unfortunately it FTBFS (electrum : Depends: python3-electrum (= 3.0.6-1~bpo9+1) but it is not going to be installed).

Fix pushed so now it should build.

#28 Updated by anonym over 1 year ago

  • Assignee changed from anonym to bertagaz
  • % Done changed from 70 to 100
  • QA Check changed from Dev Needed to Ready for QA
  • Feature Branch changed from feature/15022-electrum-3.0.6feature/15022-electrum-3.0.6 to feature/15022-electrum-3.0.6

I dogtailified electrum.feature since the steps needed substantial work any way given the bump to 3.x. Please review'n'merge!

#29 Updated by bertagaz over 1 year ago

  • Assignee changed from bertagaz to anonym

anonym wrote:

I dogtailified electrum.feature since the steps needed substantial work any way given the bump to 3.x. Please review'n'merge!

Ok, reviewed and tested. I noticed that the electrum window takes a bit of time to show up, and the test suite does not wait long enough, so the scenario fails a lot because of that.

I've pushed 6bf98ac1bf63228cd8a96c972f986cd2c50e9e9a to fix that. Please review it and if happy merge this branch. :)

#30 Updated by bertagaz over 1 year ago

  • Blocks Bug #15201: Disable Memory Hole for outgoing emails in Tails added

#31 Updated by anonym over 1 year ago

  • Status changed from In Progress to Fix committed
  • Assignee changed from anonym to bertagaz
  • QA Check changed from Ready for QA to Info Needed

bertagaz wrote:

anonym wrote:

I dogtailified electrum.feature since the steps needed substantial work any way given the bump to 3.x. Please review'n'merge!

Ok, reviewed and tested. I noticed that the electrum window takes a bit of time to show up, and the test suite does not wait long enough, so the scenario fails a lot because of that.

I've pushed 6bf98ac1bf63228cd8a96c972f986cd2c50e9e9a to fix that. Please review it and if happy merge this branch. :)

LGTM => merged!

Edit: removed irrelevant question.

#32 Updated by anonym over 1 year ago

  • Assignee deleted (bertagaz)
  • QA Check changed from Info Needed to Pass

#33 Updated by intrigeri over 1 year ago

  • Related to Bug #15390: TREZOR hardware wallet support is gone since Electrum was updated to 3.x added

#34 Updated by intrigeri over 1 year ago

  • Related to Bug #15391: Stop shipping python-qt4 that is no longer required added

#35 Updated by bertagaz over 1 year ago

  • Status changed from Fix committed to Resolved

#36 Updated by intrigeri over 1 year ago

  • Related to Bug #15452: Check the status of Electrum qrcode support since we upgraded it to 3.x added

Also available in: Atom PDF