Project

General

Profile

Feature #11529

Save data to Persistence when it is created (no need to restart)

Added by huertanix over 3 years ago. Updated 4 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
-
Category:
Persistence
Target version:
-
Start date:
06/13/2016
Due date:
% Done:

0%

Feature Branch:
feature/11529-enable-persistence-immediately,tps:11529-enable-persistence-immediately
Type of work:
Code
Blueprint:
Starter:
No
Affected tool:

Description

NOTE: This is one of a series of usability issues that arose from the TAILS OS training workshop on March 19th, 2016 in New York City.

Currently, when creating a persistent volume, one has to restart before using it, as per the documentation: https://tails.boum.org/doc/first_steps/persistence/configure/index.en.html. Unfortunately, when people I've trained create a volume, their immediate assumption was that persistence was ready as soon as they set it up. If there is a way to remove the need to restart, that would vastly improve the experience of creating persistent volumes and keep people from becoming frustrated when they create a persistent volume, set up a bunch of things, then restart only to see that they have to go through it all over again.


Related issues

Related to Tails - Feature #10647: Integrate information about persistence to persistence assistant Confirmed 11/24/2015
Related to Tails - Feature #14544: Spend software developer time on smallish UX improvements In Progress 08/31/2018
Related to Tails - Bug #16384: Force restarting after creating persistent storage Confirmed 01/23/2019
Related to Tails - Feature #15586: Instruct about the possibility of creating a persistent storage when there is none Confirmed 05/05/2018
Blocked by Tails - Feature #5881: Add a "Restart now" button to persistence setup assistant Duplicate

Associated revisions

Revision e9df52b1 (diff)
Added by segfault 6 months ago

Add crypttab entry for TailsData_unlocked (refs: #11529)

With this entry we specify the device mapper name of the unlocked
persistent volume. This fixes that the device mapper name is different
when the volume was just created (the name was automatically set to
luks-$UUID by udisks, no way to configure this) and when the volume
was unlocked by the greeter (we specify the name "TailsData_unlocked"
in the cryptsetup command).

The "_unlocked" suffix is required by live-persist, so this is a
requirement for immediately enabling the persistent partition after
creating it.

The "noauto" option ensures that the entry is ignored by
cryptsetup.target, which would otherwise try to unlock the partition
during boot.

Revision 97ac2b29 (diff)
Added by segfault 6 months ago

Revert "Add crypttab entry for TailsData_unlocked (refs: #11529)"

This does not actually change the device mapper name after creating the
partition, because udisks ignores the crypttab in that case and always
sets the name to luks-$UUID.

This reverts commit e9df52b18f0a75a04b744e4ee8f5470cd19de04f.

Revision 992da4e0 (diff)
Added by segfault 6 months ago

Allow t-p-s to activate persistence (refs: #11529)

Revision e45b9646 (diff)
Added by segfault 5 months ago

Add crypttab entry for TailsData_unlocked (refs: #11529)

With this entry we specify the device mapper name of the unlocked
persistent volume. This fixes that the device mapper name is different
when the volume was just created (the name was automatically set to
luks-$UUID by udisks, no way to configure this) and when the volume
was unlocked by the greeter (we specify the name "TailsData_unlocked"
in the cryptsetup command).

The "_unlocked" suffix is required by live-persist, so this is a
requirement for immediately enabling the persistent partition after
creating it.

The "noauto" option ensures that the entry is ignored by
cryptsetup.target, which would otherwise try to unlock the partition
during boot.

Revision b9f769b3 (diff)
Added by segfault 5 months ago

Revert "Add crypttab entry for TailsData_unlocked (refs: #11529)"

This does not actually change the device mapper name after creating the
partition, because udisks ignores the crypttab in that case and always
sets the name to luks-$UUID.

This reverts commit e9df52b18f0a75a04b744e4ee8f5470cd19de04f.

Revision 3e5ea9e4 (diff)
Added by segfault 5 months ago

Allow t-p-s to activate persistence (refs: #11529)

Revision 4db9067d (diff)
Added by segfault 5 months ago

Add new systemd unit tails-persistence-is-enabled.target (refs: #11529)

... which, when started, starts all the services that require
persistence to be set up.

Revision 0e057e12 (diff)
Added by segfault 5 months ago

Remove persistence_is_enabled_read_write (refs: #11529)

We don't support mounting the persistence read-only anymore, so the
greeter never sets TAILS_PERSISTENCE_READONLY and this check is always true.

Revision 7024f08f (diff)
Added by segfault 5 months ago

Use tails-persistence-is-enabled.target instead of persistence state file (refs: #11529)

Revision ceeee484 (diff)
Added by segfault 5 months ago

Add tails-check-if-persistence-enabled.service (refs: #11529)

This service is run automatically during boot (after the greeter) and
checks if persistence is enabled, and then starts the
tails-persistence-is-enabled.target accordingly.

Revision 37011341 (diff)
Added by segfault 5 months ago

Make persistence related user units system units (refs: #11529)

... in order to allow tails-persistence-setup to start
tails-check-if-persistence-enabled.service. This also allows
programs run as other users than amnesia to check if persistence is
enabled.

We set User=amnesia on the services, so the code will still run as
amnesia.

Also, sort unit files in 52-update-rc.d alphabetically.

Revision 35c90b3f (diff)
Added by segfault 5 months ago

Allow t-p-s to start tails-check-if-persistence-enabled.service (refs: #11529)

Revision 3df4258d (diff)
Added by segfault 5 months ago

Add bookmarks to ~/.config/gtk-3.0/bookmarks (refs: #11529)

The bookmarks are not updated in Nautilus if written to ~/.gtk-bookmarks

Revision e82144a3 (diff)
Added by segfault 5 months ago

Split add-GNOME-bookmarks into two scripts (refs: #11529)

... one to be (always) executed during startup and one to be executed
once persistence was activated.

Revision 44a6d60c (diff)
Added by segfault 5 months ago

Split create-tor-browser-directories into two scripts (refs: #11529)

... one to be (always) executed during startup and one to be executed
once persistence was activated.

Revision da7fb526 (diff)
Added by segfault 5 months ago

Make create-tor-browser-directory a build hook (refs: #11529)

Since we don't create the persistent directory anymore in this script,
there is no reason to do it during boot instead of during build.

Revision 422e9457 (diff)
Added by segfault 5 months ago

Allow t-p-s to start tails-check-if-persistence-enabled.service (refs: #11529)

Revision 813b88e2 (diff)
Added by segfault 5 months ago

Add bookmarks to ~/.config/gtk-3.0/bookmarks (refs: #11529)

The bookmarks are not updated in Nautilus if written to ~/.gtk-bookmarks

Revision f7cdc051 (diff)
Added by segfault 5 months ago

Change persistence related systemd units (refs: #11529)

- Extract the launcher-trusting part from add-GNOME-bookmarks into
its own script (tails-trust-desktop-launchers) and create its own
systemd service.

- For both add-GNOME-bookmarks and create-tor-browser-directory:

Split the script into two scripts, one containing the parts that
depend on persistence being enabled and one for the parts that
don't.

- Create systemd units for the new scripts.

- Convert persistence related user units into system units which are started
automatically when tails-persistence-is-enabled.target is started.

This allows tails-persistence-setup to trigger them by starting
tails-check-if-persistence-enabled.service.
This also allows programs run as  other users than amnesia to check if
persistence is enabled (we don't use this currently, but it might be useful someday).
We set User=amnesia on the services, so the code will still run as
amnesia.

- While editing it, sort unit files in 52-update-rc.d alphabetically.

Revision 6cc76328 (diff)
Added by segfault 5 months ago

Enable the feature-11529-enable-persistence-immediately APT overlay (refs: #11529).

Revision dd20b39b (diff)
Added by segfault 5 months ago

Change persistence related systemd units (refs: #11529)

- Extract the launcher-trusting part from add-GNOME-bookmarks into
its own script (tails-trust-desktop-launchers) and create its own
systemd service.

- For both add-GNOME-bookmarks and create-tor-browser-directory:

Split the script into two scripts, one containing the parts that
depend on persistence being enabled and one for the parts that
don't.

- Create systemd units for the new scripts.

- Convert persistence related user units into system units which are started
automatically when tails-persistence-is-enabled.target is started.

This allows tails-persistence-setup to trigger them by starting
tails-check-if-persistence-enabled.service.
This also allows programs run as  other users than amnesia to check if
persistence is enabled (we don't use this currently, but it might be useful someday).
We set User=amnesia on the services, so the code will still run as
amnesia.

- While editing it, sort unit files in 52-update-rc.d alphabetically.

Revision 1fdd442e (diff)
Added by segfault 5 months ago

Enable the feature-11529-enable-persistence-immediately APT overlay (refs: #11529).

Revision 1312dd48 (diff)
Added by segfault 5 months ago

Change persistence related systemd units (refs: #11529)

- Extract the launcher-trusting part from add-GNOME-bookmarks into
its own script (tails-trust-desktop-launchers) and create its own
systemd service.

- For both add-GNOME-bookmarks and create-tor-browser-directory:

Split the script into two scripts, one containing the parts that
depend on persistence being enabled and one for the parts that
don't.

- Create systemd units for the new scripts.

- Convert persistence related user units into system units which are started
automatically when tails-persistence-is-enabled.target is started.

This allows live-persist to trigger them by starting
tails-check-if-persistence-enabled.service.
This also allows programs run as  other users than amnesia to check if
persistence is enabled (we don't use this currently, but it might be useful
someday).
We set User=amnesia on the services, so the code is still run as amnesia.

- Start tails-check-if-persistence-enabled.service after activating volume in
live-persist.

- While editing it, sort unit files in 52-update-rc.d alphabetically.

Revision a6500b4a (diff)
Added by segfault 5 months ago

Make live-persist create /live/persistence world-readable (refs: #11529)

The greeter has umask 022, so when live-persist is called by the
greeter, it creates /live/persistence world-readable.

But t-p-s started via its Desktop launcher has umask 077, so when
live-persist is called by t-p-s, it creates /live/persistence not
world-readable.

This commit makes t-p-s call live-persist with umask 022 to have
consistent file permissions in both cases.

Revision fa54b057 (diff)
Added by segfault 5 months ago

Import new changes to t-p-s (refs: #11529)

Revision 3a9a7315 (diff)
Added by segfault 5 months ago

Change persistence related systemd units (refs: #11529)

- Extract the launcher-trusting part from add-GNOME-bookmarks into
its own script (tails-trust-desktop-launchers) and create its own
systemd service.

- For both add-GNOME-bookmarks and create-tor-browser-directory:

Split the script into two scripts, one containing the parts that
depend on persistence being enabled and one for the parts that
don't.

- Create systemd units for the new scripts.

- Convert persistence related user units into system units which are started
automatically when tails-persistence-is-enabled.target is started.

This allows live-persist to trigger them by starting
tails-check-if-persistence-enabled.service.
This also allows programs run as  other users than amnesia to check if
persistence is enabled (we don't use this currently, but it might be useful
someday).
We set User=amnesia on the services, so the code is still run as amnesia.

- Start tails-check-if-persistence-enabled.service after activating volume in
live-persist.

- While editing it, sort unit files in 52-update-rc.d alphabetically.

Revision 239dc447 (diff)
Added by segfault 5 months ago

live-persist: Start persistence related systemd services after activating volume (refs: #11529)

Revision a7efa5b9 (diff)
Added by segfault 5 months ago

Make live-persist create /live/persistence world-readable (refs: #11529)

The greeter has umask 022, so when live-persist is called by the
greeter, it creates /live/persistence world-readable.

But t-p-s started via its Desktop launcher has umask 077, so when
live-persist is called by t-p-s, it creates /live/persistence not
world-readable.

This commit makes t-p-s call live-persist with umask 022 to have
consistent file permissions in both cases.

Revision 4fd05012 (diff)
Added by segfault 5 months ago

Import new changes to t-p-s (refs: #11529)

Revision 1afb04a5 (diff)
Added by segfault 5 months ago

Add bookmarks to ~/.config/gtk-3.0/bookmarks (refs: #11529)

The bookmarks are not updated in Nautilus if written to ~/.gtk-bookmarks

Revision d9549618 (diff)
Added by segfault 5 months ago

Change persistence related systemd units (refs: #11529)

- Extract the launcher-trusting part from add-GNOME-bookmarks into
its own script (tails-trust-desktop-launchers) and create its own
systemd service.

- For both add-GNOME-bookmarks and create-tor-browser-directory:

Split the script into two scripts, one containing the parts that
depend on persistence being enabled and one for the parts that
don't.

- Create systemd units for the new scripts.

- Convert persistence related user units into system units which are started
automatically when tails-persistence-is-enabled.target is started.

This allows live-persist to trigger them by starting
tails-check-if-persistence-enabled.service.
This also allows programs run as  other users than amnesia to check if
persistence is enabled (we don't use this currently, but it might be useful
someday).
We set User=amnesia on the services, so the code is still run as amnesia.

- Start tails-check-if-persistence-enabled.service after activating volume in
live-persist.

- While editing it, sort unit files in 52-update-rc.d alphabetically.

Revision 6933a802 (diff)
Added by segfault 5 months ago

live-persist: Start persistence related systemd services after activating volume (refs: #11529)

Revision 79caab90 (diff)
Added by segfault 5 months ago

Make live-persist create /live/persistence world-readable (refs: #11529)

The greeter has umask 022, so when live-persist is called by the
greeter, it creates /live/persistence world-readable.

But t-p-s started via its Desktop launcher has umask 077, so when
live-persist is called by t-p-s, it creates /live/persistence not
world-readable.

This commit makes t-p-s call live-persist with umask 022 to have
consistent file permissions in both cases.

Revision 27e5fb7b (diff)
Added by segfault 5 months ago

Import new changes to t-p-s (refs: #11529)

Revision 94d48522 (diff)
Added by segfault 5 months ago

Make tails-rename-persistent-volume imported from t-p-s executable (refs: #11529)

Revision b8d8b061 (diff)
Added by segfault 5 months ago

Add tails-check-if-persistence-enabled.service (refs: #11529)

This service is started by live-persist after activating persistence. It
checks whether persistence was actually enabled, and then starts
tails-persistence-is-enabled.target.

Revision 5b3dbfcc (diff)
Added by segfault 5 months ago

Add bookmarks to ~/.config/gtk-3.0/bookmarks (refs: #11529)

The bookmarks are not updated in Nautilus if written to ~/.gtk-bookmarks

Revision ea76e161 (diff)
Added by segfault 5 months ago

Change persistence related systemd units (refs: #11529)

- Extract the launcher-trusting part from add-GNOME-bookmarks into
its own script (tails-trust-desktop-launchers) and create its own
systemd service.

- For both add-GNOME-bookmarks and create-tor-browser-directory:

Split the script into two scripts, one containing the parts that
depend on persistence being enabled and one for the parts that
don't.

- Create systemd units for the new scripts.

- Convert persistence related user units into system units which are started
automatically when tails-persistence-is-enabled.target is started.

This allows live-persist to trigger them by starting
tails-check-if-persistence-enabled.service.
This also allows programs run as  other users than amnesia to check if
persistence is enabled (we don't use this currently, but it might be useful
someday).
We set User=amnesia on the services, so the code is still run as amnesia.

- While editing it, sort unit files in 52-update-rc.d alphabetically.

Revision 9381b289 (diff)
Added by segfault 5 months ago

Make live-persist create /live/persistence world-readable (refs: #11529)

The greeter has umask 022, so when live-persist is called by the
greeter, it creates /live/persistence world-readable.

But t-p-s started via its Desktop launcher has umask 077, so when
live-persist is called by t-p-s, it creates /live/persistence not
world-readable.

This commit makes t-p-s call live-persist with umask 022 to have
consistent file permissions in both cases.

Revision 2a048dd8 (diff)
Added by segfault 5 months ago

Import new changes to t-p-s (refs: #11529)

Revision d8593020 (diff)
Added by segfault 5 months ago

Make tails-rename-persistent-volume imported from t-p-s executable (refs: #11529)

Revision d2b8facf (diff)
Added by segfault 4 months ago

Document dput config to use Tails section by default (refs: #11529)

History

#1 Updated by intrigeri about 3 years ago

  • Status changed from New to Confirmed

Yes, this would be great!

I suspect it would be trivial to implement a PoC for just what this ticket asks for, and then the hard part will be to figure out the list of things that can possibly go wrong (possibly nothing, possibly something nasty and non-obvious). The latter was never done AFAIK, which is why we're still requiring a reboot.

#2 Updated by Dr_Whax about 3 years ago

  • Priority changed from Normal to Elevated

#3 Updated by u over 2 years ago

  • Priority changed from Elevated to Normal

#4 Updated by u over 1 year ago

  • Related to Feature #5881: Add a "Restart now" button to persistence setup assistant added

#5 Updated by u over 1 year ago

  • Related to Feature #10647: Integrate information about persistence to persistence assistant added

#6 Updated by u over 1 year ago

  • Subject changed from Activate Persistent Volume Upon Creation to Automatically activate Persistent Volume Upon Creation

#7 Updated by huertanix over 1 year ago

huertanix wrote:

If there is a way to remove the need to restart, that would vastly improve the experience of creating persistent volumes and keep people from becoming frustrated when they create a persistent volume, set up a bunch of things, then restart only to see that they have to go through it all over again.

Note that even with messaging around the need for a reboot after setting up persistence, we still now see many users in training making the mistake of thinking persistence is active and ready as soon as they finish the initial configuration before rebooting. 100% of the users I train come from a Windows and Mac background where when they see macOS or Windows tell them "you need to reboot" they don't and experience no consequence and thus, are conditioned to ignore reboot notices unless they're forced to by someone from tech support.

#8 Updated by sajolida over 1 year ago

  • Subject changed from Automatically activate Persistent Volume Upon Creation to Activate persistent storage without the need to restart

I can't agree more with huertanix!

During the user testing of the Additional Software beta several participants failed to understand that they had to restart for the persistent storage to start storing data. Of course, we could improve the message after the persistent storage is created to explain this better but the root problem is that people expect the persistent storage to store whatever they already did as soon as they activate one of the feature.

For example, P2 failed to understand what happened even after restarting twice.

  1. He connected to his Wi-Fi.
  2. He created the persistent storage, activated the Network Connections feature, and understand that he had to restart.
  3. He restarted and wondered why the Wi-Fi was not connecting automatically.
  4. He restarted again after some time (as part of a task in the testing).
  5. He was puzzled when the Wi-Fi connected automatically on the second restart:

« On the second use it connected automatically. There's something I didn't understand... »

#9 Updated by sajolida over 1 year ago

  • Related to Feature #14544: Spend software developer time on smallish UX improvements added

#10 Updated by u about 1 year ago

  • Related to deleted (Feature #5881: Add a "Restart now" button to persistence setup assistant)

#11 Updated by u about 1 year ago

  • Blocked by Feature #5881: Add a "Restart now" button to persistence setup assistant added

#12 Updated by intrigeri 9 months ago

The cheapest possible fix (from a dev perspective) would probably be to lock down the system once persistence is set up, i.e. effectively force users to reboot.

Regarding enabling persistence immediately, that should be doable but:

  • It'll go wrong at least for files that are already open, e.g. if a Pidgin account is started already, we might copy the corresponding files to persistence (and get them back on next boot), but any further change in the Pidgin UI won't persist. That would create quite confusing UX.
  • IIRC some of our session initialization is done differently depending on whether persistence is enabled or not (e.g. the GNOME bookmarks). All this code would need to support being re-triggered and do the right thing when run a 2nd time. Otherwise the user will be left in a session that's of a 3rd kind, somewhere in between "no persistence unlocked" and "persistence unlocked", which may be confusing from a UX perspective (once we remove the need to reboot, one would expect the session to of the "persistence unlocked" kind) and increase our maintenance costs (more complex state machine).

I might be overly pessimistic but I'm worried that we won't do a good enough job at this and we'll end up setting user expectations we can't be up to, which would result in poor UX too (probably better than the current state of things, but the marginal UX improvement might not be worth the dev cost).

So at this point I'm very much tempted to lower our ambitions here and only force the user to reboot.

#13 Updated by intrigeri 9 months ago

  • Related to Bug #16384: Force restarting after creating persistent storage added

#14 Updated by intrigeri 9 months ago

  • Subject changed from Activate persistent storage without the need to restart to Consider saving data to just created persistent storage on restart

#15 Updated by sajolida 9 months ago

  • Related to Feature #15586: Instruct about the possibility of creating a persistent storage when there is none added

#16 Updated by segfault 6 months ago

I really want this feature. Mostly because of the very painful UX issue reported by sajolida and huertanix, which often results in accounts/PGP keys etc. being lost (I witnessed this myself multiple times).

But also because now that we have the USB image, IMO having to reboot after setting up persistence is the biggest remaining UX issue we have during setup of a new Tails device. And the setup is the first thing a new user experiences, so IMO we should really give our best to make this a nice experience.

intrigeri wrote:

Regarding enabling persistence immediately, that should be doable but:

  • It'll go wrong at least for files that are already open, e.g. if a Pidgin account is started already, we might copy the corresponding files to persistence (and get them back on next boot), but any further change in the Pidgin UI won't persist. That would create quite confusing UX.

We could run a pgrep thunderbird pidgin ... command before starting the persistence setup and ask the user to close all the applications we found a process for.

  • IIRC some of our session initialization is done differently depending on whether persistence is enabled or not (e.g. the GNOME bookmarks). All this code would need to support being re-triggered and do the right thing when run a 2nd time.

Agreed. But how much code is this really? Right now I can only find two places where the persistence_is_enabled* functions from config/chroot_local-includes/usr/local/lib/tails-shell-library/tails-greeter.sh are used:

config/chroot_local-includes/usr/local/lib/add-GNOME-bookmarks

and

config/chroot_local-includes/usr/local/lib/create-tor-browser-directories.

And re-running these after enabling persistence is enough to get into the same state as when booting with persistence. Also, both of these already have a systemd service, so I think we could just create a new unit tails-persistence-is-enabled.target which wants the other two services so we have these dependencies nicely encoded.

I would like to prepare a branch for that.

#17 Updated by segfault 6 months ago

I got enthusiastic and started working on this, so I'm posting some findings below, but I know that we still have finish the discussion about whether it's worth to do this at all.

When the persistent partition was just created, the device name of the unlocked volume is different than when the volume was unlocked later. In the first case, the name is luks-$UUID, in the latter TailsData_unlocked.

live-boot uses the same name for the mountpoint under /live/persistence, which has to be /live/persistence/TailsData_unlocked, we use that path everywhere.

My first idea was to set the name in /etc/crypttab, which is honored by udisks when unlocking an encrypted volume, but unfortunately, that's not the case when the volume is unlocked as part of Format. I'm thinking about preparing a patch for upstream to fix that (or at least reporting it).

A workaround I found is to use dmsetup rename to rename the device to TailsData_unlocked. After that, live-persist activate works.

#18 Updated by intrigeri 6 months ago

segfault, I'm glad you're working on this! It'll be useful to estimate the actual cost of fixing this problem. Feel free to clock the cheapest prototype you can build as FT work. If you hit stumbling blocks that can't be addressed cheaply, let's talk :)

#19 Updated by intrigeri 6 months ago

In passing, as a side-effect, other advantages of fixing this: make our test suite faster; make most persistence-related dev/doc/UX work easier, more enjoyable, and cheaper.

#20 Updated by segfault 6 months ago

  • Status changed from Confirmed to In Progress

#21 Updated by segfault 6 months ago

  • Status changed from In Progress to Confirmed
  • Feature Branch set to feature/11529-enable-persistence-immediately,tps:11529-enable-persistence-immediately

I continued working on this today, but I'm very slow because I have to write Perl.

Enabling the persistence after configuring it should work now. I also started working on the "ask user to close relevant apps" feature, but I didn't get far.

I didn't work on the tails-persistence-is-enabled.target to re-trigger the startup scripts which depend on persistence being enabled.

Another thing that's missing is that settings which were disabled are also unmounted.

#22 Updated by segfault 6 months ago

  • Status changed from Confirmed to In Progress

#23 Updated by intrigeri 6 months ago

Another thing that's missing is that settings which were disabled are also unmounted.

The discussion on #8447 might be relevant.

#24 Updated by sajolida 6 months ago

  • Subject changed from Consider saving data to just created persistent storage on restart to Save data to Persistence when it is created (no need to restart)

But also because now that we have the USB image, IMO having to reboot after setting up persistence is the biggest remaining UX issue we have during setup of a new Tails device. And the setup is the first thing a new user experiences, so IMO we should really give our best to make this a nice experience.

Hi seggie :) It's so cool that you started experimenting with this!

I agree that the Persistence onboarding could still be improved a lot, especially for newcomers. But I don't think that having to reboot in itself is the most problematic part of it.

I think that the most problematic part is loosing data, which is the root problem that huertanix was reporting about here. I've also seen repeatedly during usability tests, people wondering why they lost their Wi-Fi password when rebooting for example.

Note that we're also discussing here the case of people who are not following our installation instructions because they instruct people to create a Persistence right after the 1st boot and before having data to loose (except 1 Wi-Fi password).

In this case, another problematic part could be to learn about Persistence in the 1st place. Right now, there's nothing in the installation process itself that tells you about Persistence after you installed Tails. You have to learn about it either from the website, from friends, or by exploring the Applications menu.

To solve the data loss issue, we already have other plans that seemed technical easier than #11529:

  • #16384: Force restarting after creating persistent storage

    It would be easy for us to detect when people are restarting right
    after creating a Persistence and then we'll be in a good place to save
    the data that needs to be saved during the shutdown process.

  • #15586: Instruct about the possibility of creating a persistent
    storage when there is none.

    To prevent people from generating data that might be lost before
    creating a Persistence (and then they could be forced to restart by #16384).

Of course, I'd be delighted if you discover that #11529 is relatively easy to solve but I wanted to point out that it's not the only way to solve the most important issue here (data loss) and that we have other plans that could achieve the same (with still an extra restart, I agree).

Solving #15586 would also solve the discoverability issue, so I think it should be done anyway. On the other hand, #11529 and #16384 seems to be mutually exclusive.

NB: I'm renaming this ticket to be specifically about what segfault is trying to achieve. Please correct me if I got it wrong!

#25 Updated by segfault 5 months ago

I'm annoyed that a lot of our configuration is only applied during boot, by live-config, instead of during build.

While working on this ticket, I split some scripts which depend on persistence being set up into two, one for the part which does not depend on persistence and one which does. I would like to make the parts which does not depend on persistence build hooks (i.e. move them to config/chroot_local-hooks), so that we don't have to create additional systemd services to run them. But I can't do that because they have to create files as the amnesia user, which is only created by live-config during boot.

I took a look at the journal of a booted Tails and it seems like the whole live-config stuff takes about 15 seconds during boot. I think most of the configuration could be done at build time, which would make things easier and reduce the boot time. What's the reason why things like creating the amnesia user only happen during boot?

#26 Updated by segfault 5 months ago

I released t-p-s 2.1.1.1-1 with my changes to be able to test it on the feature branch. I hope that's OK. I wanted to encode in the version number that the release is only intended for #11529, but the release process documentation and dzil build didn't like any other characters than [0-9.].

#27 Updated by segfault 5 months ago

segfault wrote:

I released t-p-s 2.1.1.1-1 with my changes to be able to test it on the feature branch. I hope that's OK. I wanted to encode in the version number that the release is only intended for #11529, but the release process documentation and dzil build didn't like any other characters than [0-9.].

I uploaded the package with dput but it doesn't get installed on my branch and it's not in incoming/ on incoming.deb.tails.boum.org. But the distribution name (feature-11529-enable-persistence-immediately) was added to conf/distributions. What did I do wrong?

#28 Updated by intrigeri 5 months ago

What's the reason why things like creating the amnesia user only happen during boot?

I think that's because this is how Debian Live systems generally work.
I don't know if anything relies on this.

#29 Updated by intrigeri 5 months ago

I released t-p-s 2.1.1.1-1 with my changes to be able to test it on the feature branch. I hope that's OK. I wanted to encode in the version number that the release is only intended for #11529, but the release process documentation and dzil build didn't like any other characters than [0-9.].

Having non-official releases with official-looking version numbers is a bit problematic, but no big harm done.

Usually I would instead export a diff to tails.git with something like:

(cd lib && git diff --src-prefix=a/usr/share/perl5/ --dst-prefix=b/usr/share/perl5/ --relative=lib master.. . ) > ~/tails/git/config/chroot_local-patches/tps-16568.diff

#30 Updated by intrigeri 5 months ago

I uploaded the package with dput but it doesn't get installed on my branch and it's not in incoming/ on incoming.deb.tails.boum.org. But the distribution name (feature-11529-enable-persistence-immediately) was added to conf/distributions. What did I do wrong?

I see no trace of a 2.1.1.1-1 upload in the logs. I suggest you retry the upload and check /var/log/incoming-tails.log immediately after your upload seemingly succeeds.

#31 Updated by segfault 5 months ago

Oh, so apparently I forgot to set the hostname parameter of dput to tails when I uploaded the .changes file. So it was uploaded to the default host, which is ftp.upload.debian.org. I have no idea what the consequences of that are or why I'm even permitted to upload to that server.

I now updated my dput.cf to use the tails section by default.

#32 Updated by segfault 5 months ago

segfault wrote:

I now updated my dput.cf to use the tails section by default.

We might want to add that to our "Configuring dput" section on https://tails.boum.org/contribute/APT_repository/custom/ (maybe as a hint for non-Debian Maintainers):

[DEFAULT]
default_host_main = tails

#33 Updated by segfault 5 months ago

intrigeri wrote:

Usually I would instead export a diff to tails.git with something like:

 (cd lib && git diff --src-prefix=a/usr/share/perl5/ --dst-prefix=b/usr/share/perl5/ --relative=lib master.. . ) > ~/tails/git/config/chroot_local-patches/tps-16568.diff

Thanks, I used that now instead of releasing new packages.

I made some progress, on an image built from the feature branch, the persistence is now immediately activated, and all scripts which depend on persistence are automatically run after the persistence is activated.

What's missing is the check for running applications. I started to work on this, but IMO the right place to do this is in t-p-s, and I don't have the skills to implement that in perl.

#34 Updated by segfault 5 months ago

What's missing is the check for running applications. I started to work on this, but IMO the right place to do this is in t-p-s, and I don't have the skills to implement that in perl.

Here is what I had planned, but failed to implement:

  1. Have a list of process names + corresponding app names per persistent setting
  2. Before applying a new persistence config, get a list of all changed settings
  3. Check if any affected processes are running
  4. Ask the user to close all affected apps before retrying
  5. Repeat until no affected processes are running
  6. Apply new persistence config and run `live-persist activate`

#35 Updated by segfault 5 months ago

Something else that's missing is an updated design documentation (https://tails.boum.org/contribute/design/persistence/)

#36 Updated by segfault 5 months ago

And tests are missing too.

#37 Updated by intrigeri 5 months ago

Oh, so apparently I forgot to set the hostname parameter of dput to tails when I uploaded the .changes file. So it was uploaded to the default host, which is ftp.upload.debian.org.

Ah ah :)

I have no idea what the consequences of that are

@segfault, I hope this gets cleaned up automatically and no ftp-master had to do it by hand. You could check if your upload is still in the queue there. Also, one can use dcut to cancel an upload.

why I'm even permitted to upload to that server.

It's an anonymous FTP upload server. What matters is the OpenPGP signatures, not access control to the upload server.

#38 Updated by intrigeri 4 months ago

We might want to add that to our "Configuring dput" section on https://tails.boum.org/contribute/APT_repository/custom/ (maybe as a hint for non-Debian Maintainers):

> [DEFAULT]
> default_host_main = tails
> 

@segfault, please go ahead and point me to the corresponding commit once you've pushed to master; happy to review this after the fact.

#39 Updated by segfault 4 months ago

intrigeri wrote:

We might want to add that to our "Configuring dput" section on https://tails.boum.org/contribute/APT_repository/custom/ (maybe as a hint for non-Debian Maintainers):

[...]

@segfault, please go ahead and point me to the corresponding commit once you've pushed to master; happy to review this after the fact.

Done in d2b8facfbb81e2db12d3206a52df5ba3158a1415.

#40 Updated by intrigeri 4 months ago

@segfault, please go ahead and point me to the corresponding commit once you've pushed to master; happy to review this after the fact.

Done in d2b8facfbb81e2db12d3206a52df5ba3158a1415.

Great!

Also available in: Atom PDF