Project

General

Profile

Feature #17258

Enable overlayfs options that make it act more like a standards compliant filesystem

Added by segfault 4 months ago. Updated 4 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
Due date:
% Done:

0%

Feature Branch:
feature/17258-enable-overlayfs-features+force-all-tests
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

Quote from the "Non-standard behavior" section of the overlayfs documentation [1]:

The following options allow overlayfs to act more like a standards
compliant filesystem:

1) "redirect_dir" 

Enabled with the mount option or module option: "redirect_dir=on" or with
the kernel config option CONFIG_OVERLAY_FS_REDIRECT_DIR=y.

If this feature is disabled, then rename(2) on a lower or merged directory
will fail with EXDEV ("Invalid cross-device link").

2) "inode index" 

Enabled with the mount option or module option "index=on" or with the
kernel config option CONFIG_OVERLAY_FS_INDEX=y.

If this feature is disabled and a file with multiple hard links is copied
up, then this will "break" the link.  Changes will not be propagated to
other names referring to the same inode.

3) "xino" 

Enabled with the mount option "xino=auto" or "xino=on", with the module
option "xino_auto=on" or with the kernel config option
CONFIG_OVERLAY_FS_XINO_AUTO=y.  Also implicitly enabled by using the same
underlying filesystem for all layers making up the overlay.

If this feature is disabled or the underlying filesystem doesn't have
enough free bits in the inode number, then overlayfs will not be able to
guarantee that the values of st_ino and st_dev returned by stat(2) and the
value of d_ino returned by readdir(3) will act like on a normal filesystem.
E.g. the value of st_dev may be different for two objects in the same
overlay filesystem and the value of st_ino for directory objects may not be
persistent and could change even while the overlay filesystem is mounted.

[1] https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt

IMO we should try to enable these options, if they work in our use case.


Related issues

Related to Tails - Feature #8415: Migrate from aufs to overlayfs Resolved 12/18/2014

Associated revisions

Revision c0f14fad (diff)
Added by segfault 4 months ago

Enable some overlayfs features (refs: #17258)

The "redirect_dir", "index", and "xino" options make overlayfs act more
like a standards compliant filesystem.

The "metacopy" option can improve performance.

For a description of these options, see the overlayfs documentation:
https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt

Revision 7e11aa3b (diff)
Added by segfault 4 months ago

Enable some overlayfs features (refs: #17258)

The "redirect_dir", "index", and "xino" options make overlayfs act more
like a standards compliant filesystem.

The "metacopy" option can improve performance.

For a description of these options, see the overlayfs documentation:
https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt

History

#1 Updated by segfault 4 months ago

There is also the metacopy option, which is not about standard compliancy, but could improve performance in some cases:


Metadata only copy up
--------------------

When metadata only copy up feature is enabled, overlayfs will only copy
up metadata (as opposed to whole file), when a metadata specific operation
like chown/chmod is performed. Full file will be copied up later when
file is opened for WRITE operation.

In other words, this is delayed data copy up operation and data is copied
up when there is a need to actually modify data.

There are multiple ways to enable/disable this feature. A config option
CONFIG_OVERLAY_FS_METACOPY can be set/unset to enable/disable this feature
by default. Or one can enable/disable it at module load time with module
parameter metacopy=on/off. Lastly, there is also a per mount option
metacopy=on/off to enable/disable this feature per mount.

Do not use metacopy=on with untrusted upper/lower directories. Otherwise
it is possible that an attacker can create a handcrafted file with
appropriate REDIRECT and METACOPY xattrs, and gain access to file on lower
pointed by REDIRECT. This should not be possible on local system as setting
"trusted." xattrs will require CAP_SYS_ADMIN. But it should be possible
for untrusted layers like from a pen drive.

Note: redirect_dir={off|nofollow|follow(*)} conflicts with metacopy=on, and
results in an error.

(*) redirect_dir=follow only conflicts with metacopy=on if upperdir=... is
given.

#2 Updated by segfault 4 months ago

  • Status changed from Confirmed to In Progress

#3 Updated by segfault 4 months ago

I pushed a commit which enables all of the above mentioned options. Lets see if this breaks any of our automated tests.

#4 Updated by segfault 4 months ago

  • Feature Branch set to feature/17258-enable-overlayfs-features

#5 Updated by segfault 4 months ago

  • Feature Branch changed from feature/17258-enable-overlayfs-features to feature/17258-enable-overlayfs-features+force-all-tests

#6 Updated by intrigeri 4 months ago

#7 Updated by intrigeri 4 months ago

On this topic, one additional criterion for deciding whether we should enable mount option X is: does it break IUK reproducibility (#17262)? If it does, how hard/expensive is it to fix this?

I have no idea how likely this sort of problem can be but I'm working on having ways to easily tell whether that's the case :)

#8 Updated by intrigeri 4 months ago

OK, I made progress on #17262. I will report my findings in more details there but I want to give you a heads up wrt. the aspects that may affect your work on this ticket.

Most likely, to get reproducible IUKs, we'll need to drop any overlayfs xattrs from the tmpfs read-write branch before we compress its content to get the SquashFS diff we want. These xattrs seem to all be named trusted.overlay.*. To display the xattrs set on a file, one can run getfattr -d --match=- $FILE.

So I think the questions we should ask ourselves here are:

  • Do any of the mount options this ticket is about require storing info in xattrs? If not, case closed!
  • Will overlayfs behave correctly if one of the higher-level read-only branches (i.e. a SquashFS diff) has no such xattrs, while we mount the overlayfs filesystem in a way that it'll use xattrs in its top-level, read-write tmpfs branch?

#9 Updated by segfault 4 months ago

intrigeri wrote:

OK, I made progress on #17262. I will report my findings in more details there but I want to give you a heads up wrt. the aspects that may affect your work on this ticket.

Most likely, to get reproducible IUKs, we'll need to drop any overlayfs xattrs from the tmpfs read-write branch before we compress its content to get the SquashFS diff we want. These xattrs seem to all be named trusted.overlay.*. To display the xattrs set on a file, one can run getfattr -d --match=- $FILE.

So I think the questions we should ask ourselves here are:

  • Do any of the mount options this ticket is about require storing info in xattrs? If not, case closed!
  • Will overlayfs behave correctly if one of the higher-level read-only branches (i.e. a SquashFS diff) has no such xattrs, while we mount the overlayfs filesystem in a way that it'll use xattrs in its top-level, read-write tmpfs branch?

I'm a bit confused by that. I meant this ticket to be about (a) activating overlayfs options when live-boot mounts the root filesystem. We could also consider (b) activating them when we mount the overlayfs for creating the IUKs, but IMO that is less important, because we only do few, determined filesystem operations on that overlayfs, so it's easier to notice and debug in case that there are ever issues because overlayfs does not behave exactly like a standards compliant filesystem.

If we would want to do (b), then yes, dropping xattrs could be a problem, especially if we wanted to use the metacopy option (which I really think we should not in that case, because it prevents files from being copied to the upper dir, but we specifically want modified files to be copied to the upper dir).

But if we only do (a), then I don't see how dropping xattrs in the IUKs, which we use as lower dirs, can cause any issues. The way I see it, it's the usual case that lower dirs don't have overlayfs specific xattrs - those should only be created in the read-write dir. Or am I missing anything?

#10 Updated by intrigeri 4 months ago

But if we only do (a), then I don't see how dropping xattrs in the IUKs, which we use as lower dirs, can cause any issues. The way I see it, it's the usual case that lower dirs don't have overlayfs specific xattrs - those should only be created in the read-write dir. Or am I missing anything?

I agree with your reasoning and my current best answer is: I don't know :)

The reason I'm wondering is that while yeah, usually lower dirs don't have overlayfs-specific xattrs, usually lower dirs are normal filesystems, not things that contain overlayfs specific stuff used, for example, to delete files. So my gut feeling is that the way we're generating SquashFS diffs (from an overlayfs upper dir) and piling them up with overlayfs moves us quite outside of the usual case.

Also available in: Atom PDF