Project

General

Profile

Feature #11581

Feature #5451: Protect against external bus exploitation

Blocklist dangerous PCIe hotplugging modules that are not needed for supported use cases

Added by cypherpunks over 3 years ago. Updated 6 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
07/19/2016
Due date:
% Done:

10%

Feature Branch:
Type of work:
Research
Blueprint:
Starter:
No
Affected tool:

Description

The shpchp kernel module enables PCIe hotplugging, which enables DMA attacks. These are commonly used in the wild by law enforcement in order to obtain forensically valid snapshots of memory. Tails users have no need for PCIe hotplugging, so the shpchp driver should be disabled.

--- /etc/modprobe.d/no-shpchp.conf
+++ /etc/modprobe.d/no-shpchp.conf
@@ -0,0 +1 @@
+blacklist shpchp

History

#1 Updated by intrigeri over 3 years ago

  • Assignee set to cypherpunks
  • QA Check set to Info Needed

Please explain what use cases blacklisting that module would break (e.g. docking stations?).

#2 Updated by cypherpunks over 3 years ago

intrigeri wrote:

Please explain what use cases blacklisting that module would break (e.g. docking stations?).

High-availability servers that use PCIe-based SSDs or other peripherals will need to be shut down in order to have the devices replaced, rather than being opened up and serviced while still running. I don't think that will affect Tails users, since they are not HA servers. I don't believe it will affect an already installed ExpressCard (they have their own driver).

Examples of use cases of PCIe hotplugging (they're all "enterprise solutions"):

http://electronicdesign.com/boards/pci-express-hot-plug-new-era-ras
https://docs.oracle.com/cd/E23824_01/html/821-1459/devconfig2-6.html
https://serverfault.com/questions/366651/can-i-pull-a-card-from-the-pci-x-bus-without-powering-down

Unfortunately it turns out Debian already has pciehp and acpiphp built into the kernel (which are also PCIe hotplugging drivers). I'm reading through the kernel source too and it seems that it disabling shpchp only disables one type of PCIe hotplugging. Blacklisting it won't hurt, though, and will probably defeat some forensic tools. There might be a clever way to disable all PCI/PCIe hotplugging, even with the drivers being kernel builtins. I'm looking into that.

#3 Updated by emmapeel over 3 years ago

  • Assignee deleted (cypherpunks)
  • QA Check deleted (Info Needed)

information was provided... so changing QA check and assignee.

#4 Updated by emmapeel over 3 years ago

  • Status changed from New to Confirmed
  • Assignee set to Dr_Whax

(09:52:19 AM) drwhax: I guess blacklisting it doesn't hurt, considering one cant normally hotplug PCIe in laptops
(09:52:34 AM) drwhax: and for desktops, I wouldnt understand why one would want that to happen

DrWhax seems interested on it....

#5 Updated by cypherpunks about 3 years ago

Just a small correction on my part. This actually disables hotplugging for legacy PCI on boards that support it, which should still be helpful to some older systems which do not yet have PCIe devices or have devices tightly occupied.

#6 Updated by intrigeri over 2 years ago

  • Subject changed from Blacklist the PCIe hotplugging module (shpchp) to Blacklist dangerous PCIe hotplugging modules that are not needed for supported use cases
  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

Wrt. acpiphp: https://bugs.debian.org/663433 explains why it can't be autoloaded and needs to be built in. The use case that triggered that bug report and the corresponding fix (building the driver in the kernel) seems like something we should support. I've also found references that indicate this module is needed for docking some laptops (and it seems to be the case for PCI hotplug in general, e.g. some Thunderbolt docks apparently require shpchp). So it doesn't look like we can disable PCI hotplug entirely. Is it worth doing anything at all if we can't disable it fully? If yes, then what exactly should we disable while being confident we won't break laptop docking support?

#7 Updated by intrigeri over 2 years ago

  • Parent task set to #5451

#8 Updated by cypherpunks over 2 years ago

intrigeri wrote:

Wrt. acpiphp: https://bugs.debian.org/663433 explains why it can't be autoloaded and needs to be built in. The use case that triggered that bug report and the corresponding fix (building the driver in the kernel) seems like something we should support. I've also found references that indicate this module is needed for docking some laptops (and it seems to be the case for PCI hotplug in general, e.g. some Thunderbolt docks apparently require shpchp). So it doesn't look like we can disable PCI hotplug entirely. Is it worth doing anything at all if we can't disable it fully? If yes, then what exactly should we disable while being confident we won't break laptop docking support?

It is absolutely worth continuing to attempt to mitigate DMA even if PCI hotplug cannot be disabled entirely. Most motherboards do not support hotplugging for PCI, only PCIe, and attempting to hotplug PCI will likely fry both the PCI card and your board. On the majority of systems, even with acpiphp and shpchp loaded, the hardware itself will prevent the problematic situation we would like to avoid. The systems which do support PCI hotplugging, and which get the feature enabled by the modules, are simply out of luck for now.

If and when the rest of the common DMA issues are dealt with, I could try to get a patch upstream which provides a kernel boot parameter for the module which disables hotplugging, or sets the probe interval such that it never probes for new devices. As Debian has a reason to build it into the kernel, upstream Linux may be willing to accept such a feature and won't shoot it down with the reasoning "just built it as a module and blacklist it".

#9 Updated by intrigeri over 2 years ago

  • Assignee changed from Dr_Whax to cypherpunks
  • QA Check set to Info Needed

It is absolutely worth continuing to attempt to mitigate DMA even if PCI hotplug cannot be disabled entirely.

Fair enough, so we're now at the next question I've asked: "If yes, then what exactly should we disable while being confident we won't break laptop docking support?"

#10 Updated by cypherpunks over 2 years ago

intrigeri wrote:

It is absolutely worth continuing to attempt to mitigate DMA even if PCI hotplug cannot be disabled entirely.

Fair enough, so we're now at the next question I've asked: "If yes, then what exactly should we disable while being confident we won't break laptop docking support?"

I would imagine PCIe hotplugging can be disabled without breaking docking support, but I don't know how the hardware architecture behaves in regards to laptops with docking support. Worst-case scenario, hotplugging can be re-enabled at runtime if the laptop is detected as one which supports docking.

#11 Updated by intrigeri over 2 years ago

I would imagine PCIe hotplugging can be disabled without breaking docking support,

Well, at least acpiphp and shpchp can't be disabled without breaking docking support.

Worst-case scenario, hotplugging can be re-enabled at runtime if the laptop is detected as one which supports docking.

Indeed. We would need to do this early enough (e.g. initramfs) so that any device connected via the dock is available before the Greeter is started.

So, next questions are:

  • what exact kernel modules or module options should we disable while being confident we won't break laptop docking support?
  • how do we detect that a dock is supported?

#12 Updated by intrigeri over 2 years ago

cypherpunk explained on #11792#note-8 that this ticket "is primarily about internal PCI/PCIe devices and incorrectly marked as an external bus issue (even if external buses often interface with PCIe, themselves). An attack involving CardBus or Thunderbolt will not be exploiting PCIe hotplugging". Indeed, we have other, dedicated tickets for Thunderbolt and ExpressCard. But I wonder about other kinds of external PCIe interfaces, such as:

Are these related in any way to hotplugging?

#13 Updated by cypherpunks over 2 years ago

intrigeri wrote:

cypherpunk explained on #11792#note-8 that this ticket "is primarily about internal PCI/PCIe devices and incorrectly marked as an external bus issue (even if external buses often interface with PCIe, themselves). An attack involving CardBus or Thunderbolt will not be exploiting PCIe hotplugging". Indeed, we have other, dedicated tickets for Thunderbolt and ExpressCard. But I wonder about other kinds of external PCIe interfaces, such as:

Are these related in any way to hotplugging?

True external PCIe interfaces are related to hotplugging and would not work with hotplugging disabled. The same goes for ATI XGP. Support for those are mutually exclusive with the ability to prevent a device from being trivially broken into by plugging a device into the exterior.

Those devices are very rare though, so I wasn't really counting them. Does anyone who uses Tails rely on them?

#14 Updated by cypherpunks over 2 years ago

intrigeri wrote:

So, next questions are:

  • what exact kernel modules or module options should we disable while being confident we won't break laptop docking support?
  • how do we detect that a dock is supported?

If PCIe can be disabled early without breaking docking, then I believe it's the pciehp driver. It's currently a bulit-in and it doesn't have any module parameters to disable it. Note that acpiphp and pciehp are mutually exclusive. The former deals with motherboards that support hardware PCI hotplugging and send out ACPI events, and the latter deals with PCIe devices, which natively support hotplugging. It might be possible to prevent PCIe hotplugging by activating acpiphp, even on a motherboard which does not support it and will never send a PCI hotplug ACPI event, but this is just a guess and I've never tested it before.

Detecting a dock would be done through sysfs, same way as detecting any other hardware from userspace. That's the standard way to query hardware information. If done by the bootloader in order to pass on specific module parameters or something, then it'd be done through obtaining DMI info. I don't know exactly how a dock presents itself and under what name, though.

#15 Updated by intrigeri over 2 years ago

Those devices are very rare though, so I wasn't really counting them.

Fair enough.

Does anyone who uses Tails rely on them?

I have no idea. We could compute stats if we know what to grep from the logs (systemd Journal, lspci, etc.) attached to bug reports we receive.

#16 Updated by intrigeri over 2 years ago

  • Type of work changed from Code to Research

If PCIe can be disabled early without breaking docking, then I believe it's the pciehp driver. It's currently a bulit-in and it doesn't have any module parameters to disable it. Note that acpiphp and pciehp are mutually exclusive.

Both are built into the Debian kernel, so we can't blacklist them.
Is there another way to disable this support at runtime?

(Meta: setting Type of work = Research, as it seems unclear at this point how exactly we could implement what this ticket is about.)

#17 Updated by cypherpunks over 1 year ago

intrigeri wrote:

If PCIe can be disabled early without breaking docking, then I believe it's the pciehp driver. It's currently a bulit-in and it doesn't have any module parameters to disable it. Note that acpiphp and pciehp are mutually exclusive.

Both are built into the Debian kernel, so we can't blacklist them.
Is there another way to disable this support at runtime?

(Meta: setting Type of work = Research, as it seems unclear at this point how exactly we could implement what this ticket is about.)

I was unaware that Debian built two of the three hotplugging modules into the kernel when I first submitted this bug report. Where we can go from now... Submit an request upstream to create a boot parameter or sysctl to disable them at runtime, perhaps? Or go out with guns blazing and hook the kernel with kprobes!

#18 Updated by u over 1 year ago

ask upstream about it?

#19 Updated by mercedes508 over 1 year ago

  • Subject changed from Blacklist dangerous PCIe hotplugging modules that are not needed for supported use cases to Blocklist dangerous PCIe hotplugging modules that are not needed for supported use cases

#20 Updated by cypherpunks 9 months ago

I recall that there is now a way (added in the last year or two) to disable kernel drivers that are builtin, which would include those previously undisableable modules.

#21 Updated by intrigeri 6 months ago

  • Assignee deleted (cypherpunks)
  • QA Check deleted (Info Needed)

Also available in: Atom PDF