Identifying dependencies used via dlopen()

The recent XZ backdoor has sparked a lot of discussion about how the open-source community links and packages software. One possible security improvement being discussed is changing how projects like systemd link to dynamic libraries that are only used for optional functionality: using dlopen() to load those libraries only when required. This could shrink the attack surface exposed by dependencies, but the approach is not without downsides — most prominently, it makes discovering which dynamic libraries a program depends on harder. On April 11, Lennart Poettering proposed one way to eliminate that problem in a systemd RFC on GitHub.

The systemd project had actually already been moving away from directly linking optional dependencies — but not for security reasons. In Poettering's explanation of his proposal on Mastodon he noted: "The primary reason for [using dlopen()] was to make it easier to build small disk images without optional components, in particular for the purpose of initrds or container deployments." Some people have speculated that this change is what pushed "Jia Tan" to launch their attack at the beginning of April, instead of waiting until it was more robust.

There are several problems with using dlopen() for dependencies, however. One is that, unlike normal dynamic linking, using dlopen() exposes the functions provided by the dependency as void pointers, which must be cast to the correct type. If the type in the dependency does not match the type in the dependent program, this can open a potential avenue for type-confusion attacks. Several respondants to Poettering's explanation on Mastodon worried that promoting the use of dlopen() would be a detriment to security for this reason. James Henstridge said: "I imagine you could hide some interesting bugs via not-quite-compatible function signatures (e.g. cause an argument to be truncated at 32 bits)." Poettering replied:

In current systemd git we systematically use some typeof() macro magic that ensures we always cast the func ptrs returned by dlopen() to the actual prototype listed in the headers of the library in question. Thus we should get the exact same type safety guarantees as you'd get when doing regular dynamic lib linking. Took us a bit to come up with the idea that typeof() can be used for this, but it's amazing, as we don't have to repeat other libraries' prototypes in our code at all anymore.

Henstridge agreed after looking at the code that it was "quite elegant. It also neatly solves the problem of assigning a symbol to the wrong function pointer." Not all of the problems are so easily dismissed, however. The real problem, according to Poettering's announcement is the fact that using dlopen() removes information from the program's ELF headers about what its dependencies are.

Now, I think there are many pros of this approach, but there are cons too. I personally think the pros by far outweigh the cons, but the cons *do* exist. The most prominent one is that turning shared library dependencies into dlopen() dependencies somewhat hides them from the user's and tools view, as the aforementioned tools won't show them anymore. Tools that care about this information are package managers such as rpm/dpkg (which like to generate automatic package dependencies based on ELF dependencies), as well initrd generators such as dracut.

His proposed solution is to adopt a new convention for explicitly listing optional dependencies as part of the program itself. In the systemd RFC, he gave an example of a macro that could be used to embed the name of optional dependencies in a special section of the binary called ".note.uapi.dlopen". "UAPI" stands for Userspace API — referring to the Linux Userspace API Group, a relatively recent collaboration between distributions, package managers, and large software projects to define standardized interfaces for user-space software. The initial proposal for what to encode in the note section was fairly bare-bones — just a type field, the string "uapi" denoting the ELF section "vendor", and the name of the dependency in question.

Poettering was also clear that it wouldn't be useful to implement this for systemd on its own; the note would only be useful if other tooling decided to read it, and other projects choose to implement it. Mike Yuan was quick to comment positively about the possibility of adding support to mkinitcpio, Arch Linux's initramfs generation tool, and pacman, the Arch package manager.

Luca Boccassi agreed that he could "look into adding a debhelper addon for this", but wondered if there should be some way to indicate whether a dependency is truly optional or that the program will fail if the dependency is missing. Poettering responded: "If it is a hard dep, then it should not be a dlopen() one. The whole reason for using dlopen() over regular ELF shared library deps is after all that they can be weak", although he did point out that the type field means that "the door is open to extend this later."

Antonio Álvarez Feijoo raised another concern, pointing out: "Some people are very picky about the size of the initrd and don't like to include things that aren't really necessary. [...] So yes, it's great to know which libraries are necessary, but how to know what systemd component requires them?" Boccassi replied that this was an example of a situation where information on whether a dependency is required or recommended could be useful. Poettering disagreed, asserting that "which libraries to actually include in an initrd is up to local configuration or distro policy." Ultimately, consumers of the new note section can do whatever they would like with the information, including automatically generating dependencies, or merely using them as a "linter" to complain about new weak dependencies that are not already known.

I think all such approaches are better than the status quo though: we'll add a weak dep or turn a regular dep into a weak dep, and unless downstream actually read NEWS closely (which well, they aren't necessarily good at) they'll just rebuild their packages/initrd and now everything is hosed.

This appealed to Feijoo, who agreed that using the information as a sanity-check on package definitions made sense.

Carlos O'Donell asked whether Poettering cared about exposing the specific symbols and symbol versions that systemd uses, pointing out that existing ELF headers include this information. He asserted that RPM uses this information when packaging a program. Poettering said that was a good question, but replied:

To speak for the systemd usecase: even though we dlopen() quite a number of libraries these days (21 actually), and we actually do provide symbol versioning for our own libraries, we do not bother with listing symbol versions for the stuff we dlopen(). We use plain dlsym() for all of them, not dlvsym().

He went on to point out that requiring people to pin down symbol versions would be "a major additional ask".

Poettering did seem to think that there was some benefit to integrating this new proposal into the existing implementation of dynamic linking in the GNU C library (glibc). He asked O'Donell and Florian Weimer — who are both involved in the glibc project — "should we proceed with this being some independent spec somewhere that just says '.note.uapi.dlopen contains this data, use it or don't, bla bla bla'. Or did the concept of weak leaking interest you enough that you think this should be done natively in glibc, binutils and so on?" Some other operating systems — notably macOS — have a native concept of "weak linking" for optional dependencies, so the idea of incorporating this information into the build system and standard library are not new.

Zbigniew Jędrzejewski-Szmek brought up an additional question about the formatting of the new section, asking whether it would make sense to use "a compact JSON representation". Jędrzejewski-Szmek said that this could make it easy to add a human-meaningful description of what the dependency is used for. With that addition, "it should be fairly easy to integrate this in the rpm build system." Boccassi agreed that the payload should be JSON. Poettering replied: "I have nothing against using JSON for this, but it's key we can reasonably generate this from a simple no-deps C macro I'd say."

Ultimately, the idea of having a standard encoding for optional dependencies seems to have been well-received, with several package managers potentially interested in adding support. With discussion still ongoing and the final format of the added information up in the air, however, it's too soon to say exactly what form the information will take. Anything intended to help ameliorate the pain of removing traditional dynamically linked dependencies seems like a good idea, though, since they reduce the surface open to XZ-backdoor-like attacks.