Page:Ludovic Courtès - Functional Package Management with Guix.djvu/9

 declared in the derivation being built. NixOS makes one exception: it relies on the availability of  in the chroot. We remove that exception, and instead automatically patch script "shebangs" in the package’s source, as noted in Section 3.4. This turned out to be more than just a theoretical quest for "purity". First, some GNU/Linux distributions use Dash as the implementation of, while others use Bash; these are two variants of the Bourne shell, with different extensions, and in general different behavior. Second,  is typically a dynamically-linked executable. So adding  to the   is not enough; one typically needs to also add   and   to the chroot. At that point, there are many impurities, and a great potential for non-reproducibility—which defeats the purpose of the.

Several packages had to be adjusted for proper function in the absence of. In particular, libc’s  and   functions had to be changed to refer to "our" Bash instance. Likewise, GNU Make, GNU Awk, GNU Guile, and Python needed adjustment. Occasionally, occurrences of  are not be handled automatically, for instance in test suites; these have to be patched manually in the package’s recipe.

Bootstrapping
Bootstrapping in our context refers to how the distribution gets built "from nothing". Remember that the build environment of a derivation contains nothing but its declared inputs. So there’s an obvious chicken-and-egg problem: how does the first package get built? How does the first compiler get compiled? The GNU system we are building is primarily made of C code, with libc at its core. The GNU build system itself assumes the availability of a Bourne shell, traditional Unix tools provided by GNU Coreutils, GNU Awk, Findutils, sed, and grep. Furthermore, our build programs are written in Guile Scheme. Consequently, we rely on pre-built statically-linked binaries of GCC, Binutils, libc, and the other packages mentioned above to get started.

Figure 10 shows the very beginning of the dependency graph of our distribution. At this level of detail, things are slightly more complex. First, Guile itself consists of an ELF executable, along with many source and compiled Scheme files that are dynamically loaded when it runs. This gets stored in the  tarball shown in this graph. This tarball is part of Guix’s "source" distribution, and gets inserted into the store with.

But how do we write a derivation that unpacks this tarball and adds it to the store? To solve this problem, the  derivation—the first one that gets built— uses bash as its builder, which runs , which in turn calls tar to unpack the tarball. Thus, bash,,  , and   are statically-linked binaries, also part of the Guix source distribution, whose sole purpose is to allow the Guile tarball to be unpacked.

Once  is built, we have a functioning Guile that can be used to run subsequent build programs. Its first task is to download tarballs containing the other pre-built binaries—this is what the  derivations do. Guix modules such as  are used for this purpose. The  derivations import those modules in a directory in the store, using the original layout. The  derivations compile those modules, and write them in an output directory with the right layout. This corresponds to the  argument of   mentioned in Section 3.2.

Finally, the various tarballs are unpacked by the derivations, etc., at which point we have a working C GNU tool chain. The first tool that gets built with these tools (not shown here) is GNU Make, which is a prerequisite for all the following packages. Bootstrapping is complete when we have a full tool chain that does not depend on the pre-built bootstrap tools shown in Figure 10. Ways to achieve this are known, and notably documented by the Linux From Scratch project. We can formally verify this no-dependency requirement by checking whether the files of the final tool chain contain references to the  directories of the bootstrap inputs.

Obviously, Guix contains  declarations to build the bootstrap binaries shown in Figure 10. Because the final tool chain does not depend on those tools, they rarely need to be updated. Having a way to do that automatically proves to be useful, though. Coupled with Guix’s nascent support for cross-compilation, porting to a new architecture will boil down to cross-building all these bootstrap tools.



RELATED WORK
Numerous package managers for Scheme programs and libraries have been developed, including Racket’s PLaneT, Doro dango for R6RS implementations, Chicken Scheme’s "Eggs", Guildhall for Guile, and ScmPkg. Unlike GNU Guix, they are typically limited to Scheme-only code, and take the core operating system software for granted. To our knowledge, they implement the imperative package management paradigm, and do not attempt to support features such as transactional upgrades and rollbacks. Unsurprisingly, these tools rely on package descriptions that more or less resemble those described in Section 3.3; however, in the case of at least ScmPkg, Doro dango, and Guildhall, package descriptions are written in an external DSL, which happens to use s-expression syntax.

In [21], the authors illustrate how the units mechanism of MzScheme modules could be leveraged to improve operating system packaging systems. The examples therein focus on OS services, and multiple instantiation thereof, rather than on package builds and composition.

The Nix package manager is the primary source of inspiration for Guix. As noted in Section 2.3, Guix reuses the low-level build and deployment mechanisms of Nix, but differs in its programming interface and preferred implementation language for build scripts. While the Nix language relies on