From debbugs-submit-bounces@debbugs.gnu.org Mon May 11 13:06:11 2020 Received: (at submit) by debbugs.gnu.org; 11 May 2020 17:06:11 +0000 Received: from localhost ([127.0.0.1]:53123 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYBsY-0001UO-2m for submit@debbugs.gnu.org; Mon, 11 May 2020 13:06:11 -0400 Received: from lists.gnu.org ([209.51.188.17]:55082) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYBsT-0001UD-Qd for submit@debbugs.gnu.org; Mon, 11 May 2020 13:06:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59572) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYBsT-0000KH-LC for guix-patches@gnu.org; Mon, 11 May 2020 13:06:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:41485) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYBsT-0000a1-BV; Mon, 11 May 2020 13:06:05 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38072 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYBsP-0004hL-0l; Mon, 11 May 2020 13:06:04 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' Date: Mon, 11 May 2020 19:05:54 +0200 Message-Id: <20200511170554.22916-1-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Guix! ‘guix pack -RR’ is wonderful, as we know ;-), because it produces binaries that work everywhere. However, the overhead of PRoot is sometimes inappropriate, in particular for those who want to run packed software on high-performance computers, the very kind of machine that lacks Guix and unprivileged user namespaces. This patch series adds an optional “execution engine” to wrappers that uses ld.so and fakechroot LD_PRELOAD trickery. Since it’s just LD_PRELOAD, there’s very little overhead, unlike PRoot. On the flip side, it doesn’t work as well as PRoot, because it’s “just” LD_PRELOAD. For example, some of the ‘open’ calls made in libc are not intercepted; on such call is in ‘__gconv_load_cache’, which makes it fail, and in turn makes Guile fail to start in its first ‘scm_to_locale_string’ call. Things that work well include Bash and Python 3. Let me know how well it works for your favorite application! The execution engine can now be chosen at run time by setting the ‘GUIX_EXECUTION_ENGINE’. For the record, tools like udocker support a similar range of execution engines: . Feedback welcome! Thanks, Ludo’. Ludovic Courtès (3): pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable. gnu: Add fakechroot. pack: Add relocation via ld.so and fakechroot. doc/guix.texi | 43 +++- gnu/packages/aux-files/run-in-namespace.c | 250 ++++++++++++++++++++-- gnu/packages/linux.scm | 30 +++ guix/scripts/pack.scm | 65 +++++- tests/guix-pack-relocatable.sh | 23 ++ 5 files changed, 376 insertions(+), 35 deletions(-) -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 11 13:11:52 2020 Received: (at 41189) by debbugs.gnu.org; 11 May 2020 17:11:52 +0000 Received: from localhost ([127.0.0.1]:53129 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYBy4-0001cV-AI for submit@debbugs.gnu.org; Mon, 11 May 2020 13:11:52 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57330) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYBy1-0001cA-TN for 41189@debbugs.gnu.org; Mon, 11 May 2020 13:11:50 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:41566) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYBxw-0001TU-HO; Mon, 11 May 2020 13:11:44 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38172 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYBxv-0005Cj-Sf; Mon, 11 May 2020 13:11:44 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41189@debbugs.gnu.org Subject: [PATCH 1/3] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable. Date: Mon, 11 May 2020 19:11:33 +0200 Message-Id: <20200511171135.23157-1-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41189 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/aux-files/run-in-namespace.c (struct engine): New type. (exec_default): New function. (engines): New variable. (execution_engine): New function. (main): Use it instead of calling 'exec_in_user_namespace' and 'exec_with_proot' directly. * tests/guix-pack-relocatable.sh: Add test with 'GUIX_EXECUTION_ENGINE'. * doc/guix.texi (Invoking guix pack): Document 'GUIX_EXECUTION_ENGINE'. --- doc/guix.texi | 30 +++++++-- gnu/packages/aux-files/run-in-namespace.c | 78 ++++++++++++++++++++--- tests/guix-pack-relocatable.sh | 17 +++++ 3 files changed, 110 insertions(+), 15 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 0cba0ee1ec..958ed9ceec 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5185,9 +5185,9 @@ When this option is passed once, the resulting binaries require support for @dfn{user namespaces} in the kernel Linux; when passed @emph{twice}@footnote{Here's a trick to memorize it: @code{-RR}, which adds PRoot support, can be thought of as the abbreviation of ``Really -Relocatable''. Neat, isn't it?}, relocatable binaries fall to back to PRoot -if user namespaces are unavailable, and essentially work anywhere---see below -for the implications. +Relocatable''. Neat, isn't it?}, relocatable binaries fall to back to +other techniques if user namespaces are unavailable, and essentially +work anywhere---see below for the implications. For example, if you create a pack containing Bash with: @@ -5219,14 +5219,32 @@ turn it off. To produce relocatable binaries that work even in the absence of user namespaces, pass @option{--relocatable} or @option{-R} @emph{twice}. In that -case, binaries will try user namespace support and fall back to PRoot if user -namespaces are not supported. +case, binaries will try user namespace support and fall back to another +@dfn{execution engine} if user namespaces are not supported. The +following execution engines are supported: -The @uref{https://proot-me.github.io/, PRoot} program provides the necessary +@table @code +@item default +Try user namespaces and fall back to PRoot if user namespaces are not +supported (see below). + +@item userns +Run the program through user namespaces and abort if they are not +supported. + +@item proot +Run through PRoot. The @uref{https://proot-me.github.io/, PRoot} program +provides the necessary support for file system virtualization. It achieves that by using the @code{ptrace} system call on the running program. This approach has the advantage to work without requiring special kernel support, but it incurs run-time overhead every time a system call is made. +@end table + +@vindex GUIX_EXECUTION_ENGINE +When running a wrapped program, you can explicitly request one of the +execution engines listed above by setting the +@code{GUIX_EXECUTION_ENGINE} environment variable accordingly. @end quotation @cindex entry point, for Docker images diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c index 23e7875173..6beac7fd53 100644 --- a/gnu/packages/aux-files/run-in-namespace.c +++ b/gnu/packages/aux-files/run-in-namespace.c @@ -336,6 +336,71 @@ exec_with_proot (const char *store, int argc, char *argv[]) #endif + +/* Execution engines. */ + +struct engine +{ + const char *name; + void (* exec) (const char *, int, char **); +}; + +static void +buffer_stderr (void) +{ + static char stderr_buffer[4096]; + setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer); +} + +/* The default engine. */ +static void +exec_default (const char *store, int argc, char *argv[]) +{ + /* Buffer stderr so that nothing's displayed if 'exec_in_user_namespace' + fails but 'exec_with_proot' works. */ + buffer_stderr (); + + exec_in_user_namespace (store, argc, argv); +#ifdef PROOT_PROGRAM + exec_with_proot (store, argc, argv); +#endif +} + +/* List of supported engines. */ +static const struct engine engines[] = + { + { "default", exec_default }, + { "userns", exec_in_user_namespace }, +#ifdef PROOT_PROGRAM + { "proot", exec_with_proot }, +#endif + { NULL, NULL } + }; + +/* Return the "execution engine" to use. */ +static const struct engine * +execution_engine (void) +{ + const char *str = getenv ("GUIX_EXECUTION_ENGINE"); + + if (str == NULL) + str = "default"; + + try: + for (const struct engine *engine = engines; + engine->name != NULL; + engine++) + { + if (strcmp (engine->name, str) == 0) + return engine; + } + + fprintf (stderr, "%s: unsupported Guix execution engine; ignoring\n", + str); + str = "default"; + goto try; +} + int main (int argc, char *argv[]) @@ -362,22 +427,17 @@ main (int argc, char *argv[]) if (strcmp (store, "@STORE_DIRECTORY@") != 0 && lstat ("@WRAPPED_PROGRAM@", &statbuf) != 0) { - /* Buffer stderr so that nothing's displayed if 'exec_in_user_namespace' - fails but 'exec_with_proot' works. */ - static char stderr_buffer[4096]; - setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer); + const struct engine *engine = execution_engine (); + engine->exec (store, argc, argv); - exec_in_user_namespace (store, argc, argv); -#ifdef PROOT_PROGRAM - exec_with_proot (store, argc, argv); -#else + /* If we reach this point, that's because ENGINE failed to do the + job. */ fprintf (stderr, "\ This may be because \"user namespaces\" are not supported on this system.\n\ Consequently, we cannot run '@WRAPPED_PROGRAM@',\n\ unless you move it to the '@STORE_DIRECTORY@' directory.\n\ \n\ Please refer to the 'guix pack' documentation for more information.\n"); -#endif return EXIT_FAILURE; } diff --git a/tests/guix-pack-relocatable.sh b/tests/guix-pack-relocatable.sh index a3fd45623c..cb56815fed 100644 --- a/tests/guix-pack-relocatable.sh +++ b/tests/guix-pack-relocatable.sh @@ -84,6 +84,23 @@ fi grep 'GNU sed' "$test_directory/output" chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/* +case "`uname -m`" in + x86_64|i?86) + # Try '-RR' and PRoot. + tarball="`guix pack -RR -S /Bin=bin sed`" + tar tvf "$tarball" | grep /bin/proot + (cd "$test_directory"; tar xvf "$tarball") + GUIX_EXECUTION_ENGINE="proot" + export GUIX_EXECUTION_ENGINE + "$test_directory/Bin/sed" --version > "$test_directory/output" + grep 'GNU sed' "$test_directory/output" + chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/* + ;; + *) + echo "skipping PRoot test" >&2 + ;; +esac + # Ensure '-R' works with outputs other than "out". tarball="`guix pack -R -S /share=share groff:doc`" (cd "$test_directory"; tar xvf "$tarball") -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 11 13:11:53 2020 Received: (at 41189) by debbugs.gnu.org; 11 May 2020 17:11:53 +0000 Received: from localhost ([127.0.0.1]:53131 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYBy4-0001cX-PD for submit@debbugs.gnu.org; Mon, 11 May 2020 13:11:53 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57338) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYBy3-0001cC-0N for 41189@debbugs.gnu.org; Mon, 11 May 2020 13:11:51 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:41568) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYBxx-0001Tf-QM; Mon, 11 May 2020 13:11:45 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38172 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYBxw-0005Cj-Rb; Mon, 11 May 2020 13:11:45 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41189@debbugs.gnu.org Subject: [PATCH 2/3] gnu: Add fakechroot. Date: Mon, 11 May 2020 19:11:34 +0200 Message-Id: <20200511171135.23157-2-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200511171135.23157-1-ludo@gnu.org> References: <20200511171135.23157-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41189 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/linux.scm (fakechroot): New variable. --- gnu/packages/linux.scm | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index 7cf7521e24..35526b3513 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -6793,6 +6793,36 @@ have to construct the archives directly, without using the archiver.") (home-page "http://freshmeat.sourceforge.net/projects/fakeroot") (license license:gpl3+))) +(define-public fakechroot + (package + (name "fakechroot") + (version "2.20.1") + (source (origin + (method url-fetch) + (uri (string-append + "https://github.com/dex4er/fakechroot/releases/download/" + version "/fakechroot-" version ".tar.gz")) + (sha256 + (base32 + "1aijkd0b45wav25v01qhw8zxwa3pl0nnp9fabmmy1nlx7hr09gas")))) + (build-system gnu-build-system) + (arguments + ;; XXX: The tests heavily assume they run on an FHS system so for now + ;; skip them. + '(#:tests? #f + #:configure-flags '("--disable-static"))) + (synopsis "Emulate @code{chroot} by overriding file system calls") + (description + "@command{fakechroot} runs a command in an environment were is additional +possibility to use @code{chroot} command without root privileges. This is +useful for allowing users to create own chrooted environment with possibility +to install another packages without need for root privileges. + +It works by providing @file{libfakechroot.so}, a shared library meant to be +set as @code{LD_PRELOAD} to override the C library file system functions.") + (home-page "https://github.com/dex4er/fakechroot/") + (license license:lgpl2.1+))) + (define-public inputattach (package (name "inputattach") -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 11 13:12:14 2020 Received: (at 41189) by debbugs.gnu.org; 11 May 2020 17:12:14 +0000 Received: from localhost ([127.0.0.1]:53134 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYByB-0001cv-6f for submit@debbugs.gnu.org; Mon, 11 May 2020 13:12:14 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57352) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYBy4-0001cF-Rj for 41189@debbugs.gnu.org; Mon, 11 May 2020 13:11:57 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:41569) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYBxz-0001Tt-3c; Mon, 11 May 2020 13:11:47 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38172 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYBxy-0005Cj-4S; Mon, 11 May 2020 13:11:46 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41189@debbugs.gnu.org Subject: [PATCH 3/3] pack: Add relocation via ld.so and fakechroot. Date: Mon, 11 May 2020 19:11:35 +0200 Message-Id: <20200511171135.23157-3-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200511171135.23157-1-ludo@gnu.org> References: <20200511171135.23157-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41189 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Ludovic Courtès * gnu/packages/aux-files/run-in-namespace.c (HAVE_EXEC_WITH_LOADER): New macro. (bind_mount): Rename to... (mirror_directory): ... this. Add 'firmlink' argument and use it instead of calling mkdir/open/close/mount directly. (bind_mount, make_symlink): New functions. (exec_in_user_namespace): Adjust accordingly. (exec_with_loader) [HAVE_EXEC_WITH_LOADER]: New function. (exec_performance): New function. (engines): Add them. * guix/scripts/pack.scm (wrapped-package)[fakechroot-library]: New procedures. [build](elf-interpreter, elf-loader-compile-flags): New procedures. (build-wrapper): Use them. * tests/guix-pack-relocatable.sh: Test with 'GUIX_EXECUTION_ENGINE=fakechroot'. * doc/guix.texi (Invoking guix pack): Document the 'performance' and 'fakechroot' engines. --- doc/guix.texi | 13 ++ gnu/packages/aux-files/run-in-namespace.c | 174 ++++++++++++++++++++-- guix/scripts/pack.scm | 65 +++++++- tests/guix-pack-relocatable.sh | 6 + 4 files changed, 237 insertions(+), 21 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 958ed9ceec..a70a058afb 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5228,6 +5228,10 @@ following execution engines are supported: Try user namespaces and fall back to PRoot if user namespaces are not supported (see below). +@item performance +Try user namespaces and fall back to Fakechroot if user namespaces are +not supported (see below). + @item userns Run the program through user namespaces and abort if they are not supported. @@ -5239,6 +5243,15 @@ support for file system virtualization. It achieves that by using the @code{ptrace} system call on the running program. This approach has the advantage to work without requiring special kernel support, but it incurs run-time overhead every time a system call is made. + +@item fakechroot +Run through Fakechroot. @uref{https://github.com/dex4er/fakechroot/, +Fakechroot} virtualizes file system accesses by intercepting calls to C +library functions such as @code{open}, @code{stat}, @code{exec}, and so +on. Unlike PRoot, it incurs very little overhead. However, it does not +always work: for example, some file system accesses made from within the +C library are not intercepted, and file system accesses made @i{via} +direct syscalls are not intercepted either, leading to erratic behavior. @end table @vindex GUIX_EXECUTION_ENGINE diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c index 6beac7fd53..ed72a169f2 100644 --- a/gnu/packages/aux-files/run-in-namespace.c +++ b/gnu/packages/aux-files/run-in-namespace.c @@ -42,6 +42,11 @@ #include #include +/* Whether we're building the ld.so/libfakechroot wrapper. */ +#define HAVE_EXEC_WITH_LOADER \ + (defined PROGRAM_INTERPRETER) && (defined PROGRAM_RUNPATH) + + /* Like 'malloc', but abort if 'malloc' returns NULL. */ static void * xmalloc (size_t size) @@ -113,9 +118,42 @@ rm_rf (const char *directory) assert_perror (errno); } -/* Bind mount all the top-level entries in SOURCE to TARGET. */ +/* Make TARGET a bind-mount of SOURCE. Take into account ENTRY's type, which + corresponds to SOURCE. */ +static int +bind_mount (const char *source, const struct dirent *entry, + const char *target) +{ + if (entry->d_type == DT_DIR) + { + int err = mkdir (target, 0700); + if (err != 0) + return err; + } + else + close (open (target, O_WRONLY | O_CREAT)); + + return mount (source, target, "none", + MS_BIND | MS_REC | MS_RDONLY, NULL); +} + +#if HAVE_EXEC_WITH_LOADER + +/* Make TARGET a symlink to SOURCE. */ +static int +make_symlink (const char *source, const struct dirent *entry, + const char *target) +{ + return symlink (source, target); +} + +#endif + +/* Mirror with FIRMLINK all the top-level entries in SOURCE to TARGET. */ static void -bind_mount (const char *source, const char *target) +mirror_directory (const char *source, const char *target, + int (* firmlink) (const char *, const struct dirent *, + const char *)) { DIR *stream = opendir (source); @@ -150,17 +188,7 @@ bind_mount (const char *source, const char *target) else { /* Create the mount point. */ - if (entry->d_type == DT_DIR) - { - int err = mkdir (new_entry, 0700); - if (err != 0) - assert_perror (errno); - } - else - close (open (new_entry, O_WRONLY | O_CREAT)); - - int err = mount (abs_source, new_entry, "none", - MS_BIND | MS_REC | MS_RDONLY, NULL); + int err = firmlink (abs_source, entry, new_entry); /* It used to be that only directories could be bind-mounted. Thus, keep going if we fail to bind-mount a non-directory entry. @@ -244,7 +272,7 @@ exec_in_user_namespace (const char *store, int argc, char *argv[]) /* Note: Due to we cannot make NEW_ROOT a tmpfs (which would have saved the need for 'rm_rf'.) */ - bind_mount ("/", new_root); + mirror_directory ("/", new_root, bind_mount); mkdir_p (new_store); err = mount (store, new_store, "none", MS_BIND | MS_REC | MS_RDONLY, NULL); @@ -336,6 +364,106 @@ exec_with_proot (const char *store, int argc, char *argv[]) #endif + +#if HAVE_EXEC_WITH_LOADER + +static void +exec_with_loader (const char *store, int argc, char *argv[]) +{ + static const char *runpath[] = PROGRAM_RUNPATH; + char *library_path; + size_t size = 0; + + for (size_t i = 0; runpath[i] != NULL; i++) + size += strlen (store) + strlen (runpath[i]) + 1; /* upper bound */ + + library_path = xmalloc (size + 1); + library_path[0] = '\0'; + + for (size_t i = 0; runpath[i] != NULL; i++) + { + if (strncmp (runpath[i], "@STORE_DIRECTORY@", + sizeof "@STORE_DIRECTORY@" - 1) == 0) + { + strcat (library_path, store); + strcat (library_path, runpath[i] + sizeof "@STORE_DIRECTORY@"); + } + else + strcat (library_path, runpath[i]); /* possibly $ORIGIN */ + + strcat (library_path, ":"); + } + + library_path[strlen (library_path) - 1] = '\0'; /* Remove trailing colon. */ + + char *loader = concat (store, + PROGRAM_INTERPRETER + sizeof "@STORE_DIRECTORY@"); + size_t loader_specific_argc = 6; + size_t loader_argc = argc + loader_specific_argc; + char *loader_argv[loader_argc + 1]; + loader_argv[0] = argv[0]; + loader_argv[1] = "--library-path"; + loader_argv[2] = library_path; + loader_argv[3] = "--preload"; + loader_argv[4] = concat (store, + FAKECHROOT_LIBRARY + sizeof "@STORE_DIRECTORY@"); + loader_argv[5] = concat (store, + "@WRAPPED_PROGRAM@" + sizeof "@STORE_DIRECTORY@"); + + for (size_t i = 0; i < argc; i++) + loader_argv[i + loader_specific_argc] = argv[i + 1]; + + loader_argv[loader_argc] = NULL; + + /* Set up the root directory. */ + int err; + char *new_root = mkdtemp (strdup ("/tmp/guix-exec-XXXXXX")); + mirror_directory ("/", new_root, make_symlink); + + char *new_store = concat (new_root, "@STORE_DIRECTORY@"); + char *new_store_parent = dirname (strdup (new_store)); + mkdir_p (new_store_parent); + symlink (store, new_store); + + setenv ("FAKECHROOT_BASE", new_root, 1); + + pid_t child = fork (); + switch (child) + { + case 0: + err = execv (loader, loader_argv); + if (err < 0) + assert_perror (errno); + exit (EXIT_FAILURE); + break; + + case -1: + assert_perror (errno); + exit (EXIT_FAILURE); + break; + + default: + { + int status; + waitpid (child, &status, 0); + chdir ("/"); /* avoid EBUSY */ + rm_rf (new_root); + free (new_root); + + close (2); /* flushing stderr should be silent */ + + if (WIFEXITED (status)) + exit (WEXITSTATUS (status)); + else + /* Abnormal termination cannot really be reproduced, so exit + with 255. */ + exit (255); + } + } +} + +#endif + /* Execution engines. */ @@ -352,7 +480,7 @@ buffer_stderr (void) setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer); } -/* The default engine. */ +/* The default engine: choose a robust method. */ static void exec_default (const char *store, int argc, char *argv[]) { @@ -366,13 +494,29 @@ exec_default (const char *store, int argc, char *argv[]) #endif } +/* The "performance" engine: choose performance over robustness. */ +static void +exec_performance (const char *store, int argc, char *argv[]) +{ + buffer_stderr (); + + exec_in_user_namespace (store, argc, argv); +#if HAVE_EXEC_WITH_LOADER + exec_with_loader (store, argc, argv); +#endif +} + /* List of supported engines. */ static const struct engine engines[] = { { "default", exec_default }, + { "performance", exec_performance }, { "userns", exec_in_user_namespace }, #ifdef PROOT_PROGRAM { "proot", exec_with_proot }, +#endif +#if HAVE_EXEC_WITH_LOADER + { "fakechroot", exec_with_loader }, #endif { NULL, NULL } }; diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 580f696b41..3b72496a34 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -684,15 +684,26 @@ last resort for relocation." (define (proot) (specification->package "proot-static")) + (define (fakechroot-library) + (file-append (specification->package "fakechroot") + "/lib/fakechroot/libfakechroot.so")) + (define build (with-imported-modules (source-module-closure '((guix build utils) - (guix build union))) + (guix build union) + (guix build gremlin) + (guix elf))) #~(begin (use-modules (guix build utils) ((guix build union) #:select (relative-file-name)) + (guix build gremlin) + (guix elf) + (ice-9 binary-ports) (ice-9 ftw) - (ice-9 match)) + (ice-9 match) + (srfi srfi-1) + (rnrs bytevectors)) (define input ;; The OUTPUT* output of PACKAGE. @@ -711,6 +722,47 @@ last resort for relocation." (#f base) (index (string-drop base index))))) + (define (elf-interpreter elf) + ;; Return the interpreter of ELF as a string, or #f if ELF has no + ;; interpreter segment. + (match (find (lambda (segment) + (= (elf-segment-type segment) PT_INTERP)) + (elf-segments elf)) + (#f #f) ;maybe a .so + (segment + (let ((bv (make-bytevector (- (elf-segment-memsz segment) 1)))) + (bytevector-copy! (elf-bytes elf) + (elf-segment-offset segment) + bv 0 (bytevector-length bv)) + (utf8->string bv))))) + + (define (elf-loader-compile-flags program) + ;; Return the cpp flags defining macros for the ld.so/fakechroot + ;; wrapper of PROGRAM. + + ;; TODO: Handle scripts by wrapping their interpreter. + (if (elf-file? program) + (let* ((bv (call-with-input-file program get-bytevector-all)) + (elf (parse-elf bv))) + (match (elf-dynamic-info elf) + (#f '()) + (dyninfo + (let ((runpath (elf-dynamic-info-runpath dyninfo)) + (interp (elf-interpreter elf))) + (if interp + (list (string-append "-DPROGRAM_INTERPRETER=\"" + interp "\"") + (string-append "-DPROGRAM_RUNPATH={ " + (string-join + (map object->string + runpath) + ", ") + ", NULL }") + (string-append "-DFAKECHROOT_LIBRARY=\"" + #$(fakechroot-library) "\"")) + '()))))) + '())) + (define (build-wrapper program) ;; Build a user-namespace wrapper for PROGRAM. (format #t "building wrapper for '~a'...~%" program) @@ -730,10 +782,11 @@ last resort for relocation." (mkdir-p (dirname result)) (apply invoke #$compiler "-std=gnu99" "-static" "-Os" "-g0" "-Wall" "run.c" "-o" result - (if proot - (list (string-append "-DPROOT_PROGRAM=\"" - proot "\"")) - '())) + (append (if proot + (list (string-append "-DPROOT_PROGRAM=\"" + proot "\"")) + '()) + (elf-loader-compile-flags program))) (delete-file "run.c"))) (setvbuf (current-output-port) 'line) diff --git a/tests/guix-pack-relocatable.sh b/tests/guix-pack-relocatable.sh index cb56815fed..358cac5b26 100644 --- a/tests/guix-pack-relocatable.sh +++ b/tests/guix-pack-relocatable.sh @@ -94,6 +94,12 @@ case "`uname -m`" in export GUIX_EXECUTION_ENGINE "$test_directory/Bin/sed" --version > "$test_directory/output" grep 'GNU sed' "$test_directory/output" + + # Now with fakechroot. + GUIX_EXECUTION_ENGINE="fakechroot" + "$test_directory/Bin/sed" --version > "$test_directory/output" + grep 'GNU sed' "$test_directory/output" + chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/* ;; *) -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 11 17:43:53 2020 Received: (at 41189) by debbugs.gnu.org; 11 May 2020 21:43:53 +0000 Received: from localhost ([127.0.0.1]:53540 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYGDI-0004Gn-Af for submit@debbugs.gnu.org; Mon, 11 May 2020 17:43:53 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:58398 helo=us-smtp-1.mimecast.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYFoS-0003cM-8V for 41189@debbugs.gnu.org; Mon, 11 May 2020 17:18:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589231891; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=K3IZnbiv4Eim6IM9yoOY7Xde4MMdpW65/FL2N4ASoIY=; b=Ki41mDe2wjKlr37D1KIrdOg/kotjZVuRHc0FaA/1f2NHgdy3mhc/h5GHIZJzfLTpezIEmC tmBixTvMDxKG5adJjAXC88BzHno8PS0+56g+93dXTSyd3rjCLGY4LWDhisaDxnzKLKEG0r 8y7cJWewesgREkwoGOkxnHKWJF7e16c= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-490-7FdDPzpWMUO4Ff5v-UY9XQ-1; Mon, 11 May 2020 17:18:07 -0400 X-MC-Unique: 7FdDPzpWMUO4Ff5v-UY9XQ-1 Received: by mail-qt1-f197.google.com with SMTP id p31so11915681qte.1 for <41189@debbugs.gnu.org>; Mon, 11 May 2020 14:18:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:subject:from:organization:message-id:date :user-agent:mime-version:content-language:content-transfer-encoding; bh=K3IZnbiv4Eim6IM9yoOY7Xde4MMdpW65/FL2N4ASoIY=; b=W5yeU2IEw1NO0W2X3fltkFgsuk/Ui8BVvz94oXJp8jDIvsrXty/DDp3QyR9nVp0CWH uz81dgJp8kaSgFmHcOP+qWKjm4X+TyQrz1p/E3uN1a3g0faZLhb/debJBprHdG+7LF7z Uik7FjvcxyAUTxVwvLrKYuLE822XdMS/BMwe4TvFXyGbuWoD5tk4V49/z1iHeg4heFwd hs1KINiVE1xUi4m/4soFleYeIwKDaelQAc6d9LETgH/qxRq66kis0+62t3t2g8NnLP+C Vtt4K6spWiJ+2dNqP1KMbXqgUefs9Ol4IGkGQlwzGyF2Ehe5JDvN94ElTVc2TEgor297 VlcA== X-Gm-Message-State: AGi0PuZduhcm4GGVs0ZdhT7aBKeaaFjscNra/L4VI8scyKNYwAap3oGF rKtmXByNaMOWQdrTQhVzf9ny0FjczXS1NmuTyJVkaDc5xMlkgWi4xwoSjJECu7e0xHqSaPKhb5i kSzJY8KP+WNc1XHw= X-Received: by 2002:a37:67c3:: with SMTP id b186mr17575796qkc.401.1589231886885; Mon, 11 May 2020 14:18:06 -0700 (PDT) X-Google-Smtp-Source: APiQypJS68USWpfOXpPKObxtIi7dbKIjj8rBHNh2ZAs50XwNv+rzHpS/ETL1qbw5BcToAQF6JEY3DQ== X-Received: by 2002:a37:67c3:: with SMTP id b186mr17575770qkc.401.1589231886596; Mon, 11 May 2020 14:18:06 -0700 (PDT) Received: from [192.168.1.4] (198-84-170-103.cpe.teksavvy.com. [198.84.170.103]) by smtp.gmail.com with ESMTPSA id k20sm4009644qkk.30.2020.05.11.14.18.05 for <41189@debbugs.gnu.org> (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 11 May 2020 14:18:05 -0700 (PDT) To: 41189@debbugs.gnu.org Subject: [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' From: Carlos O'Donell Organization: Red Hat Message-ID: <28e3ffa2-b565-3052-e0c7-7208fab25a11@redhat.com> Date: Mon, 11 May 2020 17:18:04 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 Content-Language: en-US X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41189 X-Mailman-Approved-At: Mon, 11 May 2020 17:43:51 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) "For example, some of the ‘open’ calls made in libc are notintercepted; on such call is in ‘__gconv_load_cache’, which makesit fail, and in turn makes Guile fail to start in its first‘scm_to_locale_string’ call." -- Ludovic Courtès wrote on Mon May 11 19:05:54+0200 2020 There are two issues at hand: * Standard namespace issues (conformance) * PLT avoidance issues (performance) See: https://sourceware.org/glibc/wiki/Style_and_Conventions#Double-underscore_names_for_public_API_functions It is an internal implementation detail that open(2) is being called by the library, and as such glibc bypasses the ELF interposable symbol open, and instead calls open directly without this being visible to the application. There are many such cases where we bypass the ELF interposable symbol to provide standard namespace cleanliness, performance, and so provide consistent behaviour. Yes, in your case this means you cannot override the behaviour of the interface without using some kind of bind mount, or mount namespace (to provide an alternate view of the filesystem). We would have to argue upstream that some minimal subset of the filesystem access should be interposable via open/close/read/write, but that's going to get difficult quickly and have significant performance problems. It would be simpler, IMO, to set LOCPATH and GCONV_PATH appropriately and alter the runtime behaviour that way. If that doesn't work, perhaps because of setuid, then we can discuss further. -- Cheers, Carlos. From debbugs-submit-bounces@debbugs.gnu.org Tue May 12 06:03:37 2020 Received: (at 41189) by debbugs.gnu.org; 12 May 2020 10:03:37 +0000 Received: from localhost ([127.0.0.1]:54319 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYRlB-0006u3-FH for submit@debbugs.gnu.org; Tue, 12 May 2020 06:03:37 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:37754) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYRlA-0006tk-81 for 41189@debbugs.gnu.org; Tue, 12 May 2020 06:03:36 -0400 X-IronPort-AV: E=Sophos;i="5.73,383,1583190000"; d="scan'208";a="449285496" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 May 2020 12:03:29 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Carlos O'Donell Subject: Re: [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' References: <20200511170554.22916-1-ludo@gnu.org> <28e3ffa2-b565-3052-e0c7-7208fab25a11@redhat.com> Date: Tue, 12 May 2020 12:03:29 +0200 In-Reply-To: <28e3ffa2-b565-3052-e0c7-7208fab25a11@redhat.com> (Carlos O'Donell's message of "Mon, 11 May 2020 17:18:04 -0400") Message-ID: <87r1vpbhce.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 41189 Cc: 41189@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Carlos, Carlos O'Donell skribis: > There are two issues at hand: > * Standard namespace issues (conformance) > * PLT avoidance issues (performance) > > See: > https://sourceware.org/glibc/wiki/Style_and_Conventions#Double-underscore= _names_for_public_API_functions > > It is an internal implementation detail that open(2) is being called by > the library, and as such glibc bypasses the ELF interposable symbol > open, and instead calls open directly without this being visible to the > application. > > There are many such cases where we bypass the ELF interposable symbol to > provide standard namespace cleanliness, performance, and so provide consi= stent > behaviour. It makes sense to me, thanks for explaining. > Yes, in your case this means you cannot override the behaviour of the > interface without using some kind of bind mount, or mount namespace > (to provide an alternate view of the filesystem). Agreed, unprivileged user namespaces with bind mounts are the preferred solution; the LD_PRELOAD hack discussed here is for when they=E2=80=99re unavailable and PRoot is too slow. > We would have to argue upstream that some minimal subset of the filesystem > access should be interposable via open/close/read/write, but that's going > to get difficult quickly and have significant performance problems. Yes, understood. (I wasn=E2=80=99t going to suggest it. :-)) > It would be simpler, IMO, to set LOCPATH and GCONV_PATH appropriately and > alter the runtime behaviour that way. If that doesn't work, perhaps becau= se > of setuid, then we can discuss further. Yes, setting =E2=80=98GCONV_PATH=E2=80=99 in particular seems like somethin= g the wrapper could automatically do. The attached patch does that and now Guile runs fine with the ld.so/fakechroot =E2=80=9Cengine=E2=80=9D. One thing that won=E2=80=99t work is dlopen because our =E2=80=98--library-= path=E2=80=99 argument is computed statically based on the RUNPATH of the wrapped program. So for instance if you try to load guile-readline.so from Guile, it eventually fails because libreadline.so isn=E2=80=99t found (libreadline.so is in the RUNPATH of guile-readline.so, but the loader uses non-interposable calls here as well.) Probably no simple solution to that one. Thanks for your feedback, Carlos! Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c index ed72a169f2..c56c35a510 100644 --- a/gnu/packages/aux-files/run-in-namespace.c +++ b/gnu/packages/aux-files/run-in-namespace.c @@ -425,6 +427,15 @@ exec_with_loader (const char *store, int argc, char *argv[]) mkdir_p (new_store_parent); symlink (store, new_store); +#ifdef GCONV_DIRECTORY + /* Tell libc where to find its gconv modules. This is necessary because + gconv uses non-interposable 'open' calls. */ + char *gconv_path = concat (store, + GCONV_DIRECTORY + sizeof "@STORE_DIRECTORY@"); + setenv ("GCONV_PATH", gconv_path, 1); + free (gconv_path); +#endif + setenv ("FAKECHROOT_BASE", new_root, 1); pid_t child = fork (); diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 2d856066b2..2b37bf5027 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -739,6 +739,12 @@ last resort for relocation." bv 0 (bytevector-length bv)) (utf8->string bv))))) + (define (gconv-directory directory) + ;; Return DIRECTORY/gconv if it exists as a directory. + (let ((gconv (string-append directory "/gconv"))) + (and (directory-exists? gconv) + gconv))) + (define (elf-loader-compile-flags program) ;; Return the cpp flags defining macros for the ld.so/fakechroot ;; wrapper of PROGRAM. @@ -750,8 +756,9 @@ last resort for relocation." (match (elf-dynamic-info elf) (#f '()) (dyninfo - (let ((runpath (elf-dynamic-info-runpath dyninfo)) - (interp (elf-interpreter elf))) + (let* ((runpath (elf-dynamic-info-runpath dyninfo)) + (gconv (any gconv-directory runpath)) + (interp (elf-interpreter elf))) (if interp (list (string-append "-DPROGRAM_INTERPRETER=\"" interp "\"") @@ -762,7 +769,12 @@ last resort for relocation." ", ") ", NULL }") (string-append "-DFAKECHROOT_LIBRARY=\"" - #$(fakechroot-library) "\"")) + #$(fakechroot-library) "\"") + + (if gconv + (string-append "-DGCONV_DIRECTORY=\"" + gconv "\"") + "-UGCONV_DIRECTORY")) '()))))) '())) --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue May 12 10:58:33 2020 Received: (at 41189) by debbugs.gnu.org; 12 May 2020 14:58:33 +0000 Received: from localhost ([127.0.0.1]:56333 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYWMb-0004Qv-2C for submit@debbugs.gnu.org; Tue, 12 May 2020 10:58:33 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:40849 helo=us-smtp-delivery-1.mimecast.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYTjV-0003q5-60 for 41189@debbugs.gnu.org; Tue, 12 May 2020 08:10:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589285400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MzQgWKgdcz2E91AN0YylVPXaEZIDov321tC6PZEHeXg=; b=Bll1gP49KqqpUnF4jZGXDvI6/BcC6TeP5X5wEPYP8P75KhUBdcRMeP0M4MPCpijEelRf7V n1tfDeAMK9z/3EURAPeh5dpD+bF16YyOqxY77eHFMBDvWWPJJGb7ENGBqkau/QatsOlMXQ 7mzhoIpjmnY37DdQoYKA3mo19srtcNc= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-125-t3PV57xgNDK-EvjeeGivEg-1; Tue, 12 May 2020 08:09:56 -0400 X-MC-Unique: t3PV57xgNDK-EvjeeGivEg-1 Received: by mail-qv1-f69.google.com with SMTP id y60so11091880qvy.13 for <41189@debbugs.gnu.org>; Tue, 12 May 2020 05:09:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=MzQgWKgdcz2E91AN0YylVPXaEZIDov321tC6PZEHeXg=; b=PfmtIB7gLXsbF2UirGwYIoM8WXgJ5XnDFjvx5cj5lg0CpKyJ84prnXuUDgo49Y4rO2 KLUxxcTjWXtjTqOPyzoz2aGcz1OzPWwH9FEUVNwa1NODDI16GbXay2yeCZ6c7kaLoo37 XLo5jYG6Yh10Cbi8OFjcL2vPK1GI70KXET+wFaYTzjt1d0ncWOsthcZpsgf/Ohh8hxLD YKc8fJSBzTIuFCgndcvK3yphdZy9FPSSUz5oN9gf6p3atkpP6amC9j3cqQzfaOaOxYF9 FPgd0b6w70v5BGENusfxDotS2+xdyFEGt59IXcN5esUwqdueB/bvkVf7EcNeGt2QKAAh Ciww== X-Gm-Message-State: AGi0PuZfZcO0x25+ynm+wPr9hBVs4fmQQjSJhH3J++PFneiCtaHi3CW0 X2xz+WdEpTuDIF4cQZX/GBdkklGrp0QeruV2jNc3P0TFGMOgkt+7LBn/e+ccMKEEgW2vVRz6Q+L y/yIHZ4cGl2nDocQ= X-Received: by 2002:a37:b185:: with SMTP id a127mr19961690qkf.87.1589285396044; Tue, 12 May 2020 05:09:56 -0700 (PDT) X-Google-Smtp-Source: APiQypIurnbGhaVNjQQMw1Voo0oPHb4uk/cWWGyarxEpwngFl8mfhzCqtxqLrJ8srmuF0ReP8+V1uw== X-Received: by 2002:a37:b185:: with SMTP id a127mr19961665qkf.87.1589285395788; Tue, 12 May 2020 05:09:55 -0700 (PDT) Received: from [192.168.1.4] (198-84-170-103.cpe.teksavvy.com. [198.84.170.103]) by smtp.gmail.com with ESMTPSA id d82sm10979637qke.81.2020.05.12.05.09.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 12 May 2020 05:09:55 -0700 (PDT) Subject: Re: [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' To: =?UTF-8?Q?Ludovic_Court=c3=a8s?= References: <20200511170554.22916-1-ludo@gnu.org> <28e3ffa2-b565-3052-e0c7-7208fab25a11@redhat.com> <87r1vpbhce.fsf@gnu.org> From: Carlos O'Donell Organization: Red Hat Message-ID: Date: Tue, 12 May 2020 08:09:53 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <87r1vpbhce.fsf@gnu.org> Content-Language: en-US X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41189 X-Mailman-Approved-At: Tue, 12 May 2020 10:58:30 -0400 Cc: 41189@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 5/12/20 6:03 AM, Ludovic Courtès wrote: > One thing that won’t work is dlopen because our ‘--library-path’ > argument is computed statically based on the RUNPATH of the wrapped > program. So for instance if you try to load guile-readline.so from > Guile, it eventually fails because libreadline.so isn’t found > (libreadline.so is in the RUNPATH of guile-readline.so, but the loader > uses non-interposable calls here as well.) Probably no simple solution > to that one. There is a simple solution. You need to write a dynamic loader audit module that handles la_objsearch() and inject your lookup path. See man 7 rtld-audit. The dynamic loader audit modules allow you to alter the loader's core behaviour with a plugin. -- Cheers, Carlos. From debbugs-submit-bounces@debbugs.gnu.org Tue May 12 11:32:43 2020 Received: (at 41189) by debbugs.gnu.org; 12 May 2020 15:32:43 +0000 Received: from localhost ([127.0.0.1]:56368 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYWtY-0005LA-5q for submit@debbugs.gnu.org; Tue, 12 May 2020 11:32:43 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:23260) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYWtV-0005Kj-HM for 41189@debbugs.gnu.org; Tue, 12 May 2020 11:32:34 -0400 X-IronPort-AV: E=Sophos;i="5.73,384,1583190000"; d="scan'208";a="449355026" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 May 2020 17:32:26 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Carlos O'Donell Subject: Re: [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' References: <20200511170554.22916-1-ludo@gnu.org> <28e3ffa2-b565-3052-e0c7-7208fab25a11@redhat.com> <87r1vpbhce.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 24 =?utf-8?Q?Flor=C3=A9al?= an 228 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 12 May 2020 17:32:26 +0200 In-Reply-To: (Carlos O'Donell's message of "Tue, 12 May 2020 08:09:53 -0400") Message-ID: <87tv0l9njp.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 41189 Cc: 41189@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Carlos, Carlos O'Donell skribis: > On 5/12/20 6:03 AM, Ludovic Court=C3=A8s wrote: >> One thing that won=E2=80=99t work is dlopen because our =E2=80=98--libra= ry-path=E2=80=99 >> argument is computed statically based on the RUNPATH of the wrapped >> program. So for instance if you try to load guile-readline.so from >> Guile, it eventually fails because libreadline.so isn=E2=80=99t found >> (libreadline.so is in the RUNPATH of guile-readline.so, but the loader >> uses non-interposable calls here as well.) Probably no simple solution >> to that one. > > There is a simple solution. You need to write a dynamic loader audit modu= le > that handles la_objsearch() and inject your lookup path. See man 7 rtld-a= udit. > The dynamic loader audit modules allow you to alter the loader's core beh= aviour > with a plugin. That=E2=80=99s a great idea, and it works like a charm. Thank you! Concretely, I can do: guix pack -RR -S /bin=3Dbin -S /etc=3Detc guile guile-readline and then, on the target machine: tar xf pack.tar.gz . ./etc/profile export GUIX_EXECUTION_ENGINE=3Dfakechroot ./bin/guile -c '(use-modules (ice-9 readline))' Neat! Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable diff --git a/Makefile.am b/Makefile.am index 6cd6e79cab..f65bf5f900 100644 --- a/Makefile.am +++ b/Makefile.am @@ -338,6 +338,7 @@ AUX_FILES =3D \ gnu/packages/aux-files/linux-libre/4.9-x86_64.conf \ gnu/packages/aux-files/linux-libre/4.4-i686.conf \ gnu/packages/aux-files/linux-libre/4.4-x86_64.conf \ + gnu/packages/aux-files/pack-audit.c \ gnu/packages/aux-files/run-in-namespace.c =20 # Templates, examples. diff --git a/gnu/packages/aux-files/pack-audit.c b/gnu/packages/aux-files/p= ack-audit.c new file mode 100644 index 0000000000..374787e8b9 --- /dev/null +++ b/gnu/packages/aux-files/pack-audit.c @@ -0,0 +1,85 @@ +/* GNU Guix --- Functional package management for GNU + Copyright (C) 2020 Ludovic Court=C3=A8s + + This file is part of GNU Guix. + + GNU Guix is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or (at + your option) any later version. + + GNU Guix is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with GNU Guix. If not, see . */ + +/* This file implements part of the GNU ld.so audit interface. It is used= by + the "fakechroot" engine of the 'guix pack -RR' wrappers to make sure the + loader looks for shared objects under the "fake" root directory. */ + +#define _GNU_SOURCE 1 + +#include + +#include +#include +#include +#include + +/* The pseudo root directory and store that we are relocating to. */ +static const char *root_directory; +static char *store; + +/* The original store, "/gnu/store" by default. */ +static const char original_store[] =3D "@STORE_DIRECTORY@"; + +/* Like 'malloc', but abort if 'malloc' returns NULL. */ +static void * +xmalloc (size_t size) +{ + void *result =3D malloc (size); + assert (result !=3D NULL); + return result; +} + +unsigned int +la_version (unsigned int v) +{ + if (v !=3D LAV_CURRENT) + error (1, 0, "cannot handle interface version %u", v); + + root_directory =3D getenv ("FAKECHROOT_BASE"); + if (root_directory =3D=3D NULL) + error (1, 0, "'FAKECHROOT_BASE' is not set"); + + store =3D xmalloc (strlen (root_directory) + sizeof original_store); + strcpy (store, root_directory); + strcat (store, original_store); + + return v; +} + +/* Return NAME, a shared object file name, relocated under STORE. This + function is called by the loader whenever it looks for a shared object.= */ +char * +la_objsearch (const char *name, uintptr_t *cookie, unsigned int flag) +{ + char *result; + + if (strncmp (name, original_store, + sizeof original_store - 1) =3D=3D 0) + { + size_t len =3D strlen (name) - sizeof original_store + + strlen (store) + 1; + result =3D xmalloc (len); + strcpy (result, store); + strcat (result, name + sizeof original_store - 1); + } + else + result =3D strdup (name); + + return result; +} diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-f= iles/run-in-namespace.c index c56c35a510..5ed1c6a1bc 100644 --- a/gnu/packages/aux-files/run-in-namespace.c +++ b/gnu/packages/aux-files/run-in-namespace.c @@ -400,7 +400,7 @@ exec_with_loader (const char *store, int argc, char *ar= gv[]) =20 char *loader =3D concat (store, PROGRAM_INTERPRETER + sizeof "@STORE_DIRECTORY@"); - size_t loader_specific_argc =3D 6; + size_t loader_specific_argc =3D 8; size_t loader_argc =3D argc + loader_specific_argc; char *loader_argv[loader_argc + 1]; loader_argv[0] =3D argv[0]; @@ -409,7 +409,10 @@ exec_with_loader (const char *store, int argc, char *a= rgv[]) loader_argv[3] =3D "--preload"; loader_argv[4] =3D concat (store, FAKECHROOT_LIBRARY + sizeof "@STORE_DIRECTORY@"); - loader_argv[5] =3D concat (store, + loader_argv[5] =3D "--audit"; + loader_argv[6] =3D concat (store, + LOADER_AUDIT_MODULE + sizeof "@STORE_DIRECTORY@"); + loader_argv[7] =3D concat (store, "@WRAPPED_PROGRAM@" + sizeof "@STORE_DIRECTORY@"); =20 for (size_t i =3D 0; i < argc; i++) diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 2b37bf5027..13ede581e9 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -681,6 +681,9 @@ last resort for relocation." (define runner (local-file (search-auxiliary-file "run-in-namespace.c"))) =20 + (define audit-source + (local-file (search-auxiliary-file "pack-audit.c"))) + (define (proot) (specification->package "proot-static")) =20 @@ -691,6 +694,21 @@ last resort for relocation." "/lib/fakechroot/libfakechroot.so") #$output))) =20 + (define (audit-module) + (computed-file "pack-audit.so" + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils)) + + (copy-file #$audit-source "audit.c") + (substitute* "audit.c" + (("@STORE_DIRECTORY@") + (%store-directory))) + + (invoke #$compiler "-std=3Dgnu99" + "-shared" "-fPIC" "-Os" "-g0" + "-Wall" "audit.c" "-o" #$output))))) + (define build (with-imported-modules (source-module-closure '((guix build utils) @@ -771,6 +789,8 @@ last resort for relocation." (string-append "-DFAKECHROOT_LIBRARY=3D\"" #$(fakechroot-library) "\"= ") =20 + (string-append "-DLOADER_AUDIT_MODULE=3D\= "" + #$(audit-module) "\"") (if gconv (string-append "-DGCONV_DIRECTORY=3D\= "" gconv "\"") --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed May 13 08:52:29 2020 Received: (at 41189) by debbugs.gnu.org; 13 May 2020 12:52:29 +0000 Received: from localhost ([127.0.0.1]:57626 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqs9-0007wG-Lp for submit@debbugs.gnu.org; Wed, 13 May 2020 08:52:29 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37714) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqs7-0007vb-EI for 41189@debbugs.gnu.org; Wed, 13 May 2020 08:52:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35567) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYqs2-0001Kd-4z; Wed, 13 May 2020 08:52:22 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44970 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYqs1-0006Fm-RC; Wed, 13 May 2020 08:52:22 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41189@debbugs.gnu.org Subject: [PATCH v2 0/4] Add Fakechroot engine for 'guix pack -RR' Date: Wed, 13 May 2020 14:52:11 +0200 Message-Id: <20200513125215.27740-1-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <87tv0l9njp.fsf@inria.fr> References: <87tv0l9njp.fsf@inria.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41189 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello! This version incorporates changes discussed in this thread along with subsequent cleanups: • Set ‘GCONV_PATH’. • Build an ld.so audit module to rewrite store file names read from RUNPATH entries and the likes. • Get rid of the ‘--library-path’ argument to ld.so, which is now useless since the audit module takes care of rewriting .so file names. This simplifies both ‘exec_with_loader’ in the wrapper and ‘wrapped-package’ in ‘guix pack’. Ludo’. Ludovic Courtès (4): pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable. pack: Factorize store references in wrapper. gnu: Add fakechroot. pack: Add relocation via ld.so and fakechroot. Makefile.am | 1 + doc/guix.texi | 43 +++- gnu/packages/aux-files/pack-audit.c | 85 ++++++++ gnu/packages/aux-files/run-in-namespace.c | 251 +++++++++++++++++++--- gnu/packages/linux.scm | 30 +++ guix/scripts/pack.scm | 87 +++++++- tests/guix-pack-relocatable.sh | 23 ++ 7 files changed, 479 insertions(+), 41 deletions(-) create mode 100644 gnu/packages/aux-files/pack-audit.c -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Wed May 13 08:52:31 2020 Received: (at 41189) by debbugs.gnu.org; 13 May 2020 12:52:31 +0000 Received: from localhost ([127.0.0.1]:57630 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqsA-0007wZ-Su for submit@debbugs.gnu.org; Wed, 13 May 2020 08:52:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37718) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqs8-0007vi-GR for 41189@debbugs.gnu.org; Wed, 13 May 2020 08:52:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35568) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYqs3-0001Kl-83; Wed, 13 May 2020 08:52:23 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44970 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYqs2-0006Fm-GG; Wed, 13 May 2020 08:52:22 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41189@debbugs.gnu.org Subject: [PATCH v2 1/4] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable. Date: Wed, 13 May 2020 14:52:12 +0200 Message-Id: <20200513125215.27740-2-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200513125215.27740-1-ludo@gnu.org> References: <87tv0l9njp.fsf@inria.fr> <20200513125215.27740-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41189 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/aux-files/run-in-namespace.c (struct engine): New type. (exec_default): New function. (engines): New variable. (execution_engine): New function. (main): Use it instead of calling 'exec_in_user_namespace' and 'exec_with_proot' directly. * tests/guix-pack-relocatable.sh: Add test with 'GUIX_EXECUTION_ENGINE'. * doc/guix.texi (Invoking guix pack): Document 'GUIX_EXECUTION_ENGINE'. --- doc/guix.texi | 30 +++++++-- gnu/packages/aux-files/run-in-namespace.c | 78 ++++++++++++++++++++--- tests/guix-pack-relocatable.sh | 17 +++++ 3 files changed, 110 insertions(+), 15 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 0cba0ee1ec..958ed9ceec 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5185,9 +5185,9 @@ When this option is passed once, the resulting binaries require support for @dfn{user namespaces} in the kernel Linux; when passed @emph{twice}@footnote{Here's a trick to memorize it: @code{-RR}, which adds PRoot support, can be thought of as the abbreviation of ``Really -Relocatable''. Neat, isn't it?}, relocatable binaries fall to back to PRoot -if user namespaces are unavailable, and essentially work anywhere---see below -for the implications. +Relocatable''. Neat, isn't it?}, relocatable binaries fall to back to +other techniques if user namespaces are unavailable, and essentially +work anywhere---see below for the implications. For example, if you create a pack containing Bash with: @@ -5219,14 +5219,32 @@ turn it off. To produce relocatable binaries that work even in the absence of user namespaces, pass @option{--relocatable} or @option{-R} @emph{twice}. In that -case, binaries will try user namespace support and fall back to PRoot if user -namespaces are not supported. +case, binaries will try user namespace support and fall back to another +@dfn{execution engine} if user namespaces are not supported. The +following execution engines are supported: -The @uref{https://proot-me.github.io/, PRoot} program provides the necessary +@table @code +@item default +Try user namespaces and fall back to PRoot if user namespaces are not +supported (see below). + +@item userns +Run the program through user namespaces and abort if they are not +supported. + +@item proot +Run through PRoot. The @uref{https://proot-me.github.io/, PRoot} program +provides the necessary support for file system virtualization. It achieves that by using the @code{ptrace} system call on the running program. This approach has the advantage to work without requiring special kernel support, but it incurs run-time overhead every time a system call is made. +@end table + +@vindex GUIX_EXECUTION_ENGINE +When running a wrapped program, you can explicitly request one of the +execution engines listed above by setting the +@code{GUIX_EXECUTION_ENGINE} environment variable accordingly. @end quotation @cindex entry point, for Docker images diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c index 23e7875173..6beac7fd53 100644 --- a/gnu/packages/aux-files/run-in-namespace.c +++ b/gnu/packages/aux-files/run-in-namespace.c @@ -336,6 +336,71 @@ exec_with_proot (const char *store, int argc, char *argv[]) #endif + +/* Execution engines. */ + +struct engine +{ + const char *name; + void (* exec) (const char *, int, char **); +}; + +static void +buffer_stderr (void) +{ + static char stderr_buffer[4096]; + setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer); +} + +/* The default engine. */ +static void +exec_default (const char *store, int argc, char *argv[]) +{ + /* Buffer stderr so that nothing's displayed if 'exec_in_user_namespace' + fails but 'exec_with_proot' works. */ + buffer_stderr (); + + exec_in_user_namespace (store, argc, argv); +#ifdef PROOT_PROGRAM + exec_with_proot (store, argc, argv); +#endif +} + +/* List of supported engines. */ +static const struct engine engines[] = + { + { "default", exec_default }, + { "userns", exec_in_user_namespace }, +#ifdef PROOT_PROGRAM + { "proot", exec_with_proot }, +#endif + { NULL, NULL } + }; + +/* Return the "execution engine" to use. */ +static const struct engine * +execution_engine (void) +{ + const char *str = getenv ("GUIX_EXECUTION_ENGINE"); + + if (str == NULL) + str = "default"; + + try: + for (const struct engine *engine = engines; + engine->name != NULL; + engine++) + { + if (strcmp (engine->name, str) == 0) + return engine; + } + + fprintf (stderr, "%s: unsupported Guix execution engine; ignoring\n", + str); + str = "default"; + goto try; +} + int main (int argc, char *argv[]) @@ -362,22 +427,17 @@ main (int argc, char *argv[]) if (strcmp (store, "@STORE_DIRECTORY@") != 0 && lstat ("@WRAPPED_PROGRAM@", &statbuf) != 0) { - /* Buffer stderr so that nothing's displayed if 'exec_in_user_namespace' - fails but 'exec_with_proot' works. */ - static char stderr_buffer[4096]; - setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer); + const struct engine *engine = execution_engine (); + engine->exec (store, argc, argv); - exec_in_user_namespace (store, argc, argv); -#ifdef PROOT_PROGRAM - exec_with_proot (store, argc, argv); -#else + /* If we reach this point, that's because ENGINE failed to do the + job. */ fprintf (stderr, "\ This may be because \"user namespaces\" are not supported on this system.\n\ Consequently, we cannot run '@WRAPPED_PROGRAM@',\n\ unless you move it to the '@STORE_DIRECTORY@' directory.\n\ \n\ Please refer to the 'guix pack' documentation for more information.\n"); -#endif return EXIT_FAILURE; } diff --git a/tests/guix-pack-relocatable.sh b/tests/guix-pack-relocatable.sh index a3fd45623c..cb56815fed 100644 --- a/tests/guix-pack-relocatable.sh +++ b/tests/guix-pack-relocatable.sh @@ -84,6 +84,23 @@ fi grep 'GNU sed' "$test_directory/output" chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/* +case "`uname -m`" in + x86_64|i?86) + # Try '-RR' and PRoot. + tarball="`guix pack -RR -S /Bin=bin sed`" + tar tvf "$tarball" | grep /bin/proot + (cd "$test_directory"; tar xvf "$tarball") + GUIX_EXECUTION_ENGINE="proot" + export GUIX_EXECUTION_ENGINE + "$test_directory/Bin/sed" --version > "$test_directory/output" + grep 'GNU sed' "$test_directory/output" + chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/* + ;; + *) + echo "skipping PRoot test" >&2 + ;; +esac + # Ensure '-R' works with outputs other than "out". tarball="`guix pack -R -S /share=share groff:doc`" (cd "$test_directory"; tar xvf "$tarball") -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Wed May 13 08:52:31 2020 Received: (at 41189) by debbugs.gnu.org; 13 May 2020 12:52:31 +0000 Received: from localhost ([127.0.0.1]:57632 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqsB-0007wd-A9 for submit@debbugs.gnu.org; Wed, 13 May 2020 08:52:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37724) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqsA-0007vk-2z for 41189@debbugs.gnu.org; Wed, 13 May 2020 08:52:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35571) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYqs4-0001Ku-FZ; Wed, 13 May 2020 08:52:24 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44970 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYqs3-0006Fm-Ir; Wed, 13 May 2020 08:52:24 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41189@debbugs.gnu.org Subject: [PATCH v2 2/4] pack: Factorize store references in wrapper. Date: Wed, 13 May 2020 14:52:13 +0200 Message-Id: <20200513125215.27740-3-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200513125215.27740-1-ludo@gnu.org> References: <87tv0l9njp.fsf@inria.fr> <20200513125215.27740-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41189 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/aux-files/run-in-namespace.c (original_store): New variable. (exec_in_user_namespace, exec_with_proot, main): Use it instead of the literal "@STORE_DIRECTORY@". --- gnu/packages/aux-files/run-in-namespace.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c index 6beac7fd53..6e97359078 100644 --- a/gnu/packages/aux-files/run-in-namespace.c +++ b/gnu/packages/aux-files/run-in-namespace.c @@ -42,6 +42,10 @@ #include #include +/* The original store, "/gnu/store" by default. */ +static const char original_store[] = "@STORE_DIRECTORY@"; + + /* Like 'malloc', but abort if 'malloc' returns NULL. */ static void * xmalloc (size_t size) @@ -228,7 +232,7 @@ exec_in_user_namespace (const char *store, int argc, char *argv[]) bind-mounted in the right place. */ int err; char *new_root = mkdtemp (strdup ("/tmp/guix-exec-XXXXXX")); - char *new_store = concat (new_root, "@STORE_DIRECTORY@"); + char *new_store = concat (new_root, original_store); char *cwd = get_current_dir_name (); /* Create a child with separate namespaces and set up bind-mounts from @@ -307,11 +311,11 @@ exec_with_proot (const char *store, int argc, char *argv[]) int proot_specific_argc = 4; int proot_argc = argc + proot_specific_argc; char *proot_argv[proot_argc + 1], *proot; - char bind_spec[strlen (store) + 1 + sizeof "@STORE_DIRECTORY@"]; + char bind_spec[strlen (store) + 1 + sizeof original_store]; strcpy (bind_spec, store); strcat (bind_spec, ":"); - strcat (bind_spec, "@STORE_DIRECTORY@"); + strcat (bind_spec, original_store); proot = concat (store, PROOT_PROGRAM); @@ -413,8 +417,7 @@ main (int argc, char *argv[]) /* SELF is something like "/home/ludo/.local/gnu/store/…-foo/bin/ls" and we want to extract "/home/ludo/.local/gnu/store". */ size_t index = strlen (self) - - strlen ("@WRAPPED_PROGRAM@") - + strlen ("@STORE_DIRECTORY@"); + - strlen ("@WRAPPED_PROGRAM@") + strlen (original_store); char *store = strdup (self); store[index] = '\0'; @@ -424,7 +427,7 @@ main (int argc, char *argv[]) @WRAPPED_PROGRAM@ right away. This is not just an optimization: it's needed when running one of these wrappers from within an unshare'd namespace, because 'unshare' fails with EPERM in that context. */ - if (strcmp (store, "@STORE_DIRECTORY@") != 0 + if (strcmp (store, original_store) != 0 && lstat ("@WRAPPED_PROGRAM@", &statbuf) != 0) { const struct engine *engine = execution_engine (); -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Wed May 13 08:52:34 2020 Received: (at 41189) by debbugs.gnu.org; 13 May 2020 12:52:34 +0000 Received: from localhost ([127.0.0.1]:57635 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqsE-0007ws-JH for submit@debbugs.gnu.org; Wed, 13 May 2020 08:52:34 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37728) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqsA-0007vs-GE for 41189@debbugs.gnu.org; Wed, 13 May 2020 08:52:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35572) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYqs5-0001Kz-7w; Wed, 13 May 2020 08:52:25 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44970 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYqs4-0006Fm-Qy; Wed, 13 May 2020 08:52:25 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41189@debbugs.gnu.org Subject: [PATCH v2 3/4] gnu: Add fakechroot. Date: Wed, 13 May 2020 14:52:14 +0200 Message-Id: <20200513125215.27740-4-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200513125215.27740-1-ludo@gnu.org> References: <87tv0l9njp.fsf@inria.fr> <20200513125215.27740-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41189 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/linux.scm (fakechroot): New variable. --- gnu/packages/linux.scm | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index 7cf7521e24..35526b3513 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -6793,6 +6793,36 @@ have to construct the archives directly, without using the archiver.") (home-page "http://freshmeat.sourceforge.net/projects/fakeroot") (license license:gpl3+))) +(define-public fakechroot + (package + (name "fakechroot") + (version "2.20.1") + (source (origin + (method url-fetch) + (uri (string-append + "https://github.com/dex4er/fakechroot/releases/download/" + version "/fakechroot-" version ".tar.gz")) + (sha256 + (base32 + "1aijkd0b45wav25v01qhw8zxwa3pl0nnp9fabmmy1nlx7hr09gas")))) + (build-system gnu-build-system) + (arguments + ;; XXX: The tests heavily assume they run on an FHS system so for now + ;; skip them. + '(#:tests? #f + #:configure-flags '("--disable-static"))) + (synopsis "Emulate @code{chroot} by overriding file system calls") + (description + "@command{fakechroot} runs a command in an environment were is additional +possibility to use @code{chroot} command without root privileges. This is +useful for allowing users to create own chrooted environment with possibility +to install another packages without need for root privileges. + +It works by providing @file{libfakechroot.so}, a shared library meant to be +set as @code{LD_PRELOAD} to override the C library file system functions.") + (home-page "https://github.com/dex4er/fakechroot/") + (license license:lgpl2.1+))) + (define-public inputattach (package (name "inputattach") -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Wed May 13 08:52:44 2020 Received: (at 41189) by debbugs.gnu.org; 13 May 2020 12:52:44 +0000 Received: from localhost ([127.0.0.1]:57637 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqsI-0007x8-TU for submit@debbugs.gnu.org; Wed, 13 May 2020 08:52:44 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37734) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYqsC-0007w3-PO for 41189@debbugs.gnu.org; Wed, 13 May 2020 08:52:37 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35573) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYqs6-0001LC-HF; Wed, 13 May 2020 08:52:26 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44970 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYqs5-0006Fm-JX; Wed, 13 May 2020 08:52:25 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41189@debbugs.gnu.org Subject: [PATCH v2 4/4] pack: Add relocation via ld.so and fakechroot. Date: Wed, 13 May 2020 14:52:15 +0200 Message-Id: <20200513125215.27740-5-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200513125215.27740-1-ludo@gnu.org> References: <87tv0l9njp.fsf@inria.fr> <20200513125215.27740-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41189 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Ludovic Courtès * gnu/packages/aux-files/run-in-namespace.c (HAVE_EXEC_WITH_LOADER): New macro. (bind_mount): Rename to... (mirror_directory): ... this. Add 'firmlink' argument and use it instead of calling mkdir/open/close/mount directly. (bind_mount, make_symlink): New functions. (exec_in_user_namespace): Adjust accordingly. (exec_with_loader) [HAVE_EXEC_WITH_LOADER]: New function. (exec_performance): New function. (engines): Add them. * guix/scripts/pack.scm (wrapped-package)[fakechroot-library] [audit-module]: New procedures. [audit-source]: New variable. [build](elf-interpreter, elf-loader-compile-flags): New procedures. (build-wrapper): Use them. * tests/guix-pack-relocatable.sh: Test with 'GUIX_EXECUTION_ENGINE=fakechroot'. * doc/guix.texi (Invoking guix pack): Document the 'performance' and 'fakechroot' engines. * gnu/packages/aux-files/pack-audit.c: New file. * Makefile.am (AUX_FILES): Add it. --- Makefile.am | 1 + doc/guix.texi | 13 ++ gnu/packages/aux-files/pack-audit.c | 85 ++++++++++++ gnu/packages/aux-files/run-in-namespace.c | 160 ++++++++++++++++++++-- guix/scripts/pack.scm | 87 +++++++++++- tests/guix-pack-relocatable.sh | 6 + 6 files changed, 331 insertions(+), 21 deletions(-) create mode 100644 gnu/packages/aux-files/pack-audit.c diff --git a/Makefile.am b/Makefile.am index 6cd6e79cab..f65bf5f900 100644 --- a/Makefile.am +++ b/Makefile.am @@ -338,6 +338,7 @@ AUX_FILES = \ gnu/packages/aux-files/linux-libre/4.9-x86_64.conf \ gnu/packages/aux-files/linux-libre/4.4-i686.conf \ gnu/packages/aux-files/linux-libre/4.4-x86_64.conf \ + gnu/packages/aux-files/pack-audit.c \ gnu/packages/aux-files/run-in-namespace.c # Templates, examples. diff --git a/doc/guix.texi b/doc/guix.texi index 958ed9ceec..a70a058afb 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5228,6 +5228,10 @@ following execution engines are supported: Try user namespaces and fall back to PRoot if user namespaces are not supported (see below). +@item performance +Try user namespaces and fall back to Fakechroot if user namespaces are +not supported (see below). + @item userns Run the program through user namespaces and abort if they are not supported. @@ -5239,6 +5243,15 @@ support for file system virtualization. It achieves that by using the @code{ptrace} system call on the running program. This approach has the advantage to work without requiring special kernel support, but it incurs run-time overhead every time a system call is made. + +@item fakechroot +Run through Fakechroot. @uref{https://github.com/dex4er/fakechroot/, +Fakechroot} virtualizes file system accesses by intercepting calls to C +library functions such as @code{open}, @code{stat}, @code{exec}, and so +on. Unlike PRoot, it incurs very little overhead. However, it does not +always work: for example, some file system accesses made from within the +C library are not intercepted, and file system accesses made @i{via} +direct syscalls are not intercepted either, leading to erratic behavior. @end table @vindex GUIX_EXECUTION_ENGINE diff --git a/gnu/packages/aux-files/pack-audit.c b/gnu/packages/aux-files/pack-audit.c new file mode 100644 index 0000000000..374787e8b9 --- /dev/null +++ b/gnu/packages/aux-files/pack-audit.c @@ -0,0 +1,85 @@ +/* GNU Guix --- Functional package management for GNU + Copyright (C) 2020 Ludovic Courtès + + This file is part of GNU Guix. + + GNU Guix is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or (at + your option) any later version. + + GNU Guix is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with GNU Guix. If not, see . */ + +/* This file implements part of the GNU ld.so audit interface. It is used by + the "fakechroot" engine of the 'guix pack -RR' wrappers to make sure the + loader looks for shared objects under the "fake" root directory. */ + +#define _GNU_SOURCE 1 + +#include + +#include +#include +#include +#include + +/* The pseudo root directory and store that we are relocating to. */ +static const char *root_directory; +static char *store; + +/* The original store, "/gnu/store" by default. */ +static const char original_store[] = "@STORE_DIRECTORY@"; + +/* Like 'malloc', but abort if 'malloc' returns NULL. */ +static void * +xmalloc (size_t size) +{ + void *result = malloc (size); + assert (result != NULL); + return result; +} + +unsigned int +la_version (unsigned int v) +{ + if (v != LAV_CURRENT) + error (1, 0, "cannot handle interface version %u", v); + + root_directory = getenv ("FAKECHROOT_BASE"); + if (root_directory == NULL) + error (1, 0, "'FAKECHROOT_BASE' is not set"); + + store = xmalloc (strlen (root_directory) + sizeof original_store); + strcpy (store, root_directory); + strcat (store, original_store); + + return v; +} + +/* Return NAME, a shared object file name, relocated under STORE. This + function is called by the loader whenever it looks for a shared object. */ +char * +la_objsearch (const char *name, uintptr_t *cookie, unsigned int flag) +{ + char *result; + + if (strncmp (name, original_store, + sizeof original_store - 1) == 0) + { + size_t len = strlen (name) - sizeof original_store + + strlen (store) + 1; + result = xmalloc (len); + strcpy (result, store); + strcat (result, name + sizeof original_store - 1); + } + else + result = strdup (name); + + return result; +} diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c index 6e97359078..5a6b932b87 100644 --- a/gnu/packages/aux-files/run-in-namespace.c +++ b/gnu/packages/aux-files/run-in-namespace.c @@ -42,6 +42,11 @@ #include #include +/* Whether we're building the ld.so/libfakechroot wrapper. */ +#define HAVE_EXEC_WITH_LOADER \ + (defined PROGRAM_INTERPRETER) && (defined LOADER_AUDIT_MODULE) \ + && (defined FAKECHROOT_LIBRARY) + /* The original store, "/gnu/store" by default. */ static const char original_store[] = "@STORE_DIRECTORY@"; @@ -117,9 +122,42 @@ rm_rf (const char *directory) assert_perror (errno); } -/* Bind mount all the top-level entries in SOURCE to TARGET. */ +/* Make TARGET a bind-mount of SOURCE. Take into account ENTRY's type, which + corresponds to SOURCE. */ +static int +bind_mount (const char *source, const struct dirent *entry, + const char *target) +{ + if (entry->d_type == DT_DIR) + { + int err = mkdir (target, 0700); + if (err != 0) + return err; + } + else + close (open (target, O_WRONLY | O_CREAT)); + + return mount (source, target, "none", + MS_BIND | MS_REC | MS_RDONLY, NULL); +} + +#if HAVE_EXEC_WITH_LOADER + +/* Make TARGET a symlink to SOURCE. */ +static int +make_symlink (const char *source, const struct dirent *entry, + const char *target) +{ + return symlink (source, target); +} + +#endif + +/* Mirror with FIRMLINK all the top-level entries in SOURCE to TARGET. */ static void -bind_mount (const char *source, const char *target) +mirror_directory (const char *source, const char *target, + int (* firmlink) (const char *, const struct dirent *, + const char *)) { DIR *stream = opendir (source); @@ -154,17 +192,7 @@ bind_mount (const char *source, const char *target) else { /* Create the mount point. */ - if (entry->d_type == DT_DIR) - { - int err = mkdir (new_entry, 0700); - if (err != 0) - assert_perror (errno); - } - else - close (open (new_entry, O_WRONLY | O_CREAT)); - - int err = mount (abs_source, new_entry, "none", - MS_BIND | MS_REC | MS_RDONLY, NULL); + int err = firmlink (abs_source, entry, new_entry); /* It used to be that only directories could be bind-mounted. Thus, keep going if we fail to bind-mount a non-directory entry. @@ -248,7 +276,7 @@ exec_in_user_namespace (const char *store, int argc, char *argv[]) /* Note: Due to we cannot make NEW_ROOT a tmpfs (which would have saved the need for 'rm_rf'.) */ - bind_mount ("/", new_root); + mirror_directory ("/", new_root, bind_mount); mkdir_p (new_store); err = mount (store, new_store, "none", MS_BIND | MS_REC | MS_RDONLY, NULL); @@ -340,6 +368,92 @@ exec_with_proot (const char *store, int argc, char *argv[]) #endif + +#if HAVE_EXEC_WITH_LOADER + +/* Execute the wrapped program by invoking the loader (ld.so) directly, + passing it the audit module and preloading libfakechroot.so. */ +static void +exec_with_loader (const char *store, int argc, char *argv[]) +{ + char *loader = concat (store, + PROGRAM_INTERPRETER + sizeof original_store); + size_t loader_specific_argc = 6; + size_t loader_argc = argc + loader_specific_argc; + char *loader_argv[loader_argc + 1]; + loader_argv[0] = argv[0]; + loader_argv[1] = "--audit"; + loader_argv[2] = concat (store, + LOADER_AUDIT_MODULE + sizeof original_store); + loader_argv[3] = "--preload"; + loader_argv[4] = concat (store, + FAKECHROOT_LIBRARY + sizeof original_store); + loader_argv[5] = concat (store, + "@WRAPPED_PROGRAM@" + sizeof original_store); + + for (size_t i = 0; i < argc; i++) + loader_argv[i + loader_specific_argc] = argv[i + 1]; + + loader_argv[loader_argc] = NULL; + + /* Set up the root directory. */ + int err; + char *new_root = mkdtemp (strdup ("/tmp/guix-exec-XXXXXX")); + mirror_directory ("/", new_root, make_symlink); + + char *new_store = concat (new_root, original_store); + char *new_store_parent = dirname (strdup (new_store)); + mkdir_p (new_store_parent); + symlink (store, new_store); + +#ifdef GCONV_DIRECTORY + /* Tell libc where to find its gconv modules. This is necessary because + gconv uses non-interposable 'open' calls. */ + char *gconv_path = concat (store, + GCONV_DIRECTORY + sizeof original_store); + setenv ("GCONV_PATH", gconv_path, 1); + free (gconv_path); +#endif + + setenv ("FAKECHROOT_BASE", new_root, 1); + + pid_t child = fork (); + switch (child) + { + case 0: + err = execv (loader, loader_argv); + if (err < 0) + assert_perror (errno); + exit (EXIT_FAILURE); + break; + + case -1: + assert_perror (errno); + exit (EXIT_FAILURE); + break; + + default: + { + int status; + waitpid (child, &status, 0); + chdir ("/"); /* avoid EBUSY */ + rm_rf (new_root); + free (new_root); + + close (2); /* flushing stderr should be silent */ + + if (WIFEXITED (status)) + exit (WEXITSTATUS (status)); + else + /* Abnormal termination cannot really be reproduced, so exit + with 255. */ + exit (255); + } + } +} + +#endif + /* Execution engines. */ @@ -356,7 +470,7 @@ buffer_stderr (void) setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer); } -/* The default engine. */ +/* The default engine: choose a robust method. */ static void exec_default (const char *store, int argc, char *argv[]) { @@ -370,13 +484,29 @@ exec_default (const char *store, int argc, char *argv[]) #endif } +/* The "performance" engine: choose performance over robustness. */ +static void +exec_performance (const char *store, int argc, char *argv[]) +{ + buffer_stderr (); + + exec_in_user_namespace (store, argc, argv); +#if HAVE_EXEC_WITH_LOADER + exec_with_loader (store, argc, argv); +#endif +} + /* List of supported engines. */ static const struct engine engines[] = { { "default", exec_default }, + { "performance", exec_performance }, { "userns", exec_in_user_namespace }, #ifdef PROOT_PROGRAM { "proot", exec_with_proot }, +#endif +#if HAVE_EXEC_WITH_LOADER + { "fakechroot", exec_with_loader }, #endif { NULL, NULL } }; diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 580f696b41..23aab01701 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -681,18 +681,50 @@ last resort for relocation." (define runner (local-file (search-auxiliary-file "run-in-namespace.c"))) + (define audit-source + (local-file (search-auxiliary-file "pack-audit.c"))) + (define (proot) (specification->package "proot-static")) + (define (fakechroot-library) + (computed-file "libfakechroot.so" + #~(copy-file #$(file-append + (specification->package "fakechroot") + "/lib/fakechroot/libfakechroot.so") + #$output))) + + (define (audit-module) + ;; Return an ld.so audit module for use by the 'fakechroot' execution + ;; engine that translates file names of all the files ld.so loads. + (computed-file "pack-audit.so" + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils)) + + (copy-file #$audit-source "audit.c") + (substitute* "audit.c" + (("@STORE_DIRECTORY@") + (%store-directory))) + + (invoke #$compiler "-std=gnu99" + "-shared" "-fPIC" "-Os" "-g0" + "-Wall" "audit.c" "-o" #$output))))) + (define build (with-imported-modules (source-module-closure '((guix build utils) - (guix build union))) + (guix build union) + (guix elf))) #~(begin (use-modules (guix build utils) ((guix build union) #:select (relative-file-name)) + (guix elf) + (ice-9 binary-ports) (ice-9 ftw) - (ice-9 match)) + (ice-9 match) + (srfi srfi-1) + (rnrs bytevectors)) (define input ;; The OUTPUT* output of PACKAGE. @@ -711,6 +743,48 @@ last resort for relocation." (#f base) (index (string-drop base index))))) + (define (elf-interpreter elf) + ;; Return the interpreter of ELF as a string, or #f if ELF has no + ;; interpreter segment. + (match (find (lambda (segment) + (= (elf-segment-type segment) PT_INTERP)) + (elf-segments elf)) + (#f #f) ;maybe a .so + (segment + (let ((bv (make-bytevector (- (elf-segment-memsz segment) 1)))) + (bytevector-copy! (elf-bytes elf) + (elf-segment-offset segment) + bv 0 (bytevector-length bv)) + (utf8->string bv))))) + + (define (elf-loader-compile-flags program) + ;; Return the cpp flags defining macros for the ld.so/fakechroot + ;; wrapper of PROGRAM. + + ;; TODO: Handle scripts by wrapping their interpreter. + (if (elf-file? program) + (let* ((bv (call-with-input-file program + get-bytevector-all)) + (elf (parse-elf bv)) + (interp (elf-interpreter elf)) + (gconv (and interp + (string-append (dirname interp) + "/gconv")))) + (if interp + (list (string-append "-DPROGRAM_INTERPRETER=\"" + interp "\"") + (string-append "-DFAKECHROOT_LIBRARY=\"" + #$(fakechroot-library) "\"") + + (string-append "-DLOADER_AUDIT_MODULE=\"" + #$(audit-module) "\"") + (if gconv + (string-append "-DGCONV_DIRECTORY=\"" + gconv "\"") + "-UGCONV_DIRECTORY")) + '())) + '())) + (define (build-wrapper program) ;; Build a user-namespace wrapper for PROGRAM. (format #t "building wrapper for '~a'...~%" program) @@ -730,10 +804,11 @@ last resort for relocation." (mkdir-p (dirname result)) (apply invoke #$compiler "-std=gnu99" "-static" "-Os" "-g0" "-Wall" "run.c" "-o" result - (if proot - (list (string-append "-DPROOT_PROGRAM=\"" - proot "\"")) - '())) + (append (if proot + (list (string-append "-DPROOT_PROGRAM=\"" + proot "\"")) + '()) + (elf-loader-compile-flags program))) (delete-file "run.c"))) (setvbuf (current-output-port) 'line) diff --git a/tests/guix-pack-relocatable.sh b/tests/guix-pack-relocatable.sh index cb56815fed..358cac5b26 100644 --- a/tests/guix-pack-relocatable.sh +++ b/tests/guix-pack-relocatable.sh @@ -94,6 +94,12 @@ case "`uname -m`" in export GUIX_EXECUTION_ENGINE "$test_directory/Bin/sed" --version > "$test_directory/output" grep 'GNU sed' "$test_directory/output" + + # Now with fakechroot. + GUIX_EXECUTION_ENGINE="fakechroot" + "$test_directory/Bin/sed" --version > "$test_directory/output" + grep 'GNU sed' "$test_directory/output" + chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/* ;; *) -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Thu May 14 11:24:13 2020 Received: (at 41189-done) by debbugs.gnu.org; 14 May 2020 15:24:13 +0000 Received: from localhost ([127.0.0.1]:34495 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jZFiX-00051l-8t for submit@debbugs.gnu.org; Thu, 14 May 2020 11:24:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:43620) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jZFiV-00051U-Ky for 41189-done@debbugs.gnu.org; Thu, 14 May 2020 11:24:12 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35408) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZFiQ-0001S1-E5 for 41189-done@debbugs.gnu.org; Thu, 14 May 2020 11:24:06 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=34884 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jZFiP-00047K-VX for 41189-done@debbugs.gnu.org; Thu, 14 May 2020 11:24:06 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 41189-done@debbugs.gnu.org Subject: Re: [bug#41189] [PATCH v2 0/4] Add Fakechroot engine for 'guix pack -RR' References: <87tv0l9njp.fsf@inria.fr> <20200513125215.27740-1-ludo@gnu.org> Date: Thu, 14 May 2020 17:24:04 +0200 In-Reply-To: <20200513125215.27740-1-ludo@gnu.org> ("Ludovic \=\?utf-8\?Q\?Cou\?\= \=\?utf-8\?Q\?rt\=C3\=A8s\=22's\?\= message of "Wed, 13 May 2020 14:52:11 +0200") Message-ID: <87wo5etu97.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41189-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable. > pack: Factorize store references in wrapper. > gnu: Add fakechroot. > pack: Add relocation via ld.so and fakechroot. Pushed as 6456232164890dbf5aa20394ee24637feb4b7b9e! Ludo=E2=80=99. From unknown Mon Jun 23 13:12:13 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 12 Jun 2020 11:24:09 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator