From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 06:37:01 2019 Received: (at submit) by debbugs.gnu.org; 12 May 2019 10:37:01 +0000 Received: from localhost ([127.0.0.1]:41917 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPlqn-0002nz-3A for submit@debbugs.gnu.org; Sun, 12 May 2019 06:37:01 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38243) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPlql-0002nl-H4 for submit@debbugs.gnu.org; Sun, 12 May 2019 06:36:59 -0400 Received: from lists.gnu.org ([209.51.188.17]:47898) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPlqg-0006c1-8b for submit@debbugs.gnu.org; Sun, 12 May 2019 06:36:54 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52533) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPlqf-0006FK-5U for guix-patches@gnu.org; Sun, 12 May 2019 06:36:54 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED,BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54289) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPll4-0003YW-T3; Sun, 12 May 2019 06:31:06 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41580 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPll1-0004T3-3v; Sun, 12 May 2019 06:31:04 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH 0/8] Make 'guix system docker-image' readily usable Date: Sun, 12 May 2019 12:30:55 +0200 Message-Id: <20190512103055.16832-1-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Guix, On current master, ‘guix system docker-image’ produces an image without an entry point, so one has to carefully follow the “GUIX_NEW_SYSTEM hack” described in the manual. Furthermore, due to other issues, the resulting image doesn’t properly boot because it tries to mount file systems that it cannot mount, such as /dev/shm and /dev/pts. These patches fix both issues, such that one can just do ‘docker create’ and ‘docker start’ to get Guix System up and running in the container. I think that’s a nice improvement. :-) We discussed this Friday on IRC and people said that it’s quite unusual to provide a “full OS” (with PID 1) as a Docker image; instead, people would rather do one image per (micro)service. But anyway, that’s the purpose of ‘guix system docker-image’, and I can imagine it has use cases too. For example, it’s a simple way to get Guix set up in a container, for people who want to perform Guix builds in a container. Thoughts? Ludo’. Ludovic Courtès (8): system: Export 'operating-system-default-essential-services'. linux-container: Improve filtering of unnecessary file systems. services: 'gc-root-service-type' now has a default value. linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image OSes. linux-container: Compute essential services for THIS-OPERATING-SYSTEM. system: Add 'operating-system-with-gc-roots'. docker: 'build-docker-image' accepts an optional #:entry-point. vm: 'system-docker-image' provides an entry point. doc/guix.texi | 18 +++-- gnu/services.scm | 5 +- gnu/system.scm | 18 ++++- gnu/system/linux-container.scm | 30 ++++++--- gnu/system/vm.scm | 18 ++++- gnu/tests/docker.scm | 118 ++++++++++++++++++++++++++++++++- gnu/tests/install.scm | 11 --- guix/docker.scm | 15 +++-- 8 files changed, 195 insertions(+), 38 deletions(-) -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 06:38:20 2019 Received: (at 35697) by debbugs.gnu.org; 12 May 2019 10:38:20 +0000 Received: from localhost ([127.0.0.1]:41924 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls4-0002qy-Ft for submit@debbugs.gnu.org; Sun, 12 May 2019 06:38:20 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38668) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls3-0002qa-6U for 35697@debbugs.gnu.org; Sun, 12 May 2019 06:38:19 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54349) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPlry-0007Hv-1W; Sun, 12 May 2019 06:38:14 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41590 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPlrx-00052w-KH; Sun, 12 May 2019 06:38:13 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 35697@debbugs.gnu.org Subject: [PATCH 1/8] system: Export 'operating-system-default-essential-services'. Date: Sun, 12 May 2019 12:37:55 +0200 Message-Id: <20190512103802.17032-1-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 35697 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/system.scm (essential-services): Rename to... (operating-system-default-essential-services): ... this. ()[essential-services]: Adjust accordingly. --- gnu/system.scm | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/gnu/system.scm b/gnu/system.scm index 0489b9720d..2c4ca55ffc 100644 --- a/gnu/system.scm +++ b/gnu/system.scm @@ -72,6 +72,7 @@ operating-system-bootloader operating-system-services operating-system-essential-services + operating-system-default-essential-services operating-system-user-services operating-system-packages operating-system-host-name @@ -213,7 +214,8 @@ (essential-services operating-system-essential-services ; list of services (thunked) - (default (essential-services this-operating-system))) + (default (operating-system-default-essential-services + this-operating-system))) (services operating-system-user-services ; list of services (default %base-services)) @@ -463,7 +465,7 @@ value of the SYSTEM-SERVICE-TYPE service." ("initrd" ,initrd) ("locale" ,locale)))))) ;used by libc -(define* (essential-services os) +(define (operating-system-default-essential-services os) "Return the list of essential services for OS. These are special services that implement part of what's declared in OS are responsible for low-level bookkeeping." -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 06:38:22 2019 Received: (at 35697) by debbugs.gnu.org; 12 May 2019 10:38:22 +0000 Received: from localhost ([127.0.0.1]:41927 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls5-0002rD-QI for submit@debbugs.gnu.org; Sun, 12 May 2019 06:38:22 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38670) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls4-0002qc-6P for 35697@debbugs.gnu.org; Sun, 12 May 2019 06:38:20 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54350) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPlrz-0007IA-1l; Sun, 12 May 2019 06:38:15 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41590 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPlry-00052w-Hw; Sun, 12 May 2019 06:38:14 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 35697@debbugs.gnu.org Subject: [PATCH 2/8] linux-container: Improve filtering of unnecessary file systems. Date: Sun, 12 May 2019 12:37:56 +0200 Message-Id: <20190512103802.17032-2-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190512103802.17032-1-ludo@gnu.org> References: <20190512103802.17032-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 35697 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/system/linux-container.scm (containerized-operating-system)[user-file-systems]: Add trailing slash for the "/dev/" and "/sys/" prefixes. --- gnu/system/linux-container.scm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scm index 149c3d08a3..ded5f279fe 100644 --- a/gnu/system/linux-container.scm +++ b/gnu/system/linux-container.scm @@ -65,8 +65,8 @@ containerized OS." (string=? target "/") (and (string? source) (string-prefix? "/dev/" source)) - (string-prefix? "/dev" target) - (string-prefix? "/sys" target)))) + (string-prefix? "/dev/" target) + (string-prefix? "/sys/" target)))) (operating-system-file-systems os))) (define (mapping->fs fs) -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 06:38:22 2019 Received: (at 35697) by debbugs.gnu.org; 12 May 2019 10:38:22 +0000 Received: from localhost ([127.0.0.1]:41929 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls6-0002rF-41 for submit@debbugs.gnu.org; Sun, 12 May 2019 06:38:22 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38673) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls5-0002qe-6O for 35697@debbugs.gnu.org; Sun, 12 May 2019 06:38:21 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54351) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPls0-0007Ie-1f; Sun, 12 May 2019 06:38:16 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41590 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPlrz-00052w-IE; Sun, 12 May 2019 06:38:15 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 35697@debbugs.gnu.org Subject: [PATCH 3/8] services: 'gc-root-service-type' now has a default value. Date: Sun, 12 May 2019 12:37:57 +0200 Message-Id: <20190512103802.17032-3-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190512103802.17032-1-ludo@gnu.org> References: <20190512103802.17032-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 35697 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/services.scm (gc-root-service-type)[default-value]: New field. --- gnu/services.scm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/gnu/services.scm b/gnu/services.scm index f151bbaa9d..7de78105ff 100644 --- a/gnu/services.scm +++ b/gnu/services.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2015, 2016, 2017, 2018 Ludovic Courtès +;;; Copyright © 2015, 2016, 2017, 2018, 2019 Ludovic Courtès ;;; Copyright © 2016 Chris Marusich ;;; ;;; This file is part of GNU Guix. @@ -649,7 +649,8 @@ as Wifi cards."))) (extend append) (description "Register garbage-collector roots---i.e., store items that -will not be reclaimed by the garbage collector."))) +will not be reclaimed by the garbage collector.") + (default-value '()))) ;;; -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 06:38:24 2019 Received: (at 35697) by debbugs.gnu.org; 12 May 2019 10:38:24 +0000 Received: from localhost ([127.0.0.1]:41934 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls8-0002re-D5 for submit@debbugs.gnu.org; Sun, 12 May 2019 06:38:24 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38676) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls6-0002qh-6T for 35697@debbugs.gnu.org; Sun, 12 May 2019 06:38:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54352) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPls1-0007Iy-1Y; Sun, 12 May 2019 06:38:17 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41590 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPls0-00052w-IA; Sun, 12 May 2019 06:38:16 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 35697@debbugs.gnu.org Subject: [PATCH 4/8] linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image OSes. Date: Sun, 12 May 2019 12:37:58 +0200 Message-Id: <20190512103802.17032-4-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190512103802.17032-1-ludo@gnu.org> References: <20190512103802.17032-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 35697 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Previously, 'guix system docker-image' would end up providing an OS that would try to mount all of %CONTAINER-FILE-SYSTEMS as well as /gnu/store, which is bound to fail in unprivileged Docker. This patch makes it so that 'guix system container' still gets those file systems, but 'guix system docker-image' doesn't. * gnu/system/linux-container.scm (containerized-operating-system): Add #:extra-file-systems parameter and honor it. Do not import %STORE-MAPPING. (container-script): Add %STORE-MAPPING to MAPPINGS and pass #:extra-file-systems. --- gnu/system/linux-container.scm | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scm index ded5f279fe..5adec064f7 100644 --- a/gnu/system/linux-container.scm +++ b/gnu/system/linux-container.scm @@ -53,10 +53,12 @@ from OS that are needed on the bare metal and not in a container." (return `(("locale" ,locale)))))) base)) -(define (containerized-operating-system os mappings) +(define* (containerized-operating-system os mappings + #:key + (extra-file-systems '())) "Return an operating system based on OS for use in a Linux container environment. MAPPINGS is a list of to realize in the -containerized OS." +containerized OS. EXTRA-FILE-SYSTEMS is a list of file systems to add to OS." (define user-file-systems (remove (lambda (fs) (let ((target (file-system-mount-point fs)) @@ -88,15 +90,17 @@ containerized OS." (memq (service-kind service) useless-services)) (operating-system-user-services os))) - (file-systems (append (map mapping->fs (cons %store-mapping mappings)) - %container-file-systems + (file-systems (append (map mapping->fs mappings) + extra-file-systems user-file-systems)))) (define* (container-script os #:key (mappings '())) "Return a derivation of a script that runs OS as a Linux container. MAPPINGS is a list of objects that specify the files/directories that will be shared with the host system." - (let* ((os (containerized-operating-system os mappings)) + (let* ((os (containerized-operating-system + os (cons %store-mapping mappings) + #:extra-file-systems %container-file-systems)) (file-systems (filter file-system-needed-for-boot? (operating-system-file-systems os))) (specs (map file-system->spec file-systems))) -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 06:38:25 2019 Received: (at 35697) by debbugs.gnu.org; 12 May 2019 10:38:25 +0000 Received: from localhost ([127.0.0.1]:41936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls8-0002rl-Nu for submit@debbugs.gnu.org; Sun, 12 May 2019 06:38:24 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38679) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls7-0002qj-9a for 35697@debbugs.gnu.org; Sun, 12 May 2019 06:38:23 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54353) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPls2-0007JB-1i; Sun, 12 May 2019 06:38:18 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41590 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPls1-00052w-I1; Sun, 12 May 2019 06:38:17 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 35697@debbugs.gnu.org Subject: [PATCH 5/8] linux-container: Compute essential services for THIS-OPERATING-SYSTEM. Date: Sun, 12 May 2019 12:37:59 +0200 Message-Id: <20190512103802.17032-5-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190512103802.17032-1-ludo@gnu.org> References: <20190512103802.17032-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 35697 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Previously, the 'essential-services' would correspond to the initial, non-containerized OS. Thus, all the file systems removed in 'container-essential-services' would actually still be there because the essential services would be computed on the non-containerized OS. This is a followup to 69cae3d3356a69b7fe69481338f760545995485e. * gnu/system/linux-container.scm (container-essential-services): Call 'operating-system-default-essential-services' to get the baseline services. (containerized-operating-system): Pass THIS-OPERATING-SYSTEM, not OS, to 'container-essential-services'. Add a dummy root file system to 'file-systems'. --- gnu/system/linux-container.scm | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scm index 5adec064f7..fc2e05a5bc 100644 --- a/gnu/system/linux-container.scm +++ b/gnu/system/linux-container.scm @@ -45,7 +45,7 @@ from OS that are needed on the bare metal and not in a container." (list (service-kind %linux-bare-metal-service) firmware-service-type system-service-type))) - (operating-system-essential-services os))) + (operating-system-default-essential-services os))) (cons (service system-service-type (let ((locale (operating-system-locale-directory os))) @@ -85,14 +85,20 @@ containerized OS. EXTRA-FILE-SYSTEMS is a list of file systems to add to OS." (operating-system (inherit os) (swap-devices '()) ; disable swap - (essential-services (container-essential-services os)) + (essential-services (container-essential-services this-operating-system)) (services (remove (lambda (service) (memq (service-kind service) useless-services)) (operating-system-user-services os))) (file-systems (append (map mapping->fs mappings) extra-file-systems - user-file-systems)))) + user-file-systems + + ;; Provide a dummy root file system. + (list (file-system + (mount-point "/") + (device "none") + (type "none"))))))) (define* (container-script os #:key (mappings '())) "Return a derivation of a script that runs OS as a Linux container. -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 06:38:28 2019 Received: (at 35697) by debbugs.gnu.org; 12 May 2019 10:38:28 +0000 Received: from localhost ([127.0.0.1]:41940 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPlsC-0002sA-1N for submit@debbugs.gnu.org; Sun, 12 May 2019 06:38:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38682) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls8-0002ql-64 for 35697@debbugs.gnu.org; Sun, 12 May 2019 06:38:24 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54354) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPls3-0007JU-1G; Sun, 12 May 2019 06:38:19 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41590 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPls2-00052w-IB; Sun, 12 May 2019 06:38:18 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 35697@debbugs.gnu.org Subject: [PATCH 6/8] system: Add 'operating-system-with-gc-roots'. Date: Sun, 12 May 2019 12:38:00 +0200 Message-Id: <20190512103802.17032-6-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190512103802.17032-1-ludo@gnu.org> References: <20190512103802.17032-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 35697 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/tests/install.scm (operating-system-with-gc-roots): Move to... * gnu/system.scm (operating-system-with-gc-roots): ... here. New procedure. --- gnu/system.scm | 12 ++++++++++++ gnu/tests/install.scm | 11 ----------- 2 files changed, 12 insertions(+), 11 deletions(-) diff --git a/gnu/system.scm b/gnu/system.scm index 2c4ca55ffc..01be1243fe 100644 --- a/gnu/system.scm +++ b/gnu/system.scm @@ -109,6 +109,7 @@ operating-system-boot-script system-linux-image-file-name + operating-system-with-gc-roots boot-parameters boot-parameters? @@ -519,6 +520,17 @@ bookkeeping." (append (operating-system-user-services os) (operating-system-essential-services os)))) +(define (operating-system-with-gc-roots os roots) + "Return a variant of OS where ROOTS are registered as GC roots." + (operating-system + (inherit os) + + ;; We use this procedure for the installation OS, which already defines GC + ;; roots. Add ROOTS to those. + (services (cons (simple-service 'extra-root + gc-root-service-type roots) + (operating-system-user-services os))))) + ;;; ;;; /etc. diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm index 430a102378..7b5ee18505 100644 --- a/gnu/tests/install.scm +++ b/gnu/tests/install.scm @@ -123,17 +123,6 @@ (inherit config) (guix (current-guix)))))))) -(define (operating-system-with-gc-roots os roots) - "Return a variant of OS where ROOTS are registered as GC roots." - (operating-system - (inherit os) - - ;; We use this procedure for the installation OS, which already defines GC - ;; roots. Add ROOTS to those. - (services (cons (simple-service 'extra-root - gc-root-service-type roots) - (operating-system-user-services os))))) - (define MiB (expt 2 20)) -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 06:38:28 2019 Received: (at 35697) by debbugs.gnu.org; 12 May 2019 10:38:28 +0000 Received: from localhost ([127.0.0.1]:41942 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPlsC-0002sC-9c for submit@debbugs.gnu.org; Sun, 12 May 2019 06:38:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38686) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPls9-0002qs-5v for 35697@debbugs.gnu.org; Sun, 12 May 2019 06:38:25 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54355) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPls4-0007Ji-11; Sun, 12 May 2019 06:38:20 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41590 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPls3-00052w-Hj; Sun, 12 May 2019 06:38:19 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 35697@debbugs.gnu.org Subject: [PATCH 7/8] docker: 'build-docker-image' accepts an optional #:entry-point. Date: Sun, 12 May 2019 12:38:01 +0200 Message-Id: <20190512103802.17032-7-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190512103802.17032-1-ludo@gnu.org> References: <20190512103802.17032-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 35697 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/docker.scm (config): Add #:entry-point and honor it. (build-docker-image): Likewise. --- guix/docker.scm | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/guix/docker.scm b/guix/docker.scm index c6e9c6fee5..7fe83d9797 100644 --- a/guix/docker.scm +++ b/guix/docker.scm @@ -1,6 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2017 Ricardo Wurmus -;;; Copyright © 2017, 2018 Ludovic Courtès +;;; Copyright © 2017, 2018, 2019 Ludovic Courtès ;;; Copyright © 2018 Chris Marusich ;;; ;;; This file is part of GNU Guix. @@ -73,7 +73,7 @@ `((,(generate-tag path) . ((latest . ,id))))) ;; See https://github.com/opencontainers/image-spec/blob/master/config.md -(define (config layer time arch) +(define* (config layer time arch #:key entry-point) "Generate a minimal image configuration for the given LAYER file." ;; "architecture" must be values matching "platform.arch" in the ;; runtime-spec at @@ -81,7 +81,9 @@ `((architecture . ,arch) (comment . "Generated by GNU Guix") (created . ,time) - (config . #nil) + (config . ,(if entry-point + `((entrypoint . ,entry-point)) + #nil)) (container_config . #nil) (os . "linux") (rootfs . ((type . "layers") @@ -110,6 +112,7 @@ return \"a\"." (transformations '()) (system (utsname:machine (uname))) database + entry-point compressor (creation-time (current-time time-utc))) "Write to IMAGE a Docker image archive containing the given PATHS. PREFIX @@ -118,6 +121,9 @@ must be a store path that is a prefix of any store paths in PATHS. When DATABASE is true, copy it to /var/guix/db in the image and create /var/guix/gcroots and friends. +When ENTRY-POINT is true, it must be a list of strings; it is stored as the +entry point in the Docker image JSON structure. + SYMLINKS must be a list of (SOURCE -> TARGET) tuples describing symlinks to be created in the image, where each TARGET is relative to PREFIX. TRANSFORMATIONS must be a list of (OLD -> NEW) tuples describing how to @@ -227,7 +233,8 @@ SRFI-19 time-utc object, as the creation time in metadata." (with-output-to-file "config.json" (lambda () (scm->json (config (string-append id "/layer.tar") - time arch)))) + time arch + #:entry-point entry-point)))) (with-output-to-file "manifest.json" (lambda () (scm->json (manifest prefix id)))) -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 06:38:29 2019 Received: (at 35697) by debbugs.gnu.org; 12 May 2019 10:38:29 +0000 Received: from localhost ([127.0.0.1]:41944 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPlsC-0002sK-LN for submit@debbugs.gnu.org; Sun, 12 May 2019 06:38:29 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38687) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPlsA-0002r6-Aj for 35697@debbugs.gnu.org; Sun, 12 May 2019 06:38:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54356) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPls5-0007K6-3m; Sun, 12 May 2019 06:38:21 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41590 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPls4-00052w-HM; Sun, 12 May 2019 06:38:20 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 35697@debbugs.gnu.org Subject: [PATCH 8/8] vm: 'system-docker-image' provides an entry point. Date: Sun, 12 May 2019 12:38:02 +0200 Message-Id: <20190512103802.17032-8-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190512103802.17032-1-ludo@gnu.org> References: <20190512103802.17032-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 35697 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) This simplifies use of images created with 'guix system docker-image'. * gnu/system/vm.scm (system-docker-image)[boot-program]: New variable. [os]: Add it to the GC roots. [build]: Pass #:entry-point to 'build-docker-image'. * gnu/tests/docker.scm (run-docker-system-test): New procedure. (%test-docker-system): New variable. * doc/guix.texi (Invoking guix system): Remove GUIX_NEW_SYSTEM hack and '--entrypoint' from the example. Mention 'docker create', 'docker start', and 'docker exec'. --- doc/guix.texi | 18 ++++--- gnu/system/vm.scm | 18 ++++++- gnu/tests/docker.scm | 118 ++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 145 insertions(+), 9 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index df7208229c..da65fd8a4e 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -24497,20 +24497,26 @@ system configuration file. You can then load the image and launch a Docker container using commands like the following: @example -image_id="$(docker load < guix-system-docker-image.tar.gz)" -docker run -e GUIX_NEW_SYSTEM=/var/guix/profiles/system \\ - --entrypoint /var/guix/profiles/system/profile/bin/guile \\ - $image_id /var/guix/profiles/system/boot +image_id="`docker load < guix-system-docker-image.tar.gz`" +container_id="`docker create $image_id`" +docker start $container_id @end example This command starts a new Docker container from the specified image. It will boot the Guix system in the usual manner, which means it will start any services you have defined in the operating system -configuration. Depending on what you run in the Docker container, it +configuration. You can get an interactive shell running in the container +using @command{docker exec}: + +@example +docker exec -ti $container_id /run/current-system/profile/bin/bash --login +@end example + +Depending on what you run in the Docker container, it may be necessary to give the container additional permissions. For example, if you intend to build software using Guix inside of the Docker container, you may need to pass the @option{--privileged} option to -@code{docker run}. +@code{docker create}. @item container Return a script to run the operating system declared in @var{file} diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm index 124abd0fc9..f3027cd4ca 100644 --- a/gnu/system/vm.scm +++ b/gnu/system/vm.scm @@ -473,7 +473,7 @@ the image." (define* (system-docker-image os #:key - (name "guixsd-docker-image") + (name "guix-docker-image") register-closures?) "Build a docker image. OS is the desired . NAME is the base name to use for the output file. When REGISTER-CLOSURES? is not #f, @@ -487,7 +487,19 @@ should set REGISTER-CLOSURES? to #f." (local-file (search-path %load-path "guix/store/schema.sql")))) - (let ((os (containerized-operating-system os '())) + (define boot-program + ;; Program that runs the boot script of OS, which in turn starts shepherd. + (program-file "boot-program" + #~(let ((system (cadr (command-line)))) + (setenv "GUIX_NEW_SYSTEM" system) + (execl #$(file-append guile-2.2 "/bin/guile") + "guile" "--no-auto-compile" + (string-append system "/boot"))))) + + + (let ((os (operating-system-with-gc-roots + (containerized-operating-system os '()) + (list boot-program))) (name (string-append name ".tar.gz")) (graph "system-graph")) (define build @@ -538,9 +550,11 @@ should set REGISTER-CLOSURES? to #f." (string-append "/xchg/" #$graph) read-reference-graph))) #$os + #:entry-point '(#$boot-program #$os) #:compressor '(#+(file-append gzip "/bin/gzip") "-9n") #:creation-time (make-time time-utc 0 1) #:transformations `((,root-directory -> "")))))))) + (expression->derivation-in-linux-vm name build #:make-disk-image? #f diff --git a/gnu/tests/docker.scm b/gnu/tests/docker.scm index 25e172efae..3cd3a27884 100644 --- a/gnu/tests/docker.scm +++ b/gnu/tests/docker.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2019 Danny Milosavljevic +;;; Copyright © 2019 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -28,6 +29,7 @@ #:use-module (gnu services desktop) #:use-module (gnu packages bootstrap) ; %bootstrap-guile #:use-module (gnu packages docker) + #:use-module (gnu packages guile) #:use-module (guix gexp) #:use-module (guix grafts) #:use-module (guix monads) @@ -38,7 +40,8 @@ #:use-module (guix tests) #:use-module (guix build-system trivial) #:use-module ((guix licenses) #:prefix license:) - #:export (%test-docker)) + #:export (%test-docker + %test-docker-system)) (define %docker-os (simple-operating-system @@ -166,3 +169,116 @@ standard output device and then enters a new line.") (name "docker") (description "Test Docker container of Guix.") (value (build-tarball&run-docker-test)))) + + +(define (run-docker-system-test tarball) + "Load DOCKER-TARBALL as Docker image and run it in a Docker container, +inside %DOCKER-OS." + (define os + (marionette-operating-system + %docker-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define vm + (virtual-machine + (operating-system os) + ;; FIXME: Because we're using the volatile-root setup where the root file + ;; system is a tmpfs overlaid over a small root file system, 'docker + ;; load' must be able to store the whole image into memory, hence the + ;; huge memory requirements. We should avoid the volatile-root setup + ;; instead. + (memory-size 3000) + (port-forwardings '()))) + + (define test + (with-imported-modules '((gnu build marionette) + (guix build utils)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette) + (guix build utils)) + + (define marionette + (make-marionette (list #$vm))) + + (mkdir #$output) + (chdir #$output) + + (test-begin "docker") + + (test-assert "service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'dockerd) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-assert "load system image and run it" + (marionette-eval + `(begin + (define (slurp command . args) + ;; Return the output from COMMAND. + (let* ((port (apply open-pipe* OPEN_READ command args)) + (output (read-line port)) + (status (close-pipe port))) + output)) + + (define (docker-cli command . args) + ;; Run the given Docker COMMAND. + (apply invoke #$(file-append docker-cli "/bin/docker") + command args)) + + (define (wait-for-container-file container file) + ;; Wait for FILE to show up in CONTAINER. + (docker-cli "exec" container + #$(file-append guile-2.2 "/bin/guile") + "-c" + (object->string + `(let loop ((n 15)) + (when (zero? n) + (error "file didn't show up" ,file)) + (unless (file-exists? ,file) + (sleep 1) + (loop (- n 1))))))) + + (let* ((line (slurp #$(file-append docker-cli "/bin/docker") + "load" "-i" #$tarball)) + (repository&tag (string-drop line + (string-length + "Loaded image: "))) + (container (slurp + #$(file-append docker-cli "/bin/docker") + "create" repository&tag))) + (docker-cli "start" container) + + ;; Wait for shepherd to be ready. + (wait-for-container-file container + "/var/run/shepherd/socket") + + (docker-cli "exec" container + "/run/current-system/profile/bin/herd" + "status") + (slurp #$(file-append docker-cli "/bin/docker") + "exec" container + "/run/current-system/profile/bin/herd" + "status" "guix-daemon"))) + marionette)) + + (test-end) + (exit (= (test-runner-fail-count (test-runner-current)) 0))))) + + (gexp->derivation "docker-system-test" test)) + +(define %test-docker-system + (system-test + (name "docker-system") + (description "Run a system image as produced by @command{guix system +docker-image} inside Docker.") + (value (with-monad %store-monad + (>>= (system-docker-image (simple-operating-system)) + run-docker-system-test))))) -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Wed May 15 10:38:02 2019 Received: (at 35697-done) by debbugs.gnu.org; 15 May 2019 14:38:02 +0000 Received: from localhost ([127.0.0.1]:53143 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hQv2g-0006gG-1X for submit@debbugs.gnu.org; Wed, 15 May 2019 10:38:02 -0400 Received: from eggs.gnu.org ([209.51.188.92]:48127) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hQv2d-0006fz-Up for 35697-done@debbugs.gnu.org; Wed, 15 May 2019 10:38:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57361) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQv2Y-0000pI-Ps; Wed, 15 May 2019 10:37:54 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=42136 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hQv2Y-0000Ma-8o; Wed, 15 May 2019 10:37:54 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 35697-done@debbugs.gnu.org Subject: Re: [bug#35697] [PATCH 0/8] Make 'guix system docker-image' readily usable References: <20190512103055.16832-1-ludo@gnu.org> Date: Wed, 15 May 2019 16:37:51 +0200 In-Reply-To: <20190512103055.16832-1-ludo@gnu.org> ("Ludovic \=\?utf-8\?Q\?Cou\?\= \=\?utf-8\?Q\?rt\=C3\=A8s\=22's\?\= message of "Sun, 12 May 2019 12:30:55 +0200") Message-ID: <87mujnu6kg.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 35697-done Cc: Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Ludovic Court=C3=A8s skribis: > Ludovic Court=C3=A8s (8): > system: Export 'operating-system-default-essential-services'. > linux-container: Improve filtering of unnecessary file systems. > services: 'gc-root-service-type' now has a default value. > linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image > OSes. > linux-container: Compute essential services for THIS-OPERATING-SYSTEM. > system: Add 'operating-system-with-gc-roots'. > docker: 'build-docker-image' accepts an optional #:entry-point. > vm: 'system-docker-image' provides an entry point. Pushed! I had to rebase and adjust to changes made over the last couple of days in that area of the code. Feedback welcome! Ludo=E2=80=99. From unknown Mon Jun 23 04:11:27 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 13 Jun 2019 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator