From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 27 14:37:27 2019 Received: (at submit) by debbugs.gnu.org; 27 Jun 2019 18:37:27 +0000 Received: from localhost ([127.0.0.1]:39638 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZGw-0003ri-LZ for submit@debbugs.gnu.org; Thu, 27 Jun 2019 14:37:27 -0400 Received: from lists.gnu.org ([209.51.188.17]:49381) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZGt-0003rX-Mo for submit@debbugs.gnu.org; Thu, 27 Jun 2019 14:37:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49166) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hgZGq-0006rx-KK for guix-patches@gnu.org; Thu, 27 Jun 2019 14:37:23 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RCVD_IN_DNSWL_NONE, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hgZGo-0005kD-4o for guix-patches@gnu.org; Thu, 27 Jun 2019 14:37:20 -0400 Received: from mx.sdf.org ([205.166.94.20]:52640) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hgZGn-0005ZJ-NR for guix-patches@gnu.org; Thu, 27 Jun 2019 14:37:18 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5RIb2In003059 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO) for ; Thu, 27 Jun 2019 18:37:09 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: guix-patches@gnu.org Subject: [PATCH 0/6] Add 'guix deploy'. Date: Thu, 27 Jun 2019 14:35:28 -0400 Message-ID: <87o92ianbj.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 205.166.94.20 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hello, Guix! This patch provides the basis for 'guix deploy', implementing what I've referred to as the "simple case" in my progress reports: in-place updates to machines (physical or virtual) whose name and IP address we know well. Do note that these commits depend on Ludovic's implementation of 'remote-eval'.[1] There's certainly more to be done with this -- the GSoC period is far from over, and I'm hoping to use that time to implement more complex use-cases such as automatically provisioning virtual machines in the cloud. I'm submitting a patch series now per the recommendation of my mentors to break the project into a few chunks to submit over the duration of the summer. Quite a bit has changed since my last email about this.[2] For one, GOOPS is no longer used. Machine declarations now look just like any other sort of declaration in Guix. #+BEGIN_SRC scheme (use-modules (gnu) (guix)) (use-machine-modules ssh) (use-service-modules networking ssh) (use-package-modules bootloaders) (define %system (operating-system (host-name "gnu-deployed") (timezone "Etc/UTC") (bootloader (bootloader-configuration (bootloader grub-bootloader) (target "/dev/vda") (terminal-outputs '(console)))) (file-systems (cons (file-system (mount-point "/") (device "/dev/vda1") (type "ext4")) %base-file-systems)) (services (append (list (service dhcp-client-service-type) (service openssh-service-type (openssh-configuration (permit-root-login #t) (allow-empty-passwords? #t)))) %base-services)))) (list (machine (system %system) (environment 'managed-host) (configuration (machine-ssh-configuration (host-name "localhost") (identity "./id_rsa") (port 2222))))) #+END_SRC scheme There are a number of other differences here as well. For one, the SSH configuration now has an 'identity' field for specifying a private key to use when authenticating with the host. Any key management scheme you might have set up in '~/.ssh/config' will also work if the 'identity' field is omitted. The 'environment' field is where we declare how machines should be provisioned. In this case, the only type of provisioning that's been implemented is 'managed-host' -- the "simple case" of in-place updates to a machine that's already running GuixSD. The parameters for provisioning are given in the form of an environment-specific configuration type. In the example, this is 'machine-ssh-configuration', which describes how 'guix deploy' should make an SSH connection to the machine. I'm sure you can imagine something along the lines of a 'machine-digitalocean-configuration', describing some parameters for a droplet. There are two things in this patch series that I'd like comments on in particular. First, I still haven't figured out the whole testing situation. The tests, as of now, spin up a virtual machine, create a machine instance, deploy that to the virtual machine, and then make assertions about changes made to the system. These tests were originally in the system test suite as they deal with virtual machines, but I've since moved it into the normal Guix test suite because of how much needs to be done on the host side -- I spent an absurd amount of time trying to fit a call to 'deploy-machine' into a derivation that could be run by the system test suite, but I just wasn't able to make it work. I'm hoping someone will have thoughts about how we can test 'guix deploy'. Should we have them disabled by default? Is there some way to implement them in the a system test suite that I've overlooked? Should the tests be included at all? Second, I'd like some suggestions on how to go about the documentation. I have a cursory description of how to invoke the command-line tool, and an example of a deployment specification, but I'm wondering if the documentation should be split up into multiple sections across the manual -- especially if we're going to have multiple 'environment' types with their own configuration records down the line. I look forward to your comments. Regards, Jakob [1]: https://lists.gnu.org/archive/html/guix-patches/2019-06/msg00201.html [2]: https://lists.gnu.org/archive/html/guix-devel/2019-06/msg00078.html David Thompson (1): Take another stab at this whole guix deploy thing. Jakob L. Kreuze (5): ssh: Add 'identity' keyword to 'open-ssh-session'. gnu: Add machine type for deployment specifications. Export the (gnu machine) interface. Add 'guix deploy'. doc: Add section for 'guix deploy'. Makefile.am | 4 +- doc/guix.texi | 103 +++++++++ gnu.scm | 8 +- gnu/local.mk | 5 +- gnu/machine.scm | 89 ++++++++ gnu/machine/ssh.scm | 355 +++++++++++++++++++++++++++++++ guix/scripts/deploy.scm | 90 ++++++++ guix/ssh.scm | 3 +- tests/machine.scm | 450 ++++++++++++++++++++++++++++++++++++++++ 9 files changed, 1103 insertions(+), 4 deletions(-) create mode 100644 gnu/machine.scm create mode 100644 gnu/machine/ssh.scm create mode 100644 guix/scripts/deploy.scm create mode 100644 tests/machine.scm =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0VDHAACgkQ9Qb9Fp2P 2VoM7RAAhm+uON9VKdFJwd2u9P4E5QPel+EAqgQImJXTWxRS81T2GLsyCt9qHc+H WIRyBzW96p8V0uYI7PlLcdA851XrVychuv6oazAO2YPw0lgaPr3Gn3foJsFi+Sa9 9vfyClT2ime1HqlYHs0/H80FVSF2a/EwVygkKaLn+6UTu4hOSTEXE8uMXTplqWbw M0766ngNI2+1ECJzqoSjgpDssMZihMrv2+4jayqXC9lXtKu/D+vEXHFpoOi/9s12 z32rvooOcFqX2FOXH1DHeTcQ6Y8mK4YlPTaZHHDqlKO9uvZACEWCsf22MOq64YMa 4VqHT71qoiomSg221ZItNGWv9L8zt5YanNpvKqKFbIlH0bURiutf+6cRVJZ7ORvT ya3KTFjLo4aNoKuu35TyOSPVKWyJe3svP9zUhhBon71T+33tCzyG4MSSpiGm8GTh fe395AoHcvronrSQ/jgxgzhCZYWevtz+kPybwP67MSgaL526uMzkgNiQbw8NrfIQ 5SnfOiT0ZHzV+QsCh5htlkVCGR5HI8DTAmG3C1e8d1xUfkCX2RJHmjowbZDjNIYN YxX+NJe0V4RAVVBP8dWzdq/whznBYmLamRo1OfOxSuXru9tL2ppNItXwr7mre1zJ uBN+YA+oQrCZvdh1bZmyzJDx0z5f9qJ6lVTngXqJiI4kDTxKVCQ= =93JG -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 27 14:40:22 2019 Received: (at 36404) by debbugs.gnu.org; 27 Jun 2019 18:40:23 +0000 Received: from localhost ([127.0.0.1]:39643 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZJm-0003xD-AK for submit@debbugs.gnu.org; Thu, 27 Jun 2019 14:40:22 -0400 Received: from mx.sdf.org ([205.166.94.20]:51857) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZJh-0003x1-NG for 36404@debbugs.gnu.org; Thu, 27 Jun 2019 14:40:20 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5RIeF2W015462 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO) for <36404@debbugs.gnu.org>; Thu, 27 Jun 2019 18:40:16 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: 36404@debbugs.gnu.org Subject: [PATCH 1/6] Take another stab at this whole guix deploy thing. In-Reply-To: <87o92ianbj.fsf@sdf.lonestar.org> (Jakob L. Kreuze's message of "Thu, 27 Jun 2019 14:35:28 -0400") References: <87o92ianbj.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) Date: Thu, 27 Jun 2019 14:38:41 -0400 Message-ID: <87imsqan66.fsf@sdf.lonestar.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 2019-03-09 David Thompson * guix/scripts/deploy.scm: New file. * Makefile.am (MODULES): Add it. * gnu/machine.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. --- Makefile.am | 1 + gnu/local.mk | 3 +- gnu/machine.scm | 59 ++++++++++++++++++++++++++++++++ guix/scripts/deploy.scm | 76 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 138 insertions(+), 1 deletion(-) create mode 100644 gnu/machine.scm create mode 100644 guix/scripts/deploy.scm diff --git a/Makefile.am b/Makefile.am index 80be73e4bf..ba01264a4b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -266,6 +266,7 @@ MODULES =3D \ guix/scripts/weather.scm \ guix/scripts/container.scm \ guix/scripts/container/exec.scm \ + guix/scripts/deploy.scm \ guix.scm \ $(GNU_SYSTEM_MODULES) =20 diff --git a/gnu/local.mk b/gnu/local.mk index f5d53b49b8..f973a8d804 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -563,6 +563,7 @@ GNU_SYSTEM_MODULES =3D \ %D%/system/shadow.scm \ %D%/system/uuid.scm \ %D%/system/vm.scm \ + %D%/machine.scm \ \ %D%/build/accounts.scm \ %D%/build/activation.scm \ @@ -629,7 +630,7 @@ INSTALLER_MODULES =3D \ %D%/installer/newt/user.scm \ %D%/installer/newt/utils.scm \ %D%/installer/newt/welcome.scm \ - %D%/installer/newt/wifi.scm=09 + %D%/installer/newt/wifi.scm =20 # Always ship the installer modules but compile them only when # ENABLE_INSTALLER is true. diff --git a/gnu/machine.scm b/gnu/machine.scm new file mode 100644 index 0000000000..4fde7d5c01 --- /dev/null +++ b/gnu/machine.scm @@ -0,0 +1,59 @@ +(define-module (gnu machine) + #:use-module ((gnu packages package-management) #:select (guix)) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix inferior) + #:use-module (guix packages) + #:use-module (guix ssh) + #:use-module (guix store) + #:use-module (oop goops) + #:use-module (ssh session) + #:export ( + system + display-name + build-os + deploy-os + remote-eval + + + host-name + ssh-port + ssh-user)) + +(define-class () + (system #:getter system #:init-keyword #:system)) + +(define-method (display-name (machine )) + (operating-system-host-name (system machine))) + +(define-method (build-os (machine ) store) + (let* ((guixdrv (run-with-store store (package->derivation guix))) + (guixdir (and (build-derivations store (list guixdrv)) + (derivation->output-path guixdrv))) + (osdrv (run-with-store store (operating-system-derivation + (system machine))))) + (and (build-derivations store (list osdrv)) + (list (derivation-file-name osdrv) + (derivation->output-path osdrv))))) + +(define-method (deploy-os (machine ) store osdrv) + (error "not implemented")) + +(define-method (remote-eval (machine ) exp) + (error "not implemented")) + +(define-class () + (host-name #:getter host-name #:init-keyword #:host-name) + (ssh-port #:getter ssh-port #:init-keyword #:ssh-port #:init-form 22) + (ssh-user #:getter ssh-user #:init-keyword #:ssh-user #:init-form "root") + ;; ??? - SSH key config? + ) + +(define-method (deploy-os (machine ) store osdrvs) + (let ((session (open-ssh-session (host-name machine) + #:user (ssh-user machine) + #:port (ssh-port machine)))) + (with-store store (send-files store osdrvs + (connect-to-remote-daemon session) + #:recursive? #t)) + #t)) diff --git a/guix/scripts/deploy.scm b/guix/scripts/deploy.scm new file mode 100644 index 0000000000..bcb3a2ea4c --- /dev/null +++ b/guix/scripts/deploy.scm @@ -0,0 +1,76 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts deploy) + #:use-module (gnu machine) + #:use-module (guix ui) + #:use-module (guix scripts) + #:use-module (guix scripts build) + #:use-module (guix store) + #:use-module (ice-9 format) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-37) + #:export (guix-deploy)) + +(define (show-help) + (display (G_ "Usage: guix deploy WHATEVER\n"))) + +(define %options + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + %standard-build-options)) + +(define %default-options + '((system . ,(%current-system)) + (substitutes? . #t) + (build-hook? . #t) + (graft? . #t) + (print-build-trace? . #t) + (print-extended-build-trace? . #t) + (multiplexed-build-output? . #t) + (debug . 0) + (verbosity . 2))) + +(define (load-source-file file) + (let ((module (make-user-module '()))) + (load* file module))) + +(define (guix-deploy . args) + (define (handle-argument arg result) + (alist-cons 'file arg result)) + (let* ((opts (parse-command-line args %options (list %default-options) + #:argument-handler handle-argument)) + (file (assq-ref opts 'file)) + (machines (load-source-file file))) + (with-store store + (set-build-options-from-command-line store opts) + ;; Build all the OSes and create a mapping from machine to OS deriva= tion + ;; for use in the deploy step. + (let ((osdrvs (map (lambda (machine) + (format #t "building ~a... " (display-name mach= ine)) + (let ((osdrv (build-os machine store))) + (display "done\n") + (cons machine osdrv))) + machines))) + (for-each (lambda (machine) + (format #t "deploying to ~a... " (display-name machine= )) + (deploy-os machine store (assq-ref osdrvs machine)) + (display "done\n")) + machines))))) --=20 2.22.0 From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 27 14:41:21 2019 Received: (at 36404) by debbugs.gnu.org; 27 Jun 2019 18:41:21 +0000 Received: from localhost ([127.0.0.1]:39651 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZKj-0003zb-8j for submit@debbugs.gnu.org; Thu, 27 Jun 2019 14:41:21 -0400 Received: from mx.sdf.org ([205.166.94.20]:51603) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZKf-0003zO-Nc for 36404@debbugs.gnu.org; Thu, 27 Jun 2019 14:41:19 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5RIfG5j012369 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO) for <36404@debbugs.gnu.org>; Thu, 27 Jun 2019 18:41:17 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: 36404@debbugs.gnu.org Subject: [PATCH 2/6] ssh: Add 'identity' keyword to 'open-ssh-session'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> Date: Thu, 27 Jun 2019 14:39:41 -0400 In-Reply-To: <87imsqan66.fsf@sdf.lonestar.org> (Jakob L. Kreuze's message of "Thu, 27 Jun 2019 14:38:41 -0400") Message-ID: <87ef3ean4i.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 2019-06-26 Jakob L. Kreuze * guix/ssh.scm (open-ssh-session): Add 'identity' keyword argument. --- guix/ssh.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/guix/ssh.scm b/guix/ssh.scm index 9b9baf54ea..a2387564a4 100644 --- a/guix/ssh.scm +++ b/guix/ssh.scm @@ -57,12 +57,13 @@ (define %compression "zlib@openssh.com,zlib") -(define* (open-ssh-session host #:key user port +(define* (open-ssh-session host #:key user port identity (compression %compression)) "Open an SSH session for HOST and return it. When USER and PORT are #f, use default values or whatever '~/.ssh/config' specifies; otherwise use them. Throw an error on failure." (let ((session (make-session #:user user + #:identity identity #:host host #:port port #:timeout 10 ;seconds -- 2.22.0 From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 27 14:42:06 2019 Received: (at 36404) by debbugs.gnu.org; 27 Jun 2019 18:42:06 +0000 Received: from localhost ([127.0.0.1]:39656 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZLL-00040o-MJ for submit@debbugs.gnu.org; Thu, 27 Jun 2019 14:42:06 -0400 Received: from mx.sdf.org ([205.166.94.20]:51486) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZLH-00040a-4D for 36404@debbugs.gnu.org; Thu, 27 Jun 2019 14:41:58 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5RIfrbS019842 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO) for <36404@debbugs.gnu.org>; Thu, 27 Jun 2019 18:41:54 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: 36404@debbugs.gnu.org Subject: [PATCH 3/6] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> Date: Thu, 27 Jun 2019 14:40:18 -0400 In-Reply-To: <87ef3ean4i.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Thu, 27 Jun 2019 14:39:41 -0400") Message-ID: <87a7e2an3h.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 2019-06-26 Jakob L. Kreuze * tests/machine.scm: New file. * Makefile.am (SCM_TESTS): Add it. * gnu/machine/ssh.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. * gnu/machine.scm (machine, sshable-machine): Delete. * gnu/machine.scm: (machine): New record type. * gnu/machine.scm: (display-name, build-os, deploy-os, host-name) (ssh-port, ssh-user): Delete. * gnu/machine.scm: (remote-eval): Rewrite procedure. * gnu/machine.scm: (machine-display-name, build-machine) (deploy-machine): New procedures. All callers changed. --- Makefile.am | 3 +- gnu/local.mk | 4 +- gnu/machine.scm | 140 ++++++++----- gnu/machine/ssh.scm | 355 +++++++++++++++++++++++++++++++ guix/scripts/deploy.scm | 8 +- tests/machine.scm | 450 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 899 insertions(+), 61 deletions(-) create mode 100644 gnu/machine/ssh.scm create mode 100644 tests/machine.scm diff --git a/Makefile.am b/Makefile.am index ba01264a4b..8dbc220489 100644 --- a/Makefile.am +++ b/Makefile.am @@ -424,7 +424,8 @@ SCM_TESTS =3D \ tests/import-utils.scm \ tests/store-database.scm \ tests/store-deduplication.scm \ - tests/store-roots.scm + tests/store-roots.scm \ + tests/machine.scm =20 SH_TESTS =3D \ tests/guix-build.sh \ diff --git a/gnu/local.mk b/gnu/local.mk index f973a8d804..ad87de5ea7 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -563,7 +563,9 @@ GNU_SYSTEM_MODULES =3D \ %D%/system/shadow.scm \ %D%/system/uuid.scm \ %D%/system/vm.scm \ - %D%/machine.scm \ + \ + %D%/machine.scm \ + %D%/machine/ssh.scm \ \ %D%/build/accounts.scm \ %D%/build/activation.scm \ diff --git a/gnu/machine.scm b/gnu/machine.scm index 4fde7d5c01..900a2020dc 100644 --- a/gnu/machine.scm +++ b/gnu/machine.scm @@ -1,59 +1,89 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + (define-module (gnu machine) - #:use-module ((gnu packages package-management) #:select (guix)) #:use-module (gnu system) #:use-module (guix derivations) - #:use-module (guix inferior) - #:use-module (guix packages) - #:use-module (guix ssh) + #:use-module (guix monads) + #:use-module (guix records) #:use-module (guix store) - #:use-module (oop goops) - #:use-module (ssh session) - #:export ( - system - display-name - build-os - deploy-os - remote-eval - - - host-name - ssh-port - ssh-user)) - -(define-class () - (system #:getter system #:init-keyword #:system)) - -(define-method (display-name (machine )) - (operating-system-host-name (system machine))) - -(define-method (build-os (machine ) store) - (let* ((guixdrv (run-with-store store (package->derivation guix))) - (guixdir (and (build-derivations store (list guixdrv)) - (derivation->output-path guixdrv))) - (osdrv (run-with-store store (operating-system-derivation - (system machine))))) - (and (build-derivations store (list osdrv)) - (list (derivation-file-name osdrv) - (derivation->output-path osdrv))))) - -(define-method (deploy-os (machine ) store osdrv) - (error "not implemented")) - -(define-method (remote-eval (machine ) exp) - (error "not implemented")) - -(define-class () - (host-name #:getter host-name #:init-keyword #:host-name) - (ssh-port #:getter ssh-port #:init-keyword #:ssh-port #:init-form 22) - (ssh-user #:getter ssh-user #:init-keyword #:ssh-user #:init-form "root") - ;; ??? - SSH key config? - ) - -(define-method (deploy-os (machine ) store osdrvs) - (let ((session (open-ssh-session (host-name machine) - #:user (ssh-user machine) - #:port (ssh-port machine)))) - (with-store store (send-files store osdrvs - (connect-to-remote-daemon session) - #:recursive? #t)) - #t)) + #:export (machine + machine? + this-machine + + machine-system + machine-environment + machine-configuration + machine-display-name + + build-machine + deploy-machine + remote-eval)) + +;;; Commentary: +;;; +;;; This module provides the types used to declare individual machines in a +;;; heterogeneous Guix deployment. The interface allows users of specify s= ystem +;;; configurations and the means by which resources should be provisioned = on a +;;; per-host basis. +;;; +;;; Code: + +(define-record-type* machine + make-machine + machine? + this-machine + (system machine-system) ; + (environment machine-environment) ; symbol + (configuration machine-configuration ; configuration object + (default #f))) ; specific to environment + +(define (machine-display-name machine) + "Return the host-name identifying MACHINE." + (operating-system-host-name (machine-system machine))) + +(define (build-machine machine) + "Monadic procedure that builds the system derivation for MACHINE and ret= urning +a list containing the path of the derivation file and the path of the deri= vation +output." + (let ((os (machine-system machine))) + (mlet* %store-monad ((osdrv (operating-system-derivation os)) + (_ ((store-lift build-derivations) (list osdrv)))) + (return (list (derivation-file-name osdrv) + (derivation->output-path osdrv)))))) + +(define (remote-eval machine exp) + "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP refe= rs to +are built and deployed to MACHINE beforehand." + (case (machine-environment machine) + ((managed-host) + ((@@ (gnu machine ssh) remote-eval) machine exp)) + (else + (let ((type (machine-environment machine))) + (error "unsupported environment type" type))))) + +(define (deploy-machine machine) + "Monadic procedure transferring the new system's OS closure to the remote +MACHINE, activating it on MACHINE and switching MACHINE to the new generat= ion." + (case (machine-environment machine) + ((managed-host) + ((@@ (gnu machine ssh) deploy-machine) machine)) + (else + (let ((type (machine-environment machine))) + (error "unsupported environment type" type))))) diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm new file mode 100644 index 0000000000..a8f946e19f --- /dev/null +++ b/gnu/machine/ssh.scm @@ -0,0 +1,355 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu machine ssh) + #:use-module (gnu bootloader) + #:use-module (gnu machine) + #:autoload (gnu packages gnupg) (guile-gcrypt) + #:use-module (gnu services) + #:use-module (gnu services shepherd) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix gexp) + #:use-module (guix modules) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix ssh) + #:use-module (guix store) + #:use-module (ice-9 match) + #:use-module (srfi srfi-19) + #:export (machine-ssh-configuration + machine-ssh-configuration? + machine-ssh-configuration + + machine-ssh-configuration-host-name + machine-ssh-configuration-port + machine-ssh-configuration-user + machine-ssh-configuration-session)) + +;;; Commentary: +;;; +;;; This module implements remote evaluation and system deployment for +;;; machines that are accessable over SSH and have a known host-name. In t= he +;;; sense of the broader "machine" interface, we describe the environment = for +;;; such machines as 'managed-host. +;;; +;;; Code: + + +;;; +;;; SSH client parameter configuration. +;;; + +(define-record-type* machine-ssh-configuration + make-machine-ssh-configuration + machine-ssh-configuration? + this-machine-ssh-configuration + (host-name machine-ssh-configuration-host-name) ; string + (port machine-ssh-configuration-port ; integer + (default 22)) + (user machine-ssh-configuration-user ; string + (default "root")) + (identity machine-ssh-configuration-identity ; path to a private key + (default #f)) + (session machine-ssh-configuration-session ; session + (default #f))) + +(define (machine-ssh-session machine) + "Return the SSH session that was given in MACHINE's configuration, or cr= eate +one from the configuration's parameters if one was not provided." + (let ((config (machine-configuration machine))) + (if (machine-ssh-configuration? config) + (or (machine-ssh-configuration-session config) + (let ((host-name (machine-ssh-configuration-host-name config)) + (user (machine-ssh-configuration-user config)) + (port (machine-ssh-configuration-port config)) + (identity (machine-ssh-configuration-identity config))) + (open-ssh-session host-name + #:user user + #:port port + #:identity identity))) + (error "unsupported configuration type")))) + + +;;; +;;; Remote evaluation. +;;; + +(define (remote-eval machine exp) + "Internal implementation of 'remote-eval' for MACHINE instances with an +environment type of 'managed-host." + (unless (machine-configuration machine) + (error (format #f (G_ "no configuration specified for machine of envir= onment '~a'") + (symbol->string (machine-environment machine))))) + ((@ (guix remote) remote-eval) exp (machine-ssh-session machine))) + + +;;; +;;; System deployment. +;;; + +(define (switch-to-system machine) + "Monadic procedure creating a new generation on MACHINE and execute the +activation script for the new system configuration." + (define (remote-exp drv script) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles) + (guix utils))) + #~(begin + (use-modules (guix config) + (guix profiles) + (guix utils)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (let* ((system #$(derivation->output-path drv)) + (number (1+ (generation-number %system-profile))) + (generation (generation-file-name %system-profile numbe= r)) + (old-env (environ)) + (old-path %load-path) + (old-cpath %load-compiled-path)) + (switch-symlinks generation system) + (switch-symlinks %system-profile generation) + ;; Guard against the activation script modifying $PATH. + (dynamic-wind + (const #t) + (lambda () + (setenv "GUIX_NEW_SYSTEM" system) + ;; Guard against the activation script modifying '%load-= path'. + (dynamic-wind + (const #t) + (lambda () + ;; The activation script may write to stdout, which + ;; confuses 'remote-eval' when it attempts to read a + ;; result from the remote REPL. We work around this = by + ;; forcing the output to a string. + (with-output-to-string + (lambda () + (primitive-load #$script)))) + (lambda () + (set! %load-path old-path) + (set! %load-compiled-path old-cpath)))) + (lambda () + (environ old-env)))))))) + + (let* ((os (machine-system machine)) + (script (operating-system-activation-script os))) + (mlet* %store-monad ((drv (operating-system-derivation os))) + (remote-eval machine (remote-exp drv script))))) + +(define (upgrade-shepherd-services machine) + "Monadic procedure unloading and starting services on the remote as need= ed +to realize the MACHINE's system configuration." + (define target-services + ;; Monadic expression evaluating to a list of (name output-path) pairs= for + ;; all of MACHINE's services. + (mapm %store-monad + (lambda (service) + (mlet %store-monad ((file ((compose lower-object + shepherd-service-file) + service))) + (return (list (shepherd-service-canonical-name service) + (derivation->output-path file))))) + (service-value + (fold-services (operating-system-services (machine-system machi= ne)) + #:target-type shepherd-root-service-type)))) + + (define (remote-exp target-services) + (with-imported-modules '((gnu services herd)) + #~(begin + (use-modules (gnu services herd) + (srfi srfi-1)) + + (define running + (filter live-service-running (current-services))) + + (define (essential? service) + ;; Return #t if SERVICE is essential and should not be unloaded + ;; under any circumstance. + (memq (first (live-service-provision service)) + '(root shepherd))) + + (define (obsolete? service) + ;; Return #t if SERVICE can be safely unloaded. + (and (not (essential? service)) + (every (lambda (requirements) + (not (memq (first (live-service-provision servic= e)) + requirements))) + (map live-service-requirement running)))) + + (define to-unload + (filter obsolete? + (remove (lambda (service) + (memq (first (live-service-provision service= )) + (map first '#$target-services))) + running))) + + (define to-start + (remove (lambda (service-pair) + (memq (first service-pair) + (map (compose first live-service-provision) + running))) + '#$target-services)) + + ;; Unload obsolete services. + (for-each (lambda (service) + (false-if-exception + (unload-service service))) + to-unload) + + ;; Load the service files for any new services and start them. + (load-services/safe (map second to-start)) + (for-each start-service (map first to-start)) + + #t))) + + (mlet %store-monad ((target-services target-services)) + (remote-eval machine (remote-exp target-services)))) + +(define (machine-boot-parameters machine) + "Monadic procedure returning a list of 'boot-parameters' for the generat= ions +of MACHINE's system profile, ordered from most recent to oldest." + (define bootable-kernel-arguments + (@@ (gnu system) bootable-kernel-arguments)) + + (define remote-exp + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles))) + #~(begin + (use-modules (guix config) + (guix profiles) + (ice-9 textual-ports)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (define (read-file path) + (call-with-input-file path + (lambda (port) + (get-string-all port)))) + + (map (lambda (generation) + (let* ((system-path (generation-file-name %system-profi= le + generation)) + (boot-parameters-path (string-append system-path + "/parameter= s")) + (time (stat:mtime (lstat system-path)))) + (list generation + system-path + time + (read-file boot-parameters-path)))) + (reverse (generation-numbers %system-profile))))))) + + (mlet* %store-monad ((generations (remote-eval machine remote-exp))) + (return + (map (lambda (generation) + (match generation + ((generation system-path time serialized-params) + (let* ((params (call-with-input-string serialized-params + read-boot-parameters)) + (root (boot-parameters-root-device params)) + (label (boot-parameters-label params))) + (boot-parameters + (inherit params) + (label + (string-append label " (#" + (number->string generation) ", " + (let ((time (make-time time-utc 0 time))) + (date->string (time-utc->date time) + "~Y-~m-~d ~H:~M")) + ")")) + (kernel-arguments + (append (bootable-kernel-arguments system-path root) + (boot-parameters-kernel-arguments params)))))))) + generations)))) + +(define (install-bootloader machine) + "Create a bootloader entry for the new system generation on MACHINE, and +configure the bootloader to boot that generation by default." + (define bootloader-installer-script + (@@ (guix scripts system) bootloader-installer-script)) + + (define (remote-exp installer bootcfg bootcfg-file) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((gnu build install) + (guix store) + (guix utils))) + #~(begin + (use-modules (gnu build install) + (guix store) + (guix utils)) + (let* ((gc-root (string-append "/" %gc-roots-directory "/bootc= fg")) + (temp-gc-root (string-append gc-root ".new")) + (old-path %load-path) + (old-cpath %load-compiled-path)) + (switch-symlinks temp-gc-root gc-root) + + (unless (false-if-exception + (begin + (install-boot-config #$bootcfg #$bootcfg-file "/") + ;; Guard against the activation script modifying + ;; '%load-path'. + (dynamic-wind + (const #t) + (lambda () + ;; The installation script may write to stdou= t, + ;; which confuses 'remote-eval' when it attem= pts to + ;; read a result from the remote REPL. We work + ;; around this by forcing the output to a str= ing. + (with-output-to-string + (lambda () + (primitive-load #$installer)))) + (lambda () + (set! %load-path old-path) + (set! %load-compiled-path old-cpath))))) + (delete-file temp-gc-root) + (error "failed to install bootloader")) + + (rename-file temp-gc-root gc-root) + #t))))) + + (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine))) + (let* ((os (machine-system machine)) + (bootloader ((compose bootloader-configuration-bootloader + operating-system-bootloader) + os)) + (bootloader-target (bootloader-configuration-target + (operating-system-bootloader os))) + (installer (bootloader-installer-script + (bootloader-installer bootloader) + (bootloader-package bootloader) + bootloader-target + "/")) + (menu-entries (map boot-parameters->menu-entry boot-parameters)) + (bootcfg (operating-system-bootcfg os menu-entries)) + (bootcfg-file (bootloader-configuration-file bootloader))) + (remote-eval machine (remote-exp installer bootcfg bootcfg-file))))) + +(define (deploy-machine machine) + "Internal implementation of 'deploy-machine' for MACHINE instances with = an +environment type of 'managed-host." + (unless (machine-configuration machine) + (error (format #f (G_ "no configuration specified for machine of envir= onment '~a'") + (symbol->string (machine-environment machine))))) + (mbegin %store-monad + (switch-to-system machine) + (upgrade-shepherd-services machine) + (install-bootloader machine))) diff --git a/guix/scripts/deploy.scm b/guix/scripts/deploy.scm index bcb3a2ea4c..0be279642b 100644 --- a/guix/scripts/deploy.scm +++ b/guix/scripts/deploy.scm @@ -64,13 +64,13 @@ ;; Build all the OSes and create a mapping from machine to OS deriva= tion ;; for use in the deploy step. (let ((osdrvs (map (lambda (machine) - (format #t "building ~a... " (display-name mach= ine)) - (let ((osdrv (build-os machine store))) + (format #t "building ~a... " (machine-display-n= ame machine)) + (let ((osdrv (run-with-store store (build-machi= ne machine)))) (display "done\n") (cons machine osdrv))) machines))) (for-each (lambda (machine) - (format #t "deploying to ~a... " (display-name machine= )) - (deploy-os machine store (assq-ref osdrvs machine)) + (format #t "deploying to ~a... " (machine-display-name= machine)) + (run-with-store store (deploy-machine machine)) (display "done\n")) machines))))) diff --git a/tests/machine.scm b/tests/machine.scm new file mode 100644 index 0000000000..390c0189bb --- /dev/null +++ b/tests/machine.scm @@ -0,0 +1,450 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu tests machine) + #:use-module (gnu bootloader grub) + #:use-module (gnu bootloader) + #:use-module (gnu build marionette) + #:use-module (gnu build vm) + #:use-module (gnu machine) + #:use-module (gnu machine ssh) + #:use-module (gnu packages bash) + #:use-module (gnu packages virtualization) + #:use-module (gnu services base) + #:use-module (gnu services networking) + #:use-module (gnu services ssh) + #:use-module (gnu services) + #:use-module (gnu system file-systems) + #:use-module (gnu system vm) + #:use-module (gnu system) + #:use-module (gnu tests) + #:use-module (guix derivations) + #:use-module (guix gexp) + #:use-module (guix monads) + #:use-module (guix pki) + #:use-module (guix store) + #:use-module (guix utils) + #:use-module (ice-9 ftw) + #:use-module (ice-9 match) + #:use-module (ice-9 textual-ports) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-64) + #:use-module (ssh auth) + #:use-module (ssh channel) + #:use-module (ssh key) + #:use-module (ssh session)) + + +;;; +;;; Virtual machine scaffolding. +;;; + +(define marionette-pid (@@ (gnu build marionette) marionette-pid)) + +(define (call-with-marionette path command proc) + "Invoke PROC with a marionette running COMMAND in PATH." + (let* ((marionette (make-marionette command #:socket-directory path)) + (pid (marionette-pid marionette))) + (dynamic-wind + (lambda () + (unless marionette + (error "could not start marionette"))) + (lambda () (proc marionette)) + (lambda () + (kill pid SIGTERM))))) + +(define (dir-join . components) + "Join COMPONENTS with `file-name-separator-string'." + (string-join components file-name-separator-string)) + +(define (call-with-machine-test-directory proc) + "Run PROC with the path to a temporary directory that will be cleaned up +when PROC returns. Only files that can be passed to 'delete-file' should be +created within the temporary directory; cleanup will not recurse into +subdirectories." + (let ((path (tmpnam))) + (dynamic-wind + (lambda () + (unless (mkdir path) + (error (format #f "could not create directory '~a'" path)))) + (lambda () (proc path)) + (lambda () + (let ((children (map first (cddr (file-system-tree path))))) + (for-each (lambda (child) + (false-if-exception + (delete-file (dir-join path child)))) + children) + (rmdir path)))))) + +(define (os-for-test os) + "Return an record derived from OS that is appropriate= for +use with 'qemu-image'." + (define file-systems-to-keep + ;; Keep only file systems other than root and not normally bound to re= al + ;; devices. + (remove (lambda (fs) + (let ((target (file-system-mount-point fs)) + (source (file-system-device fs))) + (or (string=3D? target "/") + (string-prefix? "/dev/" source)))) + (operating-system-file-systems os))) + + (define root-uuid + ;; UUID of the root file system. + ((@@ (gnu system vm) operating-system-uuid) os 'dce)) + + + (operating-system + (inherit os) + ;; Assume we have an initrd with the whole QEMU shebang. + + ;; Force our own root file system. Refer to it by UUID so that + ;; it works regardless of how the image is used ("qemu -hda", + ;; Xen, etc.). + (file-systems (cons (file-system + (mount-point "/") + (device root-uuid) + (type "ext4")) + file-systems-to-keep)))) + +(define (qemu-image-for-test os) + "Return a derivation producing a QEMU disk image running OS. This proced= ure +is similar to 'system-qemu-image' in (gnu system vm), but makes use of +'os-for-test' so that callers may obtain the same system derivation that w= ill +be booted by the image." + (define root-uuid ((@@ (gnu system vm) operating-system-uuid) os 'dce)) + (let* ((os (os-for-test os)) + (bootcfg (operating-system-bootcfg os))) + (qemu-image #:os os + #:bootcfg-drv bootcfg + #:bootloader (bootloader-configuration-bootloader + (operating-system-bootloader os)) + #:disk-image-size (* 9000 (expt 2 20)) + #:file-system-type "ext4" + #:file-system-uuid root-uuid + #:inputs `(("system" ,os) + ("bootcfg" ,bootcfg)) + #:copy-inputs? #t))) + +(define (make-writable-image image) + "Return a derivation producing a script to create a writable disk image +overlay of IMAGE, writing the overlay to the the path given as a command-l= ine +argument to the script." + (define qemu-img-exec + #~(list (string-append #$qemu-minimal "/bin/qemu-img") + "create" "-f" "qcow2" + "-o" (string-append "backing_file=3D" #$image))) + + (define builder + #~(call-with-output-file #$output + (lambda (port) + (format port "#!~a~% exec ~a \"$@\"~%" + #$(file-append bash "/bin/sh") + (string-join #$qemu-img-exec " ")) + (chmod port #o555)))) + + (gexp->derivation "make-writable-image.sh" builder)) + +(define (run-os-for-test os) + "Return a derivation producing a script to run OS as a qemu guest, whose +first argument is the path to a writable disk image. Additional arguments = are +passed as-is to qemu." + (define kernel-arguments + #~(list "console=3DttyS0" + #+@(operating-system-kernel-arguments os "/dev/sda1"))) + + (define qemu-exec + #~(begin + (list (string-append #$qemu-minimal "/bin/" #$(qemu-command (%curr= ent-system))) + "-kernel" #$(operating-system-kernel-file os) + "-initrd" #$(file-append os "/initrd") + (format #f "-append ~s" + (string-join #$kernel-arguments " ")) + #$@(if (file-exists? "/dev/kvm") + '("-enable-kvm") + '()) + "-no-reboot" + "-net nic,model=3Dvirtio" + "-object" "rng-random,filename=3D/dev/urandom,id=3Dguixsd-vm= -rng" + "-device" "virtio-rng-pci,rng=3Dguixsd-vm-rng" + "-vga" "std" + "-m" "256" + "-net" "user,hostfwd=3Dtcp::2222-:22"))) + + (define builder + #~(call-with-output-file #$output + (lambda (port) + (format port "#!~a~% exec ~a -drive \"file=3D$@\"~%" + #$(file-append bash "/bin/sh") + (string-join #$qemu-exec " ")) + (chmod port #o555)))) + + (gexp->derivation "run-vm.sh" builder)) + +(define (scripts-for-test os) + "Build and return a list containing the paths of: + +- A script to make a writable disk image overlay of OS. +- A script to run that disk image overlay as a qemu guest." + (let ((virtualized-os (os-for-test os))) + (mlet* %store-monad ((osdrv (operating-system-derivation virtualized-o= s)) + (imgdrv (qemu-image-for-test os)) + + ;; Ungexping 'imgdrv' or 'osdrv' will result in an + ;; error if the derivations don't exist in the st= ore, + ;; so we ensure they're built prior to invoking + ;; 'run-vm' or 'make-image'. + (_ ((store-lift build-derivations) (list imgdrv))) + + (run-vm (run-os-for-test virtualized-os)) + (make-image + (make-writable-image (derivation->output-path im= gdrv)))) + (mbegin %store-monad + ((store-lift build-derivations) (list imgdrv make-image run-vm)) + (return (list (derivation->output-path make-image) + (derivation->output-path run-vm))))))) + +(define (call-with-marionette-and-session os proc) + "Construct a marionette backed by OS in a temporary test environment and +invoke PROC with two arguments: the marionette object, and an SSH session +connected to the marionette." + (call-with-machine-test-directory + (lambda (path) + (match (with-store store + (run-with-store store + (scripts-for-test %system))) + ((make-image run-vm) + (let ((image (dir-join path "image"))) + ;; Create the writable image overlay. + (system (string-join (list make-image image) " ")) + (call-with-marionette + path + (list run-vm image) + (lambda (marionette) + ;; XXX: The guest clearly has (gcrypt pk-crypto) since this + ;; works, but trying to import it from 'marionette-eval' fail= s as + ;; the Marionette REPL does not have 'guile-gcrypt' in its + ;; %load-path. + (marionette-eval + `(begin + (use-modules (ice-9 popen)) + (let ((port (open-pipe* OPEN_WRITE "guix" "archive" "--au= thorize"))) + (put-string port ,%signing-key) + (close port))) + marionette) + ;; XXX: This is an absolute hack to work around potential qui= rks + ;; in the operating system. For one, we invoke 'herd' from the + ;; command-line to ensure that the Shepherd socket file + ;; exists. Second, we enable 'ssh-daemon', as there's a chance + ;; the service will be disabled upon booting the image. + (marionette-eval + `(system "herd enable ssh-daemon") + marionette) + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (start-service 'ssh-daemon)) + marionette) + (call-with-connected-session/auth + (lambda (session) + (proc marionette session))))))))))) + + +;;; +;;; SSH session management. These are borrowed from (gnu tests ssh). +;;; + +(define (make-session-for-test) + "Make a session with predefined parameters for a test." + (make-session #:user "root" + #:port 2222 + #:host "localhost")) + +(define (call-with-connected-session proc) + "Call the one-argument procedure PROC with a freshly created and +connected SSH session object, return the result of the procedure call. The +session is disconnected when the PROC is finished." + (let ((session (make-session-for-test))) + (dynamic-wind + (lambda () + (let ((result (connect! session))) + (unless (equal? result 'ok) + (error "Could not connect to a server" + session result)))) + (lambda () (proc session)) + (lambda () (disconnect! session))))) + +(define (call-with-connected-session/auth proc) + "Make an authenticated session. We should be able to connect as +root with an empty password." + (call-with-connected-session + (lambda (session) + ;; Try the simple authentication methods. Dropbear requires + ;; 'none' when there are no passwords, whereas OpenSSH accepts + ;; 'password' with an empty password. + (let loop ((methods (list (cut userauth-password! <> "") + (cut userauth-none! <>)))) + (match methods + (() + (error "all the authentication methods failed")) + ((auth rest ...) + (match (pk 'auth (auth session)) + ('success + (proc session)) + ('denied + (loop rest))))))))) + + +;;; +;;; Virtual machines for use in the test suite. +;;; + +(define %system + ;; A "bare bones" operating system running both an OpenSSH daemon and the + ;; "marionette" service. + (marionette-operating-system + (operating-system + (host-name "gnu") + (timezone "Etc/UTC") + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (target "/dev/sda") + (terminal-outputs '(console)))) + (file-systems (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + %base-file-systems)) + (services + (append (list (service dhcp-client-service-type) + (service openssh-service-type + (openssh-configuration + (permit-root-login #t) + (allow-empty-passwords? #t)))) + %base-services))) + #:imported-modules '((gnu services herd) + (guix combinators)))) + +(define %signing-key + ;; The host's signing key, encoded as a string. The "marionette" will re= ject + ;; any files signed by an unauthorized host, so we'll need to send this = key + ;; over and authorize it. + (call-with-input-file %public-key-file + (lambda (port) + (get-string-all port)))) + + +(test-begin "machine") + +(define (system-generations marionette) + (marionette-eval + '(begin + (use-modules (ice-9 ftw) + (srfi srfi-1)) + (let* ((profile-dir "/var/guix/profiles/") + (entries (map first (cddr (file-system-tree profile-dir))))) + (remove (lambda (entry) + (member entry '("per-user" "system"))) + entries))) + marionette)) + +(define (running-services marionette) + (marionette-eval + '(begin + (use-modules (gnu services herd) + (srfi srfi-1)) + (map (compose first live-service-provision) + (filter live-service-running (current-services)))) + marionette)) + +(define (count-grub-cfg-entries marionette) + (marionette-eval + '(begin + (define grub-cfg + (call-with-input-file "/boot/grub/grub.cfg" + (lambda (port) + (get-string-all port)))) + + (let loop ((n 0) + (start 0)) + (let ((index (string-contains grub-cfg "menuentry" start))) + (if index + (loop (1+ n) (1+ index)) + n)))) + marionette)) + +(define %target-system + (marionette-operating-system + (operating-system + (host-name "gnu-deployed") + (timezone "Etc/UTC") + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (target "/dev/sda") + (terminal-outputs '(console)))) + (file-systems (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + %base-file-systems)) + (services + (append (list (service tor-service-type) + (service dhcp-client-service-type) + (service openssh-service-type + (openssh-configuration + (permit-root-login #t) + (allow-empty-passwords? #t)))) + %base-services))) + #:imported-modules '((gnu services herd) + (guix combinators)))) + +(call-with-marionette-and-session + (os-for-test %system) + (lambda (marionette session) + (let ((generations-prior (system-generations marionette)) + (services-prior (running-services marionette)) + (grub-entry-count-prior (count-grub-cfg-entries marionette)) + (machine (machine + (system %target-system) + (environment 'managed-host) + (configuration (machine-ssh-configuration + (host-name "localhost") + (session session)))))) + (with-store store + (run-with-store store + (build-machine machine)) + (run-with-store store + (deploy-machine machine))) + (test-equal "deployment created new generation" + (length (system-generations marionette)) + (1+ (length generations-prior))) + (test-assert "deployment started new service" + (and (not (memq 'tor services-prior)) + (memq 'tor (running-services marionette)))) + (test-equal "deployment created new menu entry" + (count-grub-cfg-entries marionette) + ;; A Grub configuration that contains a single menu entry does not = have + ;; an "old configurations" submenu. Deployment, then, would result = in + ;; this submenu being created, meaning an additional two 'menuentry' + ;; fields rather than just one. + (if (=3D grub-entry-count-prior 1) + (+ 2 grub-entry-count-prior) + (1+ grub-entry-count-prior)))))) + +(test-end "machine") --=20 2.22.0 From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 27 14:42:35 2019 Received: (at 36404) by debbugs.gnu.org; 27 Jun 2019 18:42:35 +0000 Received: from localhost ([127.0.0.1]:39663 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZLv-00042L-16 for submit@debbugs.gnu.org; Thu, 27 Jun 2019 14:42:35 -0400 Received: from mx.sdf.org ([205.166.94.20]:51328) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZLt-00042D-Eq for 36404@debbugs.gnu.org; Thu, 27 Jun 2019 14:42:33 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5RIgVoV017405 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO) for <36404@debbugs.gnu.org>; Thu, 27 Jun 2019 18:42:32 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: 36404@debbugs.gnu.org Subject: [PATCH 4/6] Export the (gnu machine) interface. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> <87a7e2an3h.fsf_-_@sdf.lonestar.org> Date: Thu, 27 Jun 2019 14:40:57 -0400 In-Reply-To: <87a7e2an3h.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Thu, 27 Jun 2019 14:40:18 -0400") Message-ID: <875zoqan2e.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) This is so machine declarations can have a simple (use-modules (gnu)) rather than having to import the machine module explicitly. 2019-06-26 Jakob L. Kreuze * gnu.scm (%public-modules): Add '(gnu machine)'. * gnu.scm (use-machine-modules): New macro. --- gnu.scm | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/gnu.scm b/gnu.scm index 2c29b6dc3f..fa643a5b92 100644 --- a/gnu.scm +++ b/gnu.scm @@ -27,7 +27,8 @@ #:use-module (guix packages) #:use-module (gnu packages) #:use-module (gnu services) - #:export (use-package-modules + #:export (use-machine-modules + use-package-modules use-service-modules use-system-modules)) @@ -45,6 +46,7 @@ (gnu system file-systems) (gnu bootloader) (gnu bootloader grub) + (gnu machine) (gnu system keyboard) (gnu system pam) (gnu system shadow) ; 'user-account' @@ -142,6 +144,10 @@ Try adding @code{(use-service-modules ~a)}.") (current-source-location)) hint))) +(define-syntax-rule (use-machine-modules module ...) + (try-use-modules package-module-hint + (gnu machine module) ...)) + (define-syntax-rule (use-package-modules module ...) (try-use-modules package-module-hint (gnu packages module) ...)) -- 2.22.0 From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 27 14:43:08 2019 Received: (at 36404) by debbugs.gnu.org; 27 Jun 2019 18:43:08 +0000 Received: from localhost ([127.0.0.1]:39669 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZMS-00043z-By for submit@debbugs.gnu.org; Thu, 27 Jun 2019 14:43:08 -0400 Received: from mx.sdf.org ([205.166.94.20]:51205) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZMR-00043r-1T for 36404@debbugs.gnu.org; Thu, 27 Jun 2019 14:43:07 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5RIh5l6024493 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO) for <36404@debbugs.gnu.org>; Thu, 27 Jun 2019 18:43:06 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: 36404@debbugs.gnu.org Subject: [PATCH 5/6] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> <87a7e2an3h.fsf_-_@sdf.lonestar.org> <875zoqan2e.fsf_-_@sdf.lonestar.org> Date: Thu, 27 Jun 2019 14:41:29 -0400 In-Reply-To: <875zoqan2e.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Thu, 27 Jun 2019 14:40:57 -0400") Message-ID: <871rzean1i.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 2019-06-26 Jakob L. Kreuze * guix/scripts/deploy.scm: Add on-line help and limit verbosity. --- guix/scripts/deploy.scm | 52 ++++++++++++++++++++++++++--------------- 1 file changed, 33 insertions(+), 19 deletions(-) diff --git a/guix/scripts/deploy.scm b/guix/scripts/deploy.scm index 0be279642b..c52434f518 100644 --- a/guix/scripts/deploy.scm +++ b/guix/scripts/deploy.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze ;;; ;;; This file is part of GNU Guix. ;;; @@ -18,17 +19,35 @@ =20 (define-module (guix scripts deploy) #:use-module (gnu machine) - #:use-module (guix ui) #:use-module (guix scripts) #:use-module (guix scripts build) #:use-module (guix store) + #:use-module (guix ui) #:use-module (ice-9 format) #:use-module (srfi srfi-1) #:use-module (srfi srfi-37) #:export (guix-deploy)) =20 +;;; Commentary: +;;; +;;; This program provides a command-line interface to (gnu machine), allow= ing +;;; users to perform remote deployments through specification files. +;;; +;;; Code: + + + (define (show-help) - (display (G_ "Usage: guix deploy WHATEVER\n"))) + (display (G_ "Usage: guix deploy [OPTION] FILE... +Perform the deployment specified by FILE.\n")) + (show-build-options-help) + (newline) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) =20 (define %options (cons* (option '(#\h "help") #f #f @@ -42,13 +61,11 @@ (substitutes? . #t) (build-hook? . #t) (graft? . #t) - (print-build-trace? . #t) - (print-extended-build-trace? . #t) - (multiplexed-build-output? . #t) (debug . 0) (verbosity . 2))) =20 (define (load-source-file file) + "Load FILE as a user module." (let ((module (make-user-module '()))) (load* file module))) =20 @@ -58,19 +75,16 @@ (let* ((opts (parse-command-line args %options (list %default-options) #:argument-handler handle-argument)) (file (assq-ref opts 'file)) - (machines (load-source-file file))) + (machines (or (and file (load-source-file file)) '()))) (with-store store (set-build-options-from-command-line store opts) - ;; Build all the OSes and create a mapping from machine to OS deriva= tion - ;; for use in the deploy step. - (let ((osdrvs (map (lambda (machine) - (format #t "building ~a... " (machine-display-n= ame machine)) - (let ((osdrv (run-with-store store (build-machi= ne machine)))) - (display "done\n") - (cons machine osdrv))) - machines))) - (for-each (lambda (machine) - (format #t "deploying to ~a... " (machine-display-name= machine)) - (run-with-store store (deploy-machine machine)) - (display "done\n")) - machines))))) + (for-each (lambda (machine) + (format #t "building ~a... " (machine-display-name machi= ne)) + (run-with-store store (build-machine machine)) + (display "done\n")) + machines) + (for-each (lambda (machine) + (format #t "deploying to ~a... " (machine-display-name m= achine)) + (run-with-store store (deploy-machine machine)) + (display "done\n")) + machines)))) --=20 2.22.0 From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 27 14:43:59 2019 Received: (at 36404) by debbugs.gnu.org; 27 Jun 2019 18:43:59 +0000 Received: from localhost ([127.0.0.1]:39672 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZNG-00045J-Q6 for submit@debbugs.gnu.org; Thu, 27 Jun 2019 14:43:59 -0400 Received: from mx.sdf.org ([205.166.94.20]:51018) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgZNE-000457-Tn for 36404@debbugs.gnu.org; Thu, 27 Jun 2019 14:43:57 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5RIhtPR004216 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO) for <36404@debbugs.gnu.org>; Thu, 27 Jun 2019 18:43:56 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: 36404@debbugs.gnu.org Subject: [PATCH 6/6] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> <87a7e2an3h.fsf_-_@sdf.lonestar.org> <875zoqan2e.fsf_-_@sdf.lonestar.org> <871rzean1i.fsf_-_@sdf.lonestar.org> Date: Thu, 27 Jun 2019 14:42:20 -0400 In-Reply-To: <871rzean1i.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Thu, 27 Jun 2019 14:41:29 -0400") Message-ID: <87woh698fn.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 2019-06-26 Jakob L. Kreuze doc/guix.texi: Add section "Invoking guix deploy". --- doc/guix.texi | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index f0d148ace0..948767d8c8 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -81,6 +81,7 @@ Documentation License''. * guix gc: (guix)Invoking guix gc. Reclaiming unused disk space. * guix pull: (guix)Invoking guix pull. Update the list of available packages. * guix system: (guix)Invoking guix system. Manage the operating system configuration. +* guix deploy: (guix)Invoking guix deploy. Manage operating system configurations for remote hosts. @end direntry @dircategory Software development @@ -269,6 +270,7 @@ System Configuration * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remote host. * Running Guix in a VM:: How to run Guix System in a virtual machine. * Defining Services:: Adding new service definitions. @@ -10303,6 +10305,7 @@ instance to support new system services. * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remote host. * Running Guix in a VM:: How to run Guix System in a virtual machine. * Defining Services:: Adding new service definitions. @end menu @@ -25399,6 +25402,106 @@ example graph. @end table +@node Invoking guix deploy +@section Invoking @code{guix deploy} + +In addition to managing a machine's configuration locally through operating +system declarations, Guix also provides the ability to managing multiple remote +hosts as a logical ``deployment''. This is done using @command{guix deploy}. + +@example +guix deploy @var{file} +@end example + +Such an invocation will deploy the machines that the code within @var{file} +evaluates to. As an example, @var{file} might contain a definition like this: + +@example +;; This is a Guix deployment of a "bare bones" setup, with +;; no X11 display server, to a machine with an SSH daemon +;; listening on localhost:2222. A configuration such as this +;; may be appropriate for virtual machine with ports +;; forwarded to the host's loopback interface. + +(use-modules (gnu) (guix)) +(use-machine-modules ssh) +(use-service-modules networking ssh) +(use-package-modules bootloaders) + +(define %system + (operating-system + (host-name "gnu-deployed") + (timezone "Etc/UTC") + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (target "/dev/vda") + (terminal-outputs '(console)))) + (file-systems (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + %base-file-systems)) + (services + (append (list (service dhcp-client-service-type) + (service openssh-service-type + (openssh-configuration + (permit-root-login #t) + (allow-empty-passwords? #t)))) + %base-services)))) + +(list (machine + (system %system) + (environment 'managed-host) + (configuration (machine-ssh-configuration + (host-name "localhost") + (identity "./id_rsa") + (port 2222))))) +@end example + +The file should evaluate to a list of machines, rather than just one. This +example, upon being deployed, will create a new generation on the remote system +realizing the operating-system configuration @var{%system}. @var{environment} +and @var{configuration} specify how the machine should be provisioned--that is, +deployment and management of computing resources. The above example does not +provision any resources -- a @code{'managed-host} is a machine that is already +up and running the Guix system. A more complex deployment may involve +i.e. starting virtual machines through a VPS provider, however, in which case a +different @var{environment} types would be used. + +@deftp {Data Type} machine +This is the data type representing a single machine in a heterogeneous Guix +deployment. + +@table @asis +@item @code{system} +The object of the operating system configuration to deploy. + +@item @code{environment} +A symbol describing how the machine should be provisioned. At the moment, only +the only supported value is @code{'managed-host}. + +@item @code{configuration} (default: @code{#f}) +An object describing the configuration for the machine's @code{environment}. If +the @code{environment} has a default configuration, @code{#f} can be used. If +@code{#f} is used for an environment with no default configuration, however, an +error will be thrown. +@end table +@end deftp + +@deftp {Data Type} machine-ssh-configuration +This is the data type representing the SSH client parameters for connecting to a +@code{'managed-host}. + +@table @asis +@item @code{host-name} +@item @code{port} (default: @code{22}) +@item @code{user} (default: @code{"root"}) +@item @code{identity} (default: @code{#f}) +If specified, the path to the SSH private key to use to authenticate with the +remote host. +@end table +@end deftp + @node Running Guix in a VM @section Running Guix in a Virtual Machine -- 2.22.0 From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 27 16:06:09 2019 Received: (at 36404) by debbugs.gnu.org; 27 Jun 2019 20:06:09 +0000 Received: from localhost ([127.0.0.1]:39798 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgaen-0004cJ-HU for submit@debbugs.gnu.org; Thu, 27 Jun 2019 16:06:09 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:50213) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgaeg-0004bZ-TX for 36404@debbugs.gnu.org; Thu, 27 Jun 2019 16:06:08 -0400 Received: by mail-wm1-f68.google.com with SMTP id c66so6912820wmf.0 for <36404@debbugs.gnu.org>; Thu, 27 Jun 2019 13:06:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=worcester-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=4FIrZ5nCS/xYxIysvOpkcu6DPkRaszjNoh+gymeFmvo=; b=nvdCz8DnpiKTuiiophGLTZvCNDejANT6n+pCfggMfQy1RJ8QserUdp6AzTJYlwnAS5 pzc18b5CfJ5tZcIrWzRlkhHZEXl7rysJaCRKVSnwGLS5Wfy82cCQz3l2QQsRVbg4XWjz CjE1h3Ao7usMrrxn1w8Wy8jPBjUGzVaBloHiX+i62qtND536Ydxq3CsRhK3y5czz36xI ElqPLPG+rAdZJfLle94tZVqIkSlxij1NkOACL8tusyHsYUse4fNmClVIxVem8+GLJYwA XbNQskRymhMt/fSCPtupkRvFWLunuRi5s+lAl58TaRnyHmYYgUyre+8Ejx/8lLHhgF6t WeBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=4FIrZ5nCS/xYxIysvOpkcu6DPkRaszjNoh+gymeFmvo=; b=WXFB/dGQZH5+3V3Y2Y9uDMyBUE/MEWCBEs6MyFZA5WLYtaPv7suhnwybF29e6Nw5q0 OIx2S3q93Ug4Nt0DRb4sM+F6hBSd6XRi9xi015N49QnGEifkcyrBDdjGa/t0D1dA7Lc4 mz1/vww6RVYIn07rDC2Ti+rS1I+3Zum8ZLS9N3Qr6/xRCvDBplSuDK8ZCrzu2n9ZkwHA OZAhaSOr+QaMc0r8M1gmcLtlgxZl4KagIwwJmJZpV333/4rjwnk+2GtwMFa1qcubQibF MQYMegrIswtDtJQZfrzYdMg5otX0Uh740zkayNG9+ToOflK3lOX98BnL6+4d8xBG+yp6 CjOA== X-Gm-Message-State: APjAAAUJhjlRok+fFycf9vpnv74+l44mPGKr6uHhnQbhpyhu0brDuCus 4c1zoVAdY+iHtCPnQlm3hNZUuTFli6GOc+2Mi57F7Q== X-Google-Smtp-Source: APXvYqxUzBxgDpWoG0Orb1g6tVsrMKDXJl77SnBd49Rf7ufMolI8dm5DWVD3DkQ5JeR9H2INo/2D/0JsB2adw/ywtX4= X-Received: by 2002:a05:600c:240e:: with SMTP id 14mr4239334wmp.30.1561665956936; Thu, 27 Jun 2019 13:05:56 -0700 (PDT) MIME-Version: 1.0 References: <87o92ianbj.fsf@sdf.lonestar.org> In-Reply-To: <87o92ianbj.fsf@sdf.lonestar.org> From: "Thompson, David" Date: Thu, 27 Jun 2019 16:05:46 -0400 Message-ID: Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. To: "Jakob L. Kreuze" Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Jakob, On Thu, Jun 27, 2019 at 2:38 PM Jakob L. Kreuze wrote: > > Hello, Guix! > > This patch provides the basis for 'guix deploy', implementing what I've > referred to as the "simple case" in my progress reports: in-place > updates to machines (physical or virtual) whose name and IP address we > know well. First of all: Wooooooooooooooo!!!!!!!!!!!!!! This is a huge first step! Second of all: Could you please squash these 5 commits into one commit? No one needs to review my WIP code that uses GOOPS that later gets dropped in one of your commits. :) Thanks! - Dave From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 28 09:36:00 2019 Received: (at 36404) by debbugs.gnu.org; 28 Jun 2019 13:36:00 +0000 Received: from localhost ([127.0.0.1]:40579 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr2m-00030E-0V for submit@debbugs.gnu.org; Fri, 28 Jun 2019 09:36:00 -0400 Received: from mx.sdf.org ([205.166.94.20]:52919) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr2h-0002zv-5n for 36404@debbugs.gnu.org; Fri, 28 Jun 2019 09:35:57 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5SDZn62025951 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 28 Jun 2019 13:35:53 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: "Thompson\, David" Subject: [PATCH 0/5] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> Date: Fri, 28 Jun 2019 09:34:11 -0400 In-Reply-To: (David Thompson's message of "Thu, 27 Jun 2019 16:05:46 -0400") Message-ID: <87imspj0ks.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hey Dave, Thanks for the initial feedback. I squashed that first commit of yours and used the opportunity to move the addition of 'deploy.scm' into the "Add 'guix deploy' commit". Here's the cleaned up patch set. Jakob L. Kreuze (5): ssh: Add 'identity' keyword to 'open-ssh-session'. gnu: Add machine type for deployment specifications. Add 'guix deploy'. Export the (gnu machine) interface. doc: Add section for 'guix deploy'. Makefile.am | 4 +- doc/guix.texi | 103 +++++++++ gnu.scm | 8 +- gnu/local.mk | 5 +- gnu/machine.scm | 89 ++++++++ gnu/machine/ssh.scm | 355 +++++++++++++++++++++++++++++++ guix/scripts/deploy.scm | 90 ++++++++ guix/ssh.scm | 3 +- tests/machine.scm | 450 ++++++++++++++++++++++++++++++++++++++++ 9 files changed, 1103 insertions(+), 4 deletions(-) create mode 100644 gnu/machine.scm create mode 100644 gnu/machine/ssh.scm create mode 100644 guix/scripts/deploy.scm create mode 100644 tests/machine.scm =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0WF1MACgkQ9Qb9Fp2P 2VoOOA//WSvYrq04AT/GXkXj00opDUmsB9cOeFFn9sSZFughYJJHhk1OpxowUzGV d9bfhtqGrBhKT1wAfMqut/6iokL34cF/BBu1y9UsimNiQzRMeFecJ7V0f3KgR/3M /g+D4Y6qRgODR/C4JpvDT8xK7SkKEXMUn1TdhYSLfOdNykg7+7ncHXv74lOuq7Jj O2zmnHh7HJ4x7xekt5VVnAABSDDGdsWBPz+QEGqVX4te5jbt/t+SoF2yT9iQQwKm zrxp/FaehJGM9pu46ynO4mzM6ibuLCJMi5UGBAhsFlPyyYSPBofiKduEQEN3reej 2LWjGCzswVOqQvnMC2QhWIMUGFBx39zrCUO1EPs78yiwHU4S2Iyu9keSnwp0pjyQ MhYCzCp83ybVnXChd44d5S01jGj1mV3qqkbOmYWRXe60hfBN7zZd15nE1AUiikK5 8Ua0nKVKHOm3N1cx5M+4UGX1A4aa0K2K83UkxDMjMQx5/+fpraI1yfyO/WixsUV+ MoSJGnSDkXo5bQ1kII23ISzq4udVpvzaGs3C7VPy6lXkhlqXdo2oiKEXpWf8mNXS O2gW481rt8SggFoglnTwEYR9ZSX7fWC1xcintX0GbLBkECWa3qeNrcGy0buhqnHL DKRxTtMoH3jJiv+X11q/YfXnlTILotsrQQShcG4VXkVZ5DgdX1s= =Alcz -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 28 09:36:48 2019 Received: (at 36404) by debbugs.gnu.org; 28 Jun 2019 13:36:48 +0000 Received: from localhost ([127.0.0.1]:40583 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr3Y-00031d-B3 for submit@debbugs.gnu.org; Fri, 28 Jun 2019 09:36:48 -0400 Received: from mx.sdf.org ([205.166.94.20]:52565) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr3W-00031U-2v for 36404@debbugs.gnu.org; Fri, 28 Jun 2019 09:36:46 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5SDaiij012381 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 28 Jun 2019 13:36:45 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: "Thompson\, David" Subject: [PATCH 1/5] ssh: Add 'identity' keyword to 'open-ssh-session'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> Date: Fri, 28 Jun 2019 09:35:06 -0400 In-Reply-To: <87imspj0ks.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 28 Jun 2019 09:34:11 -0400") Message-ID: <87ef3dj0j9.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * guix/ssh.scm (open-ssh-session): Add 'identity' keyword argument. =2D-- guix/ssh.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/guix/ssh.scm b/guix/ssh.scm index 9b9baf54ea..a2387564a4 100644 =2D-- a/guix/ssh.scm +++ b/guix/ssh.scm @@ -57,12 +57,13 @@ (define %compression "zlib@openssh.com,zlib") =20 =2D(define* (open-ssh-session host #:key user port +(define* (open-ssh-session host #:key user port identity (compression %compression)) "Open an SSH session for HOST and return it. When USER and PORT are #f,= use default values or whatever '~/.ssh/config' specifies; otherwise use them. Throw an error on failure." (let ((session (make-session #:user user + #:identity identity #:host host #:port port #:timeout 10 ;seconds =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0WF4oACgkQ9Qb9Fp2P 2VrRcA//bcOgim0LcMr9SgMA7NaRIOF6JgUtHYB2JyaHBa2s5mmnhZsXHRokRh/r DLCZ5fDpQBVvMDI28KHN8AO72C6aYz7B+PikbfNfTHf2a1yN13awEEaoMjJDQLxs Oycdz93g80TOsEaWVDR/rsqO2eTm+L5hrtT0rnLaPBZzdeWuY4T2Ky3cA1Id7ADx bttnisgLgS1b3PWbRF7fZ3BSarKCL/XHsCyWw7/RW+kHhTcAb3+ccj3fIlXlwgFY rVmLWdM3gODbuYZN2shYCH8FdYIyobxSP0qlL28psHVWnPTgXAcc6gIhcp3dedB/ IEVoBaOEWOQHuUveVeNA8rhpqVKaG4nMkPC0l4PpAJMbSoVg0bP12VQNFbmaX64U 8nLeWc/zoHCmyG8tibhb2oXW1vIIr3rer5u1DkRPVoNIMK7SgN4Mrg3YhNsimKJw S7SqHYhzVNPTr1g7uWlE2lAJEsHIxYBz3nfgJSTEbFn4SoSWWf1bXuFUEGFx5OOq Ivo/qnGX31+GQ/wMHyZ/g2WqsUh2uSmutIcIiLVVGkPkcall6hcnXHBlsrrEzeIE dxXJMJ1SrxYyNr1RfttoxYrxfTp1DhcHIyP7cbAyu4O7AuSr2UwWlHjR3cmH/6Vj qDR485hfLftTU0eRcLsuV2JmSW71Pnyz9H2JXbYlkXGPST45wH0= =jxdx -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 28 09:37:45 2019 Received: (at 36404) by debbugs.gnu.org; 28 Jun 2019 13:37:45 +0000 Received: from localhost ([127.0.0.1]:40589 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr4M-00033B-Ma for submit@debbugs.gnu.org; Fri, 28 Jun 2019 09:37:44 -0400 Received: from mx.sdf.org ([205.166.94.20]:52317) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr4H-000330-Cu for 36404@debbugs.gnu.org; Fri, 28 Jun 2019 09:37:37 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5SDbVkM017910 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 28 Jun 2019 13:37:32 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: "Thompson\, David" Subject: [PATCH 2/5] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> Date: Fri, 28 Jun 2019 09:35:53 -0400 In-Reply-To: <87ef3dj0j9.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 28 Jun 2019 09:35:06 -0400") Message-ID: <87a7e1j0hy.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable * gnu/machine.scm: New file. * gnu/machine/ssh.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. * tests/machine.scm: New file. * Makefile.am (SCM_TESTS): Add it. =2D-- Makefile.am | 3 +- gnu/local.mk | 5 +- gnu/machine.scm | 89 +++++++++ gnu/machine/ssh.scm | 355 ++++++++++++++++++++++++++++++++++ tests/machine.scm | 450 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 900 insertions(+), 2 deletions(-) create mode 100644 gnu/machine.scm create mode 100644 gnu/machine/ssh.scm create mode 100644 tests/machine.scm diff --git a/Makefile.am b/Makefile.am index 80be73e4bf..9156554635 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -423,7 +423,8 @@ SCM_TESTS =3D \ tests/import-utils.scm \ tests/store-database.scm \ tests/store-deduplication.scm \ =2D tests/store-roots.scm + tests/store-roots.scm \ + tests/machine.scm =20 SH_TESTS =3D \ tests/guix-build.sh \ diff --git a/gnu/local.mk b/gnu/local.mk index f5d53b49b8..ad87de5ea7 100644 =2D-- a/gnu/local.mk +++ b/gnu/local.mk @@ -564,6 +564,9 @@ GNU_SYSTEM_MODULES =3D \ %D%/system/uuid.scm \ %D%/system/vm.scm \ \ + %D%/machine.scm \ + %D%/machine/ssh.scm \ + \ %D%/build/accounts.scm \ %D%/build/activation.scm \ %D%/build/bootloader.scm \ @@ -629,7 +632,7 @@ INSTALLER_MODULES =3D \ %D%/installer/newt/user.scm \ %D%/installer/newt/utils.scm \ %D%/installer/newt/welcome.scm \ =2D %D%/installer/newt/wifi.scm=09 + %D%/installer/newt/wifi.scm =20 # Always ship the installer modules but compile them only when # ENABLE_INSTALLER is true. diff --git a/gnu/machine.scm b/gnu/machine.scm new file mode 100644 index 0000000000..900a2020dc =2D-- /dev/null +++ b/gnu/machine.scm @@ -0,0 +1,89 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu machine) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix store) + #:export (machine + machine? + this-machine + + machine-system + machine-environment + machine-configuration + machine-display-name + + build-machine + deploy-machine + remote-eval)) + +;;; Commentary: +;;; +;;; This module provides the types used to declare individual machines in a +;;; heterogeneous Guix deployment. The interface allows users of specify s= ystem +;;; configurations and the means by which resources should be provisioned = on a +;;; per-host basis. +;;; +;;; Code: + +(define-record-type* machine + make-machine + machine? + this-machine + (system machine-system) ; + (environment machine-environment) ; symbol + (configuration machine-configuration ; configuration object + (default #f))) ; specific to environment + +(define (machine-display-name machine) + "Return the host-name identifying MACHINE." + (operating-system-host-name (machine-system machine))) + +(define (build-machine machine) + "Monadic procedure that builds the system derivation for MACHINE and ret= urning +a list containing the path of the derivation file and the path of the deri= vation +output." + (let ((os (machine-system machine))) + (mlet* %store-monad ((osdrv (operating-system-derivation os)) + (_ ((store-lift build-derivations) (list osdrv)))) + (return (list (derivation-file-name osdrv) + (derivation->output-path osdrv)))))) + +(define (remote-eval machine exp) + "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP refe= rs to +are built and deployed to MACHINE beforehand." + (case (machine-environment machine) + ((managed-host) + ((@@ (gnu machine ssh) remote-eval) machine exp)) + (else + (let ((type (machine-environment machine))) + (error "unsupported environment type" type))))) + +(define (deploy-machine machine) + "Monadic procedure transferring the new system's OS closure to the remote +MACHINE, activating it on MACHINE and switching MACHINE to the new generat= ion." + (case (machine-environment machine) + ((managed-host) + ((@@ (gnu machine ssh) deploy-machine) machine)) + (else + (let ((type (machine-environment machine))) + (error "unsupported environment type" type))))) diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm new file mode 100644 index 0000000000..a8f946e19f =2D-- /dev/null +++ b/gnu/machine/ssh.scm @@ -0,0 +1,355 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu machine ssh) + #:use-module (gnu bootloader) + #:use-module (gnu machine) + #:autoload (gnu packages gnupg) (guile-gcrypt) + #:use-module (gnu services) + #:use-module (gnu services shepherd) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix gexp) + #:use-module (guix modules) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix ssh) + #:use-module (guix store) + #:use-module (ice-9 match) + #:use-module (srfi srfi-19) + #:export (machine-ssh-configuration + machine-ssh-configuration? + machine-ssh-configuration + + machine-ssh-configuration-host-name + machine-ssh-configuration-port + machine-ssh-configuration-user + machine-ssh-configuration-session)) + +;;; Commentary: +;;; +;;; This module implements remote evaluation and system deployment for +;;; machines that are accessable over SSH and have a known host-name. In t= he +;;; sense of the broader "machine" interface, we describe the environment = for +;;; such machines as 'managed-host. +;;; +;;; Code: + + +;;; +;;; SSH client parameter configuration. +;;; + +(define-record-type* machine-ssh-configuration + make-machine-ssh-configuration + machine-ssh-configuration? + this-machine-ssh-configuration + (host-name machine-ssh-configuration-host-name) ; string + (port machine-ssh-configuration-port ; integer + (default 22)) + (user machine-ssh-configuration-user ; string + (default "root")) + (identity machine-ssh-configuration-identity ; path to a private key + (default #f)) + (session machine-ssh-configuration-session ; session + (default #f))) + +(define (machine-ssh-session machine) + "Return the SSH session that was given in MACHINE's configuration, or cr= eate +one from the configuration's parameters if one was not provided." + (let ((config (machine-configuration machine))) + (if (machine-ssh-configuration? config) + (or (machine-ssh-configuration-session config) + (let ((host-name (machine-ssh-configuration-host-name config)) + (user (machine-ssh-configuration-user config)) + (port (machine-ssh-configuration-port config)) + (identity (machine-ssh-configuration-identity config))) + (open-ssh-session host-name + #:user user + #:port port + #:identity identity))) + (error "unsupported configuration type")))) + + +;;; +;;; Remote evaluation. +;;; + +(define (remote-eval machine exp) + "Internal implementation of 'remote-eval' for MACHINE instances with an +environment type of 'managed-host." + (unless (machine-configuration machine) + (error (format #f (G_ "no configuration specified for machine of envir= onment '~a'") + (symbol->string (machine-environment machine))))) + ((@ (guix remote) remote-eval) exp (machine-ssh-session machine))) + + +;;; +;;; System deployment. +;;; + +(define (switch-to-system machine) + "Monadic procedure creating a new generation on MACHINE and execute the +activation script for the new system configuration." + (define (remote-exp drv script) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles) + (guix utils))) + #~(begin + (use-modules (guix config) + (guix profiles) + (guix utils)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (let* ((system #$(derivation->output-path drv)) + (number (1+ (generation-number %system-profile))) + (generation (generation-file-name %system-profile numbe= r)) + (old-env (environ)) + (old-path %load-path) + (old-cpath %load-compiled-path)) + (switch-symlinks generation system) + (switch-symlinks %system-profile generation) + ;; Guard against the activation script modifying $PATH. + (dynamic-wind + (const #t) + (lambda () + (setenv "GUIX_NEW_SYSTEM" system) + ;; Guard against the activation script modifying '%load-= path'. + (dynamic-wind + (const #t) + (lambda () + ;; The activation script may write to stdout, which + ;; confuses 'remote-eval' when it attempts to read a + ;; result from the remote REPL. We work around this = by + ;; forcing the output to a string. + (with-output-to-string + (lambda () + (primitive-load #$script)))) + (lambda () + (set! %load-path old-path) + (set! %load-compiled-path old-cpath)))) + (lambda () + (environ old-env)))))))) + + (let* ((os (machine-system machine)) + (script (operating-system-activation-script os))) + (mlet* %store-monad ((drv (operating-system-derivation os))) + (remote-eval machine (remote-exp drv script))))) + +(define (upgrade-shepherd-services machine) + "Monadic procedure unloading and starting services on the remote as need= ed +to realize the MACHINE's system configuration." + (define target-services + ;; Monadic expression evaluating to a list of (name output-path) pairs= for + ;; all of MACHINE's services. + (mapm %store-monad + (lambda (service) + (mlet %store-monad ((file ((compose lower-object + shepherd-service-file) + service))) + (return (list (shepherd-service-canonical-name service) + (derivation->output-path file))))) + (service-value + (fold-services (operating-system-services (machine-system machi= ne)) + #:target-type shepherd-root-service-type)))) + + (define (remote-exp target-services) + (with-imported-modules '((gnu services herd)) + #~(begin + (use-modules (gnu services herd) + (srfi srfi-1)) + + (define running + (filter live-service-running (current-services))) + + (define (essential? service) + ;; Return #t if SERVICE is essential and should not be unloaded + ;; under any circumstance. + (memq (first (live-service-provision service)) + '(root shepherd))) + + (define (obsolete? service) + ;; Return #t if SERVICE can be safely unloaded. + (and (not (essential? service)) + (every (lambda (requirements) + (not (memq (first (live-service-provision servic= e)) + requirements))) + (map live-service-requirement running)))) + + (define to-unload + (filter obsolete? + (remove (lambda (service) + (memq (first (live-service-provision service= )) + (map first '#$target-services))) + running))) + + (define to-start + (remove (lambda (service-pair) + (memq (first service-pair) + (map (compose first live-service-provision) + running))) + '#$target-services)) + + ;; Unload obsolete services. + (for-each (lambda (service) + (false-if-exception + (unload-service service))) + to-unload) + + ;; Load the service files for any new services and start them. + (load-services/safe (map second to-start)) + (for-each start-service (map first to-start)) + + #t))) + + (mlet %store-monad ((target-services target-services)) + (remote-eval machine (remote-exp target-services)))) + +(define (machine-boot-parameters machine) + "Monadic procedure returning a list of 'boot-parameters' for the generat= ions +of MACHINE's system profile, ordered from most recent to oldest." + (define bootable-kernel-arguments + (@@ (gnu system) bootable-kernel-arguments)) + + (define remote-exp + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles))) + #~(begin + (use-modules (guix config) + (guix profiles) + (ice-9 textual-ports)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (define (read-file path) + (call-with-input-file path + (lambda (port) + (get-string-all port)))) + + (map (lambda (generation) + (let* ((system-path (generation-file-name %system-profi= le + generation)) + (boot-parameters-path (string-append system-path + "/parameter= s")) + (time (stat:mtime (lstat system-path)))) + (list generation + system-path + time + (read-file boot-parameters-path)))) + (reverse (generation-numbers %system-profile))))))) + + (mlet* %store-monad ((generations (remote-eval machine remote-exp))) + (return + (map (lambda (generation) + (match generation + ((generation system-path time serialized-params) + (let* ((params (call-with-input-string serialized-params + read-boot-parameters)) + (root (boot-parameters-root-device params)) + (label (boot-parameters-label params))) + (boot-parameters + (inherit params) + (label + (string-append label " (#" + (number->string generation) ", " + (let ((time (make-time time-utc 0 time))) + (date->string (time-utc->date time) + "~Y-~m-~d ~H:~M")) + ")")) + (kernel-arguments + (append (bootable-kernel-arguments system-path root) + (boot-parameters-kernel-arguments params)))))))) + generations)))) + +(define (install-bootloader machine) + "Create a bootloader entry for the new system generation on MACHINE, and +configure the bootloader to boot that generation by default." + (define bootloader-installer-script + (@@ (guix scripts system) bootloader-installer-script)) + + (define (remote-exp installer bootcfg bootcfg-file) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((gnu build install) + (guix store) + (guix utils))) + #~(begin + (use-modules (gnu build install) + (guix store) + (guix utils)) + (let* ((gc-root (string-append "/" %gc-roots-directory "/bootc= fg")) + (temp-gc-root (string-append gc-root ".new")) + (old-path %load-path) + (old-cpath %load-compiled-path)) + (switch-symlinks temp-gc-root gc-root) + + (unless (false-if-exception + (begin + (install-boot-config #$bootcfg #$bootcfg-file "/") + ;; Guard against the activation script modifying + ;; '%load-path'. + (dynamic-wind + (const #t) + (lambda () + ;; The installation script may write to stdou= t, + ;; which confuses 'remote-eval' when it attem= pts to + ;; read a result from the remote REPL. We work + ;; around this by forcing the output to a str= ing. + (with-output-to-string + (lambda () + (primitive-load #$installer)))) + (lambda () + (set! %load-path old-path) + (set! %load-compiled-path old-cpath))))) + (delete-file temp-gc-root) + (error "failed to install bootloader")) + + (rename-file temp-gc-root gc-root) + #t))))) + + (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine))) + (let* ((os (machine-system machine)) + (bootloader ((compose bootloader-configuration-bootloader + operating-system-bootloader) + os)) + (bootloader-target (bootloader-configuration-target + (operating-system-bootloader os))) + (installer (bootloader-installer-script + (bootloader-installer bootloader) + (bootloader-package bootloader) + bootloader-target + "/")) + (menu-entries (map boot-parameters->menu-entry boot-parameters)) + (bootcfg (operating-system-bootcfg os menu-entries)) + (bootcfg-file (bootloader-configuration-file bootloader))) + (remote-eval machine (remote-exp installer bootcfg bootcfg-file))))) + +(define (deploy-machine machine) + "Internal implementation of 'deploy-machine' for MACHINE instances with = an +environment type of 'managed-host." + (unless (machine-configuration machine) + (error (format #f (G_ "no configuration specified for machine of envir= onment '~a'") + (symbol->string (machine-environment machine))))) + (mbegin %store-monad + (switch-to-system machine) + (upgrade-shepherd-services machine) + (install-bootloader machine))) diff --git a/tests/machine.scm b/tests/machine.scm new file mode 100644 index 0000000000..390c0189bb =2D-- /dev/null +++ b/tests/machine.scm @@ -0,0 +1,450 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu tests machine) + #:use-module (gnu bootloader grub) + #:use-module (gnu bootloader) + #:use-module (gnu build marionette) + #:use-module (gnu build vm) + #:use-module (gnu machine) + #:use-module (gnu machine ssh) + #:use-module (gnu packages bash) + #:use-module (gnu packages virtualization) + #:use-module (gnu services base) + #:use-module (gnu services networking) + #:use-module (gnu services ssh) + #:use-module (gnu services) + #:use-module (gnu system file-systems) + #:use-module (gnu system vm) + #:use-module (gnu system) + #:use-module (gnu tests) + #:use-module (guix derivations) + #:use-module (guix gexp) + #:use-module (guix monads) + #:use-module (guix pki) + #:use-module (guix store) + #:use-module (guix utils) + #:use-module (ice-9 ftw) + #:use-module (ice-9 match) + #:use-module (ice-9 textual-ports) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-64) + #:use-module (ssh auth) + #:use-module (ssh channel) + #:use-module (ssh key) + #:use-module (ssh session)) + + +;;; +;;; Virtual machine scaffolding. +;;; + +(define marionette-pid (@@ (gnu build marionette) marionette-pid)) + +(define (call-with-marionette path command proc) + "Invoke PROC with a marionette running COMMAND in PATH." + (let* ((marionette (make-marionette command #:socket-directory path)) + (pid (marionette-pid marionette))) + (dynamic-wind + (lambda () + (unless marionette + (error "could not start marionette"))) + (lambda () (proc marionette)) + (lambda () + (kill pid SIGTERM))))) + +(define (dir-join . components) + "Join COMPONENTS with `file-name-separator-string'." + (string-join components file-name-separator-string)) + +(define (call-with-machine-test-directory proc) + "Run PROC with the path to a temporary directory that will be cleaned up +when PROC returns. Only files that can be passed to 'delete-file' should be +created within the temporary directory; cleanup will not recurse into +subdirectories." + (let ((path (tmpnam))) + (dynamic-wind + (lambda () + (unless (mkdir path) + (error (format #f "could not create directory '~a'" path)))) + (lambda () (proc path)) + (lambda () + (let ((children (map first (cddr (file-system-tree path))))) + (for-each (lambda (child) + (false-if-exception + (delete-file (dir-join path child)))) + children) + (rmdir path)))))) + +(define (os-for-test os) + "Return an record derived from OS that is appropriate= for +use with 'qemu-image'." + (define file-systems-to-keep + ;; Keep only file systems other than root and not normally bound to re= al + ;; devices. + (remove (lambda (fs) + (let ((target (file-system-mount-point fs)) + (source (file-system-device fs))) + (or (string=3D? target "/") + (string-prefix? "/dev/" source)))) + (operating-system-file-systems os))) + + (define root-uuid + ;; UUID of the root file system. + ((@@ (gnu system vm) operating-system-uuid) os 'dce)) + + + (operating-system + (inherit os) + ;; Assume we have an initrd with the whole QEMU shebang. + + ;; Force our own root file system. Refer to it by UUID so that + ;; it works regardless of how the image is used ("qemu -hda", + ;; Xen, etc.). + (file-systems (cons (file-system + (mount-point "/") + (device root-uuid) + (type "ext4")) + file-systems-to-keep)))) + +(define (qemu-image-for-test os) + "Return a derivation producing a QEMU disk image running OS. This proced= ure +is similar to 'system-qemu-image' in (gnu system vm), but makes use of +'os-for-test' so that callers may obtain the same system derivation that w= ill +be booted by the image." + (define root-uuid ((@@ (gnu system vm) operating-system-uuid) os 'dce)) + (let* ((os (os-for-test os)) + (bootcfg (operating-system-bootcfg os))) + (qemu-image #:os os + #:bootcfg-drv bootcfg + #:bootloader (bootloader-configuration-bootloader + (operating-system-bootloader os)) + #:disk-image-size (* 9000 (expt 2 20)) + #:file-system-type "ext4" + #:file-system-uuid root-uuid + #:inputs `(("system" ,os) + ("bootcfg" ,bootcfg)) + #:copy-inputs? #t))) + +(define (make-writable-image image) + "Return a derivation producing a script to create a writable disk image +overlay of IMAGE, writing the overlay to the the path given as a command-l= ine +argument to the script." + (define qemu-img-exec + #~(list (string-append #$qemu-minimal "/bin/qemu-img") + "create" "-f" "qcow2" + "-o" (string-append "backing_file=3D" #$image))) + + (define builder + #~(call-with-output-file #$output + (lambda (port) + (format port "#!~a~% exec ~a \"$@\"~%" + #$(file-append bash "/bin/sh") + (string-join #$qemu-img-exec " ")) + (chmod port #o555)))) + + (gexp->derivation "make-writable-image.sh" builder)) + +(define (run-os-for-test os) + "Return a derivation producing a script to run OS as a qemu guest, whose +first argument is the path to a writable disk image. Additional arguments = are +passed as-is to qemu." + (define kernel-arguments + #~(list "console=3DttyS0" + #+@(operating-system-kernel-arguments os "/dev/sda1"))) + + (define qemu-exec + #~(begin + (list (string-append #$qemu-minimal "/bin/" #$(qemu-command (%curr= ent-system))) + "-kernel" #$(operating-system-kernel-file os) + "-initrd" #$(file-append os "/initrd") + (format #f "-append ~s" + (string-join #$kernel-arguments " ")) + #$@(if (file-exists? "/dev/kvm") + '("-enable-kvm") + '()) + "-no-reboot" + "-net nic,model=3Dvirtio" + "-object" "rng-random,filename=3D/dev/urandom,id=3Dguixsd-vm= -rng" + "-device" "virtio-rng-pci,rng=3Dguixsd-vm-rng" + "-vga" "std" + "-m" "256" + "-net" "user,hostfwd=3Dtcp::2222-:22"))) + + (define builder + #~(call-with-output-file #$output + (lambda (port) + (format port "#!~a~% exec ~a -drive \"file=3D$@\"~%" + #$(file-append bash "/bin/sh") + (string-join #$qemu-exec " ")) + (chmod port #o555)))) + + (gexp->derivation "run-vm.sh" builder)) + +(define (scripts-for-test os) + "Build and return a list containing the paths of: + +- A script to make a writable disk image overlay of OS. +- A script to run that disk image overlay as a qemu guest." + (let ((virtualized-os (os-for-test os))) + (mlet* %store-monad ((osdrv (operating-system-derivation virtualized-o= s)) + (imgdrv (qemu-image-for-test os)) + + ;; Ungexping 'imgdrv' or 'osdrv' will result in an + ;; error if the derivations don't exist in the st= ore, + ;; so we ensure they're built prior to invoking + ;; 'run-vm' or 'make-image'. + (_ ((store-lift build-derivations) (list imgdrv))) + + (run-vm (run-os-for-test virtualized-os)) + (make-image + (make-writable-image (derivation->output-path im= gdrv)))) + (mbegin %store-monad + ((store-lift build-derivations) (list imgdrv make-image run-vm)) + (return (list (derivation->output-path make-image) + (derivation->output-path run-vm))))))) + +(define (call-with-marionette-and-session os proc) + "Construct a marionette backed by OS in a temporary test environment and +invoke PROC with two arguments: the marionette object, and an SSH session +connected to the marionette." + (call-with-machine-test-directory + (lambda (path) + (match (with-store store + (run-with-store store + (scripts-for-test %system))) + ((make-image run-vm) + (let ((image (dir-join path "image"))) + ;; Create the writable image overlay. + (system (string-join (list make-image image) " ")) + (call-with-marionette + path + (list run-vm image) + (lambda (marionette) + ;; XXX: The guest clearly has (gcrypt pk-crypto) since this + ;; works, but trying to import it from 'marionette-eval' fail= s as + ;; the Marionette REPL does not have 'guile-gcrypt' in its + ;; %load-path. + (marionette-eval + `(begin + (use-modules (ice-9 popen)) + (let ((port (open-pipe* OPEN_WRITE "guix" "archive" "--au= thorize"))) + (put-string port ,%signing-key) + (close port))) + marionette) + ;; XXX: This is an absolute hack to work around potential qui= rks + ;; in the operating system. For one, we invoke 'herd' from the + ;; command-line to ensure that the Shepherd socket file + ;; exists. Second, we enable 'ssh-daemon', as there's a chance + ;; the service will be disabled upon booting the image. + (marionette-eval + `(system "herd enable ssh-daemon") + marionette) + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (start-service 'ssh-daemon)) + marionette) + (call-with-connected-session/auth + (lambda (session) + (proc marionette session))))))))))) + + +;;; +;;; SSH session management. These are borrowed from (gnu tests ssh). +;;; + +(define (make-session-for-test) + "Make a session with predefined parameters for a test." + (make-session #:user "root" + #:port 2222 + #:host "localhost")) + +(define (call-with-connected-session proc) + "Call the one-argument procedure PROC with a freshly created and +connected SSH session object, return the result of the procedure call. The +session is disconnected when the PROC is finished." + (let ((session (make-session-for-test))) + (dynamic-wind + (lambda () + (let ((result (connect! session))) + (unless (equal? result 'ok) + (error "Could not connect to a server" + session result)))) + (lambda () (proc session)) + (lambda () (disconnect! session))))) + +(define (call-with-connected-session/auth proc) + "Make an authenticated session. We should be able to connect as +root with an empty password." + (call-with-connected-session + (lambda (session) + ;; Try the simple authentication methods. Dropbear requires + ;; 'none' when there are no passwords, whereas OpenSSH accepts + ;; 'password' with an empty password. + (let loop ((methods (list (cut userauth-password! <> "") + (cut userauth-none! <>)))) + (match methods + (() + (error "all the authentication methods failed")) + ((auth rest ...) + (match (pk 'auth (auth session)) + ('success + (proc session)) + ('denied + (loop rest))))))))) + + +;;; +;;; Virtual machines for use in the test suite. +;;; + +(define %system + ;; A "bare bones" operating system running both an OpenSSH daemon and the + ;; "marionette" service. + (marionette-operating-system + (operating-system + (host-name "gnu") + (timezone "Etc/UTC") + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (target "/dev/sda") + (terminal-outputs '(console)))) + (file-systems (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + %base-file-systems)) + (services + (append (list (service dhcp-client-service-type) + (service openssh-service-type + (openssh-configuration + (permit-root-login #t) + (allow-empty-passwords? #t)))) + %base-services))) + #:imported-modules '((gnu services herd) + (guix combinators)))) + +(define %signing-key + ;; The host's signing key, encoded as a string. The "marionette" will re= ject + ;; any files signed by an unauthorized host, so we'll need to send this = key + ;; over and authorize it. + (call-with-input-file %public-key-file + (lambda (port) + (get-string-all port)))) + + +(test-begin "machine") + +(define (system-generations marionette) + (marionette-eval + '(begin + (use-modules (ice-9 ftw) + (srfi srfi-1)) + (let* ((profile-dir "/var/guix/profiles/") + (entries (map first (cddr (file-system-tree profile-dir))))) + (remove (lambda (entry) + (member entry '("per-user" "system"))) + entries))) + marionette)) + +(define (running-services marionette) + (marionette-eval + '(begin + (use-modules (gnu services herd) + (srfi srfi-1)) + (map (compose first live-service-provision) + (filter live-service-running (current-services)))) + marionette)) + +(define (count-grub-cfg-entries marionette) + (marionette-eval + '(begin + (define grub-cfg + (call-with-input-file "/boot/grub/grub.cfg" + (lambda (port) + (get-string-all port)))) + + (let loop ((n 0) + (start 0)) + (let ((index (string-contains grub-cfg "menuentry" start))) + (if index + (loop (1+ n) (1+ index)) + n)))) + marionette)) + +(define %target-system + (marionette-operating-system + (operating-system + (host-name "gnu-deployed") + (timezone "Etc/UTC") + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (target "/dev/sda") + (terminal-outputs '(console)))) + (file-systems (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + %base-file-systems)) + (services + (append (list (service tor-service-type) + (service dhcp-client-service-type) + (service openssh-service-type + (openssh-configuration + (permit-root-login #t) + (allow-empty-passwords? #t)))) + %base-services))) + #:imported-modules '((gnu services herd) + (guix combinators)))) + +(call-with-marionette-and-session + (os-for-test %system) + (lambda (marionette session) + (let ((generations-prior (system-generations marionette)) + (services-prior (running-services marionette)) + (grub-entry-count-prior (count-grub-cfg-entries marionette)) + (machine (machine + (system %target-system) + (environment 'managed-host) + (configuration (machine-ssh-configuration + (host-name "localhost") + (session session)))))) + (with-store store + (run-with-store store + (build-machine machine)) + (run-with-store store + (deploy-machine machine))) + (test-equal "deployment created new generation" + (length (system-generations marionette)) + (1+ (length generations-prior))) + (test-assert "deployment started new service" + (and (not (memq 'tor services-prior)) + (memq 'tor (running-services marionette)))) + (test-equal "deployment created new menu entry" + (count-grub-cfg-entries marionette) + ;; A Grub configuration that contains a single menu entry does not = have + ;; an "old configurations" submenu. Deployment, then, would result = in + ;; this submenu being created, meaning an additional two 'menuentry' + ;; fields rather than just one. + (if (=3D grub-entry-count-prior 1) + (+ 2 grub-entry-count-prior) + (1+ grub-entry-count-prior)))))) + +(test-end "machine") =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0WF7kACgkQ9Qb9Fp2P 2VobbBAAjMOTiVYJwUesE2pyDxI0vFG0DVKtsAgNAhYNV8h8i13zz3PiU+TwRoRc V71/1TpOtpyj0+8PLNz9H+6dstDovrnTSvFxG6eAgnjQdw7pjPKGnEiWyoRRZQau Y+Hv2Rkz3xovU7dIMxbXRUzMDAcaag0ZTpnLPeuI7scP3Du+gdQv1PBMLBSdK6tk 5BV4xcD4KPB4z7AaUoi12XEdp7nlNOEzt2ezcfV54MVrzGGSFCSNTgaVHOVIQvrU Ylbn/Som3rAz1VYQRX/cYz9/iWzthQDtZtcrgdoonhyz2fqK6ChpCopHsDp3T1RO fuqS38c4Z3cb1oH6OjNStCkgOyz4rZlexJLEknE39pXFxxCzIjCyQCO9GXkXE+iV qEJCKgzcyAM3tP+CWhneJEadO7/FlitBFmo1OGE3faxhJ5kROjvcVDC/8z9x0X+Y 0tU5vAzAtDGvNNTorpvVMWcerw1keFsKcGAugPzwjRKRyUM6EyTVjQiXGzmF1ufn pUv1LVTEVpObrObTQ5iZcbsJATSY3kAJc8gV63/Hh16s2c7wmlFdzB+7D56CqVsl MMmyFyPn/ngwMVFZ0+ubb4OOChIA0tjWLOyHkEUlybPlrSL0OkEIanEqQqvvx3c8 iAEk5LLCwTI8HplBQ9zwwEnNJG9fEe1nTrq0XuaewAnTjYRIgvw= =Meqf -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 28 09:38:16 2019 Received: (at 36404) by debbugs.gnu.org; 28 Jun 2019 13:38:16 +0000 Received: from localhost ([127.0.0.1]:40593 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr4y-00034R-4c for submit@debbugs.gnu.org; Fri, 28 Jun 2019 09:38:16 -0400 Received: from mx.sdf.org ([205.166.94.20]:52114) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr4w-00034J-H4 for 36404@debbugs.gnu.org; Fri, 28 Jun 2019 09:38:15 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5SDcCZq018119 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 28 Jun 2019 13:38:13 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: "Thompson\, David" Subject: [PATCH 3/5] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> Date: Fri, 28 Jun 2019 09:36:35 -0400 In-Reply-To: <87a7e1j0hy.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 28 Jun 2019 09:35:53 -0400") Message-ID: <875zopj0gs.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable * guix/scripts/deploy.scm: New file. * Makefile.am (MODULES): Add it. =2D-- Makefile.am | 1 + guix/scripts/deploy.scm | 90 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) create mode 100644 guix/scripts/deploy.scm diff --git a/Makefile.am b/Makefile.am index 9156554635..8dbc220489 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -266,6 +266,7 @@ MODULES =3D \ guix/scripts/weather.scm \ guix/scripts/container.scm \ guix/scripts/container/exec.scm \ + guix/scripts/deploy.scm \ guix.scm \ $(GNU_SYSTEM_MODULES) =20 diff --git a/guix/scripts/deploy.scm b/guix/scripts/deploy.scm new file mode 100644 index 0000000000..c52434f518 =2D-- /dev/null +++ b/guix/scripts/deploy.scm @@ -0,0 +1,90 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts deploy) + #:use-module (gnu machine) + #:use-module (guix scripts) + #:use-module (guix scripts build) + #:use-module (guix store) + #:use-module (guix ui) + #:use-module (ice-9 format) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-37) + #:export (guix-deploy)) + +;;; Commentary: +;;; +;;; This program provides a command-line interface to (gnu machine), allow= ing +;;; users to perform remote deployments through specification files. +;;; +;;; Code: + + + +(define (show-help) + (display (G_ "Usage: guix deploy [OPTION] FILE... +Perform the deployment specified by FILE.\n")) + (show-build-options-help) + (newline) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + %standard-build-options)) + +(define %default-options + '((system . ,(%current-system)) + (substitutes? . #t) + (build-hook? . #t) + (graft? . #t) + (debug . 0) + (verbosity . 2))) + +(define (load-source-file file) + "Load FILE as a user module." + (let ((module (make-user-module '()))) + (load* file module))) + +(define (guix-deploy . args) + (define (handle-argument arg result) + (alist-cons 'file arg result)) + (let* ((opts (parse-command-line args %options (list %default-options) + #:argument-handler handle-argument)) + (file (assq-ref opts 'file)) + (machines (or (and file (load-source-file file)) '()))) + (with-store store + (set-build-options-from-command-line store opts) + (for-each (lambda (machine) + (format #t "building ~a... " (machine-display-name machi= ne)) + (run-with-store store (build-machine machine)) + (display "done\n")) + machines) + (for-each (lambda (machine) + (format #t "deploying to ~a... " (machine-display-name m= achine)) + (run-with-store store (deploy-machine machine)) + (display "done\n")) + machines)))) =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0WF+MACgkQ9Qb9Fp2P 2VpK7w/+IZxmmZxX8HVz4ycAP+qkEZDrdyFxyceCURNI1kE8nBZJmj/e07r7nD4F DtvFdh680tneTyRsrcK6OYWEJBfHVRaw/Ak/TuKag1sCzSlQ48DijRDrPi25EsZJ dc8bYf2vTahjf27ElMNYTQ0ZM9LhHnUbBgPWqN/QxpJZrAdIXJUA1VUhWncK5LYH aJ4V+d7NPiUOz9opdvrf1+awrZnmATQfmA2HwkUuwEdyAyjaEGF1drPSGtCJSYyQ ec5xPPeOglDFDOqziqMZJ25hv5JmYEwIcnYCqSY7Il/bnz2yLiy9TAK8rmE0L9P3 Q/Hf2OrLzeOkC/peLEOyCCFaesSFNNQoJMnyNmRuOMzY2uWJBMID5kf05iwq3Jez FZeSiIc4PVjYFFhVIVt13NIjyFUW/m/0xCQjLex92puouc19umZbriK2aODA9uQM 6byLmf9xU1jR4m3/0CfhMj5s093Mv9LLG6AkfaNbJPmECT4JItnUTvMwUZkkN3d6 6ErnJ4v7iHu+Pv1lxvC5qfuRddvdxFFkzL0GvzIHkg5eXLcLlHanNnHfyFW3uTPu tZGXdGTCClFGqIF9JNohg2oBkn0kDXBRDFzcxHMJAR/xCKlkZ3nie4rGgu1Sog3P y/I4aJVOp6GB53y3jZcSVN6x/rcR6Jss9OGT3uI3Izho3V7jzkk= =InAb -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 28 09:38:50 2019 Received: (at 36404) by debbugs.gnu.org; 28 Jun 2019 13:38:50 +0000 Received: from localhost ([127.0.0.1]:40599 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr5V-00035O-UK for submit@debbugs.gnu.org; Fri, 28 Jun 2019 09:38:50 -0400 Received: from mx.sdf.org ([205.166.94.20]:51920) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr5U-00035H-IP for 36404@debbugs.gnu.org; Fri, 28 Jun 2019 09:38:48 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5SDckRG006713 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 28 Jun 2019 13:38:47 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: "Thompson\, David" Subject: [PATCH 4/5] Export the (gnu machine) interface. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <875zopj0gs.fsf_-_@sdf.lonestar.org> Date: Fri, 28 Jun 2019 09:37:09 -0400 In-Reply-To: <875zopj0gs.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 28 Jun 2019 09:36:35 -0400") Message-ID: <871rzdj0fu.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * gnu.scm (%public-modules): Add '(gnu machine)'. * gnu.scm (use-machine-modules): New macro. =2D-- gnu.scm | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/gnu.scm b/gnu.scm index 2c29b6dc3f..fa643a5b92 100644 =2D-- a/gnu.scm +++ b/gnu.scm @@ -27,7 +27,8 @@ #:use-module (guix packages) #:use-module (gnu packages) #:use-module (gnu services) =2D #:export (use-package-modules + #:export (use-machine-modules + use-package-modules use-service-modules use-system-modules)) =20 @@ -45,6 +46,7 @@ (gnu system file-systems) (gnu bootloader) (gnu bootloader grub) + (gnu machine) (gnu system keyboard) (gnu system pam) (gnu system shadow) ; 'user-account' @@ -142,6 +144,10 @@ Try adding @code{(use-service-modules ~a)}.") (current-source-location)) hint))) =20 +(define-syntax-rule (use-machine-modules module ...) + (try-use-modules package-module-hint + (gnu machine module) ...)) + (define-syntax-rule (use-package-modules module ...) (try-use-modules package-module-hint (gnu packages module) ...)) =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0WGAUACgkQ9Qb9Fp2P 2VrGRQ/7BfLsxyTcRd+hQfNxfxHjken4Ji8Bo2y/YuZq+VY11pJQRK4fL1S3fQ93 HiCufo1hS0YCcmlvEWMWxi/ljRnbaI58uR94YVQ7nVNyXmM+2EO78IQR2zqISRDg Qq+WWf7ZbPMBo3ZPXdD7NLkbdQnAdv+XhPEl1bmbz4NR6xsATs+H5gQPXXN9pxk/ b+fXl3glZtSobPJYYDsa6FhFAiUr2YOSfy1qePEjQy30d61DOKSB6nhnYK4otLca /HCGbK3Town11REebphuj+3lRHvfvLNF98nWoxpH5QVBjAOBkD6vFYkX32tTLAKb mXCOedeW7xp2wFMNGDDOap80hePullBfS9YjQiAeYUsOMDiYHbGj5Fo45s2eamiX ujQ7LsQWXt2eNDM5jZCvk02Bc2qMG6uovj0pdOOfKo0lnqJr2OB2k2nIoE3APKUz DqcqbVygDP8CJGAeLg5/LgR303a+wVu4FWeXSsJ+Yg+jcudRNsK+oA9Q7cvS1SFd 8bh3BAGoRV+s+k7NSXXF6SiOtQGNHH/KCesbPU5AxbuhANApgNdyBLnnhBtysixh eL/UsTmMMe26koG9rCR8dJcIZJAuRK28s2HyKzTLlgZ9rb9KyFWjlfJq7+4YUFoN LslzjO+f1Xrb9xNUPs+2H9hCRCpIczG2vznJVPNAxjVzFIUeHH8= =Luse -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 28 09:39:30 2019 Received: (at 36404) by debbugs.gnu.org; 28 Jun 2019 13:39:30 +0000 Received: from localhost ([127.0.0.1]:40605 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr6A-00036u-7k for submit@debbugs.gnu.org; Fri, 28 Jun 2019 09:39:30 -0400 Received: from mx.sdf.org ([205.166.94.20]:51727) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hgr68-00036k-FG for 36404@debbugs.gnu.org; Fri, 28 Jun 2019 09:39:29 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5SDdQ3N004767 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 28 Jun 2019 13:39:27 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: "Thompson\, David" Subject: [PATCH 5/5] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <875zopj0gs.fsf_-_@sdf.lonestar.org> <871rzdj0fu.fsf_-_@sdf.lonestar.org> Date: Fri, 28 Jun 2019 09:37:49 -0400 In-Reply-To: <871rzdj0fu.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 28 Jun 2019 09:37:09 -0400") Message-ID: <87woh5hlua.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * doc/guix.texi: Add section "Invoking guix deploy". =2D-- doc/guix.texi | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index f0d148ace0..948767d8c8 100644 =2D-- a/doc/guix.texi +++ b/doc/guix.texi @@ -81,6 +81,7 @@ Documentation License''. * guix gc: (guix)Invoking guix gc. Reclaiming unused disk space. * guix pull: (guix)Invoking guix pull. Update the list of available= packages. * guix system: (guix)Invoking guix system. Manage the operating system = configuration. +* guix deploy: (guix)Invoking guix deploy. Manage operating system conf= igurations for remote hosts. @end direntry =20 @dircategory Software development @@ -269,6 +270,7 @@ System Configuration * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remo= te host. * Running Guix in a VM:: How to run Guix System in a virtual machin= e. * Defining Services:: Adding new service definitions. =20 @@ -10303,6 +10305,7 @@ instance to support new system services. * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remo= te host. * Running Guix in a VM:: How to run Guix System in a virtual machin= e. * Defining Services:: Adding new service definitions. @end menu @@ -25399,6 +25402,106 @@ example graph. =20 @end table =20 +@node Invoking guix deploy +@section Invoking @code{guix deploy} + +In addition to managing a machine's configuration locally through operating +system declarations, Guix also provides the ability to managing multiple r= emote +hosts as a logical ``deployment''. This is done using @command{guix deploy= }. + +@example +guix deploy @var{file} +@end example + +Such an invocation will deploy the machines that the code within @var{file} +evaluates to. As an example, @var{file} might contain a definition like th= is: + +@example +;; This is a Guix deployment of a "bare bones" setup, with +;; no X11 display server, to a machine with an SSH daemon +;; listening on localhost:2222. A configuration such as this +;; may be appropriate for virtual machine with ports +;; forwarded to the host's loopback interface. + +(use-modules (gnu) (guix)) +(use-machine-modules ssh) +(use-service-modules networking ssh) +(use-package-modules bootloaders) + +(define %system + (operating-system + (host-name "gnu-deployed") + (timezone "Etc/UTC") + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (target "/dev/vda") + (terminal-outputs '(console)))) + (file-systems (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + %base-file-systems)) + (services + (append (list (service dhcp-client-service-type) + (service openssh-service-type + (openssh-configuration + (permit-root-login #t) + (allow-empty-passwords? #t)))) + %base-services)))) + +(list (machine + (system %system) + (environment 'managed-host) + (configuration (machine-ssh-configuration + (host-name "localhost") + (identity "./id_rsa") + (port 2222))))) +@end example + +The file should evaluate to a list of machines, rather than just one. This +example, upon being deployed, will create a new generation on the remote s= ystem +realizing the operating-system configuration @var{%system}. @var{environme= nt} +and @var{configuration} specify how the machine should be provisioned--tha= t is, +deployment and management of computing resources. The above example does n= ot +provision any resources -- a @code{'managed-host} is a machine that is alr= eady +up and running the Guix system. A more complex deployment may involve +i.e. starting virtual machines through a VPS provider, however, in which c= ase a +different @var{environment} types would be used. + +@deftp {Data Type} machine +This is the data type representing a single machine in a heterogeneous Guix +deployment. + +@table @asis +@item @code{system} +The object of the operating system configuration to deploy. + +@item @code{environment} +A symbol describing how the machine should be provisioned. At the moment, = only +the only supported value is @code{'managed-host}. + +@item @code{configuration} (default: @code{#f}) +An object describing the configuration for the machine's @code{environment= }. If +the @code{environment} has a default configuration, @code{#f} can be used.= If +@code{#f} is used for an environment with no default configuration, howeve= r, an +error will be thrown. +@end table +@end deftp + +@deftp {Data Type} machine-ssh-configuration +This is the data type representing the SSH client parameters for connectin= g to a +@code{'managed-host}. + +@table @asis +@item @code{host-name} +@item @code{port} (default: @code{22}) +@item @code{user} (default: @code{"root"}) +@item @code{identity} (default: @code{#f}) +If specified, the path to the SSH private key to use to authenticate with = the +remote host. +@end table +@end deftp + @node Running Guix in a VM @section Running Guix in a Virtual Machine =20 =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0WGC0ACgkQ9Qb9Fp2P 2VpbYQ//foEWOlRxR81k1VlkerL++wel/Dh1JOJxtCso874rEpdmkvXAhzBGQs5p vXB56n6bfVAbzc875EUSBc8CT5q15l/u/sf5AoqmEy92/06txTT3MN5Y9kIBqOl6 46U7Y/mmezg84BGu1fQUnfxfQBZBH69BObtGSxbuCZBl60XScmjpZKYqEq8d+/bi wncKIEHVQTmKP5CEcc7K8cShF1m3TEHcD2dvHckPg3iC6/k8QexbjZzDMD6tGv+i PCTiWtClMqn7oEOVmM4OAFB8o/AHOFVWBy20H9WlbtniCeGd9dvCjWkiPV+GV4N5 1cGf2P1v5OBUEO3ONuhgSDXHuHXKksowJ/FRaJDUBALQUzFYTvPNjwRzulbzotqa 3kNOVgMT0TtA+t5S0VJV0pxlqCXguuFkqUQRuPiyUL7cFKbyAWCbNH866Xk6QMbU 3zhU/CpKZ9ktj1pb2/wdMXUaOng6LBOgyfTXYSibeu6wdb16q/Wq478qJqTot0LD dt/tjxSGDkIvv5XGkTa4Lo2+IjA+WMCAN6L2auqtz3MsrfyWfjefLzXUNRjRNiA2 y7uhCGBFoKsZ+22YNwjHqSrRMJlJwvHO++yK3/8xnLuTDfsQETgUCGPc2xMK+LPZ G2n1fW98fGH1YxUKc5PpDm6YRz7Rt5c30kakEzyKDgSu/FxvnUE= =LjvK -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 10:37:23 2019 Received: (at submit) by debbugs.gnu.org; 29 Jun 2019 14:37:23 +0000 Received: from localhost ([127.0.0.1]:44797 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhETj-00013t-6Z for submit@debbugs.gnu.org; Sat, 29 Jun 2019 10:37:23 -0400 Received: from lists.gnu.org ([209.51.188.17]:48382) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhETh-00013l-PT for submit@debbugs.gnu.org; Sat, 29 Jun 2019 10:37:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59760) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhETg-0005xW-FP for guix-patches@gnu.org; Sat, 29 Jun 2019 10:37:21 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hhETf-0003eB-0j for guix-patches@gnu.org; Sat, 29 Jun 2019 10:37:20 -0400 Received: from dustycloud.org ([50.116.34.160]:34712) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hhETe-0003Yv-RS; Sat, 29 Jun 2019 10:37:18 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 6E49C26630; Sat, 29 Jun 2019 10:37:10 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: guix-patches@gnu.org Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. In-reply-to: <87o92ianbj.fsf@sdf.lonestar.org> Date: Sat, 29 Jun 2019 10:37:10 -0400 Message-ID: <87o92glap5.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 50.116.34.160 X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: 36404@debbugs.gnu.org, Ludovic =?utf-8?Q?Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Jakob L. Kreuze writes: > Hello, Guix! > > This patch provides the basis for 'guix deploy', implementing what I've > referred to as the "simple case" in my progress reports: in-place > updates to machines (physical or virtual) whose name and IP address we > know well. Do note that these commits depend on Ludovic's implementation > of 'remote-eval'.[1] Horray! > #+BEGIN_SRC scheme > ;; [...] > (list (machine > (system %system) > (environment 'managed-host) > (configuration (machine-ssh-configuration > (host-name "localhost") > (identity "./id_rsa") > (port 2222))))) > #+END_SRC scheme > > The 'environment' field is where we declare how machines should be > provisioned. In this case, the only type of provisioning that's been > implemented is 'managed-host' -- the "simple case" of in-place updates > to a machine that's already running GuixSD. The parameters for > provisioning are given in the form of an environment-specific > configuration type. In the example, this is 'machine-ssh-configuration', > which describes how 'guix deploy' should make an SSH connection to the > machine. I'm sure you can imagine something along the lines of a > 'machine-digitalocean-configuration', describing some parameters for a > droplet. In the future I think it would be good to make this extensible as well. Dispatching on a symbol means that Guix must itself provide a fixed set of possible environment types. If we made this an extensible structure, akin to services or something, we could allow for more flexibility in the future. Thoughts for the future, but not a blocker on this patch. > There are two things in this patch series that I'd like comments on in > particular. > > First, I still haven't figured out the whole testing situation. The > tests, as of now, spin up a virtual machine, create a machine instance, > deploy that to the virtual machine, and then make assertions about > changes made to the system. These tests were originally in the system > test suite as they deal with virtual machines, but I've since moved it > into the normal Guix test suite because of how much needs to be done on > the host side -- I spent an absurd amount of time trying to fit a call > to 'deploy-machine' into a derivation that could be run by the system > test suite, but I just wasn't able to make it work. I'm hoping someone > will have thoughts about how we can test 'guix deploy'. Should we have > them disabled by default? Is there some way to implement them in the a > system test suite that I've overlooked? Should the tests be included at > all? Ludo, do you have comments? I suspect this is up your area of expertise. > I look forward to your comments. Yes, now for me to look at the actual patches :) From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 10:42:49 2019 Received: (at submit) by debbugs.gnu.org; 29 Jun 2019 14:42:49 +0000 Received: from localhost ([127.0.0.1]:44813 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhEYz-0001DA-5Q for submit@debbugs.gnu.org; Sat, 29 Jun 2019 10:42:49 -0400 Received: from lists.gnu.org ([209.51.188.17]:48704) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhEYv-0001Cy-Fk for submit@debbugs.gnu.org; Sat, 29 Jun 2019 10:42:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32950) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhEYu-0007rL-Gv for guix-patches@gnu.org; Sat, 29 Jun 2019 10:42:45 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hhEYt-0007fE-J4 for guix-patches@gnu.org; Sat, 29 Jun 2019 10:42:44 -0400 Received: from dustycloud.org ([50.116.34.160]:34742) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hhEYt-0007en-Em for guix-patches@gnu.org; Sat, 29 Jun 2019 10:42:43 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 3026B26630; Sat, 29 Jun 2019 10:42:41 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: guix-patches@gnu.org Subject: Re: [bug#36404] [PATCH 1/5] ssh: Add 'identity' keyword to 'open-ssh-session'. In-reply-to: <87ef3dj0j9.fsf_-_@sdf.lonestar.org> Date: Sat, 29 Jun 2019 10:42:40 -0400 Message-ID: <87lfxklafz.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 50.116.34.160 X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Jakob L. Kreuze writes: > * guix/ssh.scm (open-ssh-session): Add 'identity' keyword argument. > --- > guix/ssh.scm | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/guix/ssh.scm b/guix/ssh.scm > index 9b9baf54ea..a2387564a4 100644 > --- a/guix/ssh.scm > +++ b/guix/ssh.scm > @@ -57,12 +57,13 @@ > (define %compression > "zlib@openssh.com,zlib") > > -(define* (open-ssh-session host #:key user port > +(define* (open-ssh-session host #:key user port identity > (compression %compression)) > "Open an SSH session for HOST and return it. When USER and PORT are #f, use > default values or whatever '~/.ssh/config' specifies; otherwise use them. > Throw an error on failure." Looks good, but could you add to the docstring here explaining the new identity keyword? > (let ((session (make-session #:user user > + #:identity identity > #:host host > #:port port > #:timeout 10 ;seconds From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 17:36:56 2019 Received: (at submit) by debbugs.gnu.org; 29 Jun 2019 21:36:56 +0000 Received: from localhost ([127.0.0.1]:44979 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhL1c-00057D-BZ for submit@debbugs.gnu.org; Sat, 29 Jun 2019 17:36:55 -0400 Received: from lists.gnu.org ([209.51.188.17]:41382) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhL1Z-000574-Js for submit@debbugs.gnu.org; Sat, 29 Jun 2019 17:36:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47307) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhL1V-0003cJ-FC for guix-patches@gnu.org; Sat, 29 Jun 2019 17:36:45 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hhL1R-00056E-5V for guix-patches@gnu.org; Sat, 29 Jun 2019 17:36:41 -0400 Received: from dustycloud.org ([2600:3c02::f03c:91ff:feae:cb51]:33432) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hhL1P-00053r-8P for guix-patches@gnu.org; Sat, 29 Jun 2019 17:36:36 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 37DD426655; Sat, 29 Jun 2019 17:36:32 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: guix-patches@gnu.org Subject: Re: [bug#36404] [PATCH 2/5] gnu: Add machine type for deployment specifications. In-reply-to: <87a7e1j0hy.fsf_-_@sdf.lonestar.org> Date: Sat, 29 Jun 2019 17:36:31 -0400 Message-ID: <87k1d4kra8.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c02::f03c:91ff:feae:cb51 X-Spam-Score: 2.2 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Jakob L. Kreuze writes: > * gnu/machine.scm: New file. > * gnu/machine/ssh.scm: New file. > * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. > * tests/machine.scm: New file. > * Makefile.am (SCM_TESTS): Add it. > --- > Makefile. [...] Content analysis details: (2.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: lonestar.org] 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [2600:3c02:0:0:f03c:91ff:feae:cb51 listed in] [zen.spamhaus.org] 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [209.51.188.17 listed in list.dnswl.org] X-Debbugs-Envelope-To: submit Cc: 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Jakob L. Kreuze writes: > * gnu/machine.scm: New file. > * gnu/machine/ssh.scm: New file. > * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. > * tests/machine.scm: New file. > * Makefile.am (SCM_TESTS): Add it. > --- > Makefile.am | 3 +- > gnu/local.mk | 5 +- > gnu/machine.scm | 89 +++++++++ > gnu/machine/ssh.scm | 355 ++++++++++++++++++++++++++++++++++ > tests/machine.scm | 450 ++++++++++++++++++++++++++++++++++++++++++++ > 5 files changed, 900 insertions(+), 2 deletions(-) > create mode 100644 gnu/machine.scm > create mode 100644 gnu/machine/ssh.scm > create mode 100644 tests/machine.scm > > diff --git a/Makefile.am b/Makefile.am > index 80be73e4bf..9156554635 100644 > --- a/Makefile.am > +++ b/Makefile.am > @@ -423,7 +423,8 @@ SCM_TESTS =3D \ > tests/import-utils.scm \ > tests/store-database.scm \ > tests/store-deduplication.scm \ > - tests/store-roots.scm > + tests/store-roots.scm \ > + tests/machine.scm > > SH_TESTS =3D \ > tests/guix-build.sh \ > diff --git a/gnu/local.mk b/gnu/local.mk > index f5d53b49b8..ad87de5ea7 100644 > --- a/gnu/local.mk > +++ b/gnu/local.mk > @@ -564,6 +564,9 @@ GNU_SYSTEM_MODULES =3D \ > %D%/system/uuid.scm \ > %D%/system/vm.scm \ > \ > + %D%/machine.scm \ > + %D%/machine/ssh.scm \ > + \ > %D%/build/accounts.scm \ > %D%/build/activation.scm \ > %D%/build/bootloader.scm \ > @@ -629,7 +632,7 @@ INSTALLER_MODULES =3D \ > %D%/installer/newt/user.scm \ > %D%/installer/newt/utils.scm \ > %D%/installer/newt/welcome.scm \ > - %D%/installer/newt/wifi.scm > + %D%/installer/newt/wifi.scm > > # Always ship the installer modules but compile them only when > # ENABLE_INSTALLER is true. > diff --git a/gnu/machine.scm b/gnu/machine.scm > new file mode 100644 > index 0000000000..900a2020dc > --- /dev/null > +++ b/gnu/machine.scm > @@ -0,0 +1,89 @@ > +;;; GNU Guix --- Functional package management for GNU > +;;; Copyright =C2=A9 2019 David Thompson > +;;; Copyright =C2=A9 2019 Jakob L. Kreuze > +;;; > +;;; This file is part of GNU Guix. > +;;; > +;;; GNU Guix is free software; you can redistribute it and/or modify it > +;;; under the terms of the GNU General Public License as published by > +;;; the Free Software Foundation; either version 3 of the License, or (at > +;;; your option) any later version. > +;;; > +;;; GNU Guix is distributed in the hope that it will be useful, but > +;;; WITHOUT ANY WARRANTY; without even the implied warranty of > +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +;;; GNU General Public License for more details. > +;;; > +;;; You should have received a copy of the GNU General Public License > +;;; along with GNU Guix. If not, see . > + > +(define-module (gnu machine) > + #:use-module (gnu system) > + #:use-module (guix derivations) > + #:use-module (guix monads) > + #:use-module (guix records) > + #:use-module (guix store) > + #:export (machine > + machine? > + this-machine > + > + machine-system > + machine-environment > + machine-configuration > + machine-display-name > + > + build-machine > + deploy-machine > + remote-eval)) Maybe it would make sense to call it machine-remote-eval to distinguish it? I dunno. > + > +;;; Commentary: > +;;; > +;;; This module provides the types used to declare individual machines i= n a > +;;; heterogeneous Guix deployment. The interface allows users of specify= system > +;;; configurations and the means by which resources should be provisione= d on a > +;;; per-host basis. > +;;; > +;;; Code: > + > +(define-record-type* machine > + make-machine > + machine? > + this-machine > + (system machine-system) ; > + (environment machine-environment) ; symbol > + (configuration machine-configuration ; configuration object > + (default #f))) ; specific to environment > + > +(define (machine-display-name machine) > + "Return the host-name identifying MACHINE." > + (operating-system-host-name (machine-system machine))) > + > +(define (build-machine machine) > + "Monadic procedure that builds the system derivation for MACHINE and r= eturning > +a list containing the path of the derivation file and the path of the de= rivation > +output." > + (let ((os (machine-system machine))) > + (mlet* %store-monad ((osdrv (operating-system-derivation os)) > + (_ ((store-lift build-derivations) (list osdrv)= ))) > + (return (list (derivation-file-name osdrv) > + (derivation->output-path osdrv)))))) > + > +(define (remote-eval machine exp) > + "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP re= fers to > +are built and deployed to MACHINE beforehand." > + (case (machine-environment machine) > + ((managed-host) > + ((@@ (gnu machine ssh) remote-eval) machine exp)) @@ is a (sometimes useful) antipattern. But in general, if something is importing something with @@, it's a good indication that we should just be exporting it. What do you think? > + (else > + (let ((type (machine-environment machine))) > + (error "unsupported environment type" type))))) > + > +(define (deploy-machine machine) > + "Monadic procedure transferring the new system's OS closure to the rem= ote > +MACHINE, activating it on MACHINE and switching MACHINE to the new gener= ation." > + (case (machine-environment machine) > + ((managed-host) > + ((@@ (gnu machine ssh) deploy-machine) machine)) > + (else > + (let ((type (machine-environment machine))) > + (error "unsupported environment type" type))))) So I guess here's where we'd switch out the environment from being a symbol to being a struct or procedure (or struct containing a procedure). Maybe it wouldn't be so hard to do? In fact, now that I look at it, we could solve both problems at once: there's no need to export deploy-machine and remote-eval if they're wrapped in another structure. Instead, maybe this code could look like: #+BEGIN_SRC scheme (define (remote-eval machine exp) "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP refer= s to are built and deployed to MACHINE beforehand." (let* ((environment (machine-environment machine)) (remote-eval (environment-remote-eval environment))) (remote-eval machine exp))) (define (deploy-machine machine) "Monadic procedure transferring the new system's OS closure to the remote MACHINE, activating it on MACHINE and switching MACHINE to the new generati= on." (let* ((environment (machine-environment machine)) (deploy-machine (environment-deploy-machine environment))) (deploy-machine machine))) #+END_SRC Thoughts? > diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm > new file mode 100644 > index 0000000000..a8f946e19f > --- /dev/null > +++ b/gnu/machine/ssh.scm > @@ -0,0 +1,355 @@ > +;;; GNU Guix --- Functional package management for GNU > +;;; Copyright =C2=A9 2019 Jakob L. Kreuze > +;;; > +;;; This file is part of GNU Guix. > +;;; > +;;; GNU Guix is free software; you can redistribute it and/or modify it > +;;; under the terms of the GNU General Public License as published by > +;;; the Free Software Foundation; either version 3 of the License, or (at > +;;; your option) any later version. > +;;; > +;;; GNU Guix is distributed in the hope that it will be useful, but > +;;; WITHOUT ANY WARRANTY; without even the implied warranty of > +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +;;; GNU General Public License for more details. > +;;; > +;;; You should have received a copy of the GNU General Public License > +;;; along with GNU Guix. If not, see . > + > +(define-module (gnu machine ssh) > + #:use-module (gnu bootloader) > + #:use-module (gnu machine) > + #:autoload (gnu packages gnupg) (guile-gcrypt) > + #:use-module (gnu services) > + #:use-module (gnu services shepherd) > + #:use-module (gnu system) > + #:use-module (guix derivations) > + #:use-module (guix gexp) > + #:use-module (guix modules) > + #:use-module (guix monads) > + #:use-module (guix records) > + #:use-module (guix ssh) > + #:use-module (guix store) > + #:use-module (ice-9 match) > + #:use-module (srfi srfi-19) > + #:export (machine-ssh-configuration > + machine-ssh-configuration? > + machine-ssh-configuration > + > + machine-ssh-configuration-host-name > + machine-ssh-configuration-port > + machine-ssh-configuration-user > + machine-ssh-configuration-session)) > + > +;;; Commentary: > +;;; > +;;; This module implements remote evaluation and system deployment for > +;;; machines that are accessable over SSH and have a known host-name. In= the > +;;; sense of the broader "machine" interface, we describe the environmen= t for > +;;; such machines as 'managed-host. > +;;; > +;;; Code: > + > + > +;;; > +;;; SSH client parameter configuration. > +;;; > + > +(define-record-type* machine-ssh-configurati= on > + make-machine-ssh-configuration > + machine-ssh-configuration? > + this-machine-ssh-configuration > + (host-name machine-ssh-configuration-host-name) ; string > + (port machine-ssh-configuration-port ; integer > + (default 22)) > + (user machine-ssh-configuration-user ; string > + (default "root")) > + (identity machine-ssh-configuration-identity ; path to a private key > + (default #f)) > + (session machine-ssh-configuration-session ; session > + (default #f))) > + > +(define (machine-ssh-session machine) > + "Return the SSH session that was given in MACHINE's configuration, or = create > +one from the configuration's parameters if one was not provided." > + (let ((config (machine-configuration machine))) > + (if (machine-ssh-configuration? config) Feels like better polymorphism than this is desirable, but I'm not sure I have advice on how to do it right now. Probably services provide the right form of inspiration. At any rate, it's probably not a blocker to merging this first set, but I'd love to see if we could get something more future-extensible. > + (or (machine-ssh-configuration-session config) > + (let ((host-name (machine-ssh-configuration-host-name config= )) > + (user (machine-ssh-configuration-user config)) > + (port (machine-ssh-configuration-port config)) > + (identity (machine-ssh-configuration-identity config))) > + (open-ssh-session host-name > + #:user user > + #:port port > + #:identity identity))) > + (error "unsupported configuration type")))) > > + > +;;; > +;;; Remote evaluation. > +;;; > + > +(define (remote-eval machine exp) > + "Internal implementation of 'remote-eval' for MACHINE instances with an > +environment type of 'managed-host." > + (unless (machine-configuration machine) > + (error (format #f (G_ "no configuration specified for machine of env= ironment '~a'") > + (symbol->string (machine-environment machine))))) > + ((@ (guix remote) remote-eval) exp (machine-ssh-session machine))) Why not just import remote-eval in the define-module? > + > + > +;;; > +;;; System deployment. > +;;; > + > +(define (switch-to-system machine) > + "Monadic procedure creating a new generation on MACHINE and execute the > +activation script for the new system configuration." > + (define (remote-exp drv script) > + (with-extensions (list guile-gcrypt) It's so cool that this works across machines. Dang! > + (with-imported-modules (source-module-closure '((guix config) > + (guix profiles) > + (guix utils))) > + #~(begin > + (use-modules (guix config) > + (guix profiles) > + (guix utils)) > + > + (define %system-profile > + (string-append %state-directory "/profiles/system")) > + > + (let* ((system #$(derivation->output-path drv)) > + (number (1+ (generation-number %system-profile))) > + (generation (generation-file-name %system-profile num= ber)) > + (old-env (environ)) > + (old-path %load-path) > + (old-cpath %load-compiled-path)) > + (switch-symlinks generation system) > + (switch-symlinks %system-profile generation) > + ;; Guard against the activation script modifying $PATH. Yeah that sounds like it would be bad. But I'm curious... could you explain the specific bug it's preventing here? I'd like to know. > + (dynamic-wind > + (const #t) > + (lambda () > + (setenv "GUIX_NEW_SYSTEM" system) > + ;; Guard against the activation script modifying '%loa= d-path'. > + (dynamic-wind > + (const #t) > + (lambda () > + ;; The activation script may write to stdout, which > + ;; confuses 'remote-eval' when it attempts to read= a > + ;; result from the remote REPL. We work around thi= s by > + ;; forcing the output to a string. > + (with-output-to-string > + (lambda () > + (primitive-load #$script)))) > + (lambda () > + (set! %load-path old-path) > + (set! %load-compiled-path old-cpath)))) > + (lambda () > + (environ old-env)))))))) > + > + (let* ((os (machine-system machine)) > + (script (operating-system-activation-script os))) > + (mlet* %store-monad ((drv (operating-system-derivation os))) > + (remote-eval machine (remote-exp drv script))))) > + > +(define (upgrade-shepherd-services machine) > + "Monadic procedure unloading and starting services on the remote as ne= eded > +to realize the MACHINE's system configuration." > + (define target-services > + ;; Monadic expression evaluating to a list of (name output-path) pai= rs for > + ;; all of MACHINE's services. > + (mapm %store-monad > + (lambda (service) > + (mlet %store-monad ((file ((compose lower-object > + shepherd-service-file) > + service))) > + (return (list (shepherd-service-canonical-name service) > + (derivation->output-path file))))) > + (service-value > + (fold-services (operating-system-services (machine-system mac= hine)) > + #:target-type shepherd-root-service-type)))) > + > + (define (remote-exp target-services) > + (with-imported-modules '((gnu services herd)) > + #~(begin > + (use-modules (gnu services herd) > + (srfi srfi-1)) > + > + (define running > + (filter live-service-running (current-services))) > + > + (define (essential? service) > + ;; Return #t if SERVICE is essential and should not be unloa= ded > + ;; under any circumstance. > + (memq (first (live-service-provision service)) > + '(root shepherd))) This is a curious procedure, but I see why it exists. I guess these really are the only things? Maybe it will change at some point in the future, but seems to make sense for now. > + (define (obsolete? service) > + ;; Return #t if SERVICE can be safely unloaded. > + (and (not (essential? service)) > + (every (lambda (requirements) > + (not (memq (first (live-service-provision serv= ice)) > + requirements))) > + (map live-service-requirement running)))) Just to see if I understand it... this is kind of so we can identify and "garbage collect" services that don't apply to the new system? > + (define to-unload > + (filter obsolete? > + (remove (lambda (service) > + (memq (first (live-service-provision servi= ce)) > + (map first '#$target-services))) > + running))) > + > + (define to-start > + (remove (lambda (service-pair) > + (memq (first service-pair) > + (map (compose first live-service-provision) > + running))) > + '#$target-services)) > + > + ;; Unload obsolete services. > + (for-each (lambda (service) > + (false-if-exception > + (unload-service service))) > + to-unload) > + > + ;; Load the service files for any new services and start them. > + (load-services/safe (map second to-start)) > + (for-each start-service (map first to-start)) I'm a bit unsure from the above code... I'm guessing one of two things is happening: - Either it's starting services that haven't been started yet, but leaving alone services that are running but which aren't "new" - Or it's restarting services that are currently running Which is it? And mind adding a comment explaining it? By the way, is there anything about the dependency order in which services might need to be restarted to be considered? I'm honestly not sur= e. > + #t))) > + > + (mlet %store-monad ((target-services target-services)) > + (remote-eval machine (remote-exp target-services)))) > + > +(define (machine-boot-parameters machine) > + "Monadic procedure returning a list of 'boot-parameters' for the gener= ations > +of MACHINE's system profile, ordered from most recent to oldest." > + (define bootable-kernel-arguments > + (@@ (gnu system) bootable-kernel-arguments)) > + > + (define remote-exp > + (with-extensions (list guile-gcrypt) > + (with-imported-modules (source-module-closure '((guix config) > + (guix profiles))) > + #~(begin > + (use-modules (guix config) > + (guix profiles) > + (ice-9 textual-ports)) > + > + (define %system-profile > + (string-append %state-directory "/profiles/system")) > + > + (define (read-file path) > + (call-with-input-file path > + (lambda (port) > + (get-string-all port)))) > + > + (map (lambda (generation) > + (let* ((system-path (generation-file-name %system-pro= file > + generation)) > + (boot-parameters-path (string-append system-pa= th > + "/paramet= ers")) > + (time (stat:mtime (lstat system-path)))) > + (list generation > + system-path > + time > + (read-file boot-parameters-path)))) > + (reverse (generation-numbers %system-profile))))))) > + > + (mlet* %store-monad ((generations (remote-eval machine remote-exp))) > + (return > + (map (lambda (generation) > + (match generation > + ((generation system-path time serialized-params) > + (let* ((params (call-with-input-string serialized-params > + read-boot-parameters)) > + (root (boot-parameters-root-device params)) > + (label (boot-parameters-label params))) > + (boot-parameters > + (inherit params) > + (label > + (string-append label " (#" > + (number->string generation) ", " > + (let ((time (make-time time-utc 0 time= ))) > + (date->string (time-utc->date time) > + "~Y-~m-~d ~H:~M")) > + ")")) > + (kernel-arguments > + (append (bootable-kernel-arguments system-path root) > + (boot-parameters-kernel-arguments params)))))= ))) > + generations)))) So I guess this is derivative of some of the stuff in guix/scripts/system.scm. That makes me feel like it would be nice if it could be generalized, but I haven't spent enough time with the code to figure out if it really can be. I don't want to block the merge on that desire, though if you agree that generalization between those sections of code is desirable, maybe add a comment to that effect? > +(define (install-bootloader machine) > + "Create a bootloader entry for the new system generation on MACHINE, a= nd > +configure the bootloader to boot that generation by default." > + (define bootloader-installer-script > + (@@ (guix scripts system) bootloader-installer-script)) > + > + (define (remote-exp installer bootcfg bootcfg-file) > + (with-extensions (list guile-gcrypt) > + (with-imported-modules (source-module-closure '((gnu build install) > + (guix store) > + (guix utils))) > + #~(begin > + (use-modules (gnu build install) > + (guix store) > + (guix utils)) > + (let* ((gc-root (string-append "/" %gc-roots-directory "/boo= tcfg")) > + (temp-gc-root (string-append gc-root ".new")) > + (old-path %load-path) > + (old-cpath %load-compiled-path)) > + (switch-symlinks temp-gc-root gc-root) > + > + (unless (false-if-exception > + (begin > + (install-boot-config #$bootcfg #$bootcfg-file "= /") > + ;; Guard against the activation script modifying > + ;; '%load-path'. > + (dynamic-wind > + (const #t) > + (lambda () > + ;; The installation script may write to std= out, > + ;; which confuses 'remote-eval' when it att= empts to > + ;; read a result from the remote REPL. We w= ork > + ;; around this by forcing the output to a s= tring. > + (with-output-to-string > + (lambda () > + (primitive-load #$installer)))) > + (lambda () > + (set! %load-path old-path) > + (set! %load-compiled-path old-cpath))))) > + (delete-file temp-gc-root) > + (error "failed to install bootloader")) > + > + (rename-file temp-gc-root gc-root) > + #t))))) This code also looks very similar, but I compared them and I can see that they aren't quite the same, at least in that you had to install the dynamic-wind. But I get the feeling that it still might be possible to generalize them, so could you leave a comment here as well? Unless you think it's really not possible to generalize them to share code for reasons I'm not yet aware of. > + (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine= ))) > + (let* ((os (machine-system machine)) > + (bootloader ((compose bootloader-configuration-bootloader > + operating-system-bootloader) > + os)) > + (bootloader-target (bootloader-configuration-target > + (operating-system-bootloader os))) > + (installer (bootloader-installer-script > + (bootloader-installer bootloader) > + (bootloader-package bootloader) > + bootloader-target > + "/")) > + (menu-entries (map boot-parameters->menu-entry boot-parameter= s)) > + (bootcfg (operating-system-bootcfg os menu-entries)) > + (bootcfg-file (bootloader-configuration-file bootloader))) > + (remote-eval machine (remote-exp installer bootcfg bootcfg-file)))= )) > + > +(define (deploy-machine machine) > + "Internal implementation of 'deploy-machine' for MACHINE instances wit= h an > +environment type of 'managed-host." > + (unless (machine-configuration machine) > + (error (format #f (G_ "no configuration specified for machine of env= ironment '~a'") > + (symbol->string (machine-environment machine))))) > + (mbegin %store-monad > + (switch-to-system machine) > + (upgrade-shepherd-services machine) > + (install-bootloader machine))) > diff --git a/tests/machine.scm b/tests/machine.scm > new file mode 100644 > index 0000000000..390c0189bb > --- /dev/null > +++ b/tests/machine.scm > @@ -0,0 +1,450 @@ > +;;; GNU Guix --- Functional package management for GNU > +;;; Copyright =C2=A9 2019 Jakob L. Kreuze > +;;; > +;;; This file is part of GNU Guix. > +;;; > +;;; GNU Guix is free software; you can redistribute it and/or modify it > +;;; under the terms of the GNU General Public License as published by > +;;; the Free Software Foundation; either version 3 of the License, or (at > +;;; your option) any later version. > +;;; > +;;; GNU Guix is distributed in the hope that it will be useful, but > +;;; WITHOUT ANY WARRANTY; without even the implied warranty of > +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +;;; GNU General Public License for more details. > +;;; > +;;; You should have received a copy of the GNU General Public License > +;;; along with GNU Guix. If not, see . > + > +(define-module (gnu tests machine) > + #:use-module (gnu bootloader grub) > + #:use-module (gnu bootloader) > + #:use-module (gnu build marionette) > + #:use-module (gnu build vm) > + #:use-module (gnu machine) > + #:use-module (gnu machine ssh) > + #:use-module (gnu packages bash) > + #:use-module (gnu packages virtualization) > + #:use-module (gnu services base) > + #:use-module (gnu services networking) > + #:use-module (gnu services ssh) > + #:use-module (gnu services) > + #:use-module (gnu system file-systems) > + #:use-module (gnu system vm) > + #:use-module (gnu system) > + #:use-module (gnu tests) > + #:use-module (guix derivations) > + #:use-module (guix gexp) > + #:use-module (guix monads) > + #:use-module (guix pki) > + #:use-module (guix store) > + #:use-module (guix utils) > + #:use-module (ice-9 ftw) > + #:use-module (ice-9 match) > + #:use-module (ice-9 textual-ports) > + #:use-module (srfi srfi-1) > + #:use-module (srfi srfi-26) > + #:use-module (srfi srfi-64) > + #:use-module (ssh auth) > + #:use-module (ssh channel) > + #:use-module (ssh key) > + #:use-module (ssh session)) Hoo! That's a lot of imports! Makes sense I guess... > + > +;;; > +;;; Virtual machine scaffolding. > +;;; > + > +(define marionette-pid (@@ (gnu build marionette) marionette-pid)) > + > +(define (call-with-marionette path command proc) > + "Invoke PROC with a marionette running COMMAND in PATH." > + (let* ((marionette (make-marionette command #:socket-directory path)) > + (pid (marionette-pid marionette))) > + (dynamic-wind > + (lambda () > + (unless marionette > + (error "could not start marionette"))) > + (lambda () (proc marionette)) > + (lambda () > + (kill pid SIGTERM))))) > + > +(define (dir-join . components) > + "Join COMPONENTS with `file-name-separator-string'." > + (string-join components file-name-separator-string)) > + > +(define (call-with-machine-test-directory proc) > + "Run PROC with the path to a temporary directory that will be cleaned = up > +when PROC returns. Only files that can be passed to 'delete-file' should= be > +created within the temporary directory; cleanup will not recurse into > +subdirectories." > + (let ((path (tmpnam))) > + (dynamic-wind > + (lambda () > + (unless (mkdir path) > + (error (format #f "could not create directory '~a'" path)))) > + (lambda () (proc path)) > + (lambda () > + (let ((children (map first (cddr (file-system-tree path))))) > + (for-each (lambda (child) > + (false-if-exception > + (delete-file (dir-join path child)))) > + children) > + (rmdir path)))))) > + > +(define (os-for-test os) > + "Return an record derived from OS that is appropria= te for > +use with 'qemu-image'." > + (define file-systems-to-keep > + ;; Keep only file systems other than root and not normally bound to = real > + ;; devices. > + (remove (lambda (fs) > + (let ((target (file-system-mount-point fs)) > + (source (file-system-device fs))) > + (or (string=3D? target "/") > + (string-prefix? "/dev/" source)))) > + (operating-system-file-systems os))) > + > + (define root-uuid > + ;; UUID of the root file system. > + ((@@ (gnu system vm) operating-system-uuid) os 'dce)) > + > + > + (operating-system > + (inherit os) > + ;; Assume we have an initrd with the whole QEMU shebang. > + > + ;; Force our own root file system. Refer to it by UUID so that > + ;; it works regardless of how the image is used ("qemu -hda", > + ;; Xen, etc.). > + (file-systems (cons (file-system > + (mount-point "/") > + (device root-uuid) > + (type "ext4")) > + file-systems-to-keep)))) > + > +(define (qemu-image-for-test os) > + "Return a derivation producing a QEMU disk image running OS. This proc= edure > +is similar to 'system-qemu-image' in (gnu system vm), but makes use of > +'os-for-test' so that callers may obtain the same system derivation that= will > +be booted by the image." > + (define root-uuid ((@@ (gnu system vm) operating-system-uuid) os 'dce)) > + (let* ((os (os-for-test os)) > + (bootcfg (operating-system-bootcfg os))) > + (qemu-image #:os os > + #:bootcfg-drv bootcfg > + #:bootloader (bootloader-configuration-bootloader > + (operating-system-bootloader os)) > + #:disk-image-size (* 9000 (expt 2 20)) > + #:file-system-type "ext4" > + #:file-system-uuid root-uuid > + #:inputs `(("system" ,os) > + ("bootcfg" ,bootcfg)) > + #:copy-inputs? #t))) > + > +(define (make-writable-image image) > + "Return a derivation producing a script to create a writable disk image > +overlay of IMAGE, writing the overlay to the the path given as a command= -line > +argument to the script." > + (define qemu-img-exec > + #~(list (string-append #$qemu-minimal "/bin/qemu-img") > + "create" "-f" "qcow2" > + "-o" (string-append "backing_file=3D" #$image))) > + > + (define builder > + #~(call-with-output-file #$output > + (lambda (port) > + (format port "#!~a~% exec ~a \"$@\"~%" > + #$(file-append bash "/bin/sh") > + (string-join #$qemu-img-exec " ")) > + (chmod port #o555)))) > + > + (gexp->derivation "make-writable-image.sh" builder)) > + > +(define (run-os-for-test os) > + "Return a derivation producing a script to run OS as a qemu guest, who= se > +first argument is the path to a writable disk image. Additional argument= s are > +passed as-is to qemu." > + (define kernel-arguments > + #~(list "console=3DttyS0" > + #+@(operating-system-kernel-arguments os "/dev/sda1"))) > + > + (define qemu-exec > + #~(begin > + (list (string-append #$qemu-minimal "/bin/" #$(qemu-command (%cu= rrent-system))) > + "-kernel" #$(operating-system-kernel-file os) > + "-initrd" #$(file-append os "/initrd") > + (format #f "-append ~s" > + (string-join #$kernel-arguments " ")) > + #$@(if (file-exists? "/dev/kvm") > + '("-enable-kvm") > + '()) > + "-no-reboot" > + "-net nic,model=3Dvirtio" > + "-object" "rng-random,filename=3D/dev/urandom,id=3Dguixsd-= vm-rng" > + "-device" "virtio-rng-pci,rng=3Dguixsd-vm-rng" > + "-vga" "std" > + "-m" "256" > + "-net" "user,hostfwd=3Dtcp::2222-:22"))) > + > + (define builder > + #~(call-with-output-file #$output > + (lambda (port) > + (format port "#!~a~% exec ~a -drive \"file=3D$@\"~%" > + #$(file-append bash "/bin/sh") > + (string-join #$qemu-exec " ")) > + (chmod port #o555)))) > + > + (gexp->derivation "run-vm.sh" builder)) > + > +(define (scripts-for-test os) > + "Build and return a list containing the paths of: > + > +- A script to make a writable disk image overlay of OS. > +- A script to run that disk image overlay as a qemu guest." > + (let ((virtualized-os (os-for-test os))) > + (mlet* %store-monad ((osdrv (operating-system-derivation virtualized= -os)) > + (imgdrv (qemu-image-for-test os)) > + > + ;; Ungexping 'imgdrv' or 'osdrv' will result in= an > + ;; error if the derivations don't exist in the = store, > + ;; so we ensure they're built prior to invoking > + ;; 'run-vm' or 'make-image'. > + (_ ((store-lift build-derivations) (list imgdrv= ))) > + > + (run-vm (run-os-for-test virtualized-os)) > + (make-image > + (make-writable-image (derivation->output-path = imgdrv)))) > + (mbegin %store-monad > + ((store-lift build-derivations) (list imgdrv make-image run-vm)) > + (return (list (derivation->output-path make-image) > + (derivation->output-path run-vm))))))) > + > +(define (call-with-marionette-and-session os proc) > + "Construct a marionette backed by OS in a temporary test environment a= nd > +invoke PROC with two arguments: the marionette object, and an SSH session > +connected to the marionette." > + (call-with-machine-test-directory > + (lambda (path) > + (match (with-store store > + (run-with-store store > + (scripts-for-test %system))) > + ((make-image run-vm) > + (let ((image (dir-join path "image"))) > + ;; Create the writable image overlay. > + (system (string-join (list make-image image) " ")) > + (call-with-marionette > + path > + (list run-vm image) > + (lambda (marionette) > + ;; XXX: The guest clearly has (gcrypt pk-crypto) since this > + ;; works, but trying to import it from 'marionette-eval' fa= ils as > + ;; the Marionette REPL does not have 'guile-gcrypt' in its > + ;; %load-path. > + (marionette-eval > + `(begin > + (use-modules (ice-9 popen)) > + (let ((port (open-pipe* OPEN_WRITE "guix" "archive" "--= authorize"))) > + (put-string port ,%signing-key) > + (close port))) > + marionette) > + ;; XXX: This is an absolute hack to work around potential q= uirks > + ;; in the operating system. For one, we invoke 'herd' from = the > + ;; command-line to ensure that the Shepherd socket file > + ;; exists. Second, we enable 'ssh-daemon', as there's a cha= nce > + ;; the service will be disabled upon booting the image. > + (marionette-eval > + `(system "herd enable ssh-daemon") > + marionette) > + (marionette-eval > + '(begin > + (use-modules (gnu services herd)) > + (start-service 'ssh-daemon)) > + marionette) > + (call-with-connected-session/auth > + (lambda (session) > + (proc marionette session))))))))))) > + > + > +;;; > +;;; SSH session management. These are borrowed from (gnu tests ssh). > +;;; > + > +(define (make-session-for-test) > + "Make a session with predefined parameters for a test." > + (make-session #:user "root" > + #:port 2222 > + #:host "localhost")) > + > +(define (call-with-connected-session proc) > + "Call the one-argument procedure PROC with a freshly created and > +connected SSH session object, return the result of the procedure call. = The > +session is disconnected when the PROC is finished." > + (let ((session (make-session-for-test))) > + (dynamic-wind > + (lambda () > + (let ((result (connect! session))) > + (unless (equal? result 'ok) > + (error "Could not connect to a server" > + session result)))) > + (lambda () (proc session)) > + (lambda () (disconnect! session))))) > + > +(define (call-with-connected-session/auth proc) > + "Make an authenticated session. We should be able to connect as > +root with an empty password." > + (call-with-connected-session > + (lambda (session) > + ;; Try the simple authentication methods. Dropbear requires > + ;; 'none' when there are no passwords, whereas OpenSSH accepts > + ;; 'password' with an empty password. > + (let loop ((methods (list (cut userauth-password! <> "") > + (cut userauth-none! <>)))) > + (match methods > + (() > + (error "all the authentication methods failed")) > + ((auth rest ...) > + (match (pk 'auth (auth session)) > + ('success > + (proc session)) > + ('denied > + (loop rest))))))))) > + > + > +;;; > +;;; Virtual machines for use in the test suite. > +;;; > + > +(define %system > + ;; A "bare bones" operating system running both an OpenSSH daemon and = the > + ;; "marionette" service. > + (marionette-operating-system > + (operating-system > + (host-name "gnu") > + (timezone "Etc/UTC") > + (bootloader (bootloader-configuration > + (bootloader grub-bootloader) > + (target "/dev/sda") > + (terminal-outputs '(console)))) > + (file-systems (cons (file-system > + (mount-point "/") > + (device "/dev/vda1") > + (type "ext4")) > + %base-file-systems)) > + (services > + (append (list (service dhcp-client-service-type) > + (service openssh-service-type > + (openssh-configuration > + (permit-root-login #t) > + (allow-empty-passwords? #t)))) > + %base-services))) > + #:imported-modules '((gnu services herd) > + (guix combinators)))) > + > +(define %signing-key > + ;; The host's signing key, encoded as a string. The "marionette" will = reject > + ;; any files signed by an unauthorized host, so we'll need to send thi= s key > + ;; over and authorize it. > + (call-with-input-file %public-key-file > + (lambda (port) > + (get-string-all port)))) > + > + > +(test-begin "machine") > + > +(define (system-generations marionette) > + (marionette-eval > + '(begin > + (use-modules (ice-9 ftw) > + (srfi srfi-1)) > + (let* ((profile-dir "/var/guix/profiles/") > + (entries (map first (cddr (file-system-tree profile-dir))))) > + (remove (lambda (entry) > + (member entry '("per-user" "system"))) > + entries))) > + marionette)) > + > +(define (running-services marionette) > + (marionette-eval > + '(begin > + (use-modules (gnu services herd) > + (srfi srfi-1)) > + (map (compose first live-service-provision) > + (filter live-service-running (current-services)))) > + marionette)) > + > +(define (count-grub-cfg-entries marionette) > + (marionette-eval > + '(begin > + (define grub-cfg > + (call-with-input-file "/boot/grub/grub.cfg" > + (lambda (port) > + (get-string-all port)))) > + > + (let loop ((n 0) > + (start 0)) > + (let ((index (string-contains grub-cfg "menuentry" start))) > + (if index > + (loop (1+ n) (1+ index)) > + n)))) > + marionette)) > + > +(define %target-system > + (marionette-operating-system > + (operating-system > + (host-name "gnu-deployed") > + (timezone "Etc/UTC") > + (bootloader (bootloader-configuration > + (bootloader grub-bootloader) > + (target "/dev/sda") > + (terminal-outputs '(console)))) > + (file-systems (cons (file-system > + (mount-point "/") > + (device "/dev/vda1") > + (type "ext4")) > + %base-file-systems)) > + (services > + (append (list (service tor-service-type) > + (service dhcp-client-service-type) > + (service openssh-service-type > + (openssh-configuration > + (permit-root-login #t) > + (allow-empty-passwords? #t)))) > + %base-services))) > + #:imported-modules '((gnu services herd) > + (guix combinators)))) > + > +(call-with-marionette-and-session > + (os-for-test %system) > + (lambda (marionette session) > + (let ((generations-prior (system-generations marionette)) > + (services-prior (running-services marionette)) > + (grub-entry-count-prior (count-grub-cfg-entries marionette)) > + (machine (machine > + (system %target-system) > + (environment 'managed-host) > + (configuration (machine-ssh-configuration > + (host-name "localhost") > + (session session)))))) > + (with-store store > + (run-with-store store > + (build-machine machine)) > + (run-with-store store > + (deploy-machine machine))) > + (test-equal "deployment created new generation" > + (length (system-generations marionette)) > + (1+ (length generations-prior))) > + (test-assert "deployment started new service" > + (and (not (memq 'tor services-prior)) > + (memq 'tor (running-services marionette)))) > + (test-equal "deployment created new menu entry" > + (count-grub-cfg-entries marionette) > + ;; A Grub configuration that contains a single menu entry does no= t have > + ;; an "old configurations" submenu. Deployment, then, would resul= t in > + ;; this submenu being created, meaning an additional two 'menuent= ry' > + ;; fields rather than just one. > + (if (=3D grub-entry-count-prior 1) > + (+ 2 grub-entry-count-prior) > + (1+ grub-entry-count-prior)))))) > + > +(test-end "machine") Seems good from a quick scan, but I'll admit I didn't read these as carefully as I did the rest of the code. This patch looks great overall! I know it was a lot of work to figure out, and I'm impressed by how quickly you came up to speed on it. From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 17:37:03 2019 Received: (at 36404) by debbugs.gnu.org; 29 Jun 2019 21:37:03 +0000 Received: from localhost ([127.0.0.1]:44982 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhL1q-00057p-2e for submit@debbugs.gnu.org; Sat, 29 Jun 2019 17:37:03 -0400 Received: from dustycloud.org ([50.116.34.160]:35834) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhL1n-00057Y-IZ for 36404@debbugs.gnu.org; Sat, 29 Jun 2019 17:36:59 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 371AA26655; Sat, 29 Jun 2019 17:36:59 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> <87a7e2an3h.fsf_-_@sdf.lonestar.org> <875zoqan2e.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: guix-patches@gnu.org Subject: Re: [bug#36404] [PATCH 4/6] Export the (gnu machine) interface. In-reply-to: <875zoqan2e.fsf_-_@sdf.lonestar.org> Date: Sat, 29 Jun 2019 17:36:58 -0400 Message-ID: <87imsokr9h.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Jakob L. Kreuze writes: > This is so machine declarations can have a simple (use-modules (gnu)) > rather than having to import the machine module explicitly. +1 From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 17:38:10 2019 Received: (at submit) by debbugs.gnu.org; 29 Jun 2019 21:38:10 +0000 Received: from localhost ([127.0.0.1]:44995 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhL2v-0005AO-Uh for submit@debbugs.gnu.org; Sat, 29 Jun 2019 17:38:10 -0400 Received: from lists.gnu.org ([209.51.188.17]:44032) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhL2t-0005AH-Ue for submit@debbugs.gnu.org; Sat, 29 Jun 2019 17:38:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47561) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhL2t-000440-0f for guix-patches@gnu.org; Sat, 29 Jun 2019 17:38:07 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hhL2r-0006AG-35 for guix-patches@gnu.org; Sat, 29 Jun 2019 17:38:06 -0400 Received: from dustycloud.org ([2600:3c02::f03c:91ff:feae:cb51]:33450) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hhL2o-00067s-46 for guix-patches@gnu.org; Sat, 29 Jun 2019 17:38:03 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 7088B26655; Sat, 29 Jun 2019 17:38:01 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> <87a7e2an3h.fsf_-_@sdf.lonestar.org> <875zoqan2e.fsf_-_@sdf.lonestar.org> <871rzean1i.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: guix-patches@gnu.org Subject: Re: [bug#36404] [PATCH 5/6] Add 'guix deploy'. In-reply-to: <871rzean1i.fsf_-_@sdf.lonestar.org> Date: Sat, 29 Jun 2019 17:38:01 -0400 Message-ID: <87h888kr7q.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c02::f03c:91ff:feae:cb51 X-Spam-Score: 2.2 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Jakob L. Kreuze writes: > 2019-06-26 Jakob L. Kreuze > > * guix/scripts/deploy.scm: Add on-line help and limit verbosity. Looks good. No comments on this one. Content analysis details: (2.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: lonestar.org] -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [209.51.188.17 listed in list.dnswl.org] 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [2600:3c02:0:0:f03c:91ff:feae:cb51 listed in] [zen.spamhaus.org] X-Debbugs-Envelope-To: submit Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.2 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Jakob L. Kreuze writes: > 2019-06-26 Jakob L. Kreuze > > * guix/scripts/deploy.scm: Add on-line help and limit verbosity. Looks good. No comments on this one. Content analysis details: (1.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: lonestar.org] -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [209.51.188.17 listed in list.dnswl.org] 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [2600:3c02:0:0:f03c:91ff:feae:cb51 listed in] [zen.spamhaus.org] 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager Jakob L. Kreuze writes: > 2019-06-26 Jakob L. Kreuze > > * guix/scripts/deploy.scm: Add on-line help and limit verbosity. Looks good. No comments on this one. From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 17:43:21 2019 Received: (at submit) by debbugs.gnu.org; 29 Jun 2019 21:43:21 +0000 Received: from localhost ([127.0.0.1]:45008 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhL7x-0005Hz-Eo for submit@debbugs.gnu.org; Sat, 29 Jun 2019 17:43:21 -0400 Received: from lists.gnu.org ([209.51.188.17]:50693) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhL7w-0005Hs-HZ for submit@debbugs.gnu.org; Sat, 29 Jun 2019 17:43:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48286) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhL7u-0005CF-4J for guix-patches@gnu.org; Sat, 29 Jun 2019 17:43:20 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hhL7p-0001tH-F1 for guix-patches@gnu.org; Sat, 29 Jun 2019 17:43:15 -0400 Received: from dustycloud.org ([2600:3c02::f03c:91ff:feae:cb51]:33464) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hhL7j-0001ob-2a for guix-patches@gnu.org; Sat, 29 Jun 2019 17:43:08 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 8ABAE26655; Sat, 29 Jun 2019 17:43:04 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> <87a7e2an3h.fsf_-_@sdf.lonestar.org> <875zoqan2e.fsf_-_@sdf.lonestar.org> <871rzean1i.fsf_-_@sdf.lonestar.org> <87woh698fn.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: guix-patches@gnu.org Subject: Re: [bug#36404] [PATCH 6/6] doc: Add section for 'guix deploy'. In-reply-to: <87woh698fn.fsf_-_@sdf.lonestar.org> Date: Sat, 29 Jun 2019 17:43:04 -0400 Message-ID: <87ftnskqzb.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c02::f03c:91ff:feae:cb51 X-Spam-Score: 2.2 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Jakob L. Kreuze writes: > +The file should evaluate to a list of machines, rather than just one. This phrasing confused me for a second, because it could be just one machine in that list. Content analysis details: (2.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [209.51.188.17 listed in list.dnswl.org] 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [2600:3c02:0:0:f03c:91ff:feae:cb51 listed in] [zen.spamhaus.org] X-Debbugs-Envelope-To: submit Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.2 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Jakob L. Kreuze writes: > +The file should evaluate to a list of machines, rather than just one. This phrasing confused me for a second, because it could be just one machine in that list. Content analysis details: (1.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [209.51.188.17 listed in list.dnswl.org] 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [2600:3c02:0:0:f03c:91ff:feae:cb51 listed in] [zen.spamhaus.org] 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager Jakob L. Kreuze writes: > +The file should evaluate to a list of machines, rather than just one. This phrasing confused me for a second, because it could be just one machine in that list. How about: The file should evaluate to a list of @var{machine} objects. or: The file should evaluate to a list of @var{} objects. Not sure whether the angle brackets help or hurt. Looks great otherwise. I left a few other comments in response to the other patches; please review and make changes, but I think there isn't too much to do to get this in. IMO we should get this in as quickly as possible; I'd love to do so ideally in the next week so it doesn't stagnate and so people can start trying to use it. Really thrilling stuff Jakob; great work! It's exciting to have you as part of the Guix team. From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 18:05:20 2019 Received: (at 36404) by debbugs.gnu.org; 29 Jun 2019 22:05:20 +0000 Received: from localhost ([127.0.0.1]:45020 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhLTD-0005n6-Pu for submit@debbugs.gnu.org; Sat, 29 Jun 2019 18:05:20 -0400 Received: from sender4-of-o53.zoho.com ([136.143.188.53]:21328) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhLTA-0005mu-Sh for 36404@debbugs.gnu.org; Sat, 29 Jun 2019 18:05:17 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1561845899; cv=none; d=zoho.com; s=zohoarc; b=UnULl2GfeQodh63t6u8zPHHZK8MtvdgU5u3evdcNbQrQzg7SbAjKxTlFcpy0B/8xjGksBU8nmaTqzXrySBcy5ZbUuwuSkPNeCUFYqspMjjNJUPe8k2Li2gE1Gru5bXjjMRNjVn3B80B3Sz5WyM8873ZA8EhGuWNX5irg1V2I47M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1561845899; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To:ARC-Authentication-Results; bh=zOy0L2fK3xnmfbknwu/SZBCATcqtEaAoa7DomR+gziE=; b=OAxOrK81R13Udxpl+iQywwrktOXJHzVqDpq3FLWc2apA0k0oImTKn+du/xYnux7mcXtObi0JGKCOc3vefP8HMYJ3iuSvO/7AIwFp1S8y0ziz/JMltHMA4lq7kfR30HqFVytt04OINEegvWBUO2EalCfV720tsKthQwWTsYU0i2s= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1561845899; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:In-reply-to:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; l=628; bh=zOy0L2fK3xnmfbknwu/SZBCATcqtEaAoa7DomR+gziE=; b=eubqwY08pQHrSdhHccgj3OmCANt1Bq3/C81UaAQCAzYFkljpgD38Kn8vT/fMCtA2 egpHJywiJx/ApoEB+N/LegylMF0fFr9MaSK/+EJqnTyFh8Hlt5n/U/F/99Wpt+tjVOC J2eLSCRXkbWwwRsRGJSEluGjA+FaJcbxsCAA0gh8= Received: from localhost (p54AD40BF.dip0.t-ipconnect.de [84.173.64.191]) by mx.zohomail.com with SMTPS id 1561845897516238.03646310500574; Sat, 29 Jun 2019 15:04:57 -0700 (PDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> <87a7e2an3h.fsf_-_@sdf.lonestar.org> <875zoqan2e.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Ricardo Wurmus To: zerodaysfordays@sdf.lonestar.org Subject: Re: [bug#36404] [PATCH 4/6] Export the (gnu machine) interface. In-reply-to: <875zoqan2e.fsf_-_@sdf.lonestar.org> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Sun, 30 Jun 2019 00:04:53 +0200 Message-ID: <87a7e0vyii.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: cwebber@dustycloud.org, 36404@debbugs.gnu.org, dthompson2@worcester.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Jakob, > This is so machine declarations can have a simple (use-modules (gnu)) > rather than having to import the machine module explicitly. Do we need this at all or could =E2=80=9Cguix deploy=E2=80=9D evaluate the = machine declaration in an environment where the machine module is available? We do something like that for evaluating manifests =E2=80=93 no module relatin= g to manifest loading needs to be specified by users and yet =E2=80=9Cspecifications->manifest=E2=80=9D is available. Would it make sense to do something similar here instead of exporting (gnu machine) in (gnu)? --=20 Ricardo From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 19:44:08 2019 Received: (at 36404) by debbugs.gnu.org; 29 Jun 2019 23:44:08 +0000 Received: from localhost ([127.0.0.1]:45063 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhN0q-00082t-2x for submit@debbugs.gnu.org; Sat, 29 Jun 2019 19:44:08 -0400 Received: from mx.sdf.org ([205.166.94.20]:57154) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhN0n-00082i-LN for 36404@debbugs.gnu.org; Sat, 29 Jun 2019 19:44:06 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5TNhx7i016862 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Sat, 29 Jun 2019 23:44:04 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> Date: Sat, 29 Jun 2019 19:42:14 -0400 In-Reply-To: <87o92glap5.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sat, 29 Jun 2019 10:37:10 -0400") Message-ID: <87h888dkmh.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Hi, Chris! Christopher Lemmer Webber writes: > In the future I think it would be good to make this extensible as > well. Dispatching on a symbol means that Guix must itself provide a > fixed set of possible environment types. If we made this an extensible > structure, akin to services or something, we could allow for more > flexibility in the future. Thoughts for the future, but not a blocker > on this patch. +1. Initially, I thought the service types _were_ symbols, but I see now that they're actually procedures. Thanks for pointing that out. I'll see about implementing environment types similarly in my revised patch set, since I think that's a change that we'd want to make before any other environment types come into existence. Regards, Jakob --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0X91YACgkQ9Qb9Fp2P 2VqixxAAmSCaO7hIk7unux2l3fe6dlL40NMJP4tJ+LE/T2eBOGNaqsEhCv25BBfq UaGH/D7RcDB1DZgfO1jA9HKFj/uerjjrsNGQpcObrZ8D2PeAfUmT+bISW6qb8oKF 4KWyLZ6cBFgS0fBg4Nj6xcqP3TUxb6vMRUUFeGxdkbtYAJwz/SA9ugOEN3q5mHh4 0Wg3AlgaYnRccFr6nAWp3uyLuAZp7rODHTjej18NKyS80sWOIR2ug4hx8S2LQb4Z JJJKz6LH8VjUkS/7T1aG2fW5KmmTRgXKppdauyl9Wdtw0711Qok48csSbXZ4cDFU +2srJy4qNsqjmFXDjJkb4lZ1KpAFAA/Us/45/XfEWvqc7imoxSTv8uXInh1lOVsv DOgnwetJRZfAcT/5UGWWyfnO/xOp7PdAjrvPADh9FyPuF+iaXxWtB20jY7rsMCLd jqfXaJV7kOeYpNyLY4bBYIkQFNJVeY+t9HzFiM9DDR/ta0X5PMbhJUxnLsb8JwTz 6v2cDlYzIER0q5Uixv+rVkMRBUH2v5htGKoEM7gC7e6bdpkcrCso5EqW4H+6FWsR Ds82buuP0I9bF0LmhCOh3QMKzy3d6zBsYplTl19kEKQai3XN+q7MDtoP3Uossa72 e3PKdPMbUPVFu4Ghu2ky1NNW2nRYLun4SLDjuNIjn9M9Dh8D6qs= =nROi -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 19:47:20 2019 Received: (at 36404) by debbugs.gnu.org; 29 Jun 2019 23:47:20 +0000 Received: from localhost ([127.0.0.1]:45067 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhN3w-00087h-JU for submit@debbugs.gnu.org; Sat, 29 Jun 2019 19:47:20 -0400 Received: from mx.sdf.org ([205.166.94.20]:56094) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhN3t-00087Y-4y for 36404@debbugs.gnu.org; Sat, 29 Jun 2019 19:47:19 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5TNlFFH028968 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Sat, 29 Jun 2019 23:47:16 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 1/5] ssh: Add 'identity' keyword to 'open-ssh-session'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87lfxklafz.fsf@dustycloud.org> Date: Sat, 29 Jun 2019 19:45:34 -0400 In-Reply-To: <87lfxklafz.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sat, 29 Jun 2019 10:42:40 -0400") Message-ID: <87d0iwdkgx.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Christopher Lemmer Webber writes: > Looks good, but could you add to the docstring here explaining the new > identity keyword? Added, thanks! --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0X+B4ACgkQ9Qb9Fp2P 2VrBTRAAppBamoeasM9MsLyxszcZ1BPdlbw5R6ROzGm89xwRQy2HyrbJw4JE3WA2 aXxGUkSUzQ+w+XZMgJlVE4u+tgA+/GaP0ovmwJOd74Qq/i6ESP/Dcq2GpzMlOnnM yZ99oKSUMRLvqhwSfNO7SGMJBDpA8wXxQB2nzbTt2+C7Sdk5BzWfu3n8odMBYXf6 M1j1Mwk+iU5GzMBkNTQQzWezX3CMFe3addf6tozpVr3JCzLYbgTbmbeTXFxyY51V bKpQQI9CC+vJ9qFcEg83jet4GeVIc0mvCKUcExPmLb9ZC4pd/fttfIdHVRedBNog NDbhS9Xs5gwBQrHXFSJGVo1bUzCYneDANyhT+mtlG8vhH1xSpoqY9IkchyQ9YFvh vJF09CyWkzXezqEtupWH14RH50I+EKOJhJN8KNVE9jcqTp9rWFBz6GlsICBPMYRJ magmLT4JrIWj2XUjlcHsPv+kAMGZrerq6otuZqZ8gaMxwvJL1Rprd1acqG0YhNIa S7kpYwaesMViVqwgsHzIaMIevio359LRoV5LaWg+nhGHevysaSw2MCSrbBQdYjsn cLI6AXNdwyRhYAcqdWrePvh12OEGQ3jCx0Xl8fXc3bVL/D2mDa2tHzvAA85yesSa OipPJBsPcLIS7llbeosHWJ4NPzrW2fdsxlLJGAqRacjob1iuCsA= =IOIw -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 20:32:21 2019 Received: (at 36404) by debbugs.gnu.org; 30 Jun 2019 00:32:21 +0000 Received: from localhost ([127.0.0.1]:45082 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhNlU-0002Xr-6g for submit@debbugs.gnu.org; Sat, 29 Jun 2019 20:32:20 -0400 Received: from mx.sdf.org ([205.166.94.20]:56994) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhNlQ-0002US-KR for 36404@debbugs.gnu.org; Sat, 29 Jun 2019 20:32:18 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5U0WE6W019332 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Sun, 30 Jun 2019 00:32:15 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 2/5] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <87k1d4kra8.fsf@dustycloud.org> Date: Sat, 29 Jun 2019 20:30:28 -0400 In-Reply-To: <87k1d4kra8.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sat, 29 Jun 2019 17:36:31 -0400") Message-ID: <877e93ewyj.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Christopher Lemmer Webber writes: > Maybe it would make sense to call it machine-remote-eval to > distinguish it? I dunno. Considering the naming used for everything else that '(gnu machine)' exports, I think that makes more sense. And that way I'll be able to just import '(gnu remote ssh)' without shadowing 'remote-eval'. I went ahead and changed it. > @@ is a (sometimes useful) antipattern. But in general, if something is > importing something with @@, it's a good indication that we should just > be exporting it. What do you think? My thinking was that, when we have more than one environment type, @@ could be used with module reflection to get a specific environment's implementation of 'remote-eval'. But going back to your point in an earlier email about implementing environments as distinct types rather than symbols, it would be pretty easy to expose some sort of 'remote-eval' field on those environment types. > Maybe it wouldn't be so hard to do? > > In fact, now that I look at it, we could solve both problems at once: > there's no need to export deploy-machine and remote-eval if they're > wrapped in another structure. Instead, maybe this code could look like: > > #+BEGIN_SRC scheme > (define (remote-eval machine exp) > > "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP refers to > are built and deployed to MACHINE beforehand." > (let* ((environment (machine-environment machine)) > (remote-eval (environment-remote-eval environment))) > (remote-eval machine exp))) > > (define (deploy-machine machine) > "Monadic procedure transferring the new system's OS closure to the remote > MACHINE, activating it on MACHINE and switching MACHINE to the new generation." > (let* ((environment (machine-environment machine)) > (deploy-machine (environment-deploy-machine environment))) > (deploy-machine machine))) > #+END_SRC > > Thoughts? Whoops, wrote the above paragraph before getting here. :] > Feels like better polymorphism than this is desirable, but I'm not > sure I have advice on how to do it right now. Probably services > provide the right form of inspiration. Are you talking about service extensions? I'm starting to see your point regarding polymorphism, since SSH would be the backbone for a lot of these environment types. Does anyone else have suggestions for implementing that sort of polymorphism? > Why not just import remote-eval in the define-module? To avoid a Guile warning about shadowing symbols. This goes away with the renaming of 'remote-eval' to 'machine-remote-eval', though. > It's so cool that this works across machines. Dang! :) > Yeah that sounds like it would be bad. But I'm curious... could you > explain the specific bug it's preventing here? I'd like to know. You've found something I've overlooked. There wasn't a bug, it's something I put in since 'guix system' does it when loading the activation script. But after looking through the 'guix system' code, I noticed that there's a comment reading "[t]his is necessary to ensure that 'upgrade-shepherd-services' gets to see the right modules when it computes derivations with 'gexp->derivation'." Yet, I'm invoking my version of 'upgrade-shepherd-services' outside of that excursion. I haven't had any issues with it so far, but then again, I haven't done much with trying to register new services with 'guix deploy'. I think it's worth fixing. > Just to see if I understand it... this is kind of so we can identify > and "garbage collect" services that don't apply to the new system? Yep. > > I'm a bit unsure from the above code... I'm guessing one of two things > is happening: > > - Either it's starting services that haven't been started yet, but > leaving alone services that are running but which aren't "new" > - Or it's restarting services that are currently running > > Which is it? And mind adding a comment explaining it? The former. I've intentionally avoided restarting services since 'guix system' warns that "many essential services cannot be meaningfully restarted." (which is why 'guix system reconfigure' spits out "To complete the upgrade, run 'herd restart SERVICE' to stop, upgrade, and restart each service that was not automatically restarted." (which AFAIK is always none of them)). > By the way, is there anything about the dependency order in which > services might need to be restarted to be considered? I'm honestly not > sure. I'm not sure either. Would any Shepherd hackers out there care to chime in? > So I guess this is derivative of some of the stuff in > guix/scripts/system.scm. That makes me feel like it would be nice if > it could be generalized, but I haven't spent enough time with the code > to figure out if it really can be. > > I don't want to block the merge on that desire, though if you agree > that generalization between those sections of code is desirable, maybe > add a comment to that effect? You're right, and I agree 100%. I think I can commit to refactoring out the common code, albeit after this patch series is merged -- that's something that deserves its own commit, and it would probably take me some time to get right anyway. > This code also looks very similar, but I compared them and I can see > that they aren't quite the same, at least in that you had to install > the dynamic-wind. But I get the feeling that it still might be > possible to generalize them, so could you leave a comment here as > well? Unless you think it's really not possible to generalize them to > share code for reasons I'm not yet aware of. I think it can be generalized. In fact, 'guix system' does with 'save-load-path-excursion' and 'save-environment-excursion'. If I can't generalize the code from '(gnu machine)' and 'guix system', I'll at least see about exporting those excursions from 'guix system' (they're unexported at the moment). > Seems good from a quick scan, but I'll admit I didn't read these as > carefully as I did the rest of the code. I'm not sure it's really worth reading right now, this is the "me way" of testing everything and I suspect some significant changes are going to be made. > This patch looks great overall! I know it was a lot of work to figure > out, and I'm impressed by how quickly you came up to speed on it. Thank you :) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0YAqUACgkQ9Qb9Fp2P 2Vq2hA//V4qcPwFgkpi4KJx6aGaXzrClwV+alQtJNSM+EWHu+bqgO2fewrn+i/+Y O3g1s/cHa3734bztASd1/FF55aqjKA4NKQ9dVlMVH+6cR0SPW/aDsjF91DUeaz9e K884bnZ8wjSHZlxBYUgkQmxD0gDneLCVf08/Gq3gMgjwgdySOiBMQG8UOezB98mK Qat5G3RRU2qNHolkkaMkiybWr8yFyJyib1VylZ+oe9rXMKxqY7LfjlYwFD6fuXQ2 /+LtXhloVIsoP20Qq7el35T79L22SJ2MS7sXL+JBn7i3LdkYCkIyV13msn3A2G2z mX3YDPX0KmyVMzamcLzZPaABO0OEFYNl6MzyJCItQIIC+0cRPxU4Ea64EjtvLniM mNOUcNcJ0oOtIZJEN3VJDIuOIpIrfx6ed8FJ3Kyll2v4mWgt0r8HQ61HT131Uk9Y GMPdFEgHTZqGeF/Bb75XJFkbDstiu5EUQeG5Ik9y+IhJx/mNA3z2e7m3rZMEEFSf xJAw+V9MI/Ld3V74zHWTMxzxF+oTDXc2xMrrgUGi5+/oA/x01V7/Ow4jmHTcm6w4 xV+gjMsU1L0AFg/jEFlf76S1oszD9JPbMaKHfFJZHFMFOj2VNsVuFm9hZPq8vQff bac6yMTnVkL38XnLfZWMXbU6B3njERvE6zpc5YN9E+6CoJ6T02o= =AOZ5 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 20:37:13 2019 Received: (at 36404) by debbugs.gnu.org; 30 Jun 2019 00:37:13 +0000 Received: from localhost ([127.0.0.1]:45087 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhNqC-0002xF-Eu for submit@debbugs.gnu.org; Sat, 29 Jun 2019 20:37:13 -0400 Received: from mx.sdf.org ([205.166.94.20]:55173) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhNqA-0002x8-TO for 36404@debbugs.gnu.org; Sat, 29 Jun 2019 20:37:11 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5U0b9Ff021603 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Sun, 30 Jun 2019 00:37:09 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 6/6] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> <87a7e2an3h.fsf_-_@sdf.lonestar.org> <875zoqan2e.fsf_-_@sdf.lonestar.org> <871rzean1i.fsf_-_@sdf.lonestar.org> <87woh698fn.fsf_-_@sdf.lonestar.org> <87ftnskqzb.fsf@dustycloud.org> Date: Sat, 29 Jun 2019 20:35:27 -0400 In-Reply-To: <87ftnskqzb.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sat, 29 Jun 2019 17:43:04 -0400") Message-ID: <8736jrewq8.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Christopher Lemmer Webber writes: > This phrasing confused me for a second, because it could be just one > machine in that list. > > How about: > > The file should evaluate to a list of @var{machine} objects. > or: > The file should evaluate to a list of @var{} objects. > > Not sure whether the angle brackets help or hurt. I couldn't find anywhere else in the manual where the brackets are used for type names (at least with @var), so I went with the former. > Looks great otherwise. I left a few other comments in response to the > other patches; please review and make changes, but I think there isn't > too much to do to get this in. IMO we should get this in as quickly as > possible; I'd love to do so ideally in the next week so it doesn't > stagnate and so people can start trying to use it. Awesome, I'll get on putting together the revised patch series. > Really thrilling stuff Jakob; great work! It's exciting to have you as > part of the Guix team. Aw, shucks. That made my day. It's exciting to be a part of the Guix team! Regards, Jakob --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0YA88ACgkQ9Qb9Fp2P 2VrJDRAArTKDk76fNjfLS9Pnj9lTtURKasfClbifNG1uyElMKeEV06ifIG+7OcbQ fFrIM02mYhi2Yc4BtDn4Ten9pTfVs7jV26Y2scSjgqieLuoJXq/rESoSzjZOojqa iQkvbbc8afH8QI9nd3Uxy+n1bSWEwR2eoGlV1wkVo8q6rFrB4h5RXsBau/dHn79n dwUIGBJbYA0QlUndgsO8zWruuljZmtdD+/PqO5zpiDzUidFSGdlLM032NynkLLqB Uw0afmA//TVcqYZvdNj5032AMhMXqHECWb1rrHXkVDJaSb6Wtc5NU0RiAg/jwTvx U0ui55EN7FKRnZBrH+tmVoYpaOcQ1Zq38AluXAkSYowBxOftlpQ8cyyoiIkzPqm0 ie/w/a458xxPM1c8+OGjmER4qC4bsdXYwOM4YaN9f1GDDDYgpRJO4jfND7GuYfo+ wfHUszvX48vxTAzKZ2yHidJE8EWkjdWgtF/T2XP5NWMVtZw3hrXColwJFQyGoxCB JZ/qQfF21Y3bv0ElVRO4jq/2n246ovl/EX/XY/9be1ZTX/Qf8l3eWYgCj8yxFGP+ AsvLv6MlM2v+vk9BDySu0fPF1g09zhHPASKDpphmqfqi6xtLgVwsPVyfyQuVr5oc pUNFp54Lw4ZQQNZsTy11m7Ev5nkbh3R3L5ZCJ7h2W9se84O2mds= =jJ17 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 20:43:00 2019 Received: (at 36404) by debbugs.gnu.org; 30 Jun 2019 00:43:00 +0000 Received: from localhost ([127.0.0.1]:45092 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhNvo-00034t-00 for submit@debbugs.gnu.org; Sat, 29 Jun 2019 20:43:00 -0400 Received: from mx.sdf.org ([205.166.94.20]:53028) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhNvm-00034l-Cz for 36404@debbugs.gnu.org; Sat, 29 Jun 2019 20:42:58 -0400 Received: from Epsilon (pool-173-76-53-40.bstnma.fios.verizon.net [173.76.53.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x5U0gtaW007338 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Sun, 30 Jun 2019 00:42:57 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ricardo Wurmus Subject: Re: [bug#36404] [PATCH 4/6] Export the (gnu machine) interface. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imsqan66.fsf@sdf.lonestar.org> <87ef3ean4i.fsf_-_@sdf.lonestar.org> <87a7e2an3h.fsf_-_@sdf.lonestar.org> <875zoqan2e.fsf_-_@sdf.lonestar.org> <87a7e0vyii.fsf@elephly.net> Date: Sat, 29 Jun 2019 20:41:14 -0400 In-Reply-To: <87a7e0vyii.fsf@elephly.net> (Ricardo Wurmus's message of "Sun, 30 Jun 2019 00:04:53 +0200") Message-ID: <87y31jdhw5.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: cwebber@dustycloud.org, 36404@debbugs.gnu.org, dthompson2@worcester.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ricardo, Ricardo Wurmus writes: > Do we need this at all or could =E2=80=9Cguix deploy=E2=80=9D evaluate th= e machine > declaration in an environment where the machine module is available? > We do something like that for evaluating manifests =E2=80=93 no module > relating to manifest loading needs to be specified by users and yet > =E2=80=9Cspecifications->manifest=E2=80=9D is available. > > Would it make sense to do something similar here instead of exporting > (gnu machine) in (gnu)? Thanks for that comment; I'd completely forgotten about not having to import 'specifications->manifest'. I doubt the machine types will see much use outside of deployment specifications, so something like that would definitely make sense here. I'll add it in. Regards, Jakob --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0YBSoACgkQ9Qb9Fp2P 2VoNPhAAlvb2jH2+dMywjp/deZjKnxBHSmmiJeBNsYmX7AwbbjE7fFJlNPmtq4hw HoWr3+d98s09tpRwhrerY2dyJHRvcCWMtY2R0X0pYYxePe4UbGPkNgm2+ttn0MxO 3igIBj0FqycWPEVGHhOoWEffKW3cjeAdnWoev+J93XZcT02V40NYy839D8yj/RL5 tS1SQnux39vWfMh/5JTa9RCk5Uvq2rhrYGKIrZyFPVg13vGrfqNxMnYPwk0PPoqB zfn9/RxnUeoZvC6EfrA+uhvBNSz93mjLDbxrdxAhSoo0D1bb1xvQ68IG8oZ6ha/v 1bFBwAx6XwYOVsxKCebzv1qb5L9mtF1ZDK118uEV5Bn+VGPyVBRw4eZwf9qL+D3J qDInMpF/Ie/LEA6bzTh2DfgODRgGy7EC8eCp7GqdOSLXBEXLLvrjJv59kkexJ92b waTue6J3EnOnzNMiQ8G24xm2/ibOU6nMcR2J0Rpb4sK//jlT2XWQvSdwCLaevwh3 DfG8JKjVsy8V0lFO4D2Pmaz3fm3mw5lt9tPcRuy4nf02kwtzADxTNUhDw/X799wg xa4IWSh5drFfTBeEuPk/LTdmZrnPKRpTjYM1WZrhzeiTlENOPc99iCFanmt0xdfv Gz7EUDkLewGhZZ87l3IJHmB0gKhZ5VZHJ5t5uz+UY0NK81wwWfc= =MogR -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 00:58:59 2019 Received: (at submit) by debbugs.gnu.org; 30 Jun 2019 04:58:59 +0000 Received: from localhost ([127.0.0.1]:45226 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhRvW-000393-O1 for submit@debbugs.gnu.org; Sun, 30 Jun 2019 00:58:59 -0400 Received: from lists.gnu.org ([209.51.188.17]:39073) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhRvV-00038v-9p for submit@debbugs.gnu.org; Sun, 30 Jun 2019 00:58:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57636) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhRvU-0000aF-3G for guix-patches@gnu.org; Sun, 30 Jun 2019 00:58:57 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hhRvT-0007RL-0o for guix-patches@gnu.org; Sun, 30 Jun 2019 00:58:56 -0400 Received: from zancanaro.com.au ([45.76.117.151]:35616) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hhRvS-0007NH-MX for guix-patches@gnu.org; Sun, 30 Jun 2019 00:58:54 -0400 Received: from jolteon (210-1-202-160-cpe.spintel.net.au [210.1.202.160]) by zancanaro.com.au (Postfix) with ESMTPSA id 01E902886D; Sun, 30 Jun 2019 04:58:41 +0000 (UTC) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <87k1d4kra8.fsf@dustycloud.org> <877e93ewyj.fsf@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Carlo Zancanaro To: guix-patches@gnu.org Subject: Re: [bug#36404] [PATCH 2/5] gnu: Add machine type for deployment specifications. In-reply-to: <877e93ewyj.fsf@sdf.lonestar.org> Date: Sun, 30 Jun 2019 14:58:39 +1000 Message-ID: <8736jrvfcw.fsf@zancanaro.id.au> MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 45.76.117.151 X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: Christopher Lemmer Webber , "Jakob L. Kreuze" , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Hey Jakob/Chris, I can't comment on much of the deploy code, but I can help out with some stuff about the Shepherd. On Sun, Jun 30 2019, Jakob L. Kreuze wrote: >> I'm a bit unsure from the above code... I'm guessing one of two >> things >> is happening: >> >> - Either it's starting services that haven't been started yet, >> but >> leaving alone services that are running but which aren't >> "new" >> - Or it's restarting services that are currently running >> >> Which is it? And mind adding a comment explaining it? > > The former. I've intentionally avoided restarting services since > 'guix > system' warns that "many essential services cannot be > meaningfully > restarted." (which is why 'guix system reconfigure' spits out > "To > complete the upgrade, run 'herd restart SERVICE' to stop, > upgrade, and > restart each service that was not automatically restarted." > (which AFAIK > is always none of them)). There was discussion earlier this year around restarting services that are already running during a reconfigure[1]. I wonder if this problem is more worth solving if we're deploying to remote systems. I have a few patches in that issue to implement service restarting, but I didn't follow them up enough to get them into Guix. [1]: https://issues.guix.info/issue/33508 >> By the way, is there anything about the dependency order in >> which >> services might need to be restarted to be considered? I'm >> honestly not >> sure. > > I'm not sure either. Would any Shepherd hackers out there care > to chime > in? The Shepherd will start any necessary dependencies in an appropriate order. Carlo From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 08:28:50 2019 Received: (at 36404) by debbugs.gnu.org; 30 Jun 2019 12:28:50 +0000 Received: from localhost ([127.0.0.1]:45460 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhYws-0001Um-0N for submit@debbugs.gnu.org; Sun, 30 Jun 2019 08:28:50 -0400 Received: from dustycloud.org ([50.116.34.160]:36234) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhYwp-0001Ue-NJ for 36404@debbugs.gnu.org; Sun, 30 Jun 2019 08:28:48 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 18D9A265C8; Sun, 30 Jun 2019 08:28:46 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <87k1d4kra8.fsf@dustycloud.org> <877e93ewyj.fsf@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: "Jakob L. Kreuze" Subject: Re: [bug#36404] [PATCH 2/5] gnu: Add machine type for deployment specifications. In-reply-to: <877e93ewyj.fsf@sdf.lonestar.org> Date: Sun, 30 Jun 2019 08:28:45 -0400 Message-ID: <87blyfl0jm.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Jakob L. Kreuze writes: > Christopher Lemmer Webber writes: > >> Feels like better polymorphism than this is desirable, but I'm not >> sure I have advice on how to do it right now. Probably services >> provide the right form of inspiration. > > Are you talking about service extensions? I'm starting to see your point > regarding polymorphism, since SSH would be the backbone for a lot of > these environment types. Does anyone else have suggestions for > implementing that sort of polymorphism? Right now it looks like you're hard-coding dispatch into the procedure by doing a case analysis of what type it is, but this doesn't allow us to extend it. Here I'd look at how service-type works. Check out gnu/services.scm and then some examples of how services are defined in say, gnu/services/admin.scm or something (eg rotlog-service-type). I'm not saying structure it in exactly this way, but that seems to be the right general pattern to do extensibility in the guix'y way: - Have a common outer type (eg ) which actually sets up the structure of this service type - Then have the actual records that are specific to the service type represented as the service-value. Section 8.16.2 "Serivce Types and Services" and 6.16.3 "Service Reference" for details. Note that I wish there was a way to generalize the ideas behind this pattern rather than have it be reinvented for everything that needs them. This is part of why David and I turned to GOOPS in the initial prototype implementation; it's a lot of work figuring out how to set up extensibility in this way, at least for me. You might want to write a quick GOOPS version to understand what all the parameters are that are needed, then convert it to the services way of doing a general structure that wraps a specific structure. I suspect you won't need as much composability as services currently need, so the implementation of whatever this extensibility is is probably not as complicated as it is for services. As for how to share the ssh code, maybe just having the building-block procedures is good enough? Since all we support, so far, is this kind of ssh'ing, I don't want this to block the patch though. It could be that we file this as a bug and add a TODO above the code for the moment saying "we know this isn't right/ideal". However, there is some risk that this could result in people writing out machine configurations that later break... I dunno. Thoughts? >> Why not just import remote-eval in the define-module? > > To avoid a Guile warning about shadowing symbols. This goes away with > the renaming of 'remote-eval' to 'machine-remote-eval', though. Heh :) >> Yeah that sounds like it would be bad. But I'm curious... could you >> explain the specific bug it's preventing here? I'd like to know. > > You've found something I've overlooked. There wasn't a bug, it's > something I put in since 'guix system' does it when loading the > activation script. But after looking through the 'guix system' code, I > noticed that there's a comment reading "[t]his is necessary to ensure > that 'upgrade-shepherd-services' gets to see the right modules when it > computes derivations with 'gexp->derivation'." Yet, I'm invoking my > version of 'upgrade-shepherd-services' outside of that excursion. I > haven't had any issues with it so far, but then again, I haven't done > much with trying to register new services with 'guix deploy'. I think > it's worth fixing. Cool. Yay reviews! If you remove it, please leave a comment noting the difference between this and "guix system" and why you thought it was safe to remove. If it turns out to not be the case, there's a breadcrumb there to figure out how to add it back. >> Just to see if I understand it... this is kind of so we can identify >> and "garbage collect" services that don't apply to the new system? > > Yep. > >> >> I'm a bit unsure from the above code... I'm guessing one of two things >> is happening: >> >> - Either it's starting services that haven't been started yet, but >> leaving alone services that are running but which aren't "new" >> - Or it's restarting services that are currently running >> >> Which is it? And mind adding a comment explaining it? > > The former. I've intentionally avoided restarting services since 'guix > system' warns that "many essential services cannot be meaningfully > restarted." (which is why 'guix system reconfigure' spits out "To > complete the upgrade, run 'herd restart SERVICE' to stop, upgrade, and > restart each service that was not automatically restarted." (which AFAIK > is always none of them)). Aha. Thank you for explaining! This make ssense. >> By the way, is there anything about the dependency order in which >> services might need to be restarted to be considered? I'm honestly not >> sure. > > I'm not sure either. Would any Shepherd hackers out there care to chime > in? I guess if you aren't restarting the services, it's no longer a big deal. >> So I guess this is derivative of some of the stuff in >> guix/scripts/system.scm. That makes me feel like it would be nice if >> it could be generalized, but I haven't spent enough time with the code >> to figure out if it really can be. >> >> I don't want to block the merge on that desire, though if you agree >> that generalization between those sections of code is desirable, maybe >> add a comment to that effect? > > You're right, and I agree 100%. I think I can commit to refactoring out > the common code, albeit after this patch series is merged -- that's > something that deserves its own commit, and it would probably take me > some time to get right anyway. Great! >> This code also looks very similar, but I compared them and I can see >> that they aren't quite the same, at least in that you had to install >> the dynamic-wind. But I get the feeling that it still might be >> possible to generalize them, so could you leave a comment here as >> well? Unless you think it's really not possible to generalize them to >> share code for reasons I'm not yet aware of. > > I think it can be generalized. In fact, 'guix system' does with > 'save-load-path-excursion' and 'save-environment-excursion'. If I can't > generalize the code from '(gnu machine)' and 'guix system', I'll at > least see about exporting those excursions from 'guix system' (they're > unexported at the moment). Okay, cool. >> Seems good from a quick scan, but I'll admit I didn't read these as >> carefully as I did the rest of the code. > > I'm not sure it's really worth reading right now, this is the "me way" > of testing everything and I suspect some significant changes are going > to be made. Kk. >> This patch looks great overall! I know it was a lot of work to figure >> out, and I'm impressed by how quickly you came up to speed on it. > > Thank you :) Thank *you*! From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 08:34:48 2019 Received: (at submit) by debbugs.gnu.org; 30 Jun 2019 12:34:48 +0000 Received: from localhost ([127.0.0.1]:45473 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhZ2d-0001fQ-Kr for submit@debbugs.gnu.org; Sun, 30 Jun 2019 08:34:47 -0400 Received: from lists.gnu.org ([209.51.188.17]:57369) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhZ2Z-0001fG-5h for submit@debbugs.gnu.org; Sun, 30 Jun 2019 08:34:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37375) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhZ2X-0006nz-Qx for guix-patches@gnu.org; Sun, 30 Jun 2019 08:34:42 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hhZ2W-0002Ip-Kf for guix-patches@gnu.org; Sun, 30 Jun 2019 08:34:41 -0400 Received: from dustycloud.org ([50.116.34.160]:35436) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hhZ2W-0002ID-GI for guix-patches@gnu.org; Sun, 30 Jun 2019 08:34:40 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id BFA85265C8; Sun, 30 Jun 2019 08:34:38 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <87k1d4kra8.fsf@dustycloud.org> <877e93ewyj.fsf@sdf.lonestar.org> <8736jrvfcw.fsf@zancanaro.id.au> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: Carlo Zancanaro Subject: Re: [bug#36404] [PATCH 2/5] gnu: Add machine type for deployment specifications. In-reply-to: <8736jrvfcw.fsf@zancanaro.id.au> Date: Sun, 30 Jun 2019 08:34:38 -0400 Message-ID: <87a7dzl09t.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 50.116.34.160 X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: "Jakob L. Kreuze" , 36404@debbugs.gnu.org, guix-patches@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Carlo Zancanaro writes: > Hey Jakob/Chris, > > I can't comment on much of the deploy code, but I can help out with > some stuff about the Shepherd. > > On Sun, Jun 30 2019, Jakob L. Kreuze wrote: >>> I'm a bit unsure from the above code... I'm guessing one of two >>> things >>> is happening: >>> >>> - Either it's starting services that haven't been started yet, >>> but >>> leaving alone services that are running but which aren't >>> "new" >>> - Or it's restarting services that are currently running >>> >>> Which is it? And mind adding a comment explaining it? >> >> The former. I've intentionally avoided restarting services since >> 'guix >> system' warns that "many essential services cannot be meaningfully >> restarted." (which is why 'guix system reconfigure' spits out "To >> complete the upgrade, run 'herd restart SERVICE' to stop, upgrade, >> and >> restart each service that was not automatically restarted." (which >> AFAIK >> is always none of them)). > > There was discussion earlier this year around restarting services that > are already running during a reconfigure[1]. I wonder if this problem > is more worth solving if we're deploying to remote systems. I have a > few patches in that issue to implement service restarting, but I > didn't follow them up enough to get them into Guix. > > [1]: https://issues.guix.info/issue/33508 Wow! This seems highly desireable, especially if, as you pointed out in the issue, an update to nginx is pushed across the wire with a security update... in that case, we'd want to restart that, too. Jakob, do you mind checking out the issue above? I think it shouldn't block merging these patches but perhaps we should file an issue saying that when the shepherd issue is merged, changes should be made to guix deploy as well. What do you think? >>> By the way, is there anything about the dependency order in which >>> services might need to be restarted to be considered? I'm honestly >>> not >>> sure. >> >> I'm not sure either. Would any Shepherd hackers out there care to >> chime >> in? > > The Shepherd will start any necessary dependencies in an appropriate > order. > > Carlo Ok, good to know! From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 06:09:45 2019 Received: (at 36404) by debbugs.gnu.org; 1 Jul 2019 10:09:45 +0000 Received: from localhost ([127.0.0.1]:47695 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhtFp-0005li-2w for submit@debbugs.gnu.org; Mon, 01 Jul 2019 06:09:45 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21246) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhtFn-0005lZ-A6 for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 06:09:43 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1561975779; cv=none; d=zoho.com; s=zohoarc; b=DN8wfs5D4PcwZHyPeJrzDTT7iO0/UeoMkwnlYusEK4qff2SKIJ+PEO7rk+3E7/3fioe+Pg9nUMnzqaxaa0QR1URouEBxgeJheIoV5bSPLYe/GufRSEqMgEv6n5V90263yWdxFINoSeni7GGHU3EZQBRugGAkmUr8xcqzEGXMNgY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1561975779; h=Content-Type:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To:ARC-Authentication-Results; bh=r5gudYzbJmz83wgjSIeV7iW5yXpng94TklFhmocSSnA=; b=CW9CQj99CUF/3Ln3lEimDKBB0ade5vQHgbGY1Urf/YZYK3GK28XduJG+mrfBaZLtu1AuILECIok9JIVceUuYLTaoEPH+lHqtET9Y++4fcbexHduF2oJQ0Fu6rYhbkuWq5dIg9os4w9/tLM5XAul1EuyAOzQlKCRCzcCyzT6KnJc= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1561975779; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:In-reply-to:Date:Message-ID:MIME-Version:Content-Type; l=1195; bh=r5gudYzbJmz83wgjSIeV7iW5yXpng94TklFhmocSSnA=; b=gjhSb8Xeo6BLxCRFzroCMx1nJr4rBxCrwul/o0GIZF3cfCR+EOlguOsG13k2RWqR 31z2CxVwTLGrzh8M1f2WQ00rZD7E+HbuCRaTrr0F4B7Ud5Vxl+R4Pr26MjS41DgM3XC RWEVmMt3YyvKFeIpwel7gxpD9I7PC5p0b1CSwXDw= Received: from localhost (141.80.247.250 [141.80.247.250]) by mx.zohomail.com with SMTPS id 1561975779050506.12130615285537; Mon, 1 Jul 2019 03:09:39 -0700 (PDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Ricardo Wurmus To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. In-reply-to: <87o92glap5.fsf@dustycloud.org> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Mon, 01 Jul 2019 12:09:35 +0200 Message-ID: <871rzavzfk.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain X-ZohoMailClient: External X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Christopher Lemmer Webber writes: >> First, I still haven't figured out the whole testing situation. The >> tests, as of now, spin up a virtual machine, create a machine instance, >> deploy that to the virtual machine, and then make assertions about >> changes made to the system. These tests were originally in the system >> test suite as they deal with virtual machines, but I've since moved it >> into the normal Guix test suite because of how much needs to be done on >> the host side -- I spent an absurd amount of time trying to fit a call >> to 'deploy-machine' into a derivation that could be run by the system >> test suite, but I just wasn't able to make it work. I'm hoping someone >> will have thoughts about how we can test 'guix deploy'. Should we have >> them disabled by default? Is there some way to implement them in the a >> system test suite that I've overlooked? Should the tests be included at >> all? > > Ludo, do you have comments? I suspect this is up your area of expertise. Building and running virtual machines as part of the tests seems expensive. Would it be feasible to mock the remote interactions? -- Ricardo From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 08:49:02 2019 Received: (at 36404) by debbugs.gnu.org; 1 Jul 2019 12:49:02 +0000 Received: from localhost ([127.0.0.1]:47744 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhvjx-0003Jx-UL for submit@debbugs.gnu.org; Mon, 01 Jul 2019 08:49:02 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55237) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhvjs-0003Ja-4U for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 08:48:57 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49057) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hhvjm-0007nO-Kp; Mon, 01 Jul 2019 08:48:50 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=51670 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hhvjl-00054m-VM; Mon, 01 Jul 2019 08:48:50 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> Date: Mon, 01 Jul 2019 14:48:47 +0200 In-Reply-To: <87o92ianbj.fsf@sdf.lonestar.org> (Jakob L. Kreuze's message of "Thu, 27 Jun 2019 14:35:28 -0400") Message-ID: <87o92dor80.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Jakob & all! zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > This patch provides the basis for 'guix deploy', implementing what I've > referred to as the "simple case" in my progress reports: in-place > updates to machines (physical or virtual) whose name and IP address we > know well. Do note that these commits depend on Ludovic's implementation > of 'remote-eval'.[1] Woohoo! > There's certainly more to be done with this -- the GSoC period is far > from over, and I'm hoping to use that time to implement more complex > use-cases such as automatically provisioning virtual machines in the > cloud. I'm submitting a patch series now per the recommendation of my > mentors to break the project into a few chunks to submit over the > duration of the summer. That=E2=80=99s an impressive achievement! I=E2=80=99m all for integrating = patches piecemeal, and it=E2=80=99s great that you=E2=80=99ve managed to have sizab= le chunks already. > Quite a bit has changed since my last email about this.[2] For one, > GOOPS is no longer used. Machine declarations now look just like any > other sort of declaration in Guix. Neat. I prefer it this way, at least for consistency. If for some reason this turns out to make extensibility more cumbersome, like Chris wrote, we can rediscuss it. My feeling is that we can make do without GOOPS _and_ without reimplementing GOOPS mechanisms in a poor way, but if that=E2=80=99s not the case, we can adjust. > (list (machine > (system %system) > (environment 'managed-host) > (configuration (machine-ssh-configuration > (host-name "localhost") > (identity "./id_rsa") > (port 2222))))) > #+END_SRC scheme > > There are a number of other differences here as well. For one, the SSH > configuration now has an 'identity' field for specifying a private key > to use when authenticating with the host. Any key management scheme you > might have set up in '~/.ssh/config' will also work if the 'identity' > field is omitted. > > The 'environment' field is where we declare how machines should be > provisioned. In this case, the only type of provisioning that's been > implemented is 'managed-host' -- the "simple case" of in-place updates > to a machine that's already running GuixSD. The parameters for > provisioning are given in the form of an environment-specific > configuration type. In the example, this is 'machine-ssh-configuration', > which describes how 'guix deploy' should make an SSH connection to the > machine. I'm sure you can imagine something along the lines of a > 'machine-digitalocean-configuration', describing some parameters for a > droplet. Nice. I=E2=80=99ll take a closer look and to comment on the other issues you rais= e, but so far this looks very nice! Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 08:50:30 2019 Received: (at 36404) by debbugs.gnu.org; 1 Jul 2019 12:50:30 +0000 Received: from localhost ([127.0.0.1]:47749 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhvlN-0003MR-Cy for submit@debbugs.gnu.org; Mon, 01 Jul 2019 08:50:30 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55543) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhvlL-0003ME-PR for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 08:50:28 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49069) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hhvlF-0000Xo-99; Mon, 01 Jul 2019 08:50:21 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=51672 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hhvlD-00058z-E1; Mon, 01 Jul 2019 08:50:20 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <87h888dkmh.fsf@sdf.lonestar.org> Date: Mon, 01 Jul 2019 14:50:16 +0200 In-Reply-To: <87h888dkmh.fsf@sdf.lonestar.org> (Jakob L. Kreuze's message of "Sat, 29 Jun 2019 19:42:14 -0400") Message-ID: <87k1d1or5j.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > Christopher Lemmer Webber writes: > >> In the future I think it would be good to make this extensible as >> well. Dispatching on a symbol means that Guix must itself provide a >> fixed set of possible environment types. If we made this an extensible >> structure, akin to services or something, we could allow for more >> flexibility in the future. Thoughts for the future, but not a blocker >> on this patch. > > +1. Initially, I thought the service types _were_ symbols, but I see now > that they're actually procedures. Thanks for pointing that out. I'll see > about implementing environment types similarly in my revised patch set, > since I think that's a change that we'd want to make before any other > environment types come into existence. It=E2=80=99s a pattern similar to that of for packages. I t= hink it should provide the flexibility and extensibility we need. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 08:54:05 2019 Received: (at 36404) by debbugs.gnu.org; 1 Jul 2019 12:54:05 +0000 Received: from localhost ([127.0.0.1]:47753 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhvor-0003Rc-3l for submit@debbugs.gnu.org; Mon, 01 Jul 2019 08:54:05 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56141) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhvop-0003R9-A5 for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 08:54:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49091) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hhvoi-00035k-A3; Mon, 01 Jul 2019 08:53:56 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=51674 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hhvog-0005Oi-Ja; Mon, 01 Jul 2019 08:53:55 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> Date: Mon, 01 Jul 2019 14:53:53 +0200 In-Reply-To: <87o92glap5.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sat, 29 Jun 2019 10:37:10 -0400") Message-ID: <878sthoqzi.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org, "Jakob L. Kreuze" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Christopher Lemmer Webber skribis: > Jakob L. Kreuze writes: [...] >> There are two things in this patch series that I'd like comments on in >> particular. >> >> First, I still haven't figured out the whole testing situation. The >> tests, as of now, spin up a virtual machine, create a machine instance, >> deploy that to the virtual machine, and then make assertions about >> changes made to the system. These tests were originally in the system >> test suite as they deal with virtual machines, but I've since moved it >> into the normal Guix test suite because of how much needs to be done on >> the host side -- I spent an absurd amount of time trying to fit a call >> to 'deploy-machine' into a derivation that could be run by the system >> test suite, but I just wasn't able to make it work. I'm hoping someone >> will have thoughts about how we can test 'guix deploy'. Should we have >> them disabled by default? Is there some way to implement them in the a >> system test suite that I've overlooked? Should the tests be included at >> all? > > Ludo, do you have comments? I suspect this is up your area of expertise. As Ricardo wrote, I think that=E2=80=99s too much work to do in =E2=80=9Cma= ke check=E2=80=9D. Plus this would only run when a =E2=80=9Chost store=E2=80=9D is available, = as we can=E2=80=99t reasonably build QEMU and everything in $builddir/test-tmp. So I feel that the system test suite is a better fit, but I don=E2=80=99t f= ully understand the limitations you hit, Jakob. Do you still have a draft of a system test that you wrote and/or notes about what went wrong? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 19:52:26 2019 Received: (at submit) by debbugs.gnu.org; 1 Jul 2019 23:52:27 +0000 Received: from localhost ([127.0.0.1]:46299 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi65y-0004GK-Li for submit@debbugs.gnu.org; Mon, 01 Jul 2019 19:52:26 -0400 Received: from lists.gnu.org ([209.51.188.17]:45144) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi65x-0004GC-JU for submit@debbugs.gnu.org; Mon, 01 Jul 2019 19:52:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48715) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hi65v-0003LH-Tx for guix-patches@gnu.org; Mon, 01 Jul 2019 19:52:24 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hi65s-0007AW-33 for guix-patches@gnu.org; Mon, 01 Jul 2019 19:52:21 -0400 Received: from ol.sdf.org ([205.166.94.20]:49705 helo=mx.sdf.org) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hi65o-00074D-5B for guix-patches@gnu.org; Mon, 01 Jul 2019 19:52:17 -0400 Received: from Upsilon (mobile-107-107-57-169.mycingular.net [107.107.57.169]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x61Npp9u018172 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Mon, 1 Jul 2019 23:51:58 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 2/5] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <87k1d4kra8.fsf@dustycloud.org> <877e93ewyj.fsf@sdf.lonestar.org> <8736jrvfcw.fsf@zancanaro.id.au> <87a7dzl09t.fsf@dustycloud.org> Date: Mon, 01 Jul 2019 19:51:50 -0400 In-Reply-To: <87a7dzl09t.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sun, 30 Jun 2019 08:34:38 -0400") Message-ID: <875zolb9ex.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 205.166.94.20 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: Carlo Zancanaro , 36404@debbugs.gnu.org, guix-patches@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain Christopher Lemmer Webber writes: > Jakob, do you mind checking out the issue above? I think it shouldn't > block merging these patches but perhaps we should file an issue saying > that when the shepherd issue is merged, changes should be made to guix > deploy as well. What do you think? I took a peek and added a comment about it to machine.scm, are you suggesting that we track it on debbugs? --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0anJYACgkQ9Qb9Fp2P 2Vo0rg//SzvtNWytn3zn/yzDzO8ddYIZn5YCeOQYkHznEXHKEWQCCqfjF02LTcMu Y22kL+9jkry7mBaBMKbpJBdORWZ1tb0e1LdVf4JuWnniep1JW89B0J9e42iLt288 XiW4gx5CmS222MdsLtjdDDt9FRN99tI8qUzJp2CtcLYzyO66eBrSjC0dK5nTM9HL lk6OpsBdu5OXVYC4mmWrmOiX+fnf0UbjImqgwFEdBD8Wal+Wg8XpAeCA7lMQJTY/ 16et0A36STT2aBa6FUwOZGOSw/z7ab28aw2Wtoh6GDpEhNlwfMpSg2kh5Hinq8gE iN8muqMEd0+RhHMRnzaBawd6gWFvaHalHzBrV7t7hc2AD94ahiEICDvE54vgzHQQ 2Q27Y/fVyG24JObg0YjUnWIII3tsABbnBgwpu5SD7rG/RhMI+Ct8/pjGCLKMU0dL kX2EsGiRpae9kGga0Bc163z2Io+aGCTjBdE/PMdYtsIZf34X8+0pLvLps30iN3hP 5FZ9xHlCxC4sJIq+16qx4kKF2PfuSwcadn35ERr+5pXVTyOGPfDDZj5unB9axOc+ Av302u/8kcuz/idkvjC6cfDQKzIOn/iCm7LNsoF/mvtjGBNzFKMUu7yeYWj3iJYO Y1gLKf4WO3IreokrT7AootZFEbBNndx30uM99O0NPCJQ4NFgQnc= =pJQU -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 20:03:22 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 00:03:22 +0000 Received: from localhost ([127.0.0.1]:46304 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6GX-0004Z7-Os for submit@debbugs.gnu.org; Mon, 01 Jul 2019 20:03:22 -0400 Received: from mx.sdf.org ([205.166.94.20]:62350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6GV-0004Yy-FP for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 20:03:20 -0400 Received: from Upsilon (mobile-107-107-57-169.mycingular.net [107.107.57.169]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x6203GcI027890 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 00:03:17 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 2/5] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <87k1d4kra8.fsf@dustycloud.org> <877e93ewyj.fsf@sdf.lonestar.org> <87blyfl0jm.fsf@dustycloud.org> Date: Mon, 01 Jul 2019 20:03:15 -0400 In-Reply-To: <87blyfl0jm.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sun, 30 Jun 2019 08:28:45 -0400") Message-ID: <87y31h9ubg.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Christopher Lemmer Webber writes: > Right now it looks like you're hard-coding dispatch into the procedure > by doing a case analysis of what type it is, but this doesn't allow us > to extend it. > > Here I'd look at how service-type works. Check out gnu/services.scm > and then some examples of how services are defined in say, > gnu/services/admin.scm or something (eg rotlog-service-type). I'm not > saying structure it in exactly this way, but that seems to be the > right general pattern to do extensibility in the guix'y way: > > - Have a common outer type (eg ) which actually sets up > the structure of this service type > - Then have the actual records that are specific to the service type > represented as the service-value. > > Section 8.16.2 "Serivce Types and Services" and 6.16.3 "Service > Reference" for details. > > Note that I wish there was a way to generalize the ideas behind this > pattern rather than have it be reinvented for everything that needs > them. This is part of why David and I turned to GOOPS in the initial > prototype implementation; it's a lot of work figuring out how to set > up extensibility in this way, at least for me. You might want to write > a quick GOOPS version to understand what all the parameters are that > are needed, then convert it to the services way of doing a general > structure that wraps a specific structure. > > I suspect you won't need as much composability as services currently > need, so the implementation of whatever this extensibility is is > probably not as complicated as it is for services. > > As for how to share the ssh code, maybe just having the building-block > procedures is good enough? > > Since all we support, so far, is this kind of ssh'ing, I don't want > this to block the patch though. It could be that we file this as a bug > and add a TODO above the code for the moment saying "we know this > isn't right/ideal". However, there is some risk that this could result > in people writing out machine configurations that later break... I > dunno. > > Thoughts? Ah, so you mean having the configuration as part of the environment type rather than the machine type? I think that does make more sense... If that is what you meant, let me know and I'll send another patch implementing the change tomorrow. It should be an easy fix. > If you remove it, please leave a comment noting the difference between > this and "guix system" and why you thought it was safe to remove. If it > turns out to not be the case, there's a breadcrumb there to figure out > how to add it back. Added :] --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0an0MACgkQ9Qb9Fp2P 2VrPuA/+JJaQ4eqZHmOySQXfn2j/rcUphWRbVC/F3rVWbad6m3lWYl01NsOpNepi 8JOLgMbmeeWM2W6RjZjtH30etYPTwMoSGUHMI7p7lP9ImU5YMrD/ePsZc18wXM2N TClgUE6nX0R8ZMfS2iqVxxYllOkJsbR2+XiGSsofXfrHtFRwmjPJ65e4RCEmcaFC 7FQl7JCfuX/RGJdZjS2qsfHx1Fe6I03tsZdRMS9lldaY+Va7uoenK7hdav/i0NNV lwz0HwifUZrxqWHJgYDybARdMkUWuQBZlslyOg81vkbYOTD4kOtByKmO0hsI0dM8 R9IvACvn4aNDFzTjE3g7HcezjOOcDqMn/9i1tPr3Qn8DrkVoj2AdYF7uQ+pzXZkQ Lz4tgbtriEzTV/uGWpJ3lmxOoqKn2YQjuqdmc0wl0QtXNOQchFUIjVx4mWfK5EP6 cHls/yB7Rh9Iub79T/VliqSgkE2M/Q9fK8RX77aNRYoCi6aqcQ4s79eEvZXpG/Uf kJWAZqawtpd0N01d9iKtNXMAPLxJB25X4ftEkiCG/6q0/oaS0zZshJYIN8CcM62t K6CoGNCwZoS111X6yDp5qomL4XCm3jRmJHSUwamjHNhhHpIdrz+vUrdLEQjy4vxb mb0Sit8kM8dTIaOhM+f+G5/poavPUbG2Prd6os7sjxoWpzeWDgA= =c13S -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 20:10:37 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 00:10:37 +0000 Received: from localhost ([127.0.0.1]:46316 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6NZ-0004jd-4h for submit@debbugs.gnu.org; Mon, 01 Jul 2019 20:10:37 -0400 Received: from mx.sdf.org ([205.166.94.20]:59017) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6NW-0004jV-Qk for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 20:10:35 -0400 Received: from Upsilon (mobile-107-107-57-169.mycingular.net [107.107.57.169]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x620AVBI008320 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 00:10:33 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> Date: Mon, 01 Jul 2019 20:10:30 -0400 In-Reply-To: <878sthoqzi.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 01 Jul 2019 14:53:53 +0200") Message-ID: <87r2799tzd.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , Ricardo Wurmus , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Ludovic + Ricardo! Ricardo Wurmus writes: > Building and running virtual machines as part of the tests > seems expensive. Would it be feasible to mock the remote > interactions? I agree 100%. I've decoupled it from my patch series for now. We can always add it back later when it's implemented in a less expensive way. As for mocking -- I do like that idea, but that would only really be testing that calls to 'deploy-machine' et al. don't fail rather than ensuring that the implementation of 'guix deploy' does what it's supposed to do. The current tests make assertions about changes to the virtual machine. Ludovic Court=C3=A8s writes: > As Ricardo wrote, I think that=E2=80=99s too much work to do in =E2=80=9C= make check=E2=80=9D. > Plus this would only run when a =E2=80=9Chost store=E2=80=9D is available= , as we can=E2=80=99t > reasonably build QEMU and everything in $builddir/test-tmp. > > So I feel that the system test suite is a better fit, but I don=E2=80=99t > fully understand the limitations you hit, Jakob. > > Do you still have a draft of a system test that you wrote and/or notes > about what went wrong? Yep, I have an unsquashed commit history on my personal branch with all renditions of the test suite. I can pull it out tomorrow and write a detailed report on the issues I ran into. Thanks for both of your comments! --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0aoPYACgkQ9Qb9Fp2P 2VqHCg/+LJNO1ZF6NrWWS71gXjJlIgllQWYkerDPKE/kq8QhB17lTXkXEWsp+C6u CM12n7kszLHHRTG/y2N7+noQVy2YbOsIoODfJ3RK/oJ+tLwlcH6Ce6XajV9dKis/ 4f5FCRwHtr+cqDqnNzwnUTzzpLsHur/HmFWySgt4gpeijO3ran1Wsgoq7Bodk/tS pjVc3SuxK0fDuc3zg/b9ZiPwq2N1UdbAjOXF00OkOpttDitFNSK/Za0zUxTcxwPW rVngx8YGG1ExRfm+1jqOsmESpt6DP7mfp82JALZnB/cQC284ylYfVfWNNqt7jatv aTGiRKjOomrwkBa1R6zkbaQjiWk+LxXJGFlzjP4yWnmKZNxG0DniAjD4qCXu0KBw gklHMBxj7Tsijp7VntR2cpBB/TbsjazYHtZDbNZAYuxobbNUFPbrLHQeCOwfqyl+ SDzWQdw4vtnvJ6dGigEb2cCgsmHwPgRIZUN1ktFiRiUw7p7HVD3O0RxW8oG+Tgkf XM32kaj63Y11kb3iJIpphOq/l4Zjk/H/kdgYGBt0y3kSl5GjCo2/FzwGj05Yq1gY mdd3N3M++QVMzNy+Nz52c/cHjPgnRbfd8f74LnwOvpcEvcGJENh2dpRfJnTfKJbb +4C/nIY9M0eMgjOCCpAo3NQFmcGOjAkFk3NQpRs8R205PFmMJ/s= =9Bbp -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 20:14:28 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 00:14:28 +0000 Received: from localhost ([127.0.0.1]:46327 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6RI-0004pt-5J for submit@debbugs.gnu.org; Mon, 01 Jul 2019 20:14:28 -0400 Received: from mx.sdf.org ([205.166.94.20]:57592) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6RH-0004pl-7G for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 20:14:27 -0400 Received: from Upsilon (mobile-107-107-57-169.mycingular.net [107.107.57.169]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x620ENc0010741 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 00:14:25 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 0/4] Add 'guix deploy'. In-Reply-To: <878sthoqzi.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 01 Jul 2019 14:53:53 +0200") References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) Date: Mon, 01 Jul 2019 20:14:22 -0400 Message-ID: <87imsl9tsx.fsf_-_@sdf.lonestar.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Huge thanks to everyone who commented on the first two renditions of this patch series. Here's a summary of the changes I've incorporated: % The 'environment' field of is now an instance of -- a record similar to . See the manual page for an example of how this looks in a deployment specification. % Deployment specifications are loaded in an environment with '(gnu)', '(gnu machine)', and '(gnu machine ssh)'. '(gnu machine)' and its descendant modules are no longer exported from '(gnu)'. % Environment and load path excursions have been removed from the deployment internals for 'managed-host-environment-type'. 'remote-eval' spawns a new Guile REPL with each invocation, so modifications to $PATH et al. aren't really relevant -- at least not with how 'deploy-managed-host' is implemented. % Wording in the manual section has been updated. % The docstring for 'open-ssh-session' has been updated. % Tests have been decoupled from the commit adding '(gnu machine)' and omit= ted from this patch series. I will add them back in a future patch. Jakob L. Kreuze (4): ssh: Add 'identity' keyword to 'open-ssh-session'. gnu: Add machine type for deployment specifications. Add 'guix deploy'. doc: Add section for 'guix deploy'. Makefile.am | 4 +- doc/guix.texi | 101 ++++++++++++ gnu/local.mk | 5 +- gnu/machine.scm | 118 +++++++++++++ gnu/machine/ssh.scm | 355 ++++++++++++++++++++++++++++++++++++++++ guix/scripts/deploy.scm | 90 ++++++++++ guix/ssh.scm | 10 +- 7 files changed, 677 insertions(+), 6 deletions(-) create mode 100644 gnu/machine.scm create mode 100644 gnu/machine/ssh.scm create mode 100644 guix/scripts/deploy.scm =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0aod4ACgkQ9Qb9Fp2P 2VpBcBAArpK08La4zdZh9gsZZXGwWEMd2Qm+Qrf/rElJmpyCG1kibg+miI7QZREf Z8VuRt3GN2Tdch0govFghwC+STDF+uZkbPc0Tv7Mi6D8aBxphjJtHvRlcTu39C8o Owo9nVME4kioe3oSykJl5tDNY4f23d40/Dhv14/vSXsheuWG8rbvAZrc2JZaEC+U fJRUlxK+TJOY4rYck5e93eKr6KC/t/gTb0WKapTI/ZDPzJ92Co5N0G+FQge/wxsI +VtAHRD7Tw/7I4x2Y75ZUAc52VEIXejJ9jijhNqGrcw2GCCBlmHN/Gf9j7rEhGqU 3cQCgQ6PPepKfKpfoXZllKHCd+q9Eh1ZsF7ukiqT9y4qaRZXxS+VBHZ9MXuRW5Jp 93aqMfrn2v1Q8JBfnhgIYAMJDg8+6NB8xHzz6mZbHgvMj/7gYIU2nVv+kQrac6oS t+jVgrDGZ7P8xIBDIINXTtMWObAUj5ZgbjCevcoSZ1cq9VJqT7ser1+C0Roi2sny mys69X0kHDxPlSxE7D/lX8jnVIcfZfB/Wa5EfqURYPonN++1oJ42CHsvhw+D3Y5g 236SLu6idRGsAjCuzao8v6h1GOjM46hMUG4q0/x4TfrKk6u1V6aDz5Mry7gjql8O 07SNpnu6tFeEDDI1D12KL7Imq2Dwp4MIN1tmg3eFP6bH8P+Tsdg= =IIRT -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 20:16:22 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 00:16:22 +0000 Received: from localhost ([127.0.0.1]:46331 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6T5-0004tP-Kk for submit@debbugs.gnu.org; Mon, 01 Jul 2019 20:16:22 -0400 Received: from mx.sdf.org ([205.166.94.20]:56364) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6T3-0004tD-Qh for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 20:16:18 -0400 Received: from Upsilon (mobile-107-107-57-169.mycingular.net [107.107.57.169]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x620GEbf004314 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 00:16:16 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 1/4] ssh: Add 'identity' keyword to 'open-ssh-session'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> Date: Mon, 01 Jul 2019 20:16:13 -0400 In-Reply-To: <87imsl9tsx.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Mon, 01 Jul 2019 20:14:22 -0400") Message-ID: <87ef399tpu.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * guix/ssh.scm (open-ssh-session): Add 'identity' keyword argument =2D-- guix/ssh.scm | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/guix/ssh.scm b/guix/ssh.scm index 9b9baf54e..9bf10b9a0 100644 =2D-- a/guix/ssh.scm +++ b/guix/ssh.scm @@ -57,12 +57,14 @@ (define %compression "zlib@openssh.com,zlib") =20 =2D(define* (open-ssh-session host #:key user port +(define* (open-ssh-session host #:key user port identity (compression %compression)) =2D "Open an SSH session for HOST and return it. When USER and PORT are #= f, use =2Ddefault values or whatever '~/.ssh/config' specifies; otherwise use them. =2DThrow an error on failure." + "Open an SSH session for HOST and return it. IDENTITY specifies the pat= h of +a private key to use for authenticating with the host. When USER, PORT, or +IDENTITY are #f, use default values or whatever '~/.ssh/config' specifies; +otherwise use them. Throw an error on failure." (let ((session (make-session #:user user + #:identity identity #:host host #:port port #:timeout 10 ;seconds =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0aok0ACgkQ9Qb9Fp2P 2VrLag//W5i6JFyqJbMWgFFMBBhFS2APNje5iw8QAfJnhhh+ybKBIleLXT0O1Mhj nK8EuFdb6BOg+RsFkgxgQxnuQ1h3J7FIv4baTZlkoy/FmUtXuMFgJQsGpF7JqmQY HdgZyJqZc+kqQxqxn1dE0f8KZCWMyBJJq1SV8G5t5qdexMTNxtwN36HTakWmM8UU lKPuUxmii4b51FlBj2zlbM/mbiUkEgdbxHi0+ePPSbhMHBr37TmEIrkbhlTIn/NO f7NTB1MmxITI55dLXiHikE/xrBQaxaYc8/HOU7ff7Ky1lNMFmmEy0/hrS+W9VAVp upuLgz8s6qVDg0dRGhaVI/3fmpKGfxL3npD4xtFg7WtD/ttR7Jtw2qzhsdAHHOYh vIXR0xTn4sDxuSytW5caXfiHFkusBlqf6YI81Kj5zBAQZoMUwvnRc6q6oQbsWzWX 0x6TbWi72/5L3aW5rVonsH2Q0Aw7M8HXdrcEX5o0dYM15GSUhsqKwB0LKwP86bxN WF+Qu3nPBSbfaA4+GGymlRbCKolnHGYEp+UtqDDxpreJ8hcrlRT+HluW/YmbhaZg 0o6/TNzrFi1CFkRJemQJLI4K3bUaOcfQXXIwZVA+bmwaIMt95+MPh+udWpslGsLc IefOh8sF3tMrJJ6mCvZ2Bm4yn2DxoIZ1r52+aYtGuks6AcPbRww= =Eta+ -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 20:17:21 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 00:17:21 +0000 Received: from localhost ([127.0.0.1]:46335 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6U1-0004v1-8i for submit@debbugs.gnu.org; Mon, 01 Jul 2019 20:17:21 -0400 Received: from mx.sdf.org ([205.166.94.20]:55471) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6Tv-0004up-91 for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 20:17:13 -0400 Received: from Upsilon (mobile-107-107-57-169.mycingular.net [107.107.57.169]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x620H4es006465 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 00:17:06 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 2/4] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> Date: Mon, 01 Jul 2019 20:17:03 -0400 In-Reply-To: <87ef399tpu.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Mon, 01 Jul 2019 20:16:13 -0400") Message-ID: <87a7dx9tog.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable * gnu/machine.scm: New file. * gnu/machine/ssh.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. =2D-- Makefile.am | 3 +- gnu/local.mk | 5 +- gnu/machine.scm | 118 ++++++++++++++ gnu/machine/ssh.scm | 363 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 487 insertions(+), 2 deletions(-) create mode 100644 gnu/machine.scm create mode 100644 gnu/machine/ssh.scm diff --git a/Makefile.am b/Makefile.am index 42307abae..f10c000ea 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -425,7 +425,8 @@ SCM_TESTS =3D \ tests/import-utils.scm \ tests/store-database.scm \ tests/store-deduplication.scm \ =2D tests/store-roots.scm + tests/store-roots.scm \ + tests/machine.scm =20 SH_TESTS =3D \ tests/guix-build.sh \ diff --git a/gnu/local.mk b/gnu/local.mk index 81de156cf..0e17af953 100644 =2D-- a/gnu/local.mk +++ b/gnu/local.mk @@ -562,6 +562,9 @@ GNU_SYSTEM_MODULES =3D \ %D%/system/uuid.scm \ %D%/system/vm.scm \ \ + %D%/machine.scm \ + %D%/machine/ssh.scm \ + \ %D%/build/accounts.scm \ %D%/build/activation.scm \ %D%/build/bootloader.scm \ @@ -627,7 +630,7 @@ INSTALLER_MODULES =3D \ %D%/installer/newt/user.scm \ %D%/installer/newt/utils.scm \ %D%/installer/newt/welcome.scm \ =2D %D%/installer/newt/wifi.scm=09 + %D%/installer/newt/wifi.scm =20 # Always ship the installer modules but compile them only when # ENABLE_INSTALLER is true. diff --git a/gnu/machine.scm b/gnu/machine.scm new file mode 100644 index 000000000..3dfcab797 =2D-- /dev/null +++ b/gnu/machine.scm @@ -0,0 +1,118 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu machine) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix store) + #:use-module ((guix utils) #:select (source-properties->location)) + #:export (environment-type + environment-type? + environment-type-name + environment-type-description + environment-type-location + + machine + machine? + this-machine + + machine-system + machine-environment + machine-configuration + machine-display-name + + build-machine + deploy-machine + machine-remote-eval)) + +;;; Commentary: +;;; +;;; This module provides the types used to declare individual machines in a +;;; heterogeneous Guix deployment. The interface allows users of specify s= ystem +;;; configurations and the means by which resources should be provisioned = on a +;;; per-host basis. +;;; +;;; Code: + + +;;; +;;; Declarations for resources that can be provisioned. +;;; + +(define-record-type* environment-type + make-environment-type + environment-type? + + ;; Interface to the environment type's deployment code. Each procedure + ;; should take the same arguments as the top-level procedure of this file + ;; that shares the same name. For example, 'machine-remote-eval' should = be + ;; of the form '(machine-remote-eval machine exp)'. + (machine-remote-eval environment-type-machine-remote-eval) ; procedure + (deploy-machine environment-type-deploy-machine) ; procedure + + ;; Metadata. + (name environment-type-name) ; symbol + (description environment-type-description ; string + (default #f)) + (location environment-type-location ; + (default (and=3D> (current-source-location) + source-properties->location)) + (innate))) + + +;;; +;;; Declarations for machines in a deployment. +;;; + +(define-record-type* machine + make-machine + machine? + this-machine + (system machine-system) ; + (environment machine-environment) ; symbol + (configuration machine-configuration ; configuration object + (default #f))) ; specific to environment + +(define (machine-display-name machine) + "Return the host-name identifying MACHINE." + (operating-system-host-name (machine-system machine))) + +(define (build-machine machine) + "Monadic procedure that builds the system derivation for MACHINE and ret= urning +a list containing the path of the derivation file and the path of the deri= vation +output." + (let ((os (machine-system machine))) + (mlet* %store-monad ((osdrv (operating-system-derivation os)) + (_ ((store-lift build-derivations) (list osdrv)))) + (return (list (derivation-file-name osdrv) + (derivation->output-path osdrv)))))) + +(define (machine-remote-eval machine exp) + "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP refe= rs to +are built and deployed to MACHINE beforehand." + (let ((environment (machine-environment machine))) + ((environment-type-machine-remote-eval environment) machine exp))) + +(define (deploy-machine machine) + "Monadic procedure transferring the new system's OS closure to the remote +MACHINE, activating it on MACHINE and switching MACHINE to the new generat= ion." + (let ((environment (machine-environment machine))) + ((environment-type-deploy-machine environment) machine))) diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm new file mode 100644 index 000000000..6ce106bb2 =2D-- /dev/null +++ b/gnu/machine/ssh.scm @@ -0,0 +1,363 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu machine ssh) + #:use-module (gnu bootloader) + #:use-module (gnu machine) + #:autoload (gnu packages gnupg) (guile-gcrypt) + #:use-module (gnu services) + #:use-module (gnu services shepherd) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix gexp) + #:use-module (guix i18n) + #:use-module (guix modules) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix remote) + #:use-module (guix ssh) + #:use-module (guix store) + #:use-module (ice-9 match) + #:use-module (srfi srfi-19) + #:export (managed-host-environment-type + + machine-ssh-configuration + machine-ssh-configuration? + machine-ssh-configuration + + machine-ssh-configuration-host-name + machine-ssh-configuration-port + machine-ssh-configuration-user + machine-ssh-configuration-session)) + +;;; Commentary: +;;; +;;; This module implements remote evaluation and system deployment for +;;; machines that are accessable over SSH and have a known host-name. In t= he +;;; sense of the broader "machine" interface, we describe the environment = for +;;; such machines as 'managed-host. +;;; +;;; Code: + + +;;; +;;; Parameters for the SSH client. +;;; + +(define-record-type* machine-ssh-configuration + make-machine-ssh-configuration + machine-ssh-configuration? + this-machine-ssh-configuration + (host-name machine-ssh-configuration-host-name) ; string + (port machine-ssh-configuration-port ; integer + (default 22)) + (user machine-ssh-configuration-user ; string + (default "root")) + (identity machine-ssh-configuration-identity ; path to a private key + (default #f)) + (session machine-ssh-configuration-session ; session + (default #f))) + +(define (machine-ssh-session machine) + "Return the SSH session that was given in MACHINE's configuration, or cr= eate +one from the configuration's parameters if one was not provided." + (let ((config (machine-configuration machine))) + (if (machine-ssh-configuration? config) + (or (machine-ssh-configuration-session config) + (let ((host-name (machine-ssh-configuration-host-name config)) + (user (machine-ssh-configuration-user config)) + (port (machine-ssh-configuration-port config)) + (identity (machine-ssh-configuration-identity config))) + (open-ssh-session host-name + #:user user + #:port port + #:identity identity))) + (error "unsupported configuration type")))) + + +;;; +;;; Remote evaluation. +;;; + +(define (managed-host-remote-eval machine exp) + "Internal implementation of 'machine-remote-eval' for MACHINE instances = with +an environment type of 'managed-host." + (maybe-raise-missing-configuration-error machine) + (remote-eval exp (machine-ssh-session machine))) + + +;;; +;;; System deployment. +;;; + +(define (switch-to-system machine) + "Monadic procedure creating a new generation on MACHINE and execute the +activation script for the new system configuration." + (define (remote-exp drv script) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles) + (guix utils))) + #~(begin + (use-modules (guix config) + (guix profiles) + (guix utils)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (let* ((system #$(derivation->output-path drv)) + (number (1+ (generation-number %system-profile))) + (generation (generation-file-name %system-profile numbe= r))) + (switch-symlinks generation system) + (switch-symlinks %system-profile generation) + ;; The implementation of 'guix system reconfigure' saves the + ;; load path and environment here. This is unnecessary here + ;; because each invocation of 'remote-eval' runs in a distin= ct + ;; Guile REPL. + (setenv "GUIX_NEW_SYSTEM" system) + ;; The activation script may write to stdout, which confuses + ;; 'remote-eval' when it attempts to read a result from the + ;; remote REPL. We work around this by forcing the output to= a + ;; string. + (with-output-to-string + (lambda () + (primitive-load #$script)))))))) + + (let* ((os (machine-system machine)) + (script (operating-system-activation-script os))) + (mlet* %store-monad ((drv (operating-system-derivation os))) + (machine-remote-eval machine (remote-exp drv script))))) + +;; XXX: Currently, this does NOT attempt to restart running services. This= is +;; also the case with 'guix system reconfigure'. +;; +;; See . +(define (upgrade-shepherd-services machine) + "Monadic procedure unloading and starting services on the remote as need= ed +to realize the MACHINE's system configuration." + (define target-services + ;; Monadic expression evaluating to a list of (name output-path) pairs= for + ;; all of MACHINE's services. + (mapm %store-monad + (lambda (service) + (mlet %store-monad ((file ((compose lower-object + shepherd-service-file) + service))) + (return (list (shepherd-service-canonical-name service) + (derivation->output-path file))))) + (service-value + (fold-services (operating-system-services (machine-system machi= ne)) + #:target-type shepherd-root-service-type)))) + + (define (remote-exp target-services) + (with-imported-modules '((gnu services herd)) + #~(begin + (use-modules (gnu services herd) + (srfi srfi-1)) + + (define running + (filter live-service-running (current-services))) + + (define (essential? service) + ;; Return #t if SERVICE is essential and should not be unloaded + ;; under any circumstance. + (memq (first (live-service-provision service)) + '(root shepherd))) + + (define (obsolete? service) + ;; Return #t if SERVICE can be safely unloaded. + (and (not (essential? service)) + (every (lambda (requirements) + (not (memq (first (live-service-provision servic= e)) + requirements))) + (map live-service-requirement running)))) + + (define to-unload + (filter obsolete? + (remove (lambda (service) + (memq (first (live-service-provision service= )) + (map first '#$target-services))) + running))) + + (define to-start + (remove (lambda (service-pair) + (memq (first service-pair) + (map (compose first live-service-provision) + running))) + '#$target-services)) + + ;; Unload obsolete services. + (for-each (lambda (service) + (false-if-exception + (unload-service service))) + to-unload) + + ;; Load the service files for any new services and start them. + (load-services/safe (map second to-start)) + (for-each start-service (map first to-start)) + + #t))) + + (mlet %store-monad ((target-services target-services)) + (machine-remote-eval machine (remote-exp target-services)))) + +(define (machine-boot-parameters machine) + "Monadic procedure returning a list of 'boot-parameters' for the generat= ions +of MACHINE's system profile, ordered from most recent to oldest." + (define bootable-kernel-arguments + (@@ (gnu system) bootable-kernel-arguments)) + + (define remote-exp + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles))) + #~(begin + (use-modules (guix config) + (guix profiles) + (ice-9 textual-ports)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (define (read-file path) + (call-with-input-file path + (lambda (port) + (get-string-all port)))) + + (map (lambda (generation) + (let* ((system-path (generation-file-name %system-profi= le + generation)) + (boot-parameters-path (string-append system-path + "/parameter= s")) + (time (stat:mtime (lstat system-path)))) + (list generation + system-path + time + (read-file boot-parameters-path)))) + (reverse (generation-numbers %system-profile))))))) + + (mlet* %store-monad ((generations (machine-remote-eval machine remote-ex= p))) + (return + (map (lambda (generation) + (match generation + ((generation system-path time serialized-params) + (let* ((params (call-with-input-string serialized-params + read-boot-parameters)) + (root (boot-parameters-root-device params)) + (label (boot-parameters-label params))) + (boot-parameters + (inherit params) + (label + (string-append label " (#" + (number->string generation) ", " + (let ((time (make-time time-utc 0 time))) + (date->string (time-utc->date time) + "~Y-~m-~d ~H:~M")) + ")")) + (kernel-arguments + (append (bootable-kernel-arguments system-path root) + (boot-parameters-kernel-arguments params)))))))) + generations)))) + +(define (install-bootloader machine) + "Create a bootloader entry for the new system generation on MACHINE, and +configure the bootloader to boot that generation by default." + (define bootloader-installer-script + (@@ (guix scripts system) bootloader-installer-script)) + + (define (remote-exp installer bootcfg bootcfg-file) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((gnu build install) + (guix store) + (guix utils))) + #~(begin + (use-modules (gnu build install) + (guix store) + (guix utils)) + (let* ((gc-root (string-append "/" %gc-roots-directory "/bootc= fg")) + (temp-gc-root (string-append gc-root ".new"))) + + (switch-symlinks temp-gc-root gc-root) + + (unless (false-if-exception + (begin + ;; The implementation of 'guix system reconfigure' + ;; saves the load path here. This is unnecessary = here + ;; because each invocation of 'remote-eval' runs = in a + ;; distinct Guile REPL. + (install-boot-config #$bootcfg #$bootcfg-file "/") + ;; The installation script may write to stdout, w= hich + ;; confuses 'remote-eval' when it attempts to rea= d a + ;; result from the remote REPL. We work around th= is + ;; by forcing the output to a string. + (with-output-to-string + (lambda () + (primitive-load #$installer))))) + (delete-file temp-gc-root) + (error "failed to install bootloader")) + + (rename-file temp-gc-root gc-root) + #t))))) + + (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine))) + (let* ((os (machine-system machine)) + (bootloader ((compose bootloader-configuration-bootloader + operating-system-bootloader) + os)) + (bootloader-target (bootloader-configuration-target + (operating-system-bootloader os))) + (installer (bootloader-installer-script + (bootloader-installer bootloader) + (bootloader-package bootloader) + bootloader-target + "/")) + (menu-entries (map boot-parameters->menu-entry boot-parameters)) + (bootcfg (operating-system-bootcfg os menu-entries)) + (bootcfg-file (bootloader-configuration-file bootloader))) + (machine-remote-eval machine (remote-exp installer bootcfg bootcfg-f= ile))))) + +(define (deploy-managed-host machine) + "Internal implementation of 'deploy-machine' for MACHINE instances with = an +environment type of 'managed-host." + (maybe-raise-missing-configuration-error machine) + (mbegin %store-monad + (switch-to-system machine) + (upgrade-shepherd-services machine) + (install-bootloader machine))) + + +;;; +;;; Environment type. +;;; + +(define managed-host-environment-type + (environment-type + (machine-remote-eval managed-host-remote-eval) + (deploy-machine deploy-managed-host) + (name 'managed-host-environment-type) + (description "Provisioning for machines that are accessable ove= r SSH +and have a known host-name. This entails little more than maintaining an S= SH +connection to the host."))) + +(define (maybe-raise-missing-configuration-error machine) + "Raise an error if MACHINE's configuration is #f." + (let ((environment (machine-environment machine))) + (unless (machine-configuration machine) + (error (format #f (G_ "no configuration specified for environment '~= a'") + (symbol->string (environment-type-name environment)))= )))) =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0aon8ACgkQ9Qb9Fp2P 2VonoxAAheBG0IqFzgdSpo9+0txDp8aE8fnDqO8e17PNbi8q5YwdFUV8GCiGgM9a Y82YgEN4+Ksl/ywbs5AdpFuQZjLr8ZHSYWblQG1k8HY+7ZY2tEjykvybds7t4fyE viVpfYUA+CTnnFM1b3yRGpWtwKh2Bj0kuoenRqml3L50PiI6soMmdB4a1Vsg5iNz OxJeNcl3KhW1XTMHgwSliIakeziCpthJwzo2ahIGGRha5wI4d1j3OVIg1CNnowmi pyZfyLD0JBiwHq9s407m0VfOJ6Sa7ZA+8lnZQnuEqDYcj5t41FQ+lgdke+gOai2k BsWEiH2unhb7s6j4vGJsQ4cgTKY/QV7AtutaEiwJbn0YK6vsh/zO3aqHeSP941Fh qK2ouLdDTkObQ0bYxijzp6h8oyiq4Fk34Itz7dGUgXZgkYbKxx0lowTNALnd0K6G Z6IxtoogPMWU99I43Y1711TWQoBo/9BboTvA5hXLDJcxFSIINy/jgqbxcwB/XtkX IIRRyiVgXPdJG3rpz8bxPFyck6qKQqm5eqjeHHOzW+GfbnxEui9uQrZLBYhXWT11 vTtY85u+F2+qjT2mwmiDZJEdXxgn0O5iMAK0RgXx97AaOxEJ+HGcHCkNKGT0ye0J apXnvSA2qTBTuXYSd8/5WG5LwjitTXsxsqCXTfvLrHJndTKMUqk= =Ku2d -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 20:18:00 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 00:18:00 +0000 Received: from localhost ([127.0.0.1]:46338 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6Uh-0004vy-RT for submit@debbugs.gnu.org; Mon, 01 Jul 2019 20:18:00 -0400 Received: from mx.sdf.org ([205.166.94.20]:54914) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6Uf-0004vq-UG for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 20:17:59 -0400 Received: from Upsilon (mobile-107-107-57-169.mycingular.net [107.107.57.169]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x620Hsak027915 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 00:17:56 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 3/4] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> Date: Mon, 01 Jul 2019 20:17:53 -0400 In-Reply-To: <87a7dx9tog.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Mon, 01 Jul 2019 20:17:03 -0400") Message-ID: <875zol9tn2.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable * guix/scripts/deploy.scm: New file. * Makefile.am (MODULES): Add it. =2D-- Makefile.am | 1 + guix/scripts/deploy.scm | 90 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) create mode 100644 guix/scripts/deploy.scm diff --git a/Makefile.am b/Makefile.am index f10c000ea..4d3024e58 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -267,6 +267,7 @@ MODULES =3D \ guix/scripts/weather.scm \ guix/scripts/container.scm \ guix/scripts/container/exec.scm \ + guix/scripts/deploy.scm \ guix.scm \ $(GNU_SYSTEM_MODULES) =20 diff --git a/guix/scripts/deploy.scm b/guix/scripts/deploy.scm new file mode 100644 index 000000000..4fb1babe8 =2D-- /dev/null +++ b/guix/scripts/deploy.scm @@ -0,0 +1,90 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts deploy) + #:use-module (gnu machine) + #:use-module (guix scripts) + #:use-module (guix scripts build) + #:use-module (guix store) + #:use-module (guix ui) + #:use-module (ice-9 format) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-37) + #:export (guix-deploy)) + +;;; Commentary: +;;; +;;; This program provides a command-line interface to (gnu machine), allow= ing +;;; users to perform remote deployments through specification files. +;;; +;;; Code: + + + +(define (show-help) + (display (G_ "Usage: guix deploy [OPTION] FILE... +Perform the deployment specified by FILE.\n")) + (show-build-options-help) + (newline) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + %standard-build-options)) + +(define %default-options + '((system . ,(%current-system)) + (substitutes? . #t) + (build-hook? . #t) + (graft? . #t) + (debug . 0) + (verbosity . 2))) + +(define (load-source-file file) + "Load FILE as a user module." + (let ((module (make-user-module '((gnu) (gnu machine) (gnu machine ssh))= ))) + (load* file module))) + +(define (guix-deploy . args) + (define (handle-argument arg result) + (alist-cons 'file arg result)) + (let* ((opts (parse-command-line args %options (list %default-options) + #:argument-handler handle-argument)) + (file (assq-ref opts 'file)) + (machines (or (and file (load-source-file file)) '()))) + (with-store store + (set-build-options-from-command-line store opts) + (for-each (lambda (machine) + (format #t "building ~a... " (machine-display-name machi= ne)) + (run-with-store store (build-machine machine)) + (display "done\n")) + machines) + (for-each (lambda (machine) + (format #t "deploying to ~a... " (machine-display-name m= achine)) + (run-with-store store (deploy-machine machine)) + (display "done\n")) + machines)))) =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0aorEACgkQ9Qb9Fp2P 2Vpb6w/9GUSI9Yyt/ldvb+iJmsBANdGToqYSf8XMcBfcPIW/k6tcO6nYkVQeaVdd 0+ghKnQB9BSvN6GS+6AjVB26rLZtr8E79XzhncCElPcxuLMbcN/8espusKGVJTqO MUDbHguwhG2OsxBRUuhf0TFN+92BmUwPsdLRq7zQUTLRHRAhLcpg2CvdfPOxY2lC HiIhMnIAgB0DHpzJTN9k5wrKaB0PjzSDLSZ1d0T8/dvbqCI5rdkCOeyNzz7YdTkE 3/LM8pbsOkpReq8CcMhinfQs30I/vMLkO+Xc5PcINi8qyAMw04CgAKbMGHEiFita mjvvDMw0eNdJoy8w7um6l1OVgrFkhQP0axVW7NVavWJle7Sd4bazT8k4ozoRBHqV ARt6io93f9KgUevFAl/+i3Qj0lwPvK+oR1331eoWgNleKc030IV805hzcy0i3WaD mqhiC0J0EtsiGGN9eCr334peD4fANxJB+jB2K/C5BtCDYYmrBmIvQXprdvKld2k2 Tij5skoFolwJG9rJ41eCLCH2q9HpDJ7qZtDs2nVBjoxXthWGMebBGSa7XinUFeQM u1IandCeMs/zmng5BwcehkZqE8jObVUusF6BoyGchLGfjsJgno/1llyCoGDx4KFX F5K57mnxCofvNuEZltQEVtT9MRxfFHM2IiBXKf2KbhYQJdVK8j4= =d9YP -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 01 20:19:08 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 00:19:08 +0000 Received: from localhost ([127.0.0.1]:46344 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6Vo-0004yA-Ex for submit@debbugs.gnu.org; Mon, 01 Jul 2019 20:19:08 -0400 Received: from mx.sdf.org ([205.166.94.20]:54124) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hi6Vm-0004y2-TV for 36404@debbugs.gnu.org; Mon, 01 Jul 2019 20:19:07 -0400 Received: from Upsilon (mobile-107-107-57-169.mycingular.net [107.107.57.169]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x620J0b9024446 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 00:19:05 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 4/4] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> Date: Mon, 01 Jul 2019 20:18:58 -0400 In-Reply-To: <875zol9tn2.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Mon, 01 Jul 2019 20:17:53 -0400") Message-ID: <871rz99tl9.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * doc/guix.texi: Add section "Invoking guix deploy". =2D-- doc/guix.texi | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index 9dc1d2a9c..23b7416ab 100644 =2D-- a/doc/guix.texi +++ b/doc/guix.texi @@ -81,6 +81,7 @@ Documentation License''. * guix gc: (guix)Invoking guix gc. Reclaiming unused disk space. * guix pull: (guix)Invoking guix pull. Update the list of available= packages. * guix system: (guix)Invoking guix system. Manage the operating system = configuration. +* guix deploy: (guix)Invoking guix deploy. Manage operating system conf= igurations for remote hosts. @end direntry =20 @dircategory Software development @@ -269,6 +270,7 @@ System Configuration * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remo= te host. * Running Guix in a VM:: How to run Guix System in a virtual machin= e. * Defining Services:: Adding new service definitions. =20 @@ -10302,6 +10304,7 @@ instance to support new system services. * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remo= te host. * Running Guix in a VM:: How to run Guix System in a virtual machin= e. * Defining Services:: Adding new service definitions. @end menu @@ -25335,6 +25338,104 @@ example graph. =20 @end table =20 +@node Invoking guix deploy +@section Invoking @code{guix deploy} + +In addition to managing a machine's configuration locally through operating +system declarations, Guix also provides the ability to managing multiple r= emote +hosts as a logical ``deployment''. This is done using @command{guix deploy= }. + +@example +guix deploy @var{file} +@end example + +Such an invocation will deploy the machines that the code within @var{file} +evaluates to. As an example, @var{file} might contain a definition like th= is: + +@example +;; This is a Guix deployment of a "bare bones" setup, with +;; no X11 display server, to a machine with an SSH daemon +;; listening on localhost:2222. A configuration such as this +;; may be appropriate for virtual machine with ports +;; forwarded to the host's loopback interface. + +(use-service-modules networking ssh) +(use-package-modules bootloaders) + +(define %system + (operating-system + (host-name "gnu-deployed") + (timezone "Etc/UTC") + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (target "/dev/vda") + (terminal-outputs '(console)))) + (file-systems (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + %base-file-systems)) + (services + (append (list (service dhcp-client-service-type) + (service openssh-service-type + (openssh-configuration + (permit-root-login #t) + (allow-empty-passwords? #t)))) + %base-services)))) + +(list (machine + (system %system) + (environment managed-host-environment-type) + (configuration (machine-ssh-configuration + (host-name "localhost") + (identity "./id_rsa") + (port 2222))))) +@end example + +The file should evaluate to a list of @var{machine} objects. This example, +upon being deployed, will create a new generation on the remote system +realizing the operating-system configuration @var{%system}. @var{environme= nt} +and @var{configuration} specify how the machine should be provisioned--that +is, deployment and management of computing resources. The above example do= es +not provision any resources -- a @code{'managed-host} is a machine that is +already up and running the Guix system. A more complex deployment may invo= lve +i.e. starting virtual machines through a VPS provider, however, in which c= ase +a different @var{environment} types would be used. + +@deftp {Data Type} machine +This is the data type representing a single machine in a heterogeneous Guix +deployment. + +@table @asis +@item @code{system} +The object of the operating system configuration to deploy. + +@item @code{environment} +A symbol describing how the machine should be provisioned. At the moment, = only +the only supported value is @code{'managed-host}. + +@item @code{configuration} (default: @code{#f}) +An object describing the configuration for the machine's @code{environment= }. If +the @code{environment} has a default configuration, @code{#f} can be used.= If +@code{#f} is used for an environment with no default configuration, howeve= r, an +error will be thrown. +@end table +@end deftp + +@deftp {Data Type} machine-ssh-configuration +This is the data type representing the SSH client parameters for connectin= g to a +@code{'managed-host}. + +@table @asis +@item @code{host-name} +@item @code{port} (default: @code{22}) +@item @code{user} (default: @code{"root"}) +@item @code{identity} (default: @code{#f}) +If specified, the path to the SSH private key to use to authenticate with = the +remote host. +@end table +@end deftp + @node Running Guix in a VM @section Running Guix in a Virtual Machine =20 =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0aovIACgkQ9Qb9Fp2P 2VqrKw//XG5YtyceJtVABdPA9H1PE3BQzRwOFHLMqkGNf079p/u/oySS14CAsVbf 2/tae/GbFWXNbt8My2qddf2xb5cxTdofw8QtoLDstQm587Hp4fND12yxaYkQ+nnv qW/1xPp06X8SKE0OUDKWgFkcjTiQNe4ljzISrNe9asuWmgkfcWjDO3k8+bcmtvn3 Sj5IEQZH6J6e0u77A3NPIbOQzPiJ7Dw6Hk5UGkAL9D/Y3MZxQo3JbeRXOXcAyCO2 F82y51hhJJlkCxfrAaOxWvATfUTqele7pVG1scBhkJe2UMTjKimhNq7ODuydxLVO BEHp0kmrd0ws452jf7K92ChWppcEvQdR4XQZB3qFJtiNF9vXCxDft9HQqXKF4/NZ 4vEA107+joRzOL1Pt54hKhEbGGZBE2JazYTKoB0JAWASHeXgNFTNxFQUyr5Yx+2J XA+2/QtG7y5qzQQK9js8shPtX2brj6wNBsuQcgaIlmbumOKrMQBAeGvPtGpvfTjm wrgKAt9x/unt+F1dT86U1Pqp2UXNzlklZC+RPVo/+NVE453u0+yiCSKkn27GBycJ b1pEhHR9iA2yuaE0HQjnGJ+gYKHjbXfbaCRH10szBi3D/8A2gruqmtmtcqVBqGIM ZANcVa5zTdghClTnpIhaEZh8EY41zDkXtMcndlPMECmFTs4bP04= =q5t2 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 00:16:44 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 04:16:45 +0000 Received: from localhost ([127.0.0.1]:46442 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiADk-0004JH-JQ for submit@debbugs.gnu.org; Tue, 02 Jul 2019 00:16:44 -0400 Received: from pb-smtp2.pobox.com ([64.147.108.71]:56208) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiADg-0004J7-MT for 36404@debbugs.gnu.org; Tue, 02 Jul 2019 00:16:43 -0400 Received: from pb-smtp2.pobox.com (unknown [127.0.0.1]) by pb-smtp2.pobox.com (Postfix) with ESMTP id 4339616ADB1; Tue, 2 Jul 2019 00:16:40 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:in-reply-to:references:date:message-id:mime-version :content-type; s=sasl; bh=tVLV08Bis6KDnPtptcGVEvia3so=; b=IMES0o VL+7GLLNZEyIEsRhoNO+8v1dZBNLfYKLsC/3fvdi2/H/H2gvYfnYeME1B5SlfLAq 9yM6mpsnoZvjpmbRU52DPLo7D/4RimrgVrwfyV1Qg9VmAUQOx3LSg2ssPWLd2aq0 kEpYuedvZrE+X9nrxOl812OJz0xUFVrC/BexA= Received: from pb-smtp2.nyi.icgroup.com (unknown [127.0.0.1]) by pb-smtp2.pobox.com (Postfix) with ESMTP id 3BFB516ADB0; Tue, 2 Jul 2019 00:16:40 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=kyleam.com; h=from:to:cc:subject:in-reply-to:references:date:message-id:mime-version:content-type; s=mesmtp; bh=uai1QXaGjhfmgb5M7tq6SbPs44WSoooQeJIUdCOVcpI=; b=SZems/8wWEoZloD2yKwLq8VL8ezQn5eIE87qso7YKww0dhhUVt7Wrca3idpm/3V/Tte0Nw7214FN8y0HJlfCDNUvseoEDz6WnTlXzTW9Bqj4epE1Add4LktV6mdnUopOOZGoxHW4gFFVLenKt9Jd6o+xpxUuUhcKp8e5BnM/T4s= Received: from localhost (unknown [71.233.97.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pb-smtp2.pobox.com (Postfix) with ESMTPSA id A9D8C16ADAF; Tue, 2 Jul 2019 00:16:39 -0400 (EDT) From: Kyle Meyer To: "Jakob L. Kreuze" , Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 4/4] doc: Add section for 'guix deploy'. In-Reply-To: <871rz99tl9.fsf_-_@sdf.lonestar.org> References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> Date: Tue, 02 Jul 2019 00:16:38 -0400 Message-ID: <875zoldqah.fsf@kyleam.com> MIME-Version: 1.0 Content-Type: text/plain X-Pobox-Relay-ID: 30A1EF18-9C80-11E9-83EB-72EEE64BB12D-24757444!pb-smtp2.pobox.com X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Jakob, Thanks for working on this and for all the effort you've put into sharing your progress with the list. It's been interesting to watch from the sidelines. I don't know enough to say anything useful about the code changes, but here are a few minor comments about the guix.texi changes. zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes: [...] > +The file should evaluate to a list of @var{machine} objects. This example, > +upon being deployed, will create a new generation on the remote system > +realizing the operating-system configuration @var{%system}. @var{environment} > +and @var{configuration} specify how the machine should be provisioned--that nitpick: Following the style used elsewhere in guix.texi, I think this "--" and the closing one below should be "---" with no surrounding spaces. > +is, deployment and management of computing resources. The above example does > +not provision any resources -- a @code{'managed-host} is a machine that is > +already up and running the Guix system. A more complex deployment may involve > +i.e. starting virtual machines through a VPS provider, however, in which case > +a different @var{environment} types would be used. This last sentence doesn't quite parse for me. Perhaps A more complex deployment may involve, for example, starting virtual machines through a VPS provider. In such as case, a different @var{environment} type would be used. ? [...] > +@item @code{environment} > +A symbol describing how the machine should be provisioned. At the moment, only > +the only supported value is @code{'managed-host}. Repeated "only". Also, as a meta nit: It'd be helpful if you'd mark updated patch series with the iteration count (e.g., v3). You can do this with git-format-patch's --reroll-count option. -- Kyle From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 12:45:41 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 16:45:41 +0000 Received: from localhost ([127.0.0.1]:47788 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiLuW-0005vz-V5 for submit@debbugs.gnu.org; Tue, 02 Jul 2019 12:45:41 -0400 Received: from ol.sdf.org ([205.166.94.20]:53852 helo=mx.sdf.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiLuV-0005vr-02 for 36404@debbugs.gnu.org; Tue, 02 Jul 2019 12:45:39 -0400 Received: from Upsilon (mobile-166-172-60-116.mycingular.net [166.172.60.116]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x62GjYib000872 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 16:45:35 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Kyle Meyer Subject: Re: [bug#36404] [PATCH 4/4] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> Date: Tue, 02 Jul 2019 12:45:30 -0400 In-Reply-To: <875zoldqah.fsf@kyleam.com> (Kyle Meyer's message of "Tue, 02 Jul 2019 00:16:38 -0400") Message-ID: <87muhwtmfp.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Hi Kyle, Kyle Meyer writes: > nitpick: Following the style used elsewhere in guix.texi, I think this > "--" and the closing one below should be "---" with no surrounding > spaces. Yep, you're right. I cracked open the Texinfo manual and there indeed is a difference in how the two are parsed: the former is treated as an en dash while the latter is treated as an em dash. Thanks for pointing that out! > This last sentence doesn't quite parse for me. Perhaps > > A more complex deployment may involve, for example, starting virtual > machines through a VPS provider. In such as case, a different > @var{environment} type would be used. > > ? That's much clearer wording. I think I might use that verbatim. Also, I should probably be using two spaces to end my sentences as you did there :) > Repeated "only". Nice catch! > Also, as a meta nit: It'd be helpful if you'd mark updated patch > series with the iteration count (e.g., v3). You can do this with > git-format-patch's --reroll-count option. Oh awesome, I'll be sure to start using that. Thanks for the comments and the kind words, Jakob --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0biioACgkQ9Qb9Fp2P 2VosmQ/8C7edC4+i8E5rnEg9wJQv3s47owIC6C02e+2o4kuiSmKfVMKHpMClbMSN nO8R0VVAMIQn/mieNEnAeXxGrb4qWeOsDMW20sdkt2ZWbU43Jfzgd1Dtzvo4aX0T iuBV4BTP0lj2Ders/xnVyvoT9JEL4GaYOh4xUcUtkmrmDsg7vLpr+7ABQosXFLtg /nlzZ2/3yEbNQpZpZ8urG5nSv/qmSG2rlUt6Eyl3vXjNv79qRS+1pJn7GFsXpZus 06mzkjm/T9mPV1tYESAuXCDln8rWT3pV46P/SdpDkWhC0oJGbt68qBNV8u5BNUcD aZAHciBbttNm7PEfq+XJMrSTVpzbXljrDkgtdJKyS5bGWvZT8TmtZKhVUVYBWEPG RRscfq+QeynA1nyRMnz3Wn1zsi3xp8NDZ/AZMFT+4Qebmq3TbDntwzVjv8n/48cD 0czkcqhn/WINwgpK0DoeZudCsU138iptybWwW1fCPGobTq5/RIPU8MQcgLurtuz0 bS0WFdQhogwdHUoblAjABdjcVzApH/rqBV8kuMbrt0TNHwkRk1xYwZ837oyUgZfq tiDHPlYPGlO9iDyPSWBg2JD6ywDms5P5XOONY4LOtXrucP2kYrBfkhfkQXf6LcnS gg384I6Lm2/OWhEQkhro4T8yVM0ITdXHE0v/2/wz3HXOL4SqEx8= =3aiB -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:02:10 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 17:02:10 +0000 Received: from localhost ([127.0.0.1]:47805 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMAT-0006OQ-Kn for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:02:10 -0400 Received: from pb-smtp20.pobox.com ([173.228.157.52]:58512) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMAQ-0006OA-QA for 36404@debbugs.gnu.org; Tue, 02 Jul 2019 13:02:08 -0400 Received: from pb-smtp20.pobox.com (unknown [127.0.0.1]) by pb-smtp20.pobox.com (Postfix) with ESMTP id DD1717388F; Tue, 2 Jul 2019 13:02:05 -0400 (EDT) (envelope-from kyle@kyleam.com) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:in-reply-to:references:date:message-id:mime-version :content-type; s=sasl; bh=FQrt9tmirMT8tFn+gfTXzH/UkeI=; b=ThZPpp KgV/PPVhxebu5KkkS0we59d2VpqZZhhs4XDeGPFaSf330T4DiSZUNFGsGANOawpW /URs0rCTKsroEv62hcyYjTiZqVok0qP87hV2DAzoGi4z+BBp6K6fO34jbAVeTU9o FtHzv1cZ2nHaxIdEUwYdHg2Exx79/8y0cllwo= Received: from pb-smtp20.sea.icgroup.com (unknown [127.0.0.1]) by pb-smtp20.pobox.com (Postfix) with ESMTP id D580C7388E; Tue, 2 Jul 2019 13:02:05 -0400 (EDT) (envelope-from kyle@kyleam.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=kyleam.com; h=from:to:cc:subject:in-reply-to:references:date:message-id:mime-version:content-type; s=mesmtp; bh=Gz6HfiwPh9Z7FYTZH3NV3ZkI9Voj3q38lx53jDvF6r4=; b=JtDHiB29BQbXOwl1inDYinSDv4zcOyJ2H7o1K2qGVcM4ZCH9LfK31bUi+wy2KVuvvG6/7D61OHCz5LeV2fJHWbz2nhnphnlQSxxsW3zcwMEnMS+bChLdUpXLC3XQzx2JXOQ9Dnz1f5Yrwz5TF7ihjjzdH5SVOXMGzc18ZmqCbKA= Received: from localhost (unknown [71.233.97.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pb-smtp20.pobox.com (Postfix) with ESMTPSA id 1C5927388D; Tue, 2 Jul 2019 13:02:03 -0400 (EDT) (envelope-from kyle@kyleam.com) From: Kyle Meyer To: "Jakob L. Kreuze" Subject: Re: [bug#36404] [PATCH 4/4] doc: Add section for 'guix deploy'. In-Reply-To: <87muhwtmfp.fsf@sdf.lonestar.org> References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> Date: Tue, 02 Jul 2019 13:02:01 -0400 Message-ID: <871rz874l2.fsf@kyleam.com> MIME-Version: 1.0 Content-Type: text/plain X-Pobox-Relay-ID: 1D1EC162-9CEB-11E9-A6F7-B0405B776F7B-24757444!pb-smtp20.pobox.com X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 36404 Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes: [...] >> This last sentence doesn't quite parse for me. Perhaps >> >> A more complex deployment may involve, for example, starting virtual >> machines through a VPS provider. In such as case, a different >> @var{environment} type would be used. >> >> ? > > That's much clearer wording. I think I might use that verbatim. Reading that again, I see my suggestion has a typo: s/as/a/ From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:55:24 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 17:55:24 +0000 Received: from localhost ([127.0.0.1]:47876 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN00-0008Se-75 for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:55:24 -0400 Received: from mx.sdf.org ([205.166.94.20]:51327) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMzu-0008SR-Ua for 36404@debbugs.gnu.org; Tue, 02 Jul 2019 13:55:20 -0400 Received: from Upsilon (mobile-166-172-60-116.mycingular.net [166.172.60.116]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x62HtFKJ029505 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 17:55:16 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v4 0/4] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> Date: Tue, 02 Jul 2019 13:55:10 -0400 In-Reply-To: <871rz874l2.fsf@kyleam.com> (Kyle Meyer's message of "Tue, 02 Jul 2019 13:02:01 -0400") Message-ID: <877e90tj7l.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , Kyle Meyer , "Thompson, David" , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Kyle Meyer writes: > Reading that again, I see my suggestion has a typo: s/as/a/ It's alright, I picked up on it as I was copying it in :) Anyway, here's another revision updating the documentation. I had a go at making the environment types more like the service types, in the sense that configuration objects would be tied to the environment rather than the machine, but that would involve introducing an 'environment' object, which I thought to be too verbose. #+BEGIN_SRC scheme (list (machine (system %system) (environment (environment managed-host-environment-type (machine-ssh-configuration (host-name "localhost") (identity "./id_rsa") (port 2222)))))) #+END_SRC I suppose this could be avoided if I were to expose a different constructor for 'machine'. #+BEGIN_SRC scheme (list (machine %system (environment managed-host-environment-type (machine-ssh-configuration (host-name "localhost") (identity "./id_rsa") (port 2222))))) #+END_SRC I don't know if that's any better. Thoughts? Jakob L. Kreuze (4): ssh: Add 'identity' keyword to 'open-ssh-session'. gnu: Add machine type for deployment specifications. Add 'guix deploy'. doc: Add section for 'guix deploy'. Makefile.am | 4 +- doc/guix.texi | 107 ++++++++++++ gnu/local.mk | 5 +- gnu/machine.scm | 118 +++++++++++++ gnu/machine/ssh.scm | 363 ++++++++++++++++++++++++++++++++++++++++ guix/scripts/deploy.scm | 90 ++++++++++ guix/ssh.scm | 10 +- 7 files changed, 691 insertions(+), 6 deletions(-) create mode 100644 gnu/machine.scm create mode 100644 gnu/machine/ssh.scm create mode 100644 guix/scripts/deploy.scm =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0bmn8ACgkQ9Qb9Fp2P 2Vooqg/+JWmbVF7X042S8YYqyaT07ruGh0SzcSZJsE0AvnBSVOWpteoIF5C1eC3W 7A/wuQpHlXyjrBscmL42xq7KaeJ7apM5snRehWsq5ps7s1A53p3+f0X5QhHHO7nF sSJXY6NBPBaMECcebr+DFo57PAnBzsf//aoBT+NrUZa9y8flAZ77nddXpUi9F4jD 9kKGXaK+zfs45cmGD1unhiNvICL/lwNzfHU9Paejzam2qSKcpwvW6ikgXz/xomZg YO5C3rMMMBSGC6PWPjoqNrYG6Q6lWELKRvA78z3En1dN2V20RvvyYEw7Z4oR2syr D/4GEf2h5ZYAYEpmXfsgC0Z5mjGBMmZIMwRxtSygEP+LA9MzYG6sb9VsaCLvwze0 bO8cMC2lkzhX71RJ6a2Tw96mNEVF2yq0mK31HofsYXCDKWhzNJg5YPW4Nfj9FX4k hsXWE+x1DYE4/oCUnsK7cde+DSF3zILQsF5qdAVfDL2BkJ+Yn+dlkwBN0x/7uHdt t8cPC9dQZtQ6M0EfrgGH7mvWjSTMp3TZeNR0V6rDlPKe4DwBgV+m68stqQyf3whX iSqYqAY/7l1pRIBsIZQjwVQvbFC3VOe+IUPiwDdXtGJ8fD47grzzrllfsuJ/Qqj2 OUKa6VeQIPJ6x/zIYqV1cMwAvGxIr1WOdJjP0InfYa+QDKDSyO0= =coDt -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:56:17 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 17:56:17 +0000 Received: from localhost ([127.0.0.1]:47885 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN0q-0008Ub-Tr for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:56:17 -0400 Received: from mx.sdf.org ([205.166.94.20]:51017) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN0p-0008UT-5O for 36404@debbugs.gnu.org; Tue, 02 Jul 2019 13:56:15 -0400 Received: from Upsilon (mobile-166-172-60-116.mycingular.net [166.172.60.116]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x62HuCIv022015 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 17:56:13 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v4 1/4] ssh: Add 'identity' keyword to 'open-ssh-session'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> Date: Tue, 02 Jul 2019 13:56:12 -0400 In-Reply-To: <877e90tj7l.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Tue, 02 Jul 2019 13:55:10 -0400") Message-ID: <8736jotj5v.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * guix/ssh.scm (open-ssh-session): Add 'identity' keyword argument. =2D-- guix/ssh.scm | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/guix/ssh.scm b/guix/ssh.scm index 9b9baf54e..9bf10b9a0 100644 =2D-- a/guix/ssh.scm +++ b/guix/ssh.scm @@ -57,12 +57,14 @@ (define %compression "zlib@openssh.com,zlib") =20 =2D(define* (open-ssh-session host #:key user port +(define* (open-ssh-session host #:key user port identity (compression %compression)) =2D "Open an SSH session for HOST and return it. When USER and PORT are #= f, use =2Ddefault values or whatever '~/.ssh/config' specifies; otherwise use them. =2DThrow an error on failure." + "Open an SSH session for HOST and return it. IDENTITY specifies the pat= h of +a private key to use for authenticating with the host. When USER, PORT, or +IDENTITY are #f, use default values or whatever '~/.ssh/config' specifies; +otherwise use them. Throw an error on failure." (let ((session (make-session #:user user + #:identity identity #:host host #:port port #:timeout 10 ;seconds =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0bmrwACgkQ9Qb9Fp2P 2VrXzA/9EfDzArREAhXNMtKsjhKrckNiyb51u6n5cxjIcH6Qk+g8lKMMGP2AgA0i B9tVrw/bQ7xzN2a2WJhwSI5A54J6DbEAO8AQ9VWzRUAzKmsecJ6DwPt5r7nIXGdV PnF5/n5bOKWc9FE+bTi5LoWIV6AKYSWzULzjxOdkJIXszH5XlHkR0QOJkhySqHMS cAXGOgsTy8sE80oC+b9dnJuQyvL15H8zTr94kWXfcvWjwAcOZlkH8bIxDh9GiLun LA5qQs/AyA8aEjYbg3Y7FgaNoq1wAjd1++UrKl8PXZl75DPjCNWOmsQ1CK9aDZWk FF9KYPfR0cBXeH66tHasAnJ0UOA+pJgfmRYgE3lDwueC9B7U7vqSyG5RlVq7l7Wv eJN2Di4WP2IHMKKEEOSSlQExmOIdRo5ID2VYYXw2bGF0Q79Pwi3Xa/lNqjCjvlwT yLidZH79af9k+Ot2kvBgbcRvkwdG5jbKX1/5wRYNL9GDQHOodPRdoMGABlNVesub aLjlVNNnVekag3uWln+UrTdBqgPxUVwnrGsX3KdmdwlU99HXWVOCSDbjKWqV91qr gmRqkSOXASMoPZckYQyOyYbW3p496CIxXyHM42Xt0CKGUiJKe3gf7uvqBhvhZ3e9 yuxYWs16uzVnzV0CMZUTZRR2EJayAQoCovYfJI/riR/vDuy+W4E= =wCyF -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:57:13 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 17:57:14 +0000 Received: from localhost ([127.0.0.1]:47889 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN1g-0008W5-B0 for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:57:13 -0400 Received: from mx.sdf.org ([205.166.94.20]:50767) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN1d-0008Vv-4i for 36404@debbugs.gnu.org; Tue, 02 Jul 2019 13:57:06 -0400 Received: from Upsilon (mobile-166-172-60-116.mycingular.net [166.172.60.116]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x62Huw2Y007022 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 17:57:00 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v4 2/4] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> Date: Tue, 02 Jul 2019 13:56:58 -0400 In-Reply-To: <8736jotj5v.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Tue, 02 Jul 2019 13:56:12 -0400") Message-ID: <87y31gs4k5.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable * gnu/machine.scm: New file. * gnu/machine/ssh.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. =2D-- Makefile.am | 3 +- gnu/local.mk | 5 +- gnu/machine.scm | 118 ++++++++++++++ gnu/machine/ssh.scm | 363 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 487 insertions(+), 2 deletions(-) create mode 100644 gnu/machine.scm create mode 100644 gnu/machine/ssh.scm diff --git a/Makefile.am b/Makefile.am index 42307abae..f10c000ea 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -425,7 +425,8 @@ SCM_TESTS =3D \ tests/import-utils.scm \ tests/store-database.scm \ tests/store-deduplication.scm \ =2D tests/store-roots.scm + tests/store-roots.scm \ + tests/machine.scm =20 SH_TESTS =3D \ tests/guix-build.sh \ diff --git a/gnu/local.mk b/gnu/local.mk index 81de156cf..0e17af953 100644 =2D-- a/gnu/local.mk +++ b/gnu/local.mk @@ -562,6 +562,9 @@ GNU_SYSTEM_MODULES =3D \ %D%/system/uuid.scm \ %D%/system/vm.scm \ \ + %D%/machine.scm \ + %D%/machine/ssh.scm \ + \ %D%/build/accounts.scm \ %D%/build/activation.scm \ %D%/build/bootloader.scm \ @@ -627,7 +630,7 @@ INSTALLER_MODULES =3D \ %D%/installer/newt/user.scm \ %D%/installer/newt/utils.scm \ %D%/installer/newt/welcome.scm \ =2D %D%/installer/newt/wifi.scm=09 + %D%/installer/newt/wifi.scm =20 # Always ship the installer modules but compile them only when # ENABLE_INSTALLER is true. diff --git a/gnu/machine.scm b/gnu/machine.scm new file mode 100644 index 000000000..3dfcab797 =2D-- /dev/null +++ b/gnu/machine.scm @@ -0,0 +1,118 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu machine) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix store) + #:use-module ((guix utils) #:select (source-properties->location)) + #:export (environment-type + environment-type? + environment-type-name + environment-type-description + environment-type-location + + machine + machine? + this-machine + + machine-system + machine-environment + machine-configuration + machine-display-name + + build-machine + deploy-machine + machine-remote-eval)) + +;;; Commentary: +;;; +;;; This module provides the types used to declare individual machines in a +;;; heterogeneous Guix deployment. The interface allows users of specify s= ystem +;;; configurations and the means by which resources should be provisioned = on a +;;; per-host basis. +;;; +;;; Code: + + +;;; +;;; Declarations for resources that can be provisioned. +;;; + +(define-record-type* environment-type + make-environment-type + environment-type? + + ;; Interface to the environment type's deployment code. Each procedure + ;; should take the same arguments as the top-level procedure of this file + ;; that shares the same name. For example, 'machine-remote-eval' should = be + ;; of the form '(machine-remote-eval machine exp)'. + (machine-remote-eval environment-type-machine-remote-eval) ; procedure + (deploy-machine environment-type-deploy-machine) ; procedure + + ;; Metadata. + (name environment-type-name) ; symbol + (description environment-type-description ; string + (default #f)) + (location environment-type-location ; + (default (and=3D> (current-source-location) + source-properties->location)) + (innate))) + + +;;; +;;; Declarations for machines in a deployment. +;;; + +(define-record-type* machine + make-machine + machine? + this-machine + (system machine-system) ; + (environment machine-environment) ; symbol + (configuration machine-configuration ; configuration object + (default #f))) ; specific to environment + +(define (machine-display-name machine) + "Return the host-name identifying MACHINE." + (operating-system-host-name (machine-system machine))) + +(define (build-machine machine) + "Monadic procedure that builds the system derivation for MACHINE and ret= urning +a list containing the path of the derivation file and the path of the deri= vation +output." + (let ((os (machine-system machine))) + (mlet* %store-monad ((osdrv (operating-system-derivation os)) + (_ ((store-lift build-derivations) (list osdrv)))) + (return (list (derivation-file-name osdrv) + (derivation->output-path osdrv)))))) + +(define (machine-remote-eval machine exp) + "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP refe= rs to +are built and deployed to MACHINE beforehand." + (let ((environment (machine-environment machine))) + ((environment-type-machine-remote-eval environment) machine exp))) + +(define (deploy-machine machine) + "Monadic procedure transferring the new system's OS closure to the remote +MACHINE, activating it on MACHINE and switching MACHINE to the new generat= ion." + (let ((environment (machine-environment machine))) + ((environment-type-deploy-machine environment) machine))) diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm new file mode 100644 index 000000000..6ce106bb2 =2D-- /dev/null +++ b/gnu/machine/ssh.scm @@ -0,0 +1,363 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu machine ssh) + #:use-module (gnu bootloader) + #:use-module (gnu machine) + #:autoload (gnu packages gnupg) (guile-gcrypt) + #:use-module (gnu services) + #:use-module (gnu services shepherd) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix gexp) + #:use-module (guix i18n) + #:use-module (guix modules) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix remote) + #:use-module (guix ssh) + #:use-module (guix store) + #:use-module (ice-9 match) + #:use-module (srfi srfi-19) + #:export (managed-host-environment-type + + machine-ssh-configuration + machine-ssh-configuration? + machine-ssh-configuration + + machine-ssh-configuration-host-name + machine-ssh-configuration-port + machine-ssh-configuration-user + machine-ssh-configuration-session)) + +;;; Commentary: +;;; +;;; This module implements remote evaluation and system deployment for +;;; machines that are accessable over SSH and have a known host-name. In t= he +;;; sense of the broader "machine" interface, we describe the environment = for +;;; such machines as 'managed-host. +;;; +;;; Code: + + +;;; +;;; Parameters for the SSH client. +;;; + +(define-record-type* machine-ssh-configuration + make-machine-ssh-configuration + machine-ssh-configuration? + this-machine-ssh-configuration + (host-name machine-ssh-configuration-host-name) ; string + (port machine-ssh-configuration-port ; integer + (default 22)) + (user machine-ssh-configuration-user ; string + (default "root")) + (identity machine-ssh-configuration-identity ; path to a private key + (default #f)) + (session machine-ssh-configuration-session ; session + (default #f))) + +(define (machine-ssh-session machine) + "Return the SSH session that was given in MACHINE's configuration, or cr= eate +one from the configuration's parameters if one was not provided." + (let ((config (machine-configuration machine))) + (if (machine-ssh-configuration? config) + (or (machine-ssh-configuration-session config) + (let ((host-name (machine-ssh-configuration-host-name config)) + (user (machine-ssh-configuration-user config)) + (port (machine-ssh-configuration-port config)) + (identity (machine-ssh-configuration-identity config))) + (open-ssh-session host-name + #:user user + #:port port + #:identity identity))) + (error "unsupported configuration type")))) + + +;;; +;;; Remote evaluation. +;;; + +(define (managed-host-remote-eval machine exp) + "Internal implementation of 'machine-remote-eval' for MACHINE instances = with +an environment type of 'managed-host." + (maybe-raise-missing-configuration-error machine) + (remote-eval exp (machine-ssh-session machine))) + + +;;; +;;; System deployment. +;;; + +(define (switch-to-system machine) + "Monadic procedure creating a new generation on MACHINE and execute the +activation script for the new system configuration." + (define (remote-exp drv script) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles) + (guix utils))) + #~(begin + (use-modules (guix config) + (guix profiles) + (guix utils)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (let* ((system #$(derivation->output-path drv)) + (number (1+ (generation-number %system-profile))) + (generation (generation-file-name %system-profile numbe= r))) + (switch-symlinks generation system) + (switch-symlinks %system-profile generation) + ;; The implementation of 'guix system reconfigure' saves the + ;; load path and environment here. This is unnecessary here + ;; because each invocation of 'remote-eval' runs in a distin= ct + ;; Guile REPL. + (setenv "GUIX_NEW_SYSTEM" system) + ;; The activation script may write to stdout, which confuses + ;; 'remote-eval' when it attempts to read a result from the + ;; remote REPL. We work around this by forcing the output to= a + ;; string. + (with-output-to-string + (lambda () + (primitive-load #$script)))))))) + + (let* ((os (machine-system machine)) + (script (operating-system-activation-script os))) + (mlet* %store-monad ((drv (operating-system-derivation os))) + (machine-remote-eval machine (remote-exp drv script))))) + +;; XXX: Currently, this does NOT attempt to restart running services. This= is +;; also the case with 'guix system reconfigure'. +;; +;; See . +(define (upgrade-shepherd-services machine) + "Monadic procedure unloading and starting services on the remote as need= ed +to realize the MACHINE's system configuration." + (define target-services + ;; Monadic expression evaluating to a list of (name output-path) pairs= for + ;; all of MACHINE's services. + (mapm %store-monad + (lambda (service) + (mlet %store-monad ((file ((compose lower-object + shepherd-service-file) + service))) + (return (list (shepherd-service-canonical-name service) + (derivation->output-path file))))) + (service-value + (fold-services (operating-system-services (machine-system machi= ne)) + #:target-type shepherd-root-service-type)))) + + (define (remote-exp target-services) + (with-imported-modules '((gnu services herd)) + #~(begin + (use-modules (gnu services herd) + (srfi srfi-1)) + + (define running + (filter live-service-running (current-services))) + + (define (essential? service) + ;; Return #t if SERVICE is essential and should not be unloaded + ;; under any circumstance. + (memq (first (live-service-provision service)) + '(root shepherd))) + + (define (obsolete? service) + ;; Return #t if SERVICE can be safely unloaded. + (and (not (essential? service)) + (every (lambda (requirements) + (not (memq (first (live-service-provision servic= e)) + requirements))) + (map live-service-requirement running)))) + + (define to-unload + (filter obsolete? + (remove (lambda (service) + (memq (first (live-service-provision service= )) + (map first '#$target-services))) + running))) + + (define to-start + (remove (lambda (service-pair) + (memq (first service-pair) + (map (compose first live-service-provision) + running))) + '#$target-services)) + + ;; Unload obsolete services. + (for-each (lambda (service) + (false-if-exception + (unload-service service))) + to-unload) + + ;; Load the service files for any new services and start them. + (load-services/safe (map second to-start)) + (for-each start-service (map first to-start)) + + #t))) + + (mlet %store-monad ((target-services target-services)) + (machine-remote-eval machine (remote-exp target-services)))) + +(define (machine-boot-parameters machine) + "Monadic procedure returning a list of 'boot-parameters' for the generat= ions +of MACHINE's system profile, ordered from most recent to oldest." + (define bootable-kernel-arguments + (@@ (gnu system) bootable-kernel-arguments)) + + (define remote-exp + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles))) + #~(begin + (use-modules (guix config) + (guix profiles) + (ice-9 textual-ports)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (define (read-file path) + (call-with-input-file path + (lambda (port) + (get-string-all port)))) + + (map (lambda (generation) + (let* ((system-path (generation-file-name %system-profi= le + generation)) + (boot-parameters-path (string-append system-path + "/parameter= s")) + (time (stat:mtime (lstat system-path)))) + (list generation + system-path + time + (read-file boot-parameters-path)))) + (reverse (generation-numbers %system-profile))))))) + + (mlet* %store-monad ((generations (machine-remote-eval machine remote-ex= p))) + (return + (map (lambda (generation) + (match generation + ((generation system-path time serialized-params) + (let* ((params (call-with-input-string serialized-params + read-boot-parameters)) + (root (boot-parameters-root-device params)) + (label (boot-parameters-label params))) + (boot-parameters + (inherit params) + (label + (string-append label " (#" + (number->string generation) ", " + (let ((time (make-time time-utc 0 time))) + (date->string (time-utc->date time) + "~Y-~m-~d ~H:~M")) + ")")) + (kernel-arguments + (append (bootable-kernel-arguments system-path root) + (boot-parameters-kernel-arguments params)))))))) + generations)))) + +(define (install-bootloader machine) + "Create a bootloader entry for the new system generation on MACHINE, and +configure the bootloader to boot that generation by default." + (define bootloader-installer-script + (@@ (guix scripts system) bootloader-installer-script)) + + (define (remote-exp installer bootcfg bootcfg-file) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((gnu build install) + (guix store) + (guix utils))) + #~(begin + (use-modules (gnu build install) + (guix store) + (guix utils)) + (let* ((gc-root (string-append "/" %gc-roots-directory "/bootc= fg")) + (temp-gc-root (string-append gc-root ".new"))) + + (switch-symlinks temp-gc-root gc-root) + + (unless (false-if-exception + (begin + ;; The implementation of 'guix system reconfigure' + ;; saves the load path here. This is unnecessary = here + ;; because each invocation of 'remote-eval' runs = in a + ;; distinct Guile REPL. + (install-boot-config #$bootcfg #$bootcfg-file "/") + ;; The installation script may write to stdout, w= hich + ;; confuses 'remote-eval' when it attempts to rea= d a + ;; result from the remote REPL. We work around th= is + ;; by forcing the output to a string. + (with-output-to-string + (lambda () + (primitive-load #$installer))))) + (delete-file temp-gc-root) + (error "failed to install bootloader")) + + (rename-file temp-gc-root gc-root) + #t))))) + + (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine))) + (let* ((os (machine-system machine)) + (bootloader ((compose bootloader-configuration-bootloader + operating-system-bootloader) + os)) + (bootloader-target (bootloader-configuration-target + (operating-system-bootloader os))) + (installer (bootloader-installer-script + (bootloader-installer bootloader) + (bootloader-package bootloader) + bootloader-target + "/")) + (menu-entries (map boot-parameters->menu-entry boot-parameters)) + (bootcfg (operating-system-bootcfg os menu-entries)) + (bootcfg-file (bootloader-configuration-file bootloader))) + (machine-remote-eval machine (remote-exp installer bootcfg bootcfg-f= ile))))) + +(define (deploy-managed-host machine) + "Internal implementation of 'deploy-machine' for MACHINE instances with = an +environment type of 'managed-host." + (maybe-raise-missing-configuration-error machine) + (mbegin %store-monad + (switch-to-system machine) + (upgrade-shepherd-services machine) + (install-bootloader machine))) + + +;;; +;;; Environment type. +;;; + +(define managed-host-environment-type + (environment-type + (machine-remote-eval managed-host-remote-eval) + (deploy-machine deploy-managed-host) + (name 'managed-host-environment-type) + (description "Provisioning for machines that are accessable ove= r SSH +and have a known host-name. This entails little more than maintaining an S= SH +connection to the host."))) + +(define (maybe-raise-missing-configuration-error machine) + "Raise an error if MACHINE's configuration is #f." + (let ((environment (machine-environment machine))) + (unless (machine-configuration machine) + (error (format #f (G_ "no configuration specified for environment '~= a'") + (symbol->string (environment-type-name environment)))= )))) =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0bmuoACgkQ9Qb9Fp2P 2VpOWg//eNi/nh7TRhHoUxPP87RLBEUSqJsrqbSM/gfwFciPL6CvhVTtFOxsCaLA xMU2FVJRKoeBF7hk4FVCw4sHZsiMSBxyC1RdSwVebOjpxAugSNMpod+lr/j0f27G /xHHrXvxir9BRBwqMqqMyxi+lNC4RAVNxUeAKiPKOvCkqBbLuG4lb5tnX+QGSIJi Yn6chM+nsDWEs/QqQNEeg0ztfAx5mXhf5hCPjPFW8Sk2b2PGnfAWUIyDrVzKpV1d dCva0Ozc5IvJhT5zYNbrb4HaCtm1FwmZFuqMnkxICiNoYeGGbPxAfwU/5rytDVM4 UfLfW3Be0+wQhrBqBbqFc7G2Fo23z4scRaatA3QAM1Ls2eWtv2aK7jixbUgadk6K gPOW/5zWzIIOQiyHXiFvJwIQHX7wJ31LJyI7mfSdKmEcdGeKOI1Lb7HZtNyPgWdt L1k92ED2VbVo66hHQpcg9i18kKp0Z0wFxk8Pqkb9LWN59mNJF91OoN6d7yxcWzBz 0mn536ERwAMQUJIH3voQ1Aowccl2xsGBztLQ/y3PANvqTJ+1yUbUBLGssW8L5+Pq 2KcffmklULmP9k0WlMXUjnkUXTzv2H+TiIMP5sWa2Bh3gEVNH2Er+cZcjLtaEfwE Aan3yUQU9aCTXBS4oSLCOjurVx+nWN5Pl2Qmjo8NUp0FF3WvszM= =cqnd -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:57:49 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 17:57:49 +0000 Received: from localhost ([127.0.0.1]:47892 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN2K-00005T-W8 for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:57:49 -0400 Received: from mx.sdf.org ([205.166.94.20]:50584) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN2J-00005L-Rm for 36404@debbugs.gnu.org; Tue, 02 Jul 2019 13:57:48 -0400 Received: from Upsilon (mobile-166-172-60-116.mycingular.net [166.172.60.116]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x62Hvilw027866 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 17:57:46 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v4 3/4] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> Date: Tue, 02 Jul 2019 13:57:44 -0400 In-Reply-To: <87y31gs4k5.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Tue, 02 Jul 2019 13:56:58 -0400") Message-ID: <87tvc4s4iv.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable * guix/scripts/deploy.scm: New file. * Makefile.am (MODULES): Add it. =2D-- Makefile.am | 1 + guix/scripts/deploy.scm | 90 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) create mode 100644 guix/scripts/deploy.scm diff --git a/Makefile.am b/Makefile.am index f10c000ea..4d3024e58 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -267,6 +267,7 @@ MODULES =3D \ guix/scripts/weather.scm \ guix/scripts/container.scm \ guix/scripts/container/exec.scm \ + guix/scripts/deploy.scm \ guix.scm \ $(GNU_SYSTEM_MODULES) =20 diff --git a/guix/scripts/deploy.scm b/guix/scripts/deploy.scm new file mode 100644 index 000000000..4fb1babe8 =2D-- /dev/null +++ b/guix/scripts/deploy.scm @@ -0,0 +1,90 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts deploy) + #:use-module (gnu machine) + #:use-module (guix scripts) + #:use-module (guix scripts build) + #:use-module (guix store) + #:use-module (guix ui) + #:use-module (ice-9 format) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-37) + #:export (guix-deploy)) + +;;; Commentary: +;;; +;;; This program provides a command-line interface to (gnu machine), allow= ing +;;; users to perform remote deployments through specification files. +;;; +;;; Code: + + + +(define (show-help) + (display (G_ "Usage: guix deploy [OPTION] FILE... +Perform the deployment specified by FILE.\n")) + (show-build-options-help) + (newline) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + %standard-build-options)) + +(define %default-options + '((system . ,(%current-system)) + (substitutes? . #t) + (build-hook? . #t) + (graft? . #t) + (debug . 0) + (verbosity . 2))) + +(define (load-source-file file) + "Load FILE as a user module." + (let ((module (make-user-module '((gnu) (gnu machine) (gnu machine ssh))= ))) + (load* file module))) + +(define (guix-deploy . args) + (define (handle-argument arg result) + (alist-cons 'file arg result)) + (let* ((opts (parse-command-line args %options (list %default-options) + #:argument-handler handle-argument)) + (file (assq-ref opts 'file)) + (machines (or (and file (load-source-file file)) '()))) + (with-store store + (set-build-options-from-command-line store opts) + (for-each (lambda (machine) + (format #t "building ~a... " (machine-display-name machi= ne)) + (run-with-store store (build-machine machine)) + (display "done\n")) + machines) + (for-each (lambda (machine) + (format #t "deploying to ~a... " (machine-display-name m= achine)) + (run-with-store store (deploy-machine machine)) + (display "done\n")) + machines)))) =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0bmxkACgkQ9Qb9Fp2P 2VqKqw/+NHA+54PlDRYND6cpCNmyeV8GuKmrVILGEv+ExAXPRxJGZ27eT/XOelTo ky67cyKdSx/N6uOhIJgDBnTNXjzN7wk30Ui+lTeSFTXw2vT5gMcoYklyNkJjQO7+ KEBvzWbCwdwBN08HrlGs/qirgPjfJZy2+UWRZM6ntHLSCLG2TpT/cmmCfiY7zAkD wud0DxO6Ki7cms5ojX7ytFKOPh864lSgbRqVdkfteOZCNGkdy47TmvntKPBNxsc3 cl+qNbsVBqJZcOprDZrz8YIPyHbTtIup3iZ3teboR93KQcViA5OkbfJtcztfbCKO SYZasKjm9Pa/B2EnEKdCxPs4APLuLMt0YlUFeRB5QdeJ0e8OBf02u2DvdTyEOmJ5 /DgVR8DBVqau9oamFDFSlbluJ/vUYpZT2SJrDwn63Ds7dGxZmkapTpwzktFmmC5I Z2bUFVKRuDGk8zID4dbTh3lsqXFeQtUzz1eSbrIUdOd1VlJvt3zvLWPccJG9v3Md wmjjWnbl1VyUhLdPmKAulw5Xz1/x4pNIGqvCv2rFSre9a7kyo7eMTvSGLPbSCzfF uPN5hptEOakb6+2VbYrwyuPEn8w+Y0J+nj0xPdVakFCV4lI5hDohg9IYN5tyT80u 2voqvcjSs1abClEPZdXBSLY5Zjv1DYCXgnIXSyjScmq2fLuew4M= =dQx7 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:58:45 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 17:58:46 +0000 Received: from localhost ([127.0.0.1]:47897 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN3F-00007L-Fh for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:58:45 -0400 Received: from mx.sdf.org ([205.166.94.20]:50318) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN3D-00007A-8C for 36404@debbugs.gnu.org; Tue, 02 Jul 2019 13:58:43 -0400 Received: from Upsilon (mobile-166-172-60-116.mycingular.net [166.172.60.116]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x62HwdYS007134 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 17:58:41 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v4 4/4] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87tvc4s4iv.fsf_-_@sdf.lonestar.org> Date: Tue, 02 Jul 2019 13:58:38 -0400 In-Reply-To: <87tvc4s4iv.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Tue, 02 Jul 2019 13:57:44 -0400") Message-ID: <87muhws4hd.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * doc/guix.texi: Add section "Invoking guix deploy". =2D-- doc/guix.texi | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index 9dc1d2a9c..0827a2bde 100644 =2D-- a/doc/guix.texi +++ b/doc/guix.texi @@ -81,6 +81,7 @@ Documentation License''. * guix gc: (guix)Invoking guix gc. Reclaiming unused disk space. * guix pull: (guix)Invoking guix pull. Update the list of available= packages. * guix system: (guix)Invoking guix system. Manage the operating system = configuration. +* guix deploy: (guix)Invoking guix deploy. Manage operating system conf= igurations for remote hosts. @end direntry =20 @dircategory Software development @@ -269,6 +270,7 @@ System Configuration * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remo= te host. * Running Guix in a VM:: How to run Guix System in a virtual machin= e. * Defining Services:: Adding new service definitions. =20 @@ -10302,6 +10304,7 @@ instance to support new system services. * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remo= te host. * Running Guix in a VM:: How to run Guix System in a virtual machin= e. * Defining Services:: Adding new service definitions. @end menu @@ -25335,6 +25338,110 @@ example graph. =20 @end table =20 +@node Invoking guix deploy +@section Invoking @code{guix deploy} + +We've already seen @code{operating-system} declarations used to manage a +machine's configuration locally. Suppose you need to configure multiple +machines, though---perhaps you're managing a service on the web that's +comprised of several servers. @command{guix deploy} enables you to use th= ose +same @code{operating-system} declarations to manage multiple remote hosts = at +once as a logical ``deployment''. + +@example +guix deploy @var{file} +@end example + +Such an invocation will deploy the machines that the code within @var{file} +evaluates to. As an example, @var{file} might contain a definition like t= his: + +@example +;; This is a Guix deployment of a "bare bones" setup, with +;; no X11 display server, to a machine with an SSH daemon +;; listening on localhost:2222. A configuration such as this +;; may be appropriate for virtual machine with ports +;; forwarded to the host's loopback interface. + +(use-service-modules networking ssh) +(use-package-modules bootloaders) + +(define %system + (operating-system + (host-name "gnu-deployed") + (timezone "Etc/UTC") + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (target "/dev/vda") + (terminal-outputs '(console)))) + (file-systems (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + %base-file-systems)) + (services + (append (list (service dhcp-client-service-type) + (service openssh-service-type + (openssh-configuration + (permit-root-login #t) + (allow-empty-passwords? #t)))) + %base-services)))) + +(list (machine + (system %system) + (environment managed-host-environment-type) + (configuration (machine-ssh-configuration + (host-name "localhost") + (identity "./id_rsa") + (port 2222))))) +@end example + +The file should evaluate to a list of @var{machine} objects. This example, +upon being deployed, will create a new generation on the remote system +realizing the @code{operating-system} declaration @var{%system}. +@var{environment} and @var{configuration} specify how the machine should be +provisioned---that is, how the computing resources should be created and +managed. The above example does not create any resources, as a +@code{'managed-host} is a machine that is already running the Guix system = and +available over the network. This is a particularly simple case; a more +complex deployment may involve, for example, starting virtual machines thr= ough +a VPS provider. In such a case, a different @var{environment} type would = be +used. + +@deftp {Data Type} machine +This is the data type representing a single machine in a heterogeneous Guix +deployment. + +@table @asis +@item @code{system} +The object of the operating system configuration to deploy. + +@item @code{environment} +An @code{environment-type} describing how the machine should be provisione= d. +At the moment, the only supported value is +@code{managed-host-environment-type}. + +@item @code{configuration} (default: @code{#f}) +An object describing the configuration for the machine's @code{environment= }. +If the @code{environment} has a default configuration, @code{#f} maybe use= d. +If @code{#f} is used for an environment with no default configuration, +however, an error will be thrown. +@end table +@end deftp + +@deftp {Data Type} machine-ssh-configuration +This is the data type representing the SSH client parameters for a machine +with an @code{environment} of @code{managed-host-environment-type}. + +@table @asis +@item @code{host-name} +@item @code{port} (default: @code{22}) +@item @code{user} (default: @code{"root"}) +@item @code{identity} (default: @code{#f}) +If specified, the path to the SSH private key to use to authenticate with = the +remote host. +@end table +@end deftp + @node Running Guix in a VM @section Running Guix in a Virtual Machine =20 =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0bm08ACgkQ9Qb9Fp2P 2VrnMxAAmq0pb9Ik0Lqu0C0xYGhHJS88lrnZL13WUrbpXKeUVAg8Q7dzjKJ0U2eW rR+AAdfLDHDXNuNsFeJoEO18wyg0iPSjByYeWmiAcEdA9h273xhiXr7vJj6fo6Z9 4YMz4OMIPrXPyYvEFatkH0w8u97i5SSST2OY9I/BwPkUp7PIlVtm6rFK9i02TDvd YAyupkSGyE2LXO54j16Rr4xKscgeoe/GWfKthfFWAbIko8awEyvsdMnc7HPAldcB oiJTdXL84Ex6GG777+aQvWGq++0NxhYzk+Uk2LNShTr+j2XkdjkQoAXuEEhUUaK3 9vIf6wDcfB9l2jJrFjiuzFLzsEyyQQduFDbtA1qRBWWuwG4B+cgfy7IWSAvbbHnT /T7Swp187dsZ/X2oYdhpUMCpqJanEecn/uIhT0hEnS+IckDjQubS8kctzUOFCOWP /kR68UhZLqR9YQrmFGDER0r/DRRMGrDqMNjbrV7xavVdxx5+k+05SiWIvsnP2BMI +G7XdyeMs1e7tS3QCAVyukI9qYNOJ5RLmhLgk4QccEJJw9urx+09vXKJLCACwjeC EP7Xp2QVIMDh0LmBOSzVAjgzc2mpqyGMwYzNLY7mg00JPRcWePfcJ/78l7IRSz+B cZ3pvnph4U08jYi8UK7rrU0Qrl29rhUlBp+wx5FtbqBz6U63rZU= =C5Tg -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 18:15:41 2019 Received: (at 36404) by debbugs.gnu.org; 2 Jul 2019 22:15:41 +0000 Received: from localhost ([127.0.0.1]:48096 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiR3s-0004J0-K3 for submit@debbugs.gnu.org; Tue, 02 Jul 2019 18:15:41 -0400 Received: from mx.sdf.org ([205.166.94.20]:54871) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiR3o-0004Iq-8w for 36404@debbugs.gnu.org; Tue, 02 Jul 2019 18:15:39 -0400 Received: from Upsilon (mobile-107-107-57-226.mycingular.net [107.107.57.226]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x62MF3e3029512 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Tue, 2 Jul 2019 22:15:27 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> Date: Tue, 02 Jul 2019 18:14:43 -0400 In-Reply-To: <87r2799tzd.fsf@sdf.lonestar.org> (Jakob L. Kreuze's message of "Mon, 01 Jul 2019 20:10:30 -0400") Message-ID: <87d0isrsmk.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludovic, zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes: > Yep, I have an unsquashed commit history on my personal branch with > all renditions of the test suite. I can pull it out tomorrow and write > a detailed report on the issues I ran into. So we begin as I did about a month ago with a very na=C3=AFve test, ensuring that we can create a 'machine' object. This isn't particularly hard to pull off in the system test suite. #+BEGIN_SRC scheme (define (run-sshable-machine-test) (define os (marionette-operating-system (simple-operating-system (service dhcp-client-service-type) (service openssh-service-type (openssh-configuration (permit-root-login #t) (allow-empty-passwords? #t)))) #:imported-modules '((gnu services herd) (guix combinators)))) (define vm (virtual-machine (operating-system os) (port-forwardings '((2222 . 22))))) (define test (with-extensions (list guile-bytestructures guile-gcrypt guile-git guile-ssh guile-sqlite3 guix) (with-imported-modules '((gnu build marionette) (gnu) (gnu machine) (gnu machine ssh) (guix remote)) #~(begin (use-modules (gnu build marionette) (gnu) (gnu machine) (gnu machine ssh) (srfi srfi-64)) (use-service-modules networking ssh) (define %system (operating-system (host-name "gnu-deployed") (timezone "Etc/UTC") (bootloader (bootloader-configuration (bootloader grub-bootloader) (target "/dev/vda") (terminal-outputs '(console)))) (file-systems (cons (file-system (mount-point "/") (device "/dev/vda1") (type "ext4")) %base-file-systems)) (services (append (list (service dhcp-client-service-type) (service openssh-service-type (openssh-configuration (permit-root-login #t) (allow-empty-passwords? #t)))) %base-services)))) (define %machine (machine (system %system) (environment managed-host-environment-type) (configuration (machine-ssh-configuration (host-name "localhost") (port 2222))))) (define marionette (make-marionette (list #$vm))) (mkdir #$output) (chdir #$output) (test-begin "remote-eval") (test-assert "machine instance was created" %machine) (test-end) (exit (=3D (test-runner-fail-count (test-runner-current)) 0))))= )) (gexp->derivation "sshable-machine" test)) (define %test-sshable-machine (system-test (name "sshable-machine") (description "Create a machine object") (value (run-sshable-machine-test)))) #+END_SRC For onlookers unfamiliar with the system test suite, this is mostly boilerplate. The important code begins at 'define %system' and ends at 'test-end'. Wonderful! We've ensured that we can import '(gnu machine)', and that we can create instances of 'machine'. Where to now? How about testing 'remote-eval'? (This snippet requires more changes to the surrounding code to work. If you need a reproducible version, let me know.) #+BEGIN_SRC scheme (test-assert "can invoke machine-remote-eval" (with-store store (run-with-store store (machine-remote-eval %machine #~#t)))) #+END_SRC Alas, this doesn't work in the context of a derivation. #+BEGIN_SRC scheme (srfi-34 #) #+END_SRC This is around when I began to pester you on IRC with questions that I realize are kind of silly now. In general, system tests can't use the store. The only workaround that I'm aware of is 'gnu/tests/install.scm', which makes use of the store monad to perform store operations before running the test. For example: #+BEGIN_SRC scheme (define %test-iso-image-installer (system-test (name "iso-image-installer") (description "") (value (mlet* %store-monad ((image (run-install %minimal-os-on-vda %minimal-os-on-vda-source #:script %simple-installation-script-for-/dev/vda #:installation-disk-image-file-system-ty= pe "iso9660")) (command (qemu-command/writable-image image))) (run-basic-test %minimal-os-on-vda command name))))) #+END_SRC This is a bit less complicated than system deployment, since the tests only need the store to build the virtual machine image. Deployment to a machine requires that the machine is /up/, but if you look at the initial, na=C3=AFve test, you can see that the virtual machine isn't started until the test derivation runs -- which is after everything in the store monad is run. c6e01898[1] has a version that starts the virtual machine while the store monad is running so it can deploy to it. This is an absolute mess, as seen in 'call-with-marionette'. Also, the use of 'dynamic-wind' in that rendition causes the SSH session to close during deployment, which is why that test fails. (I didn't figure that out until around the time I began reimplementing the tests in the normal test suite.) In theory, _I could fix that issue and implement the tests this way_. Another possibility would be to spawn two virtual machines and have one deploy to the other. This is implemented in 358f1287[2], which I believe I would also be able to adapt now that I know I need to create writable disk images for the virtual machines. Before I go ahead with either, though, I'd like to know if either is the "right way". Or if there's something better than what I'm suggesting. Regards, Jakob [1]: https://git.sr.ht/~jakob/guix/tree/c6e01898dc774eef318c042595d6490e50e= 19486/gnu/tests/machine.scm [2]: https://git.sr.ht/~jakob/guix/tree/358f12871326085c3e108181887ea36a857= 7de73/gnu/tests/machine.scm --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0b11MACgkQ9Qb9Fp2P 2VqPqhAAixvg0Fo4O3GLrUUhk57YHGEutqPaUmCmfHoWUcXc6JoPG2exTg/fWhe6 OgvEXwdqFMMh6IOUBx66T6Wid0ZyZwwZOufXzeX6U9fXn1e0KutmnCNFKKc5+TT9 EStbkEY5au9zDRK61OY2OOe6HW3EJIe0DscvAkiEs+cQWW/rbDRyl69EB+tEiyzP CoPgeyhPWvI9rffIeAy3ztGxljUszS7enOt7glJOZlX1S9ogsCp2mhz2HvH8kCtI XpZMBicL3+T0vfvXfqgPiV2mSR+gYaBalE141mPaU4fPfay1ncSEHxFPHNmDTNax cPk5XURJUITK+Zn5oLdFABsnOzfJ/TImXGEQM0QycTDFWUCxELHEErlJw4KCKfuV WNyxC2hsPquA5IHQGgVkG0+upUSO5gIj933ShBzlQBuvtYiE/oDjZjwzxmxvEfTm m2d9QSn+yjwq5d1suKE6Chph9hYSvaXUbwTZIAXF6C3dzpPe2wj4jbASqWCMgAdZ YwXumfBghQ6n5/Ylwtavdq85C9RY5+ELQ0lz4BL2kX5oKWbOncS+xTG0m+AB3z6i BE7JgNg+FlzNa++vuVC+lq0KpfBXoRCJSHw3nfywYlhp1Y2cMyk2HocV4gruSklv 6LNOO7bmSMf9U3ZzXDdsmwKpr4/MmjRZ0SlPRIxDeTbhjWFsxtE= =Tr26 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 19:08:03 2019 Received: (at 36404) by debbugs.gnu.org; 3 Jul 2019 23:08:03 +0000 Received: from localhost ([127.0.0.1]:50170 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hioM6-0004o7-Jx for submit@debbugs.gnu.org; Wed, 03 Jul 2019 19:08:02 -0400 Received: from dustycloud.org ([50.116.34.160]:52282) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hioM4-0004nh-BC for 36404@debbugs.gnu.org; Wed, 03 Jul 2019 19:08:00 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 62D7426650; Wed, 3 Jul 2019 19:07:58 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87tvc4s4iv.fsf_-_@sdf.lonestar.org> <87muhws4hd.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: "Jakob L. Kreuze" Subject: Re: [bug#36404] [PATCH v4 4/4] doc: Add section for 'guix deploy'. In-reply-to: <87muhws4hd.fsf_-_@sdf.lonestar.org> Date: Wed, 03 Jul 2019 19:07:51 -0400 Message-ID: <87ftnmn2d4.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , "Thompson, David" , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Jakob L. Kreuze writes: > * doc/guix.texi: Add section "Invoking guix deploy". > --- > doc/guix.texi | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 107 insertions(+) > > diff --git a/doc/guix.texi b/doc/guix.texi > index 9dc1d2a9c..0827a2bde 100644 > --- a/doc/guix.texi > +++ b/doc/guix.texi > @@ -81,6 +81,7 @@ Documentation License''. > * guix gc: (guix)Invoking guix gc. Reclaiming unused disk space. > * guix pull: (guix)Invoking guix pull. Update the list of available packages. > * guix system: (guix)Invoking guix system. Manage the operating system configuration. > +* guix deploy: (guix)Invoking guix deploy. Manage operating system configurations for remote hosts. > @end direntry > > @dircategory Software development > @@ -269,6 +270,7 @@ System Configuration > * Initial RAM Disk:: Linux-Libre bootstrapping. > * Bootloader Configuration:: Configuring the boot loader. > * Invoking guix system:: Instantiating a system configuration. > +* Invoking guix deploy:: Deploying a system configuration to a remote host. > * Running Guix in a VM:: How to run Guix System in a virtual machine. > * Defining Services:: Adding new service definitions. > > @@ -10302,6 +10304,7 @@ instance to support new system services. > * Initial RAM Disk:: Linux-Libre bootstrapping. > * Bootloader Configuration:: Configuring the boot loader. > * Invoking guix system:: Instantiating a system configuration. > +* Invoking guix deploy:: Deploying a system configuration to a remote host. > * Running Guix in a VM:: How to run Guix System in a virtual machine. > * Defining Services:: Adding new service definitions. > @end menu > @@ -25335,6 +25338,110 @@ example graph. > > @end table > > +@node Invoking guix deploy > +@section Invoking @code{guix deploy} > + > +We've already seen @code{operating-system} declarations used to manage a > +machine's configuration locally. Suppose you need to configure multiple > +machines, though---perhaps you're managing a service on the web that's > +comprised of several servers. @command{guix deploy} enables you to use those > +same @code{operating-system} declarations to manage multiple remote hosts at > +once as a logical ``deployment''. > + > +@example > +guix deploy @var{file} > +@end example > + > +Such an invocation will deploy the machines that the code within @var{file} > +evaluates to. As an example, @var{file} might contain a definition like this: > + > +@example > +;; This is a Guix deployment of a "bare bones" setup, with > +;; no X11 display server, to a machine with an SSH daemon > +;; listening on localhost:2222. A configuration such as this > +;; may be appropriate for virtual machine with ports > +;; forwarded to the host's loopback interface. > + > +(use-service-modules networking ssh) > +(use-package-modules bootloaders) > + > +(define %system > + (operating-system > + (host-name "gnu-deployed") > + (timezone "Etc/UTC") > + (bootloader (bootloader-configuration > + (bootloader grub-bootloader) > + (target "/dev/vda") > + (terminal-outputs '(console)))) > + (file-systems (cons (file-system > + (mount-point "/") > + (device "/dev/vda1") > + (type "ext4")) > + %base-file-systems)) > + (services > + (append (list (service dhcp-client-service-type) > + (service openssh-service-type > + (openssh-configuration > + (permit-root-login #t) > + (allow-empty-passwords? #t)))) > + %base-services)))) > + > +(list (machine > + (system %system) > + (environment managed-host-environment-type) > + (configuration (machine-ssh-configuration > + (host-name "localhost") > + (identity "./id_rsa") > + (port 2222))))) > +@end example > + > +The file should evaluate to a list of @var{machine} objects. This example, > +upon being deployed, will create a new generation on the remote system > +realizing the @code{operating-system} declaration @var{%system}. > +@var{environment} and @var{configuration} specify how the machine should be > +provisioned---that is, how the computing resources should be created and > +managed. The above example does not create any resources, as a > +@code{'managed-host} is a machine that is already running the Guix system and > +available over the network. This is a particularly simple case; a more > +complex deployment may involve, for example, starting virtual machines through > +a VPS provider. In such a case, a different @var{environment} type would be > +used. > + > +@deftp {Data Type} machine > +This is the data type representing a single machine in a heterogeneous Guix > +deployment. > + > +@table @asis > +@item @code{system} > +The object of the operating system configuration to deploy. > + > +@item @code{environment} > +An @code{environment-type} describing how the machine should be provisioned. > +At the moment, the only supported value is > +@code{managed-host-environment-type}. > + > +@item @code{configuration} (default: @code{#f}) > +An object describing the configuration for the machine's @code{environment}. > +If the @code{environment} has a default configuration, @code{#f} maybe used. > +If @code{#f} is used for an environment with no default configuration, > +however, an error will be thrown. > +@end table > +@end deftp > + > +@deftp {Data Type} machine-ssh-configuration > +This is the data type representing the SSH client parameters for a machine > +with an @code{environment} of @code{managed-host-environment-type}. > + > +@table @asis > +@item @code{host-name} > +@item @code{port} (default: @code{22}) > +@item @code{user} (default: @code{"root"}) > +@item @code{identity} (default: @code{#f}) > +If specified, the path to the SSH private key to use to authenticate with the > +remote host. > +@end table > +@end deftp > + > @node Running Guix in a VM > @section Running Guix in a Virtual Machine All looks good to me. >From my perspective, this is ready to merge. Which means that we need to merge Ludo's remote-eval too. Of course others may catch things, but I'd say let's not take too long... we should get this in and let people start playing with it. :) From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 05:19:19 2019 Received: (at 36404) by debbugs.gnu.org; 4 Jul 2019 09:19:19 +0000 Received: from localhost ([127.0.0.1]:50349 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hixtc-0001te-S2 for submit@debbugs.gnu.org; Thu, 04 Jul 2019 05:19:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49923) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hixtZ-0001tR-FV for 36404@debbugs.gnu.org; Thu, 04 Jul 2019 05:19:14 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54709) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hixtT-0007Ux-4k; Thu, 04 Jul 2019 05:19:07 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=42298 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hixtS-0003pg-Ke; Thu, 04 Jul 2019 05:19:06 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH v4 2/4] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 16 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 04 Jul 2019 11:19:03 +0200 In-Reply-To: <87y31gs4k5.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Tue, 02 Jul 2019 13:56:58 -0400") Message-ID: <87r27688e0.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Jakob and all! Apologies for not moving as fast as you do! :-) zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > +(define (switch-to-system machine) > + "Monadic procedure creating a new generation on MACHINE and execute the > +activation script for the new system configuration." [...] > +(define (upgrade-shepherd-services machine) > + "Monadic procedure unloading and starting services on the remote as ne= eded > +to realize the MACHINE's system configuration." [...] > +(define (machine-boot-parameters machine) > + "Monadic procedure returning a list of 'boot-parameters' for the gener= ations > +of MACHINE's system profile, ordered from most recent to oldest." [...] > +(define (install-bootloader machine) > + "Create a bootloader entry for the new system generation on MACHINE, a= nd > +configure the bootloader to boot that generation by default." To me the end goal was to move these =E2=80=9Ceffectful=E2=80=9D bits into = a script, such that both =E2=80=98guix system reconfigure=E2=80=99 and =E2=80=98guix = deploy=E2=80=99 would only have to run that script, locally or remotely. That would avoid duplicating these somewhat tricky procedures. Now, perhaps we can start like this, and leave factorization for later? I just want to make sure we don=E2=80=99t forget about that and let it evol= ve into something we have a hard time maintaining. WDYT? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 05:21:05 2019 Received: (at 36404) by debbugs.gnu.org; 4 Jul 2019 09:21:05 +0000 Received: from localhost ([127.0.0.1]:50354 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hixvN-0001x1-7k for submit@debbugs.gnu.org; Thu, 04 Jul 2019 05:21:05 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50195) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hixvJ-0001w9-Sf for 36404@debbugs.gnu.org; Thu, 04 Jul 2019 05:21:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54720) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hixvE-00005h-DN; Thu, 04 Jul 2019 05:20:56 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=42300 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hixvD-00040z-VL; Thu, 04 Jul 2019 05:20:56 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH v4 4/4] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87tvc4s4iv.fsf_-_@sdf.lonestar.org> <87muhws4hd.fsf_-_@sdf.lonestar.org> <87ftnmn2d4.fsf@dustycloud.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 16 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 04 Jul 2019 11:20:54 +0200 In-Reply-To: <87ftnmn2d4.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Wed, 03 Jul 2019 19:07:51 -0400") Message-ID: <87muhu88ax.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: "Jakob L. Kreuze" , "Thompson, David" , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Christopher Lemmer Webber skribis: > From my perspective, this is ready to merge. Which means that we need > to merge Ludo's remote-eval too. Yes, sorry for the delay on this. I started wondering about details of the =E2=80=98lower-gexp=E2=80=99 API; I=E2=80=99ll try to get to it today= =E2=80=A6 Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 08:49:00 2019 Received: (at submit) by debbugs.gnu.org; 4 Jul 2019 12:49:00 +0000 Received: from localhost ([127.0.0.1]:50536 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hj1Aa-0000gn-Gi for submit@debbugs.gnu.org; Thu, 04 Jul 2019 08:49:00 -0400 Received: from lists.gnu.org ([209.51.188.17]:51960) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hj1AY-0000gf-3F for submit@debbugs.gnu.org; Thu, 04 Jul 2019 08:48:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43061) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hj1AW-0000GC-Nk for guix-patches@gnu.org; Thu, 04 Jul 2019 08:48:57 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hj1AV-0004jJ-NQ for guix-patches@gnu.org; Thu, 04 Jul 2019 08:48:56 -0400 Received: from dustycloud.org ([50.116.34.160]:50960) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hj1AV-0004iS-7u for guix-patches@gnu.org; Thu, 04 Jul 2019 08:48:55 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 50061265C9; Thu, 4 Jul 2019 08:48:52 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <87k1d4kra8.fsf@dustycloud.org> <877e93ewyj.fsf@sdf.lonestar.org> <8736jrvfcw.fsf@zancanaro.id.au> <87a7dzl09t.fsf@dustycloud.org> <875zolb9ex.fsf@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: "Jakob L. Kreuze" Subject: Re: [bug#36404] [PATCH 2/5] gnu: Add machine type for deployment specifications. In-reply-to: <875zolb9ex.fsf@sdf.lonestar.org> Date: Thu, 04 Jul 2019 08:48:51 -0400 Message-ID: <87ef36m0cs.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 50.116.34.160 X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: Carlo Zancanaro , 36404@debbugs.gnu.org, guix-patches@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Jakob L. Kreuze writes: > Christopher Lemmer Webber writes: > >> Jakob, do you mind checking out the issue above? I think it shouldn't >> block merging these patches but perhaps we should file an issue saying >> that when the shepherd issue is merged, changes should be made to guix >> deploy as well. What do you think? > > I took a peek and added a comment about it to machine.scm, are you > suggesting that we track it on debbugs? Yeha, it will help us be less likely to forget it as well as having a nicer place to track it... I think? :) From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 12:00:10 2019 Received: (at 36404) by debbugs.gnu.org; 4 Jul 2019 16:00:10 +0000 Received: from localhost ([127.0.0.1]:51754 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hj49a-0003a4-9f for submit@debbugs.gnu.org; Thu, 04 Jul 2019 12:00:10 -0400 Received: from ol.sdf.org ([205.166.94.20]:51494 helo=mx.sdf.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hj49W-0003Yw-ET for 36404@debbugs.gnu.org; Thu, 04 Jul 2019 12:00:09 -0400 Received: from Upsilon (mobile-107-107-56-10.mycingular.net [107.107.56.10]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x64FxtGK011820 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Thu, 4 Jul 2019 16:00:02 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v4 2/4] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87r27688e0.fsf@gnu.org> Date: Thu, 04 Jul 2019 11:59:49 -0400 In-Reply-To: <87r27688e0.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 04 Jul 2019 11:19:03 +0200") Message-ID: <87o9293i4q.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludovic, Ludovic Court=C3=A8s writes: > To me the end goal was to move these =E2=80=9Ceffectful=E2=80=9D bits int= o a script, > such that both =E2=80=98guix system reconfigure=E2=80=99 and =E2=80=98gui= x deploy=E2=80=99 would only > have to run that script, locally or remotely. That would avoid > duplicating these somewhat tricky procedures. Ah, that's starting to ring a bell now. I believe you mentioned that when 'guix deploy' was initially being proposed, but at the time I didn't quite register that we'd be extracting the behavior in that way. > Now, perhaps we can start like this, and leave factorization for > later? I just want to make sure we don=E2=80=99t forget about that and le= t it > evolve into something we have a hard time maintaining. > > WDYT? I agree. I'm getting the impression that people don't want this to sit in review limbo for too long, and in terms of "commit history hygiene," I think it would be better to recognize refactoring out the common behavior as a distinct change. Thanks! Regards, Jakob --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0eIncACgkQ9Qb9Fp2P 2Vod0w/+Idur/TVyhglj3m8rFifhzLqZylE909dWHY7ThBDUDkTv+hRVMwE15pBq 9aIGf3fiok5jguA9y9KbdaAaXvblSr03tjT1SJ4g64/mnKL35ApR1JYuJ46c7ISl JIDoUjfHYJ/j+sRsa0bpZF7zDTndocl2JGOYFWSQQXcRv5jl7tmuU4Qyty1bd/bt ZkoJ6tgUj63GpT6pZvThqQadU8dYgQFguyEVf+01Iu0MhWaOTbr0EW27kCw+r3p3 jAQo+8aMxvC2DUPm/Wy5i8yTQlDj03zhqDk4JC78g0m6FRpPkNUdbzYKukUNYb/o Ic5zGO0+XS4TfNQV2x3oXENj77FXjaT5j+e75bO7OFZvMhFuVL+oNCMKNFpV93Mw pruw9lrUNbEeqNFcgiKGCzuhicyrCkelTPLz4F8JZewqPVcZQLeqs5zmHGtal8dt mfbYfeYtkXy2TEl+0FsR/Q7QM3Xg6If080aB7NlMQ7czAKd3qDg3VeNeqfMw2ddT uEAZ6mxuPj8QLnokol5aHxkCC9NoFT6Ft1Ls9cEPLT5cjnrbHM/X/Ko29YqRh0QR OxGnGTo0HM711PdXzgRb6aPxiEhQ9zXg8w7KN6+ezOnSjgaJuks4rj9Q6Dn7D6Mw TrqbGoRpwZlkdmVY2tGF9sDjUA4jq9p+TLnSsGmJHCmMgimVjGQ= =FIsV -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 12:05:33 2019 Received: (at submit) by debbugs.gnu.org; 4 Jul 2019 16:05:33 +0000 Received: from localhost ([127.0.0.1]:51766 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hj4Em-0003jK-PZ for submit@debbugs.gnu.org; Thu, 04 Jul 2019 12:05:33 -0400 Received: from lists.gnu.org ([209.51.188.17]:52233) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hj4El-0003jC-01 for submit@debbugs.gnu.org; Thu, 04 Jul 2019 12:05:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43583) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hj4Ei-0006Dj-5W for guix-patches@gnu.org; Thu, 04 Jul 2019 12:05:30 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_20,RCVD_IN_DNSWL_NONE, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hj4Eg-0000BI-F4 for guix-patches@gnu.org; Thu, 04 Jul 2019 12:05:28 -0400 Received: from ol.sdf.org ([205.166.94.20]:50431 helo=mx.sdf.org) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hj4Ef-000067-Vz for guix-patches@gnu.org; Thu, 04 Jul 2019 12:05:26 -0400 Received: from Upsilon (mobile-107-107-56-10.mycingular.net [107.107.56.10]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x64G5EwG022186 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Thu, 4 Jul 2019 16:05:16 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 2/5] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87imspj0ks.fsf_-_@sdf.lonestar.org> <87ef3dj0j9.fsf_-_@sdf.lonestar.org> <87a7e1j0hy.fsf_-_@sdf.lonestar.org> <87k1d4kra8.fsf@dustycloud.org> <877e93ewyj.fsf@sdf.lonestar.org> <8736jrvfcw.fsf@zancanaro.id.au> <87a7dzl09t.fsf@dustycloud.org> <875zolb9ex.fsf@sdf.lonestar.org> <87ef36m0cs.fsf@dustycloud.org> Date: Thu, 04 Jul 2019 12:05:12 -0400 In-Reply-To: <87ef36m0cs.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Thu, 04 Jul 2019 08:48:51 -0400") Message-ID: <87ftnl3hvr.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 205.166.94.20 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: 36404@debbugs.gnu.org, guix-patches@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain Christopher Lemmer Webber writes: > Yeha, it will help us be less likely to forget it as well as having a > nicer place to track it... I think? :) Sounds good to me. I'll file it as soon as this patch gets merged upstream, since we have Carlo's ticket for tracking it in 'guix system reconfigure'. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0eI7gACgkQ9Qb9Fp2P 2Vp5kQ/8C6eYqo42GJGKaui49ir41JMzLNsE0y8BZq66iP/ZWxcnV6nn3EklNODJ azXQ9cb7Ok3LnvlcC+3VUmHnFAkArmECmM4/Haj1QLGiwSaX6o6zGxbcsIeEwyrp nOA3XnxCebcWoIk+7x56KM0LCsChyuXk1Grbe325f/7bt2CrHJntJdL30mTAegnW HeccJNSxbqjp9NGfgVOMD866R8CT0Z5KgS3YtPdzJel8j4LelMaVPbD1rft7+CoT nf3nI/xeG4dgbpP7M7C2nNN5J539hvTX4gGCtV+HRJ39G49J4WFfjl3Q8oQxSx/i ARbcMViQzA/U3bRbj3AVET1LmW1uCl7u1oHB/dc4irPoNtgaoTAmp7MtbQtnCp9c o3syF59ySMbBon5i0wH/e//9qUay/OXuDUSU8Yo4FvdOqKzPkvfCdY87f/LPFl17 Jqfqz4jyTpdsH0mvf8DCo+r9/KeqH7aWtagZsppinkDZ8enV2fpNnvV2YWDwoOxk a+jIdFULmUM9ja3hplf6QzQzkwCUOGiM3jQy238vXWsv3vfVCncMm/O7htFBJ1jz cbVhiyToLPS80/WyTF7qidjzmKxjlHT2mbR+fUCNhHDYg6PaUbXMUJeQDyNgCofs 7gZKMMDreH82loeiEuKiDKKNPydSKl1AqX6VxXQyElVnxP+v/+E= =Yxtz -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 12:48:36 2019 Received: (at 36404) by debbugs.gnu.org; 4 Jul 2019 16:48:36 +0000 Received: from localhost ([127.0.0.1]:51828 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hj4uQ-0000hx-Ey for submit@debbugs.gnu.org; Thu, 04 Jul 2019 12:48:36 -0400 Received: from mx.sdf.org ([205.166.94.20]:56212) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hj4uO-0000hp-MW for 36404@debbugs.gnu.org; Thu, 04 Jul 2019 12:48:33 -0400 Received: from Upsilon (mobile-107-107-56-10.mycingular.net [107.107.56.10]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x64GmKt2010860 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Thu, 4 Jul 2019 16:48:25 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> Date: Thu, 04 Jul 2019 12:48:15 -0400 In-Reply-To: <87d0isrsmk.fsf@sdf.lonestar.org> (Jakob L. Kreuze's message of "Tue, 02 Jul 2019 18:14:43 -0400") Message-ID: <878std3fw0.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Hi Ludovic + Ricardo, Something hit me today. There aren't any tests for 'guix system reconfigure'. There are for 'guix system init' in 'gnu/tests/install.scm', but not for 'guix system reconfigure', which makes me think that I'm going about testing this the wrong way. I feel I should begin by isolate the behavior that's common between 'guix system reconfigure' and 'guix deploy' as you suggested, and then writing tests for that common code in the system test suite. Then, as Ricardo suggested, mocking can be used for the parts that are specific only to 'guix deploy'. I will look into this today and report back. Regards, Jakob --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0eLc8ACgkQ9Qb9Fp2P 2VpQlQ//aGcTvwx5ZgaLunEUdAf0zMUhf364Xncz9kfQsCMvVr6iaAbppbkMAb1x XEk8MYeow+MENm4sACkP154lrgdF7AY+P44rHgAnPwrTtfEfXIPUU5Xhjy6o8MlN 1ahhSA/6cTubl5grBZXexUVvYycZtkYN3zs3uojiFHW5gYiJgqurGfHQrJDcvgVb kL9qYWGj7+HEUULfDDz9wui47AbNAD0QtTPe+cOSYhste9ywQZohEGPJ74nANDl7 0riRA+21HoxQsMYlfax7fSx+T4aqKaihIMh23WFScxXvofL/5WiHxkWrI1xqVb7X /pKmSb+nL+adihCjU+szCqYqVDT5L26NTBYGGQjBT3ffVfmRLjZDtFi2rMOUezPT l4gCQ/im6OKt+zuenw1ofEFEQO2+uHTv3jPPsdrIlX0Dezk+agYkw/5SqxZuZ2fT LyPyeRWp++ZLTTSq1eXzo2KIrKNvvdWZ6eHXLk5aiQRUcjcPtDcyePs2s6Gm9DH7 fqWrc3OEuPacTwGIa1Xr4t6KPry+LpUEroiVhYOsG3lxgrbAEBKKdFNvqMzsOQey TwaJA9QC/ig9XcADKz6VY9dGtRqiPY6Vc4jyWgS9wYw7/VIyD3jRmwSCgW2T42Vm f/S20tOvHpg3vcEn0HS/PLLKs1a29kltCUKUFJqlDdlfiEjXeew= =TpuR -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 21:24:05 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 01:24:05 +0000 Received: from localhost ([127.0.0.1]:52146 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjCxI-0000VJ-Mm for submit@debbugs.gnu.org; Thu, 04 Jul 2019 21:24:04 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:54976) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjCxE-0000Uj-Pk for 36404@debbugs.gnu.org; Thu, 04 Jul 2019 21:24:03 -0400 Received: by mail-wm1-f68.google.com with SMTP id p74so4139736wme.4 for <36404@debbugs.gnu.org>; Thu, 04 Jul 2019 18:24:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=worcester-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=QGPidIKfhHcl5J1UdW7NQd6hYjMMxTJzkZyOTydxMrg=; b=WL0gm/t9nIm9rV63iP5syMNMuVzlOrCn+uW2z+NwS3LcwgGR3KBidvheBVyYTjdvNe 9MNAT6AvoGZ2uU3/YtEkmCbtawE/OkzThHdEuY3gXm6IZMBIz+7IAS45Lv1dqJ1pz0s5 bemrOtDtwS/UqXDCXWf5q68SaBwED1lFUWdgt6EyHDuaSLr9qRnNVVgaax3lvUhnmkHP 1cjDG1NAuZSJwYEWQAQAuu1UzQPQOS3UbvBb6nHP9hkfztDe1dz8KVFzl+ndqs5wp7cZ dZtIetI13A7ggNDaHf0LthXDrScY9araPy3qdHVCwkcCcjK6tD6K9GpP1L/8RwWTrH5W DwmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=QGPidIKfhHcl5J1UdW7NQd6hYjMMxTJzkZyOTydxMrg=; b=gUXI9hpRcgaYEWHmxEcTsdTfjphxkdnd3ty/C2GEC+vkLGXsloCN79QMHb8vHWqMF6 UwIWWeIp+iCrDkJfVW+RdzZTrG6VBheHRj2z6qLwnz9VBQcZYYtv88CboCnS/I2NKbFe jBPObfe3GFjcVVtSTWXecXV51QEiQZiNSw7RM+xfal/H/SEIJQAXRvvDD02LQRBjeEtG O3VvGbEWbfzVMj7r2msFHyYmUQkhjbk0L9pNegj2oMm/bujwaqw2NSW0UuxP0V9kQosh zAte9q1+WSk7X5cO/rduN8bL4+FaRhNggzgQkD2Jly7r5UvLLEK1SX/KBy/B7f2h9uV5 r8dw== X-Gm-Message-State: APjAAAW9HHsIpKDiyBR0eud8NSCERyds3OrKclV2nJk8Y2KaEK/WHT8k Ae7Xo7mL3NgGrCyRkPhcGVwnwarHu+sjJVz1c0O3dw== X-Google-Smtp-Source: APXvYqx/fXm+ix+JnId1MxpqaTL6HK6KIkSfgfNLB8gyfQb7nEFavcBF2H6sm1cyfNpcibUYPkWM7HbR1vK00EoaFE0= X-Received: by 2002:a7b:c651:: with SMTP id q17mr134449wmk.136.1562289834863; Thu, 04 Jul 2019 18:23:54 -0700 (PDT) MIME-Version: 1.0 References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> In-Reply-To: <8736jotj5v.fsf_-_@sdf.lonestar.org> From: "Thompson, David" Date: Thu, 4 Jul 2019 21:23:44 -0400 Message-ID: Subject: Re: [bug#36404] [PATCH v4 1/4] ssh: Add 'identity' keyword to 'open-ssh-session'. To: "Jakob L. Kreuze" Content-Type: text/plain; charset="UTF-8" X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , =?UTF-8?Q?Ludovic_Court=C3=A8s?= , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.1 (-) On Tue, Jul 2, 2019 at 1:56 PM Jakob L. Kreuze wrote: > > * guix/ssh.scm (open-ssh-session): Add 'identity' keyword argument. > --- > guix/ssh.scm | 10 ++++++---- > 1 file changed, 6 insertions(+), 4 deletions(-) > > diff --git a/guix/ssh.scm b/guix/ssh.scm > index 9b9baf54e..9bf10b9a0 100644 > --- a/guix/ssh.scm > +++ b/guix/ssh.scm > @@ -57,12 +57,14 @@ > (define %compression > "zlib@openssh.com,zlib") > > -(define* (open-ssh-session host #:key user port > +(define* (open-ssh-session host #:key user port identity > (compression %compression)) > - "Open an SSH session for HOST and return it. When USER and PORT are #f, use > -default values or whatever '~/.ssh/config' specifies; otherwise use them. > -Throw an error on failure." > + "Open an SSH session for HOST and return it. IDENTITY specifies the path of Replace "path" with "file name". Lots of people use them interchangeably, but GNU makes a clear distinction between the two terms. > +a private key to use for authenticating with the host. When USER, PORT, or > +IDENTITY are #f, use default values or whatever '~/.ssh/config' specifies; > +otherwise use them. Throw an error on failure." > (let ((session (make-session #:user user > + #:identity identity > #:host host > #:port port > #:timeout 10 ;seconds > -- > 2.22.0 From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 21:32:57 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 01:32:57 +0000 Received: from localhost ([127.0.0.1]:52154 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjD5t-0000jt-1X for submit@debbugs.gnu.org; Thu, 04 Jul 2019 21:32:57 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:42668) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjD5p-0000ja-Uf for 36404@debbugs.gnu.org; Thu, 04 Jul 2019 21:32:55 -0400 Received: by mail-wr1-f65.google.com with SMTP id a10so7099423wrp.9 for <36404@debbugs.gnu.org>; Thu, 04 Jul 2019 18:32:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=worcester-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=gYlxv3h7Dh3p7B6wmxTxzSDXQbkRfE5vZA4r0s4uO6o=; b=Nmk+1Phj4OV8IbIC32cQpShoH0yDpRLrTa0n4amHZUqtGsVimB0h+nDRD0C/EgzEYZ nUmEG1jH945FFNFA6XE/GSgzz3amySrRZRkJPlWcUp94xK0alYDlYTunxtF0hOoVyWxf K2x6z79h7JQWSMtIRzI4uX3drSKPm2I+YNY0KFHWvx4xhkWcQ6Anog3dPUVSRV456Pbu uNSeI6NNrezXa4roUDjPS/BmFXWRhzrY4dEuJ9MxZicVg2GutKZg8PfAQPd4hFbrrp7r cDx1NZiP0JI3JU5V2Mp4tPjff3HcyzjP5WGvaGCGFj5np3qOhmR6gor2On6xlfOXtMcD gb5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gYlxv3h7Dh3p7B6wmxTxzSDXQbkRfE5vZA4r0s4uO6o=; b=gTASpvUHq/2wpQsekF+z8A21L6VN5EgTlKg4IlL45unkke0nhi+U8sm1qih/u1vgZF mwDV1PxAt2AzdGZorgp/MhsRugxLy5q5KrZgIC8Db0yO8WFpOvITDF8d232UG3IzNpnx loyt2O4lFG6CQpsWD+zGlkYRFM7tDcCxPBHJs823g8SsVtt0iAsK8hUzwDg6dIbHc72r gry8NI9YBP0iis7LD429E+PKW3dAIiwJwp2vchKCHSmNn9nnlzqQ6TeZRJs0V7KkOCUD aIevjU//IdTXaIytCx1ljw+q5KYDKHcRFbN5ZRwAI+FeEOJMoUtZHpdqjgI/kCRB6+JL jB9g== X-Gm-Message-State: APjAAAVFnwPYVEaCX8Y75kZXt/B1PoLsmTfUvhJ3l6SLBDkp54OtlC91 aYxxnSsna98sdFZuG5iTvOpEwfy3hk/oIXhntCnUHQ== X-Google-Smtp-Source: APXvYqx+T0z65j5Orz5c+2GylfjsBMW0tXEborjpsYzPzrmMN6UhQeT0KmpSe6MrXzaom8hxdWzK1AEZP7Ed99SaiGE= X-Received: by 2002:adf:dd4b:: with SMTP id u11mr751877wrm.189.1562290367927; Thu, 04 Jul 2019 18:32:47 -0700 (PDT) MIME-Version: 1.0 References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> In-Reply-To: <87y31gs4k5.fsf_-_@sdf.lonestar.org> From: "Thompson, David" Date: Thu, 4 Jul 2019 21:32:37 -0400 Message-ID: Subject: Re: [bug#36404] [PATCH v4 2/4] gnu: Add machine type for deployment specifications. To: "Jakob L. Kreuze" Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , =?UTF-8?Q?Ludovic_Court=C3=A8s?= , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Tue, Jul 2, 2019 at 1:57 PM Jakob L. Kreuze wrote: > > * gnu/machine.scm: New file. > * gnu/machine/ssh.scm: New file. > * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. I'm OK with refactoring the reconfigure code in a future patch set. This patch looks good to me! - Dave From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 21:35:24 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 01:35:24 +0000 Received: from localhost ([127.0.0.1]:52159 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjD8G-0000o2-HD for submit@debbugs.gnu.org; Thu, 04 Jul 2019 21:35:24 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:33865) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjD8F-0000np-AW for 36404@debbugs.gnu.org; Thu, 04 Jul 2019 21:35:24 -0400 Received: by mail-wr1-f68.google.com with SMTP id u18so8231808wru.1 for <36404@debbugs.gnu.org>; Thu, 04 Jul 2019 18:35:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=worcester-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=DVAbCxDENyUmIvA21ss/WkAOnoZjvTzRK7T2FVZ+NE8=; b=ta9Jj8rtdj1o7eyFJ9XOchV4XxcQX085MBsq4uD74dhtpvALRIYziO3nzPRS5pUIlA gncLv5GoLnzJY8bLW6nkvDMuUMm0A2HLyoBcYpJzu7GSW8A3FonnlZvM29swNYho3lgV xojxEOw3cVwZYTIUV8usswG9sOi7p5KwQkFqmlSySKxjTOEFLCPBazDJqE2G3EJcaCo2 TIDyzy8xqLoJm1mdG9IZSkftmlygCOL2cfOIXyVzKGx6TSzzYKncyNsMEzvb5DpBB4f7 DAEiT/M1lIsjwXyDF4pLyTuYySI4WNT7VqTZPdvPd660GJXhhX4WrxUPA5LnEOes3L47 jsZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=DVAbCxDENyUmIvA21ss/WkAOnoZjvTzRK7T2FVZ+NE8=; b=nQTB3COLHBqGj+PL2TgXkVT2JEGWtj62RVyEavihhZDMZX6kcJbL8eBLOV0qp03CHf ZDf6l6kZZtDUtz/OanqafChq6fSWq719tMBk0WLHUu3gWm7/RCI7Pk7PJQlC6o/rRiOE nYMwE6332ceMXeBdnBkwAKTpwT6b7gNJEC2kG2V2GhBc4XHZABUhWJ1HEFs+tqvRZ45u vNPRXG6qzR32xpquZFYhlDQfS05XFRnwW6epjQMFhRRhVvlJCbzNCM5ftDm2lqbw5vE1 LPgjW4gtBKIGWCXcfraYI4Py5tITcy1gCN9Z5plz5ArobENI6BqS4jCE718v0dcER/ik HT7A== X-Gm-Message-State: APjAAAWkcMyazWVmX+9oev4v/n6t5jfqd3diPCkPcGuFKmw9FIpinZdv 4VQLThMYIUTwy7aJTMBP0OGIVUmsEsg+MNai8KtjBmdJ X-Google-Smtp-Source: APXvYqw8MuIyMxzuz06iF7t3yR0NRfMSwZgh81frdekDzltlIlbQCAF3LdiSFh5bHFc8ENoxDpPqlMYh8huqr5buZc8= X-Received: by 2002:a5d:4311:: with SMTP id h17mr880516wrq.9.1562290517615; Thu, 04 Jul 2019 18:35:17 -0700 (PDT) MIME-Version: 1.0 References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87tvc4s4iv.fsf_-_@sdf.lonestar.org> In-Reply-To: <87tvc4s4iv.fsf_-_@sdf.lonestar.org> From: "Thompson, David" Date: Thu, 4 Jul 2019 21:35:06 -0400 Message-ID: Subject: Re: [bug#36404] [PATCH v4 3/4] Add 'guix deploy'. To: "Jakob L. Kreuze" Content-Type: text/plain; charset="UTF-8" X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , =?UTF-8?Q?Ludovic_Court=C3=A8s?= , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.1 (-) On Tue, Jul 2, 2019 at 1:57 PM Jakob L. Kreuze wrote: > > * guix/scripts/deploy.scm: New file. > * Makefile.am (MODULES): Add it. Looks good to me! - Dave From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 04 21:39:40 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 01:39:40 +0000 Received: from localhost ([127.0.0.1]:52164 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjDCO-0000u6-4m for submit@debbugs.gnu.org; Thu, 04 Jul 2019 21:39:40 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:38615) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjDCM-0000tt-Ac for 36404@debbugs.gnu.org; Thu, 04 Jul 2019 21:39:38 -0400 Received: by mail-wm1-f65.google.com with SMTP id s15so7699161wmj.3 for <36404@debbugs.gnu.org>; Thu, 04 Jul 2019 18:39:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=worcester-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=gk/ZCYI/PS9RLf60yDemWxoIsiTdz/HX1ucYsPF7hNE=; b=W2aZsqdVF3GqaA2aavylKsOsMyhjz5wvrJZqytYS6TBpab+72Rf585eR/i39EKvub0 fEKb7D3vDdVOaSmniIKkaeUbgTbO1tosU9GiXi7kWTkLsHKayfnm5E3Xfi7UZaUvoMU1 liAB5R8F8S7CQcxUEmrgKdu1c/dsmvwGgkJ7mHy7whPEafQc75V1QQyf53mhFnaA3Pos 8Go4D14iYhuCufwoN4t8K9EQSWoOqFymIGF0GFM2VTISlbwWVjijcT0grIAanGETYmPo IvHIqMpWigtHJ/w5qeyfCBwl1WYFn+NSZHsK1C3CI1cPhQ72A4K16LKDY1FZ101eMLYY jjLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gk/ZCYI/PS9RLf60yDemWxoIsiTdz/HX1ucYsPF7hNE=; b=gQc9OFnWiiAHCitGEez/xOhK4/Q/xoWD+pEECPAUYMzUIBqNqKAvDHsrWyB8Ux2oPf xJCFinGpbun7xzLOnNQI0bP5X/dOo0uJsmU2tzCILmwdZEjxpJItozH5YrEKp8Ymab8r a1OxGmfrwcMgqv/ya2PDx4sqcX23nPWp6J9pFt0DFBEzq5ayoK+QyBBiP4sWvaeU0mKD 1lnZPO96tmWJy45KriBeCArOmQdgogEbIB8cuN0CZ0UsZmtxXuythZk7xNz1m0HrwTx6 2/0wcVFGUcEC0b9YJH0onnANCmi2rmBoqrkDK6OY497Ra1sKbYmt4PXMs7kJ3TFDJmKN zf2A== X-Gm-Message-State: APjAAAV9I7l2nxkSmVRg62k2ytE88acYrQjxPrmasYfNCS5w3j0p3Nty aKp/dnKcW+XwHgvbJzBoAPqJbr4ZNwhSNk111rr/ow== X-Google-Smtp-Source: APXvYqxDFY+8cGhSkZavMUggqWKqsmr5yu6C4glPwQe0DvLRMgvS+6ueb4kkB+DiF6Q22EN+YHnunvwl5ca26Q2IB+4= X-Received: by 2002:a1c:2dd2:: with SMTP id t201mr550811wmt.109.1562290772264; Thu, 04 Jul 2019 18:39:32 -0700 (PDT) MIME-Version: 1.0 References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87tvc4s4iv.fsf_-_@sdf.lonestar.org> <87muhws4hd.fsf_-_@sdf.lonestar.org> In-Reply-To: <87muhws4hd.fsf_-_@sdf.lonestar.org> From: "Thompson, David" Date: Thu, 4 Jul 2019 21:39:21 -0400 Message-ID: Subject: Re: [bug#36404] [PATCH v4 4/4] doc: Add section for 'guix deploy'. To: "Jakob L. Kreuze" Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , =?UTF-8?Q?Ludovic_Court=C3=A8s?= , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Tue, Jul 2, 2019 at 1:58 PM Jakob L. Kreuze wrote: > > * doc/guix.texi: Add section "Invoking guix deploy". Looks good to me. Congratulations in advance for getting 'guix deploy' to master! - Dave From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 04:00:31 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 08:00:31 +0000 Received: from localhost ([127.0.0.1]:52302 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJ8w-0003iP-OB for submit@debbugs.gnu.org; Fri, 05 Jul 2019 04:00:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39085) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJ8t-0003i3-VO for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 04:00:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44456) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjJ8h-00012x-9q; Fri, 05 Jul 2019 04:00:17 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=39232 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjJ8b-00018O-Dc; Fri, 05 Jul 2019 04:00:13 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH 0/6] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 17 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 05 Jul 2019 10:00:03 +0200 In-Reply-To: <878std3fw0.fsf@sdf.lonestar.org> (Jakob L. Kreuze's message of "Thu, 04 Jul 2019 12:48:15 -0400") Message-ID: <87wogwoqrg.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Jakob, zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > Something hit me today. There aren't any tests for 'guix system > reconfigure'. There are for 'guix system init' in > 'gnu/tests/install.scm', but not for 'guix system reconfigure', which > makes me think that I'm going about testing this the wrong way. I feel I > should begin by isolate the behavior that's common between 'guix system > reconfigure' and 'guix deploy' as you suggested, and then writing tests > for that common code in the system test suite. That would be great, especially factorizing these bits. Note that writing tests could be tricky because it=E2=80=99s about testing = the effect of these reconfigure actions. At any rate, let us know how it goes! > Then, as Ricardo suggested, mocking can be used for the parts that are > specific only to 'guix deploy'. Sounds good. Thank you! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 04:10:48 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 08:10:48 +0000 Received: from localhost ([127.0.0.1]:52311 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJIt-0003xA-UN for submit@debbugs.gnu.org; Fri, 05 Jul 2019 04:10:48 -0400 Received: from eggs.gnu.org ([209.51.188.92]:40866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJIs-0003wu-D9 for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 04:10:46 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44553) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjJIl-0007x5-Ub; Fri, 05 Jul 2019 04:10:39 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=39262 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjJIl-0008Ac-3x; Fri, 05 Jul 2019 04:10:39 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: "Thompson\, David" Subject: Re: [bug#36404] [PATCH v4 2/4] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 10:10:37 +0200 In-Reply-To: (David Thompson's message of "Thu, 4 Jul 2019 21:32:37 -0400") Message-ID: <87imsgoq9u.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , "Jakob L. Kreuze" , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, "Thompson, David" skribis: > On Tue, Jul 2, 2019 at 1:57 PM Jakob L. Kreuze > wrote: >> >> * gnu/machine.scm: New file. >> * gnu/machine/ssh.scm: New file. >> * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. > > I'm OK with refactoring the reconfigure code in a future patch set. OK, sounds good to me! Thanks for your feedback, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 04:17:59 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 08:17:59 +0000 Received: from localhost ([127.0.0.1]:52320 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJPr-00047q-2s for submit@debbugs.gnu.org; Fri, 05 Jul 2019 04:17:59 -0400 Received: from eggs.gnu.org ([209.51.188.92]:42026) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJPo-00047d-Q2 for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 04:17:57 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44772) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjJPj-0002rp-Cc; Fri, 05 Jul 2019 04:17:51 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=39264 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjJPi-0000FA-Sy; Fri, 05 Jul 2019 04:17:51 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH v4 3/4] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87tvc4s4iv.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 10:17:49 +0200 In-Reply-To: <87tvc4s4iv.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Tue, 02 Jul 2019 13:57:44 -0400") Message-ID: <87a7dsopxu.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > * guix/scripts/deploy.scm: New file. > * Makefile.am (MODULES): Add it. Overall LGTM, just a couple of minor points: > +++ b/guix/scripts/deploy.scm Please add this file to po/guix/POTFILES.in so it can be subject to localization. > +(define %default-options > + '((system . ,(%current-system)) > + (substitutes? . #t) > + (build-hook? . #t) > + (graft? . #t) > + (debug . 0) > + (verbosity . 2))) =E2=80=98verbosity=E2=80=99 should probably be 1 (only =E2=80=98guix build= =E2=80=99 and =E2=80=98guix system build=E2=80=99 default to 2.) > + (for-each (lambda (machine) > + (format #t "building ~a... " (machine-display-name mac= hine)) > + (run-with-store store (build-machine machine)) > + (display "done\n")) > + machines) > + (for-each (lambda (machine) > + (format #t "deploying to ~a... " (machine-display-name= machine)) > + (run-with-store store (deploy-machine machine)) > + (display "done\n")) > + machines)))) For i18n purposes and also to get consistent output, please avoid =E2=80=98format #t=E2=80=99 and instead write: (info (G_ "deploying ~a=E2=80=A6~%") (machine-display-name machine)) I think you can omit the =E2=80=9Cdone=E2=80=9D message. As a matter of style, it=E2=80=99s clearer IMO to have only one =E2=80=98ru= n-with-store=E2=80=99 call in the whole program. Also, the separate =E2=80=98build-machine=E2=80=99 phase is not needed=E2= =80=94more on that in another message. Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 04:24:22 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 08:24:22 +0000 Received: from localhost ([127.0.0.1]:52328 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJW1-0004He-88 for submit@debbugs.gnu.org; Fri, 05 Jul 2019 04:24:21 -0400 Received: from eggs.gnu.org ([209.51.188.92]:42838) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJVz-0004HS-GG for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 04:24:19 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44828) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjJVt-0006co-BJ; Fri, 05 Jul 2019 04:24:13 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=39266 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjJVs-0000hm-Rt; Fri, 05 Jul 2019 04:24:13 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH v4 2/4] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 10:24:10 +0200 In-Reply-To: <87y31gs4k5.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Tue, 02 Jul 2019 13:56:58 -0400") Message-ID: <87y31cnb2t.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > +(define (build-machine machine) > + "Monadic procedure that builds the system derivation for MACHINE and r= eturning > +a list containing the path of the derivation file and the path of the de= rivation > +output." > + (let ((os (machine-system machine))) > + (mlet* %store-monad ((osdrv (operating-system-derivation os)) > + (_ ((store-lift build-derivations) (list osdrv)= ))) > + (return (list (derivation-file-name osdrv) > + (derivation->output-path osdrv)))))) > + > +(define (machine-remote-eval machine exp) > + "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP re= fers to > +are built and deployed to MACHINE beforehand." > + (let ((environment (machine-environment machine))) > + ((environment-type-machine-remote-eval environment) machine exp))) > + > +(define (deploy-machine machine) > + "Monadic procedure transferring the new system's OS closure to the rem= ote > +MACHINE, activating it on MACHINE and switching MACHINE to the new gener= ation." > + (let ((environment (machine-environment machine))) > + ((environment-type-deploy-machine environment) machine))) In the SSH case, =E2=80=98deploy-machine=E2=80=99 should roughly translate = to: (remote-eval #~(switch-to-system #$os) machine) Thus, =E2=80=98build-machine=E2=80=99 is unnecessary: the actual build of O= S is automatically triggered by =E2=80=98remote-eval=E2=80=99, either locally or= remotely, depending on #:build-locally?. So I believe you can remove =E2=80=98build-machine=E2=80=99 altogether. > + (error "unsupported configuration type")))) It=E2=80=99s a bit verbose, but I=E2=80=99d suggest using SRFI-34/35 instea= d, like so: (raise (condition (&message (message "unsupported machine configuration type")))) That way, if you also add the file to po/guix/POTFILES.in, i18n will do its magic. :-) Otherwise it looks great to me! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 04:29:19 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 08:29:19 +0000 Received: from localhost ([127.0.0.1]:52336 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJap-0004P3-Gy for submit@debbugs.gnu.org; Fri, 05 Jul 2019 04:29:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:43956) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjJao-0004Oq-Ft for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 04:29:18 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44893) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjJaf-0003Uo-SM; Fri, 05 Jul 2019 04:29:09 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=39328 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjJaf-0000M9-DQ; Fri, 05 Jul 2019 04:29:09 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH v4 4/4] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87tvc4s4iv.fsf_-_@sdf.lonestar.org> <87muhws4hd.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 10:29:06 +0200 In-Reply-To: <87muhws4hd.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Tue, 02 Jul 2019 13:58:38 -0400") Message-ID: <87lfxcnaul.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > * doc/guix.texi: Add section "Invoking guix deploy". Yay! You can add a copyright line for you at the top of guix.texi. > +@section Invoking @code{guix deploy} > + > +We've already seen @code{operating-system} declarations used to manage a > +machine's configuration locally. Suppose you need to configure multiple > +machines, though---perhaps you're managing a service on the web that's > +comprised of several servers. @command{guix deploy} enables you to use = those > +same @code{operating-system} declarations to manage multiple remote host= s at > +once as a logical ``deployment''. Perhaps add something like: @quotation Note The functionality described in this section is still under development and is subject to change. Get in touch with us on @email{guix-devel@@gnu.org}! @end quotation That way, if we make a Guix release before this is all stabilized, we make sure people have appropriate expectations. :-) > +complex deployment may involve, for example, starting virtual machines t= hrough > +a VPS provider. In such a case, a different @var{environment} type woul= d be ^^^ I would write =E2=80=9CVirtual Private Server (VPS)=E2=80=9D. I hope the nitpicking level is acceptable, let me know. I=E2=80=99m really excited to see this land in master! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 06:32:50 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 10:32:50 +0000 Received: from localhost ([127.0.0.1]:52393 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjLWI-0007OS-SM for submit@debbugs.gnu.org; Fri, 05 Jul 2019 06:32:50 -0400 Received: from dustycloud.org ([50.116.34.160]:53342) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjLWD-0007OE-Uk for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 06:32:44 -0400 Received: from jasmine (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id D634126620; Fri, 5 Jul 2019 06:32:37 -0400 (EDT) From: Christopher Lemmer Webber To: "Jakob L. Kreuze" Subject: Re: [bug#36404] [PATCH v4 2/4] gnu: Add machine type for deployment specifications. Message-ID: <87h882n2g1.fsf@dustycloud.org> Date: Fri, 05 Jul 2019 06:32:37 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , "Thompson, David" , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.or= g> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tp= u.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2= .fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.= fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.co= m> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org= > <87y31gs4k5.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 In-reply-to: <87y31gs4k5.fsf_-_@sdf.lonestar.org> Jakob L. Kreuze writes: > +(define-record-type* environment-type > + make-environment-type > + environment-type? > + > + ;; Interface to the environment type's deployment code. Each procedure > + ;; should take the same arguments as the top-level procedure of this f= ile > + ;; that shares the same name. For example, 'machine-remote-eval' shoul= d be > + ;; of the form '(machine-remote-eval machine exp)'. > + (machine-remote-eval environment-type-machine-remote-eval) ; procedure > + (deploy-machine environment-type-deploy-machine) ; procedure > + > + ;; Metadata. > + (name environment-type-name) ; symbol > + (description environment-type-description ; string > + (default #f)) > + (location environment-type-location ; > + (default (and=3D> (current-source-location) > + source-properties->location)) > + (innate))) Yeah! I think this is much nicer. :) > + > + > +;;; > +;;; Declarations for machines in a deployment. > +;;; > + > +(define-record-type* machine > + make-machine > + machine? > + this-machine > + (system machine-system) ; > + (environment machine-environment) ; symbol > + (configuration machine-configuration ; configuration object > + (default #f))) ; specific to environment > + > +(define (machine-display-name machine) > + "Return the host-name identifying MACHINE." > + (operating-system-host-name (machine-system machine))) > + > +(define (build-machine machine) > + "Monadic procedure that builds the system derivation for MACHINE and r= eturning > +a list containing the path of the derivation file and the path of the de= rivation > +output." > + (let ((os (machine-system machine))) > + (mlet* %store-monad ((osdrv (operating-system-derivation os)) > + (_ ((store-lift build-derivations) (list osdrv)= ))) > + (return (list (derivation-file-name osdrv) > + (derivation->output-path osdrv)))))) > + > +(define (machine-remote-eval machine exp) > + "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP re= fers to > +are built and deployed to MACHINE beforehand." > + (let ((environment (machine-environment machine))) > + ((environment-type-machine-remote-eval environment) machine exp))) > + > +(define (deploy-machine machine) > + "Monadic procedure transferring the new system's OS closure to the rem= ote > +MACHINE, activating it on MACHINE and switching MACHINE to the new gener= ation." > + (let ((environment (machine-environment machine))) > + ((environment-type-deploy-machine environment) machine))) Oooooh so much cleaner. Nice nice nice! I like this. > diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm > new file mode 100644 > index 000000000..6ce106bb2 > --- /dev/null > +++ b/gnu/machine/ssh.scm > @@ -0,0 +1,363 @@ > +;;; GNU Guix --- Functional package management for GNU > +;;; Copyright =C2=A9 2019 Jakob L. Kreuze > +;;; > +;;; This file is part of GNU Guix. > +;;; > +;;; GNU Guix is free software; you can redistribute it and/or modify it > +;;; under the terms of the GNU General Public License as published by > +;;; the Free Software Foundation; either version 3 of the License, or (at > +;;; your option) any later version. > +;;; > +;;; GNU Guix is distributed in the hope that it will be useful, but > +;;; WITHOUT ANY WARRANTY; without even the implied warranty of > +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +;;; GNU General Public License for more details. > +;;; > +;;; You should have received a copy of the GNU General Public License > +;;; along with GNU Guix. If not, see . > + > +(define-module (gnu machine ssh) > + #:use-module (gnu bootloader) > + #:use-module (gnu machine) > + #:autoload (gnu packages gnupg) (guile-gcrypt) > + #:use-module (gnu services) > + #:use-module (gnu services shepherd) > + #:use-module (gnu system) > + #:use-module (guix derivations) > + #:use-module (guix gexp) > + #:use-module (guix i18n) > + #:use-module (guix modules) > + #:use-module (guix monads) > + #:use-module (guix records) > + #:use-module (guix remote) > + #:use-module (guix ssh) > + #:use-module (guix store) > + #:use-module (ice-9 match) > + #:use-module (srfi srfi-19) > + #:export (managed-host-environment-type > + > + machine-ssh-configuration > + machine-ssh-configuration? > + machine-ssh-configuration > + > + machine-ssh-configuration-host-name > + machine-ssh-configuration-port > + machine-ssh-configuration-user > + machine-ssh-configuration-session)) > + > +;;; Commentary: > +;;; > +;;; This module implements remote evaluation and system deployment for > +;;; machines that are accessable over SSH and have a known host-name. In= the > +;;; sense of the broader "machine" interface, we describe the environmen= t for > +;;; such machines as 'managed-host. > +;;; > +;;; Code: > + > + > +;;; > +;;; Parameters for the SSH client. > +;;; > + > +(define-record-type* machine-ssh-configurati= on > + make-machine-ssh-configuration > + machine-ssh-configuration? > + this-machine-ssh-configuration > + (host-name machine-ssh-configuration-host-name) ; string > + (port machine-ssh-configuration-port ; integer > + (default 22)) > + (user machine-ssh-configuration-user ; string > + (default "root")) > + (identity machine-ssh-configuration-identity ; path to a private key > + (default #f)) > + (session machine-ssh-configuration-session ; session > + (default #f))) > + > +(define (machine-ssh-session machine) > + "Return the SSH session that was given in MACHINE's configuration, or = create > +one from the configuration's parameters if one was not provided." > + (let ((config (machine-configuration machine))) > + (if (machine-ssh-configuration? config) > + (or (machine-ssh-configuration-session config) > + (let ((host-name (machine-ssh-configuration-host-name config= )) > + (user (machine-ssh-configuration-user config)) > + (port (machine-ssh-configuration-port config)) > + (identity (machine-ssh-configuration-identity config))) > + (open-ssh-session host-name > + #:user user > + #:port port > + #:identity identity))) > + (error "unsupported configuration type")))) > + > + > +;;; > +;;; Remote evaluation. > +;;; > + > +(define (managed-host-remote-eval machine exp) > + "Internal implementation of 'machine-remote-eval' for MACHINE instance= s with > +an environment type of 'managed-host." > + (maybe-raise-missing-configuration-error machine) > + (remote-eval exp (machine-ssh-session machine))) > + > + > +;;; > +;;; System deployment. > +;;; > + > +(define (switch-to-system machine) > + "Monadic procedure creating a new generation on MACHINE and execute the > +activation script for the new system configuration." > + (define (remote-exp drv script) > + (with-extensions (list guile-gcrypt) > + (with-imported-modules (source-module-closure '((guix config) > + (guix profiles) > + (guix utils))) > + #~(begin > + (use-modules (guix config) > + (guix profiles) > + (guix utils)) > + > + (define %system-profile > + (string-append %state-directory "/profiles/system")) > + > + (let* ((system #$(derivation->output-path drv)) > + (number (1+ (generation-number %system-profile))) > + (generation (generation-file-name %system-profile num= ber))) > + (switch-symlinks generation system) > + (switch-symlinks %system-profile generation) > + ;; The implementation of 'guix system reconfigure' saves t= he > + ;; load path and environment here. This is unnecessary here > + ;; because each invocation of 'remote-eval' runs in a dist= inct > + ;; Guile REPL. > + (setenv "GUIX_NEW_SYSTEM" system) > + ;; The activation script may write to stdout, which confus= es > + ;; 'remote-eval' when it attempts to read a result from the > + ;; remote REPL. We work around this by forcing the output = to a > + ;; string. > + (with-output-to-string > + (lambda () > + (primitive-load #$script)))))))) > + > + (let* ((os (machine-system machine)) > + (script (operating-system-activation-script os))) > + (mlet* %store-monad ((drv (operating-system-derivation os))) > + (machine-remote-eval machine (remote-exp drv script))))) > + > +;; XXX: Currently, this does NOT attempt to restart running services. Th= is is > +;; also the case with 'guix system reconfigure'. > +;; > +;; See . > +(define (upgrade-shepherd-services machine) > + "Monadic procedure unloading and starting services on the remote as ne= eded > +to realize the MACHINE's system configuration." > + (define target-services > + ;; Monadic expression evaluating to a list of (name output-path) pai= rs for > + ;; all of MACHINE's services. > + (mapm %store-monad > + (lambda (service) > + (mlet %store-monad ((file ((compose lower-object > + shepherd-service-file) > + service))) > + (return (list (shepherd-service-canonical-name service) > + (derivation->output-path file))))) > + (service-value > + (fold-services (operating-system-services (machine-system mac= hine)) > + #:target-type shepherd-root-service-type)))) > + > + (define (remote-exp target-services) > + (with-imported-modules '((gnu services herd)) > + #~(begin > + (use-modules (gnu services herd) > + (srfi srfi-1)) > + > + (define running > + (filter live-service-running (current-services))) > + > + (define (essential? service) > + ;; Return #t if SERVICE is essential and should not be unloa= ded > + ;; under any circumstance. > + (memq (first (live-service-provision service)) > + '(root shepherd))) > + > + (define (obsolete? service) > + ;; Return #t if SERVICE can be safely unloaded. > + (and (not (essential? service)) > + (every (lambda (requirements) > + (not (memq (first (live-service-provision serv= ice)) > + requirements))) > + (map live-service-requirement running)))) > + > + (define to-unload > + (filter obsolete? > + (remove (lambda (service) > + (memq (first (live-service-provision servi= ce)) > + (map first '#$target-services))) > + running))) > + > + (define to-start > + (remove (lambda (service-pair) > + (memq (first service-pair) > + (map (compose first live-service-provision) > + running))) > + '#$target-services)) > + > + ;; Unload obsolete services. > + (for-each (lambda (service) > + (false-if-exception > + (unload-service service))) > + to-unload) > + > + ;; Load the service files for any new services and start them. > + (load-services/safe (map second to-start)) > + (for-each start-service (map first to-start)) > + > + #t))) > + > + (mlet %store-monad ((target-services target-services)) > + (machine-remote-eval machine (remote-exp target-services)))) > + > +(define (machine-boot-parameters machine) > + "Monadic procedure returning a list of 'boot-parameters' for the gener= ations > +of MACHINE's system profile, ordered from most recent to oldest." > + (define bootable-kernel-arguments > + (@@ (gnu system) bootable-kernel-arguments)) > + > + (define remote-exp > + (with-extensions (list guile-gcrypt) > + (with-imported-modules (source-module-closure '((guix config) > + (guix profiles))) > + #~(begin > + (use-modules (guix config) > + (guix profiles) > + (ice-9 textual-ports)) > + > + (define %system-profile > + (string-append %state-directory "/profiles/system")) > + > + (define (read-file path) > + (call-with-input-file path > + (lambda (port) > + (get-string-all port)))) > + > + (map (lambda (generation) > + (let* ((system-path (generation-file-name %system-pro= file > + generation)) > + (boot-parameters-path (string-append system-pa= th > + "/paramet= ers")) > + (time (stat:mtime (lstat system-path)))) > + (list generation > + system-path > + time > + (read-file boot-parameters-path)))) > + (reverse (generation-numbers %system-profile))))))) > + > + (mlet* %store-monad ((generations (machine-remote-eval machine remote-= exp))) > + (return > + (map (lambda (generation) > + (match generation > + ((generation system-path time serialized-params) > + (let* ((params (call-with-input-string serialized-params > + read-boot-parameters)) > + (root (boot-parameters-root-device params)) > + (label (boot-parameters-label params))) > + (boot-parameters > + (inherit params) > + (label > + (string-append label " (#" > + (number->string generation) ", " > + (let ((time (make-time time-utc 0 time= ))) > + (date->string (time-utc->date time) > + "~Y-~m-~d ~H:~M")) > + ")")) > + (kernel-arguments > + (append (bootable-kernel-arguments system-path root) > + (boot-parameters-kernel-arguments params)))))= ))) > + generations)))) > + > +(define (install-bootloader machine) > + "Create a bootloader entry for the new system generation on MACHINE, a= nd > +configure the bootloader to boot that generation by default." > + (define bootloader-installer-script > + (@@ (guix scripts system) bootloader-installer-script)) > + > + (define (remote-exp installer bootcfg bootcfg-file) > + (with-extensions (list guile-gcrypt) > + (with-imported-modules (source-module-closure '((gnu build install) > + (guix store) > + (guix utils))) > + #~(begin > + (use-modules (gnu build install) > + (guix store) > + (guix utils)) > + (let* ((gc-root (string-append "/" %gc-roots-directory "/boo= tcfg")) > + (temp-gc-root (string-append gc-root ".new"))) > + > + (switch-symlinks temp-gc-root gc-root) > + > + (unless (false-if-exception > + (begin > + ;; The implementation of 'guix system reconfigu= re' > + ;; saves the load path here. This is unnecessar= y here > + ;; because each invocation of 'remote-eval' run= s in a > + ;; distinct Guile REPL. > + (install-boot-config #$bootcfg #$bootcfg-file "= /") > + ;; The installation script may write to stdout,= which > + ;; confuses 'remote-eval' when it attempts to r= ead a > + ;; result from the remote REPL. We work around = this > + ;; by forcing the output to a string. > + (with-output-to-string > + (lambda () > + (primitive-load #$installer))))) > + (delete-file temp-gc-root) > + (error "failed to install bootloader")) > + > + (rename-file temp-gc-root gc-root) > + #t))))) > + > + (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine= ))) > + (let* ((os (machine-system machine)) > + (bootloader ((compose bootloader-configuration-bootloader > + operating-system-bootloader) > + os)) > + (bootloader-target (bootloader-configuration-target > + (operating-system-bootloader os))) > + (installer (bootloader-installer-script > + (bootloader-installer bootloader) > + (bootloader-package bootloader) > + bootloader-target > + "/")) > + (menu-entries (map boot-parameters->menu-entry boot-parameter= s)) > + (bootcfg (operating-system-bootcfg os menu-entries)) > + (bootcfg-file (bootloader-configuration-file bootloader))) > + (machine-remote-eval machine (remote-exp installer bootcfg bootcfg= -file))))) > + > +(define (deploy-managed-host machine) > + "Internal implementation of 'deploy-machine' for MACHINE instances wit= h an > +environment type of 'managed-host." > + (maybe-raise-missing-configuration-error machine) > + (mbegin %store-monad > + (switch-to-system machine) > + (upgrade-shepherd-services machine) > + (install-bootloader machine))) > + > + > +;;; > +;;; Environment type. > +;;; > + > +(define managed-host-environment-type > + (environment-type > + (machine-remote-eval managed-host-remote-eval) > + (deploy-machine deploy-managed-host) > + (name 'managed-host-environment-type) > + (description "Provisioning for machines that are accessable o= ver SSH > +and have a known host-name. This entails little more than maintaining an= SSH > +connection to the host."))) > + > +(define (maybe-raise-missing-configuration-error machine) > + "Raise an error if MACHINE's configuration is #f." > + (let ((environment (machine-environment machine))) > + (unless (machine-configuration machine) > + (error (format #f (G_ "no configuration specified for environment = '~a'") > + (symbol->string (environment-type-name environment)= )))))) Yeah ok! This looks good to me. I think my issues are all addressed here. From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 14:53:48 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 18:53:48 +0000 Received: from localhost ([127.0.0.1]:53728 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTLA-0003WN-3Z for submit@debbugs.gnu.org; Fri, 05 Jul 2019 14:53:48 -0400 Received: from mx.sdf.org ([205.166.94.20]:59308) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTL6-0003WC-Tx for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 14:53:46 -0400 Received: from Upsilon (mobile-166-172-60-210.mycingular.net [166.172.60.210]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65IrYCA028684 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 18:53:40 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v5 0/4] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87y31cnb2t.fsf@gnu.org> Date: Fri, 05 Jul 2019 14:53:27 -0400 In-Reply-To: <87y31cnb2t.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Fri, 05 Jul 2019 10:24:10 +0200") Message-ID: <87ftnkgvo8.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable "Thompson, David" writes: > Replace "path" with "file name". Lots of people use them > interchangeably, but GNU makes a clear distinction between the two > terms. Ah, good to know. Updated. Ludovic Court=C3=A8s writes: > Please add this file to po/guix/POTFILES.in so it can be subject to > localization. > >> +(define %default-options >> + '((system . ,(%current-system)) >> + (substitutes? . #t) >> + (build-hook? . #t) >> + (graft? . #t) >> + (debug . 0) >> + (verbosity . 2))) > > =E2=80=98verbosity=E2=80=99 should probably be 1 (only =E2=80=98guix buil= d=E2=80=99 and =E2=80=98guix system > build=E2=80=99 default to 2.) > >> + (for-each (lambda (machine) >> + (format #t "building ~a... " (machine-display-name ma= chine)) >> + (run-with-store store (build-machine machine)) >> + (display "done\n")) >> + machines) >> + (for-each (lambda (machine) >> + (format #t "deploying to ~a... " (machine-display-nam= e machine)) >> + (run-with-store store (deploy-machine machine)) >> + (display "done\n")) >> + machines)))) > > For i18n purposes and also to get consistent output, please avoid > =E2=80=98format #t=E2=80=99 and instead write: > > (info (G_ "deploying ~a=E2=80=A6~%") (machine-display-name machine)) > > I think you can omit the =E2=80=9Cdone=E2=80=9D message. > > As a matter of style, it=E2=80=99s clearer IMO to have only one =E2=80=98= run-with-store=E2=80=99 > call in the whole program. As in, create a monadic expression with 'mapm' to evaluate the multiple calls to '(deploy-machine machine)' in sequence, and then pass that to 'run-with-store'? > In the SSH case, =E2=80=98deploy-machine=E2=80=99 should roughly translat= e to: >=20 > (remote-eval #~(switch-to-system #$os) machine) >=20 > Thus, =E2=80=98build-machine=E2=80=99 is unnecessary: the actual build of= OS is > automatically triggered by =E2=80=98remote-eval=E2=80=99, either locally = or remotely, > depending on #:build-locally?. >=20 > So I believe you can remove =E2=80=98build-machine=E2=80=99 altogether. Thanks for pointing that out; I meant to ask about that since it's kinda vestigial at this point, but wasn't sure if it would be better to have it f= or the UI. But I went ahead and removed it, since we already have code for showing what derivations are going to be built, etc. > It=E2=80=99s a bit verbose, but I=E2=80=99d suggest using SRFI-34/35 inst= ead, like so: >=20 > (raise (condition > (&message (message "unsupported machine configuration type")))) >=20 > That way, if you also add the file to po/guix/POTFILES.in, i18n will do > its magic. :-) In the end, I generalized the various configuration-related error messages into a 'maybe-raise-unsupported-configuration-error' that uses SRFI-35. Hopefully that's alright -- I believe the manual specifies the behavior enough that one more detailed message is better than two. > Yay! > > You can add a copyright line for you at the top of guix.texi. > >> +@section Invoking @code{guix deploy} >> + >> +We've already seen @code{operating-system} declarations used to manage a >> +machine's configuration locally. Suppose you need to configure multiple >> +machines, though---perhaps you're managing a service on the web that's >> +comprised of several servers. @command{guix deploy} enables you to use= those >> +same @code{operating-system} declarations to manage multiple remote hos= ts at >> +once as a logical ``deployment''. > > Perhaps add something like: > > @quotation Note > The functionality described in this section is still under development > and is subject to change. Get in touch with us on > @email{guix-devel@@gnu.org}! > @end quotation > > That way, if we make a Guix release before this is all stabilized, > we make sure people have appropriate expectations. :-) I like it! >> +complex deployment may involve, for example, starting virtual machines = through >> +a VPS provider. In such a case, a different @var{environment} type wou= ld be > ^^^ > I would write =E2=80=9CVirtual Private Server (VPS)=E2=80=9D. > > I hope the nitpicking level is acceptable, let me know. I=E2=80=99m real= ly > excited to see this land in master! Oh, I appreciate this level of attention to detail. The hardest part of technical writing for me is having my writing fit in with the writing around it when contributing to an existing document, so these kinds of comments fr= om someone more familiar with the manual are great. Jakob L. Kreuze (4): ssh: Add 'identity' keyword to 'open-ssh-session'. gnu: Add machine type for deployment specifications. Add 'guix deploy'. doc: Add section for 'guix deploy'. Makefile.am | 4 +- doc/guix.texi | 114 +++++++++++++ gnu/local.mk | 5 +- gnu/machine.scm | 107 ++++++++++++ gnu/machine/ssh.scm | 369 ++++++++++++++++++++++++++++++++++++++++ guix/scripts/deploy.scm | 84 +++++++++ guix/ssh.scm | 10 +- po/guix/POTFILES.in | 2 + 8 files changed, 689 insertions(+), 6 deletions(-) create mode 100644 gnu/machine.scm create mode 100644 gnu/machine/ssh.scm create mode 100644 guix/scripts/deploy.scm =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0fnKoACgkQ9Qb9Fp2P 2Vp83Q//UHJcBe4p8/NmtJu3ziBkP6eO9JoNSVeXXy/jUR9eWAU9G5/4hHCo/ISi kaYG5f9hlR2qL7hDEVOVqdP+xjmVevq/GsLFBOkeEuB9YRthOom2L9Ulh79UL5ql y4GqS+8T8k14EblWB/B4eSMGSYcKtK/hdOAeuQ1AH+sVZ7EuwfmpU5zR1efXccJo aGQBSeksxgh+/lqjadIbJyN0nT8Ll9voOC3xYDAWvpz2L8ovr0xLkaZXqQKMfp5h kdBeu+4AwI4m5mN3L5VxQsC/YWdfuqjga7DzvJYUrr5rgn+wuggyQeI2LyEjGQn8 y04JkqXfXsyCO8o4Desp8vNtNW0JoMN1SD75l4s/MldJX8qdpd9uW40s3U7ntJkq zGSGtAvh8kWHzbot0cD0o3PGMhnfeck3dxweg1SFJyITEeipMTteIEAP269S1h99 j5JSSIFvOv3xWoTCzOifOeQ3C5wom5QPpQWhhrJ930xQhuafEOLfj8IDlWfeg7Q5 MSr0yQUFHQOtIVObrLE/cdRAr5RfegkZY5CE0tWwGkFDnkWLTc3mFQ5v7tIYMXnH PL+OgiRk6joSrvMaTRTUV8p/cl0a+avx7ayJI1H5tPy4KiBwRUqco3AzoP+IkIjt xiU0Vv8t5qvO5SjbtfK9xgcCirIYVhyvVTTQZbnpacQ/YMVM6SI= =sbDX -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 14:54:39 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 18:54:40 +0000 Received: from localhost ([127.0.0.1]:53732 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTLz-0003Xu-KV for submit@debbugs.gnu.org; Fri, 05 Jul 2019 14:54:39 -0400 Received: from mx.sdf.org ([205.166.94.20]:59097) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTLx-0003Xm-5J for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 14:54:37 -0400 Received: from Upsilon (mobile-166-172-60-210.mycingular.net [166.172.60.210]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65IsYbm019608 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 18:54:35 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v5 1/4] ssh: Add 'identity' keyword to 'open-ssh-session'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87y31cnb2t.fsf@gnu.org> <87ftnkgvo8.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 14:54:32 -0400 In-Reply-To: <87ftnkgvo8.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 14:53:27 -0400") Message-ID: <878stcgvmf.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * guix/ssh.scm (open-ssh-session): Add 'identity' keyword argument. =2D-- guix/ssh.scm | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/guix/ssh.scm b/guix/ssh.scm index 9b9baf54e..ede00133c 100644 =2D-- a/guix/ssh.scm +++ b/guix/ssh.scm @@ -57,12 +57,14 @@ (define %compression "zlib@openssh.com,zlib") =20 =2D(define* (open-ssh-session host #:key user port +(define* (open-ssh-session host #:key user port identity (compression %compression)) =2D "Open an SSH session for HOST and return it. When USER and PORT are #= f, use =2Ddefault values or whatever '~/.ssh/config' specifies; otherwise use them. =2DThrow an error on failure." + "Open an SSH session for HOST and return it. IDENTITY specifies the file +name of a private key to use for authenticating with the host. When USER, +PORT, or IDENTITY are #f, use default values or whatever '~/.ssh/config' +specifies; otherwise use them. Throw an error on failure." (let ((session (make-session #:user user + #:identity identity #:host host #:port port #:timeout 10 ;seconds =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0fnOgACgkQ9Qb9Fp2P 2Vpu6w//R4MuzdpOGHrsX8xif8jBrL74E8IH7bGBZk4axoZB83aDgbnmcoo9nW8h O9+p+DzJN3uGrYUUsbHaDbAHYWlAlvZEFACzHSW80GkYAUd4UbuDNYq7MHOAnyIs WepDaA4bTrGJW+Qy/P6EAcYrduZ548dGtG/wT8xSGC+KVh0oyDLaQfFWlsE3Noe/ OgyZSDDNu7PBX77zYMll4JuIroTiwznHtLRlDS6KbWWNkX4AYvqudkSLix0lsOpI qCsGXu1peXHN9RLzooQ9etR3aKFVak185gNFu36tm2MXqeNOp1qrQBPeCchn4QBB yp6/ZaTYlm218enakMbzPonadwzSLmeqtzGBYxLTUQDKtlWljaYErOaq5yvShia4 MuAbknjuDPNMtDUkbtaFs6zraqKkR2J/7lbuRvLazcNZuLWLXwxIdqWG5EAbzGXF vUo3DsZ5z4UPk4ja6oHXUId4soNAhgy0lwWv4clSogmXyY3SoObKxsNk+2N3pj5O T5PJsH3W9cHGhNykkhFF1VWumQOBnoFcHnF67ZLbILGU2jjo9YU1DkJoPOIcFO8w Vmr9vNp0hQgaCcsMV/CZXrSB6DXZ8eaNMgvo2BsSqhQWgVY3sYZzoLPmZBThYQuN OElnS+Cj6BzzI7QkBjH0F4+q3I7D/qihZYBR+Irx6P1ApLKGi4M= =/Is9 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 14:55:39 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 18:55:39 +0000 Received: from localhost ([127.0.0.1]:53736 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTMr-0003Zf-0c for submit@debbugs.gnu.org; Fri, 05 Jul 2019 14:55:39 -0400 Received: from mx.sdf.org ([205.166.94.20]:58883) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTMm-0003ZS-OY for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 14:55:31 -0400 Received: from Upsilon (mobile-166-172-60-210.mycingular.net [166.172.60.210]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65ItMuK001574 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 18:55:24 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v5 2/4] gnu: Add machine type for deployment specifications. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87y31cnb2t.fsf@gnu.org> <87ftnkgvo8.fsf_-_@sdf.lonestar.org> <878stcgvmf.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 14:55:21 -0400 In-Reply-To: <878stcgvmf.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 14:54:32 -0400") Message-ID: <874l40gvl2.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable * gnu/machine.scm: New file. * gnu/machine/ssh.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. =2D-- Makefile.am | 3 +- gnu/local.mk | 5 +- gnu/machine.scm | 107 +++++++++++++ gnu/machine/ssh.scm | 369 ++++++++++++++++++++++++++++++++++++++++++++ po/guix/POTFILES.in | 1 + 5 files changed, 483 insertions(+), 2 deletions(-) create mode 100644 gnu/machine.scm create mode 100644 gnu/machine/ssh.scm diff --git a/Makefile.am b/Makefile.am index 42307abae..f10c000ea 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -425,7 +425,8 @@ SCM_TESTS =3D \ tests/import-utils.scm \ tests/store-database.scm \ tests/store-deduplication.scm \ =2D tests/store-roots.scm + tests/store-roots.scm \ + tests/machine.scm =20 SH_TESTS =3D \ tests/guix-build.sh \ diff --git a/gnu/local.mk b/gnu/local.mk index 81de156cf..0e17af953 100644 =2D-- a/gnu/local.mk +++ b/gnu/local.mk @@ -562,6 +562,9 @@ GNU_SYSTEM_MODULES =3D \ %D%/system/uuid.scm \ %D%/system/vm.scm \ \ + %D%/machine.scm \ + %D%/machine/ssh.scm \ + \ %D%/build/accounts.scm \ %D%/build/activation.scm \ %D%/build/bootloader.scm \ @@ -627,7 +630,7 @@ INSTALLER_MODULES =3D \ %D%/installer/newt/user.scm \ %D%/installer/newt/utils.scm \ %D%/installer/newt/welcome.scm \ =2D %D%/installer/newt/wifi.scm=09 + %D%/installer/newt/wifi.scm =20 # Always ship the installer modules but compile them only when # ENABLE_INSTALLER is true. diff --git a/gnu/machine.scm b/gnu/machine.scm new file mode 100644 index 000000000..0b79402b0 =2D-- /dev/null +++ b/gnu/machine.scm @@ -0,0 +1,107 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu machine) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix store) + #:use-module ((guix utils) #:select (source-properties->location)) + #:export (environment-type + environment-type? + environment-type-name + environment-type-description + environment-type-location + + machine + machine? + this-machine + + machine-system + machine-environment + machine-configuration + machine-display-name + + deploy-machine + machine-remote-eval)) + +;;; Commentary: +;;; +;;; This module provides the types used to declare individual machines in a +;;; heterogeneous Guix deployment. The interface allows users of specify s= ystem +;;; configurations and the means by which resources should be provisioned = on a +;;; per-host basis. +;;; +;;; Code: + + +;;; +;;; Declarations for resources that can be provisioned. +;;; + +(define-record-type* environment-type + make-environment-type + environment-type? + + ;; Interface to the environment type's deployment code. Each procedure + ;; should take the same arguments as the top-level procedure of this file + ;; that shares the same name. For example, 'machine-remote-eval' should = be + ;; of the form '(machine-remote-eval machine exp)'. + (machine-remote-eval environment-type-machine-remote-eval) ; procedure + (deploy-machine environment-type-deploy-machine) ; procedure + + ;; Metadata. + (name environment-type-name) ; symbol + (description environment-type-description ; string + (default #f)) + (location environment-type-location ; + (default (and=3D> (current-source-location) + source-properties->location)) + (innate))) + + +;;; +;;; Declarations for machines in a deployment. +;;; + +(define-record-type* machine + make-machine + machine? + this-machine + (system machine-system) ; + (environment machine-environment) ; symbol + (configuration machine-configuration ; configuration object + (default #f))) ; specific to environment + +(define (machine-display-name machine) + "Return the host-name identifying MACHINE." + (operating-system-host-name (machine-system machine))) + +(define (machine-remote-eval machine exp) + "Evaluate EXP, a gexp, on MACHINE. Ensure that all the elements EXP refe= rs to +are built and deployed to MACHINE beforehand." + (let ((environment (machine-environment machine))) + ((environment-type-machine-remote-eval environment) machine exp))) + +(define (deploy-machine machine) + "Monadic procedure transferring the new system's OS closure to the remote +MACHINE, activating it on MACHINE and switching MACHINE to the new generat= ion." + (let ((environment (machine-environment machine))) + ((environment-type-deploy-machine environment) machine))) diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm new file mode 100644 index 000000000..a7d1a967a =2D-- /dev/null +++ b/gnu/machine/ssh.scm @@ -0,0 +1,369 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu machine ssh) + #:use-module (gnu bootloader) + #:use-module (gnu machine) + #:autoload (gnu packages gnupg) (guile-gcrypt) + #:use-module (gnu services) + #:use-module (gnu services shepherd) + #:use-module (gnu system) + #:use-module (guix derivations) + #:use-module (guix gexp) + #:use-module (guix i18n) + #:use-module (guix modules) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix remote) + #:use-module (guix ssh) + #:use-module (guix store) + #:use-module (ice-9 match) + #:use-module (srfi srfi-19) + #:use-module (srfi srfi-35) + #:export (managed-host-environment-type + + machine-ssh-configuration + machine-ssh-configuration? + machine-ssh-configuration + + machine-ssh-configuration-host-name + machine-ssh-configuration-port + machine-ssh-configuration-user + machine-ssh-configuration-session)) + +;;; Commentary: +;;; +;;; This module implements remote evaluation and system deployment for +;;; machines that are accessable over SSH and have a known host-name. In t= he +;;; sense of the broader "machine" interface, we describe the environment = for +;;; such machines as 'managed-host. +;;; +;;; Code: + + +;;; +;;; Parameters for the SSH client. +;;; + +(define-record-type* machine-ssh-configuration + make-machine-ssh-configuration + machine-ssh-configuration? + this-machine-ssh-configuration + (host-name machine-ssh-configuration-host-name) ; string + (port machine-ssh-configuration-port ; integer + (default 22)) + (user machine-ssh-configuration-user ; string + (default "root")) + (identity machine-ssh-configuration-identity ; path to a private key + (default #f)) + (session machine-ssh-configuration-session ; session + (default #f))) + +(define (machine-ssh-session machine) + "Return the SSH session that was given in MACHINE's configuration, or cr= eate +one from the configuration's parameters if one was not provided." + (maybe-raise-unsupported-configuration-error machine) + (let ((config (machine-configuration machine))) + (or (machine-ssh-configuration-session config) + (let ((host-name (machine-ssh-configuration-host-name config)) + (user (machine-ssh-configuration-user config)) + (port (machine-ssh-configuration-port config)) + (identity (machine-ssh-configuration-identity config))) + (open-ssh-session host-name + #:user user + #:port port + #:identity identity))))) + + +;;; +;;; Remote evaluation. +;;; + +(define (managed-host-remote-eval machine exp) + "Internal implementation of 'machine-remote-eval' for MACHINE instances = with +an environment type of 'managed-host." + (maybe-raise-unsupported-configuration-error machine) + (remote-eval exp (machine-ssh-session machine))) + + +;;; +;;; System deployment. +;;; + +(define (switch-to-system machine) + "Monadic procedure creating a new generation on MACHINE and execute the +activation script for the new system configuration." + (define (remote-exp drv script) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles) + (guix utils))) + #~(begin + (use-modules (guix config) + (guix profiles) + (guix utils)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (let* ((system #$drv) + (number (1+ (generation-number %system-profile))) + (generation (generation-file-name %system-profile numbe= r))) + (switch-symlinks generation system) + (switch-symlinks %system-profile generation) + ;; The implementation of 'guix system reconfigure' saves the + ;; load path and environment here. This is unnecessary here + ;; because each invocation of 'remote-eval' runs in a distin= ct + ;; Guile REPL. + (setenv "GUIX_NEW_SYSTEM" system) + ;; The activation script may write to stdout, which confuses + ;; 'remote-eval' when it attempts to read a result from the + ;; remote REPL. We work around this by forcing the output to= a + ;; string. + (with-output-to-string + (lambda () + (primitive-load #$script)))))))) + + (let* ((os (machine-system machine)) + (script (operating-system-activation-script os))) + (mlet* %store-monad ((drv (operating-system-derivation os))) + (machine-remote-eval machine (remote-exp drv script))))) + +;; XXX: Currently, this does NOT attempt to restart running services. This= is +;; also the case with 'guix system reconfigure'. +;; +;; See . +(define (upgrade-shepherd-services machine) + "Monadic procedure unloading and starting services on the remote as need= ed +to realize the MACHINE's system configuration." + (define target-services + ;; Monadic expression evaluating to a list of (name output-path) pairs= for + ;; all of MACHINE's services. + (mapm %store-monad + (lambda (service) + (mlet %store-monad ((file ((compose lower-object + shepherd-service-file) + service))) + (return (list (shepherd-service-canonical-name service) + (derivation->output-path file))))) + (service-value + (fold-services (operating-system-services (machine-system machi= ne)) + #:target-type shepherd-root-service-type)))) + + (define (remote-exp target-services) + (with-imported-modules '((gnu services herd)) + #~(begin + (use-modules (gnu services herd) + (srfi srfi-1)) + + (define running + (filter live-service-running (current-services))) + + (define (essential? service) + ;; Return #t if SERVICE is essential and should not be unloaded + ;; under any circumstance. + (memq (first (live-service-provision service)) + '(root shepherd))) + + (define (obsolete? service) + ;; Return #t if SERVICE can be safely unloaded. + (and (not (essential? service)) + (every (lambda (requirements) + (not (memq (first (live-service-provision servic= e)) + requirements))) + (map live-service-requirement running)))) + + (define to-unload + (filter obsolete? + (remove (lambda (service) + (memq (first (live-service-provision service= )) + (map first '#$target-services))) + running))) + + (define to-start + (remove (lambda (service-pair) + (memq (first service-pair) + (map (compose first live-service-provision) + running))) + '#$target-services)) + + ;; Unload obsolete services. + (for-each (lambda (service) + (false-if-exception + (unload-service service))) + to-unload) + + ;; Load the service files for any new services and start them. + (load-services/safe (map second to-start)) + (for-each start-service (map first to-start)) + + #t))) + + (mlet %store-monad ((target-services target-services)) + (machine-remote-eval machine (remote-exp target-services)))) + +(define (machine-boot-parameters machine) + "Monadic procedure returning a list of 'boot-parameters' for the generat= ions +of MACHINE's system profile, ordered from most recent to oldest." + (define bootable-kernel-arguments + (@@ (gnu system) bootable-kernel-arguments)) + + (define remote-exp + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles))) + #~(begin + (use-modules (guix config) + (guix profiles) + (ice-9 textual-ports)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (define (read-file path) + (call-with-input-file path + (lambda (port) + (get-string-all port)))) + + (map (lambda (generation) + (let* ((system-path (generation-file-name %system-profi= le + generation)) + (boot-parameters-path (string-append system-path + "/parameter= s")) + (time (stat:mtime (lstat system-path)))) + (list generation + system-path + time + (read-file boot-parameters-path)))) + (reverse (generation-numbers %system-profile))))))) + + (mlet* %store-monad ((generations (machine-remote-eval machine remote-ex= p))) + (return + (map (lambda (generation) + (match generation + ((generation system-path time serialized-params) + (let* ((params (call-with-input-string serialized-params + read-boot-parameters)) + (root (boot-parameters-root-device params)) + (label (boot-parameters-label params))) + (boot-parameters + (inherit params) + (label + (string-append label " (#" + (number->string generation) ", " + (let ((time (make-time time-utc 0 time))) + (date->string (time-utc->date time) + "~Y-~m-~d ~H:~M")) + ")")) + (kernel-arguments + (append (bootable-kernel-arguments system-path root) + (boot-parameters-kernel-arguments params)))))))) + generations)))) + +(define (install-bootloader machine) + "Create a bootloader entry for the new system generation on MACHINE, and +configure the bootloader to boot that generation by default." + (define bootloader-installer-script + (@@ (guix scripts system) bootloader-installer-script)) + + (define (remote-exp installer bootcfg bootcfg-file) + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((gnu build install) + (guix store) + (guix utils))) + #~(begin + (use-modules (gnu build install) + (guix store) + (guix utils)) + (let* ((gc-root (string-append "/" %gc-roots-directory "/bootc= fg")) + (temp-gc-root (string-append gc-root ".new"))) + + (switch-symlinks temp-gc-root gc-root) + + (unless (false-if-exception + (begin + ;; The implementation of 'guix system reconfigure' + ;; saves the load path here. This is unnecessary = here + ;; because each invocation of 'remote-eval' runs = in a + ;; distinct Guile REPL. + (install-boot-config #$bootcfg #$bootcfg-file "/") + ;; The installation script may write to stdout, w= hich + ;; confuses 'remote-eval' when it attempts to rea= d a + ;; result from the remote REPL. We work around th= is + ;; by forcing the output to a string. + (with-output-to-string + (lambda () + (primitive-load #$installer))))) + (delete-file temp-gc-root) + (error "failed to install bootloader")) + + (rename-file temp-gc-root gc-root) + #t))))) + + (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine))) + (let* ((os (machine-system machine)) + (bootloader ((compose bootloader-configuration-bootloader + operating-system-bootloader) + os)) + (bootloader-target (bootloader-configuration-target + (operating-system-bootloader os))) + (installer (bootloader-installer-script + (bootloader-installer bootloader) + (bootloader-package bootloader) + bootloader-target + "/")) + (menu-entries (map boot-parameters->menu-entry boot-parameters)) + (bootcfg (operating-system-bootcfg os menu-entries)) + (bootcfg-file (bootloader-configuration-file bootloader))) + (machine-remote-eval machine (remote-exp installer bootcfg bootcfg-f= ile))))) + +(define (deploy-managed-host machine) + "Internal implementation of 'deploy-machine' for MACHINE instances with = an +environment type of 'managed-host." + (maybe-raise-unsupported-configuration-error machine) + (mbegin %store-monad + (switch-to-system machine) + (upgrade-shepherd-services machine) + (install-bootloader machine))) + + +;;; +;;; Environment type. +;;; + +(define managed-host-environment-type + (environment-type + (machine-remote-eval managed-host-remote-eval) + (deploy-machine deploy-managed-host) + (name 'managed-host-environment-type) + (description "Provisioning for machines that are accessable ove= r SSH +and have a known host-name. This entails little more than maintaining an S= SH +connection to the host."))) + +(define (maybe-raise-unsupported-configuration-error machine) + "Raise an error if MACHINE's configuration is not an instance of +." + (let ((config (machine-configuration machine)) + (environment (environment-type-name (machine-environment machine))= )) + (unless (and config (machine-ssh-configuration? config)) + (raise (condition + (&message + (message (format #f (G_ "unsupported machine configuration = '~a' +for environment of type '~a'") + config + environment)))))))) diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.in index ceee589b2..bcd6f7637 100644 =2D-- a/po/guix/POTFILES.in +++ b/po/guix/POTFILES.in @@ -36,6 +36,7 @@ gnu/installer/steps.scm gnu/installer/timezone.scm gnu/installer/user.scm gnu/installer/utils.scm +gnu/machine/ssh.scm guix/scripts.scm guix/scripts/build.scm guix/discovery.scm =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0fnRkACgkQ9Qb9Fp2P 2VrJ0Q//QCaPECMneKZ+8KvvwoQm+oLeHe2OQUIKGZQTIa8qUQrdL0eR1R/I6Oda e4dQWq6AwjW/FxJ+X0eWXNpYAo4wdidk0CP0qhKUYqwDaO2ww81bYvtAizvjFwkC xhKbnI7A2jTvMHLppQpgZpcJdrG4pJdzIlNdPG5XWFOcFiIaR/qLuOsELgljK8Qz PEqkzDh48gXoPA1pujK0CrIPhaka/axpuVjGjeXFGY+RWUIz4g6YBZbKJe2X7dU3 OvXLrgYCro7yA7d5IRjCYnVu23w+t51EM8Pc543sIIEC1/dxiiaOB76JoEgLrzpS MoQ21IODgwx8G+nBjrxDj2CCKbiE6SNu2BDQS5o11q7D9NvOxFNvOrAExVhK+K85 bSQhEk1mKXIA4tUB2g/CjrKovtMp/zC4SmsVTI8hBkT8IbmxvUwGQRKWN5sl83F5 AIK8b2ccKVKIhHQq9guIrgFAxDZR/GJYtrfnEenBAsiHFxq2kGimmeK6QxJoknG4 bpmQqPGFL9IEI0F7t170uXQKxejp7Kkw6dJJTDA7moG94BZ8VQmUXtikW9uJZBAv UhveGWf1QHxLvySqnVVt1E1dN/LHt6svgRf6REmfCC7cJM9AcBzGlTARiWkHFZgT dGe9FLk4FSLN2TaF8pA0dnpqWq/p+FFWjZ0JKX73sREQNoV0WJQ= =nt9L -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 14:56:16 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 18:56:16 +0000 Received: from localhost ([127.0.0.1]:53740 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTNY-0003bQ-8o for submit@debbugs.gnu.org; Fri, 05 Jul 2019 14:56:16 -0400 Received: from mx.sdf.org ([205.166.94.20]:58701) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTNV-0003bF-95 for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 14:56:14 -0400 Received: from Upsilon (mobile-166-172-60-210.mycingular.net [166.172.60.210]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65Iu8vI028936 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 18:56:10 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v5 3/4] Add 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87y31cnb2t.fsf@gnu.org> <87ftnkgvo8.fsf_-_@sdf.lonestar.org> <878stcgvmf.fsf_-_@sdf.lonestar.org> <874l40gvl2.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 14:56:07 -0400 In-Reply-To: <874l40gvl2.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 14:55:21 -0400") Message-ID: <87zhlsfgzc.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable * guix/scripts/deploy.scm: New file. * Makefile.am (MODULES): Add it. =2D-- Makefile.am | 1 + guix/scripts/deploy.scm | 84 +++++++++++++++++++++++++++++++++++++++++ po/guix/POTFILES.in | 1 + 3 files changed, 86 insertions(+) create mode 100644 guix/scripts/deploy.scm diff --git a/Makefile.am b/Makefile.am index f10c000ea..4d3024e58 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -267,6 +267,7 @@ MODULES =3D \ guix/scripts/weather.scm \ guix/scripts/container.scm \ guix/scripts/container/exec.scm \ + guix/scripts/deploy.scm \ guix.scm \ $(GNU_SYSTEM_MODULES) =20 diff --git a/guix/scripts/deploy.scm b/guix/scripts/deploy.scm new file mode 100644 index 000000000..978cfb2a8 =2D-- /dev/null +++ b/guix/scripts/deploy.scm @@ -0,0 +1,84 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 David Thompson +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts deploy) + #:use-module (gnu machine) + #:use-module (guix scripts) + #:use-module (guix scripts build) + #:use-module (guix store) + #:use-module (guix ui) + #:use-module (ice-9 format) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-37) + #:export (guix-deploy)) + +;;; Commentary: +;;; +;;; This program provides a command-line interface to (gnu machine), allow= ing +;;; users to perform remote deployments through specification files. +;;; +;;; Code: + + + +(define (show-help) + (display (G_ "Usage: guix deploy [OPTION] FILE... +Perform the deployment specified by FILE.\n")) + (show-build-options-help) + (newline) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + %standard-build-options)) + +(define %default-options + '((system . ,(%current-system)) + (substitutes? . #t) + (build-hook? . #t) + (graft? . #t) + (debug . 0) + (verbosity . 1))) + +(define (load-source-file file) + "Load FILE as a user module." + (let ((module (make-user-module '((gnu) (gnu machine) (gnu machine ssh))= ))) + (load* file module))) + +(define (guix-deploy . args) + (define (handle-argument arg result) + (alist-cons 'file arg result)) + (let* ((opts (parse-command-line args %options (list %default-options) + #:argument-handler handle-argument)) + (file (assq-ref opts 'file)) + (machines (or (and file (load-source-file file)) '()))) + (with-store store + (set-build-options-from-command-line store opts) + (for-each (lambda (machine) + (info (G_ "deploying to ~a...") (machine-display-name ma= chine)) + (run-with-store store (deploy-machine machine))) + machines)))) diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.in index bcd6f7637..f5fc4956b 100644 =2D-- a/po/guix/POTFILES.in +++ b/po/guix/POTFILES.in @@ -67,6 +67,7 @@ guix/scripts/pack.scm guix/scripts/weather.scm guix/scripts/describe.scm guix/scripts/processes.scm +guix/scripts/deploy.scm guix/gnu-maintenance.scm guix/scripts/container.scm guix/scripts/container/exec.scm =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0fnUcACgkQ9Qb9Fp2P 2VoQ1RAAqYwk30Cf1qm0bkXMTHmjzsgIai+BVOWDeT05Vo2xbe4FjqJbXwaJFmns 7BmKOKoJVBcp8BWJP2/ODB/huq3YOeZcEU7Y4PqSOPjj1O63NLs7O42locdLwLVD If0Nwo/9IXxfHMsTEEQYPJ5FXeEyI7UL8+uA+1f9v3dKqc4VSFjx/nF6eYfj8XuQ Q7TMUXL4go1WMupT3YfGF7xTyYpKJ/+CPWn1kHTxqmh5DWdy6ul+QAdeYH1iSHBU /zFc2dK5/WIx2pvGN1Pcmae7qtUYphQyJaRlAJlXH8GT6AOAvvXff5HFRXzNf2Cp PhvyITmYIIcSjv3pyS+5ZZZIFRqYdFd8/i5fUlZ434rSZQph15w+WgbbSn+0LKa3 44R43saolhfZtYAmaIqL0GmEyIqYvj4hNBQLYYruKRfsg//JX1uMpG5L0US/fAR/ kZQ9ZmNTttq/azHVx0ibJrlvCEnapbjU4Qi9Sn5wG5xeVRyTb9EsZr7v12o9fU01 RqaEOH5mj0XzfMAMqBerBwYQITYdxa5poPprPIb/Nu7NK12iMl7SeHhGsy+Nxbi3 1VqKpLdblxBdEfozgoksywhtT+scTn9fqIxby03vxqNO2OoO3/6fifUmW2loumFY fe9LdfzwHSZjzwu6woBCcvIHvfp/6Ipeqd6rw+q+1or6uh0S41A= =CkXA -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 14:57:48 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 18:57:48 +0000 Received: from localhost ([127.0.0.1]:53744 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTP1-0003dx-SO for submit@debbugs.gnu.org; Fri, 05 Jul 2019 14:57:48 -0400 Received: from mx.sdf.org ([205.166.94.20]:58405) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTOz-0003do-Fu for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 14:57:46 -0400 Received: from Upsilon (mobile-166-172-60-210.mycingular.net [166.172.60.210]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65Ivems014593 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 18:57:42 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH v5 4/4] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87y31cnb2t.fsf@gnu.org> <87ftnkgvo8.fsf_-_@sdf.lonestar.org> <878stcgvmf.fsf_-_@sdf.lonestar.org> <874l40gvl2.fsf_-_@sdf.lonestar.org> <87zhlsfgzc.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 14:57:39 -0400 In-Reply-To: <87zhlsfgzc.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 14:56:07 -0400") Message-ID: <87v9wgfgws.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org, "Thompson, David" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * doc/guix.texi: Add section "Invoking guix deploy". =2D-- doc/guix.texi | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index 9dc1d2a9c..8d9b7c575 100644 =2D-- a/doc/guix.texi +++ b/doc/guix.texi @@ -65,6 +65,7 @@ Copyright @copyright{} 2018 Alex Vong@* Copyright @copyright{} 2019 Josh Holland@* Copyright @copyright{} 2019 Diego Nicola Barbato@* Copyright @copyright{} 2019 Ivan Petkov@* +Copyright @copyright{} 2019 Jakob L. Kreuze@* =20 Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or @@ -81,6 +82,7 @@ Documentation License''. * guix gc: (guix)Invoking guix gc. Reclaiming unused disk space. * guix pull: (guix)Invoking guix pull. Update the list of available= packages. * guix system: (guix)Invoking guix system. Manage the operating system = configuration. +* guix deploy: (guix)Invoking guix deploy. Manage operating system conf= igurations for remote hosts. @end direntry =20 @dircategory Software development @@ -269,6 +271,7 @@ System Configuration * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remo= te host. * Running Guix in a VM:: How to run Guix System in a virtual machin= e. * Defining Services:: Adding new service definitions. =20 @@ -10302,6 +10305,7 @@ instance to support new system services. * Initial RAM Disk:: Linux-Libre bootstrapping. * Bootloader Configuration:: Configuring the boot loader. * Invoking guix system:: Instantiating a system configuration. +* Invoking guix deploy:: Deploying a system configuration to a remo= te host. * Running Guix in a VM:: How to run Guix System in a virtual machin= e. * Defining Services:: Adding new service definitions. @end menu @@ -25335,6 +25339,116 @@ example graph. =20 @end table =20 +@node Invoking guix deploy +@section Invoking @code{guix deploy} + +We've already seen @code{operating-system} declarations used to manage a +machine's configuration locally. Suppose you need to configure multiple +machines, though---perhaps you're managing a service on the web that's +comprised of several servers. @command{guix deploy} enables you to use th= ose +same @code{operating-system} declarations to manage multiple remote hosts = at +once as a logical ``deployment''. + +@quotation Note +The functionality described in this section is still under development +and is subject to change. Get in touch with us on +@email{guix-devel@@gnu.org}! +@end quotation + +@example +guix deploy @var{file} +@end example + +Such an invocation will deploy the machines that the code within @var{file} +evaluates to. As an example, @var{file} might contain a definition like t= his: + +@example +;; This is a Guix deployment of a "bare bones" setup, with +;; no X11 display server, to a machine with an SSH daemon +;; listening on localhost:2222. A configuration such as this +;; may be appropriate for virtual machine with ports +;; forwarded to the host's loopback interface. + +(use-service-modules networking ssh) +(use-package-modules bootloaders) + +(define %system + (operating-system + (host-name "gnu-deployed") + (timezone "Etc/UTC") + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (target "/dev/vda") + (terminal-outputs '(console)))) + (file-systems (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + %base-file-systems)) + (services + (append (list (service dhcp-client-service-type) + (service openssh-service-type + (openssh-configuration + (permit-root-login #t) + (allow-empty-passwords? #t)))) + %base-services)))) + +(list (machine + (system %system) + (environment managed-host-environment-type) + (configuration (machine-ssh-configuration + (host-name "localhost") + (identity "./id_rsa") + (port 2222))))) +@end example + +The file should evaluate to a list of @var{machine} objects. This example, +upon being deployed, will create a new generation on the remote system +realizing the @code{operating-system} declaration @var{%system}. +@var{environment} and @var{configuration} specify how the machine should be +provisioned---that is, how the computing resources should be created and +managed. The above example does not create any resources, as a +@code{'managed-host} is a machine that is already running the Guix system = and +available over the network. This is a particularly simple case; a more +complex deployment may involve, for example, starting virtual machines thr= ough +a Virtual Private Server (VPS) provider. In such a case, a different +@var{environment} type would be used. + +@deftp {Data Type} machine +This is the data type representing a single machine in a heterogeneous Guix +deployment. + +@table @asis +@item @code{system} +The object of the operating system configuration to deploy. + +@item @code{environment} +An @code{environment-type} describing how the machine should be provisione= d. +At the moment, the only supported value is +@code{managed-host-environment-type}. + +@item @code{configuration} (default: @code{#f}) +An object describing the configuration for the machine's @code{environment= }. +If the @code{environment} has a default configuration, @code{#f} maybe use= d. +If @code{#f} is used for an environment with no default configuration, +however, an error will be thrown. +@end table +@end deftp + +@deftp {Data Type} machine-ssh-configuration +This is the data type representing the SSH client parameters for a machine +with an @code{environment} of @code{managed-host-environment-type}. + +@table @asis +@item @code{host-name} +@item @code{port} (default: @code{22}) +@item @code{user} (default: @code{"root"}) +@item @code{identity} (default: @code{#f}) +If specified, the path to the SSH private key to use to authenticate with = the +remote host. +@end table +@end deftp + @node Running Guix in a VM @section Running Guix in a Virtual Machine =20 =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0fnaMACgkQ9Qb9Fp2P 2VpurA//cTywMLZlTw1osGBxBZ/8mlX1H/KXoUbJEe6bj6WURBPiME+ou6Y3wW6S 8BbSKMo1ooyPo1mqz/UU3w4f9FPsLBGnI4pramAAPdHYMyPdeSuBtLazPzfbItPK YIAY81Xwq2OyqbpvyDh5wWd31I3PqkJj39Os0Ryi241L/t+EbbLRXQc9XBRKFKwI 1e8cplkB9U/o1j9Ho4vRybAGGV+cNgH1kEONbiMObHXhftvx7SKA/idgKz1wUw0y UFLvTWT3oKWb+hGduMgw1g9PqVXzXfuAK3QcDmunVPzQVWPL13PPVwqBzCv4x/Vf jP4O+gMN7rAWCzdW9eya8ucb5e5A7ecre7xbUEHiN0ddAxCS3E97p3VfH4enzkHy gXxDJd9pxzCq4j2VxiMyDnkKxehBDgrhm+grfD8ZYDY/rSPbRJxuUHEbC7F8AJag HBZe+OH1lvtVig2Tbk9qjq/WBLGHYeomCqCxgRkjnT2MB7DQ73lssioEXrR6bJ98 4738ImDNWJRsdvLxDUOE/tD6r8mPuYFNU44c/Let2Tx9WO9b5o0OHh4SAA/dMvkS QS9AxYvwaOcFQq3asp2NCaklMLb/uV4VLVG3r1nJvXmrRKB23Scfb4O+tEAdIlHM apokTYuULYnDYulEl67X7VzhUU3WcBTAXBFnW26W/QUBDVfX4+o= =xPO0 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 18:13:54 2019 Received: (at 36404-done) by debbugs.gnu.org; 5 Jul 2019 22:13:54 +0000 Received: from localhost ([127.0.0.1]:53911 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjWSo-00050w-El for submit@debbugs.gnu.org; Fri, 05 Jul 2019 18:13:54 -0400 Received: from dustycloud.org ([50.116.34.160]:54480) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjWSm-00050l-Ai for 36404-done@debbugs.gnu.org; Fri, 05 Jul 2019 18:13:52 -0400 Received: from twig (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 766AF26650; Fri, 5 Jul 2019 18:13:51 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87y31cnb2t.fsf@gnu.org> <87ftnkgvo8.fsf_-_@sdf.lonestar.org> <878stcgvmf.fsf_-_@sdf.lonestar.org> <874l40gvl2.fsf_-_@sdf.lonestar.org> <87zhlsfgzc.fsf_-_@sdf.lonestar.org> <87v9wgfgws.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: "Jakob L. Kreuze" Subject: Re: [bug#36404] [PATCH v5 4/4] doc: Add section for 'guix deploy'. In-reply-to: <87v9wgfgws.fsf_-_@sdf.lonestar.org> Date: Sat, 06 Jul 2019 02:14:29 -0400 Message-ID: <87bly7n0ze.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 36404-done Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , "Thompson, David" , 36404-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Since those changes seemed to reflect everyone's requests, I've pushed it to git master. Huge congrats to Jakob! I'm stoked about it. Now who can race to the finish line to be the first one using these tools for their server deployment? :) From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 19:26:02 2019 Received: (at 36404-done) by debbugs.gnu.org; 5 Jul 2019 23:26:02 +0000 Received: from localhost ([127.0.0.1]:53964 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXac-0006zy-8D for submit@debbugs.gnu.org; Fri, 05 Jul 2019 19:26:02 -0400 Received: from mx.sdf.org ([205.166.94.20]:55798) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXaa-0006zU-BP for 36404-done@debbugs.gnu.org; Fri, 05 Jul 2019 19:26:01 -0400 Received: from Upsilon (mobile-166-171-185-104.mycingular.net [166.171.185.104]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65NPkfx012302 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 23:25:56 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH v5 4/4] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87y31cnb2t.fsf@gnu.org> <87ftnkgvo8.fsf_-_@sdf.lonestar.org> <878stcgvmf.fsf_-_@sdf.lonestar.org> <874l40gvl2.fsf_-_@sdf.lonestar.org> <87zhlsfgzc.fsf_-_@sdf.lonestar.org> <87v9wgfgws.fsf_-_@sdf.lonestar.org> <87bly7n0ze.fsf@dustycloud.org> Date: Fri, 05 Jul 2019 19:25:26 -0400 In-Reply-To: <87bly7n0ze.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sat, 06 Jul 2019 02:14:29 -0400") Message-ID: <87lfxcf4ih.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404-done Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , "Thompson, David" , 36404-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Christopher Lemmer Webber writes: > Since those changes seemed to reflect everyone's requests, I've pushed > it to git master. > > Huge congrats to Jakob! I'm stoked about it. > > Now who can race to the finish line to be the first one using these > tools for their server deployment? :) Thanks for committing the patch series, this is very exciting! --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0f3GcACgkQ9Qb9Fp2P 2Vr/yQ/9H/x9Exe9NSVrm7jZzATzMEAN9+wrI8MKD3jkpvq57uG6NsT0c+Rcsmfd PRcnKRFq8WzFlPvXEufzGBk4tdsYzHWt/Hg20vQXahzTqOTbQTZDs1XDi9lqS4wS SceJ7VWFJteKGyp4Deh3bRq8y/qd2xE22j9jU+pkaC6iGx3NT/P7EtYXLTkE7bDu +xU4J89oDQt5Pkr9TxqqxZoyVfGO0Wgqs6PMc2Kjl13ZGjLFl96BApCuJkP61Hsd HXwTyim6+i7rjhg8srAm5dBIKY3aV1Ide9KEi9YiDrJJsll6Ji1a5nZcbJEBTmGs GGqI9Oj/YrR/JZnSl/h27271Z2wx/q4ayPHJpQQAhTov3Q7cm5DIsMBXg3VNH2nN tU+oYUYHkbt1J8efh2zekskivFNYxPZSNHmGF8NO3vB9AghNM3Klb/i013xbseUE cNPireP6Gdkn0M8YaFvWhGiTEXgfTYi3KwT0guX0gMYUtzrL0ooWK3fKF+L+PKED PuIltVA7/CpYSMFCU48ibbwy2ZN6PuBPdiW4qsAnSm8UPtePhMjH6xz+G7z6Yeuk /gZbfROZctZd4TxMWqzxazBJBapSiwX+0WS6oJ14DVeENolsJT0jyifzudycn+Lj dnroPNweqEf6ozce9spQyZYpNXoeiHdyh1iDe9tvilEV/QmT8vY= =QZut -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 19:46:03 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 23:46:03 +0000 Received: from localhost ([127.0.0.1]:53982 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXtz-0007ZJ-7A for submit@debbugs.gnu.org; Fri, 05 Jul 2019 19:46:03 -0400 Received: from mx.sdf.org ([205.166.94.20]:51184) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXtx-0007YY-Vy for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 19:46:02 -0400 Received: from Upsilon (mobile-166-171-185-104.mycingular.net [166.171.185.104]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65NjmLd011216 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 23:45:52 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 0/3] Refactor out common behavior for system reconfiguration. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> Date: Fri, 05 Jul 2019 19:45:41 -0400 In-Reply-To: <87wogwoqrg.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Fri, 05 Jul 2019 10:00:03 +0200") Message-ID: <87bly8f3kq.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Note that writing tests could be tricky because it=E2=80=99s about testin= g the > effect of these reconfigure actions. At any rate, let us know how it > goes! This is a _very_ preliminary patch series. I'm not nearly done with it yet; the procedures in guix/scripts/system.scm that I've replaced have some handling for i.e. installing the bootloader configuration without running the installer script, which my reimplementations don't yet support. I'm sending this tonight to make sure I'm on the right track: is this sort of what you meant by extracting the common behavior into scripts? Also, I didn't include any tests as part of this series, but implementing reconfiguration like this does, indeed, make testing for 'guix deploy' much, much easier. And we'll get some tests for the behavior of 'guix system reconfigure' out of it, too! Jakob L. Kreuze (3): guix system: Add 'reconfigure' module. machine: Reimplement 'managed-host-environment-type' deployment. guix system: Reimplement 'reconfigure'. Makefile.am | 1 + gnu/machine/ssh.scm | 235 ++++++++-------------------- guix/scripts/system.scm | 162 ++++++------------- guix/scripts/system/reconfigure.scm | 157 +++++++++++++++++++ 4 files changed, 270 insertions(+), 285 deletions(-) create mode 100644 guix/scripts/system/reconfigure.scm =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0f4SYACgkQ9Qb9Fp2P 2Vpi5A/9HUXchiXj+CK9NqXFQPqESOW3hZAJnUKlzNvi83E07Czv6JTCnohPaiPG ZqVAkmY2rH+gNPn8uiEDgRViW1pSqB72UOO+C7POtoCu0hw7RDlslEidL+Lf55/H L1Abv4vYOLqd5Snry5vCWpQFCgeNW27yaZqqY4gcRR4acYvYBWUkWk383OG04E0n 5UNp+Dq9pxxg7aBHt3o8DTDXLOMr8kB1E9DJjmJV+BteDONSEIy8WdBwAdjtcI9n h6Jjpz82/qY/4wdejNNoDfwbEXcZ7AItUa8mqyn+fk3AALKTJRvrAUWmmV/aX9a1 3sfcwn08n7SHN5aRfFCHYuMcrmXBUJIPl80xH225JjEWe3nnRCLUZOsneZOODRF/ VOaQXVZys6bUhYeQqUUkdOqcp930t3zvt8KUzkyyGPe3A2YRCPZtIXd0Ubq4bPYR 2CuNSMw2kuda/YAGErXmDVxH145ccH1OaIJ2OVz6ymTT/7zApQOrVJcYVGrZ9O31 Op/+qjtDp1k18c8y+GTKB5QvtyoZdjLM0/c+KDGtbBrlWZoPb5jjk6AIUiSVYuV0 4RBKjQL3tLD/vj6NV99XFEYpyFLL9GpKJ2QmEz/mMUOmfPQjmtfSoba6REArDMBZ C8uBvCdPm+05yFQjq9kJIWSjwuou6aWcpRRH85SoOgSXfUyaoms= =0FvD -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 19:47:08 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 23:47:08 +0000 Received: from localhost ([127.0.0.1]:53986 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXv1-0007cK-Ix for submit@debbugs.gnu.org; Fri, 05 Jul 2019 19:47:08 -0400 Received: from mx.sdf.org ([205.166.94.20]:51005) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXuy-0007c9-6X for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 19:47:05 -0400 Received: from Upsilon (mobile-166-171-185-104.mycingular.net [166.171.185.104]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65Nkoxq002677 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 23:46:56 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 1/3] guix system: Add 'reconfigure' module. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 19:46:44 -0400 In-Reply-To: <87bly8f3kq.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 19:45:41 -0400") Message-ID: <877e8wf3iz.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable * guix/scripts/system/reconfigure.scm: New file. * Makefile.am (MODULES): Add it. * guix/scripts/system.scm (bootloader-installer-script): Export variable. =2D-- Makefile.am | 1 + guix/scripts/system.scm | 1 + guix/scripts/system/reconfigure.scm | 157 ++++++++++++++++++++++++++++ 3 files changed, 159 insertions(+) create mode 100644 guix/scripts/system/reconfigure.scm diff --git a/Makefile.am b/Makefile.am index 4d3024e58..1934a21b1 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -245,6 +245,7 @@ MODULES =3D \ guix/scripts/describe.scm \ guix/scripts/system.scm \ guix/scripts/system/search.scm \ + guix/scripts/system/reconfigure.scm \ guix/scripts/lint.scm \ guix/scripts/challenge.scm \ guix/scripts/import/crate.scm \ diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index 60c1ca5c9..21858ee7d 100644 =2D-- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -70,6 +70,7 @@ #:use-module (ice-9 match) #:use-module (rnrs bytevectors) #:export (guix-system + bootloader-installer-script read-operating-system)) =20 diff --git a/guix/scripts/system/reconfigure.scm b/guix/scripts/system/reco= nfigure.scm new file mode 100644 index 000000000..f4ca6b4b1 =2D-- /dev/null +++ b/guix/scripts/system/reconfigure.scm @@ -0,0 +1,157 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 Jakob L. Kreuze +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts system reconfigure) + #:autoload (gnu packages gnupg) (guile-gcrypt) + #:use-module (guix gexp) + #:use-module (guix modules) + #:export (switch-to-system + upgrade-shepherd-services + install-bootloader)) + +;;; Commentary: +;;; +;;; This module implements the "effectful" parts of system +;;; reconfiguration. Although building a system derivation is a pure +;;; operation, a number of impure operations must be carried out for the +;;; system configuration to be realized -- chiefly, creation of generation +;;; symlinks and invocation of activation scripts. +;;; +;;; Code: + +(define (switch-to-system system-derivation activation-script) + "Return a G-Expression that, upon being evaluated, will create a new +generation for SYSTEM-DERIVATION and execute ACTIVATION-SCRIPT." + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((guix config) + (guix profiles) + (guix utils))) + #~(begin + (use-modules (guix config) + (guix profiles) + (guix utils)) + + (define %system-profile + (string-append %state-directory "/profiles/system")) + + (let* ((system #$system-derivation) + (number (1+ (generation-number %system-profile))) + (generation (generation-file-name %system-profile number)= )) + (switch-symlinks generation system) + (switch-symlinks %system-profile generation) + ;; The implementation of 'guix system reconfigure' saves the + ;; load path and environment here. This is unnecessary here + ;; because each invocation of 'remote-eval' runs in a distinct + ;; Guile REPL. + (setenv "GUIX_NEW_SYSTEM" system) + ;; The activation script may write to stdout, which confuses + ;; 'remote-eval' when it attempts to read a result from the + ;; remote REPL. We work around this by forcing the output to a + ;; string. + (with-output-to-string + (lambda () + (primitive-load #$activation-script)))))))) + +;; XXX: Currently, this does NOT attempt to restart running services. See +;; for details. +(define (upgrade-shepherd-services target-services) + "Return a G-Expression that, upon being evaluated, will use TARGET-SERVI= CES, +a list of (shepherd-service-canonical-name, shepherd-service-file) pairs to +determine which services are obsolete and need to be unloaded, as well as +which services are new and need to be started." + (with-imported-modules '((gnu services herd)) + #~(begin + (use-modules (gnu services herd) + (srfi srfi-1)) + + (define running + (filter live-service-running (current-services))) + + (define (essential? service) + ;; Return #t if SERVICE is essential and should not be unloaded + ;; under any circumstance. + (memq (first (live-service-provision service)) + '(root shepherd))) + + (define (obsolete? service) + ;; Return #t if SERVICE can be safely unloaded. + (and (not (essential? service)) + (every (lambda (requirements) + (not (memq (first (live-service-provision service)) + requirements))) + (map live-service-requirement running)))) + + (define to-unload + (filter obsolete? + (remove (lambda (service) + (memq (first (live-service-provision service)) + (map first '#$target-services))) + running))) + + (define to-start + (remove (lambda (service-pair) + (memq (first service-pair) + (map (compose first live-service-provision) + running))) + '#$target-services)) + + ;; Unload obsolete services. + (for-each (lambda (service) + (false-if-exception + (unload-service service))) + to-unload) + + ;; Load the service files for any new services and start them. + (load-services/safe (map second to-start)) + (for-each start-service (map first to-start))))) + +(define (install-bootloader installer-script bootcfg bootcfg-file target) + "Return a G-Expression that, upon being evaluated, will install BOOTCFG = to +BOOTCFG-FILE, a target path, on TARGET, a mount point, and subsequently run +INSTALLER-SCRIPT." + (with-extensions (list guile-gcrypt) + (with-imported-modules (source-module-closure '((gnu build install) + (guix store) + (guix utils))) + #~(begin + (use-modules (gnu build install) + (guix store) + (guix utils)) + (let* ((gc-root (string-append "/" %gc-roots-directory "/bootcfg= ")) + (temp-gc-root (string-append gc-root ".new"))) + + (switch-symlinks temp-gc-root gc-root) + + (unless (false-if-exception + (begin + ;; The implementation of 'guix system reconfigure' + ;; saves the load path here. This is unnecessary he= re + ;; because each invocation of 'remote-eval' runs in= a + ;; distinct Guile REPL. + (install-boot-config #$bootcfg #$bootcfg-file #$tar= get) + ;; The installation script may write to stdout, whi= ch + ;; confuses 'remote-eval' when it attempts to read a + ;; result from the remote REPL. We work around this + ;; by forcing the output to a string. + (with-output-to-string + (lambda () + (primitive-load #$installer-script))))) + (delete-file temp-gc-root) + (error "failed to install bootloader")) + + (rename-file temp-gc-root gc-root)))))) =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0f4WQACgkQ9Qb9Fp2P 2Vo9Hw/+KM7O6w9UYcD/ts2eVeOj2PiJtd3GMRWHP+k+AUUk4dqGV2IujmTx0eWK ZlWMahR19Xvf+HXSUF3RuJU2d2zhgb2Rtk8QjTFXYUPouSVn2nPekFySK57JkG4V MIN0ijzqGoudzx5letv0wJoPqb/fpfaDdZF1CzAwDLY91l7sCz/V04LN67N0LPLU /XXaL5vgAzNdFV5oC3Dg0Mt7tNFho6O0pwtRaJ5jbUwpPUo0fV3WX9BEU8idNXRX v1uh8Lz1v8CLNxneunOSAcQkcY8NGzuawLWM7r0vDqBScz/bb68yRBpDzdTpR/8Y +kTcV8eM2kwchiP2LPpq9b4YNji7/rUxbdAnpshWn9IamZoOY+hLtdo7W+mvmOKz lEFcIs7h43pWrQQ3re/1AhTc+O+vOGC5qJ+nvduoAGXVKFy/TvaiX9irt6SUqBrZ yozjeOjpqfoon5KmsAF0cxkbFZLQoDZMve8xW7aQoKPClxHem6Ny0rPISm8DCV8A mYtHVbHoxlNPL8wd8/DT4T5c4EqOEEzuxStpSJTu2+4LmJ7df+GLdbmKjP/1Qusg vTKGjjRI7ZemccTMJSOcygUbwrvye0uVIa10Jv2OGq4sioE7kJBFw/856r13/vhq urFO2yxsgTZijI0sdvgeTcaQtDyHLQH2U1Qn47SsSuajVzELR9M= =0KCQ -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 19:48:03 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 23:48:03 +0000 Received: from localhost ([127.0.0.1]:53990 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXvv-0007e6-9m for submit@debbugs.gnu.org; Fri, 05 Jul 2019 19:48:03 -0400 Received: from mx.sdf.org ([205.166.94.20]:50884) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXvq-0007dX-On for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 19:48:01 -0400 Received: from Upsilon (mobile-166-171-185-104.mycingular.net [166.171.185.104]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65Nlqi5009219 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 23:47:53 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 2/3] machine: Reimplement 'managed-host-environment-type' deployment. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> <877e8wf3iz.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 19:47:50 -0400 In-Reply-To: <877e8wf3iz.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 19:46:44 -0400") Message-ID: <8736jkf3h5.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * gnu/machine/ssh.scm (switch-to-system, upgrade-shepherd-services) (install-bootloader): Delete variable. * gnu/machine/ssh.scm (deploy-managed-host): Rewrite procedure. =2D-- gnu/machine/ssh.scm | 235 ++++++++++++-------------------------------- 1 file changed, 61 insertions(+), 174 deletions(-) diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm index a7d1a967a..72e6407f0 100644 =2D-- a/gnu/machine/ssh.scm +++ b/gnu/machine/ssh.scm @@ -30,10 +30,13 @@ #:use-module (guix monads) #:use-module (guix records) #:use-module (guix remote) + #:use-module (guix scripts system) + #:use-module (guix scripts system reconfigure) #:use-module (guix ssh) #:use-module (guix store) #:use-module (ice-9 match) #:use-module (srfi srfi-19) + #:use-module (srfi srfi-26) #:use-module (srfi srfi-35) #:export (managed-host-environment-type =20 @@ -105,118 +108,6 @@ an environment type of 'managed-host." ;;; System deployment. ;;; =20 =2D(define (switch-to-system machine) =2D "Monadic procedure creating a new generation on MACHINE and execute the =2Dactivation script for the new system configuration." =2D (define (remote-exp drv script) =2D (with-extensions (list guile-gcrypt) =2D (with-imported-modules (source-module-closure '((guix config) =2D (guix profiles) =2D (guix utils))) =2D #~(begin =2D (use-modules (guix config) =2D (guix profiles) =2D (guix utils)) =2D =2D (define %system-profile =2D (string-append %state-directory "/profiles/system")) =2D =2D (let* ((system #$drv) =2D (number (1+ (generation-number %system-profile))) =2D (generation (generation-file-name %system-profile num= ber))) =2D (switch-symlinks generation system) =2D (switch-symlinks %system-profile generation) =2D ;; The implementation of 'guix system reconfigure' saves t= he =2D ;; load path and environment here. This is unnecessary here =2D ;; because each invocation of 'remote-eval' runs in a dist= inct =2D ;; Guile REPL. =2D (setenv "GUIX_NEW_SYSTEM" system) =2D ;; The activation script may write to stdout, which confus= es =2D ;; 'remote-eval' when it attempts to read a result from the =2D ;; remote REPL. We work around this by forcing the output = to a =2D ;; string. =2D (with-output-to-string =2D (lambda () =2D (primitive-load #$script)))))))) =2D =2D (let* ((os (machine-system machine)) =2D (script (operating-system-activation-script os))) =2D (mlet* %store-monad ((drv (operating-system-derivation os))) =2D (machine-remote-eval machine (remote-exp drv script))))) =2D =2D;; XXX: Currently, this does NOT attempt to restart running services. Th= is is =2D;; also the case with 'guix system reconfigure'. =2D;; =2D;; See . =2D(define (upgrade-shepherd-services machine) =2D "Monadic procedure unloading and starting services on the remote as ne= eded =2Dto realize the MACHINE's system configuration." =2D (define target-services =2D ;; Monadic expression evaluating to a list of (name output-path) pai= rs for =2D ;; all of MACHINE's services. =2D (mapm %store-monad =2D (lambda (service) =2D (mlet %store-monad ((file ((compose lower-object =2D shepherd-service-file) =2D service))) =2D (return (list (shepherd-service-canonical-name service) =2D (derivation->output-path file))))) =2D (service-value =2D (fold-services (operating-system-services (machine-system mac= hine)) =2D #:target-type shepherd-root-service-type)))) =2D =2D (define (remote-exp target-services) =2D (with-imported-modules '((gnu services herd)) =2D #~(begin =2D (use-modules (gnu services herd) =2D (srfi srfi-1)) =2D =2D (define running =2D (filter live-service-running (current-services))) =2D =2D (define (essential? service) =2D ;; Return #t if SERVICE is essential and should not be unloa= ded =2D ;; under any circumstance. =2D (memq (first (live-service-provision service)) =2D '(root shepherd))) =2D =2D (define (obsolete? service) =2D ;; Return #t if SERVICE can be safely unloaded. =2D (and (not (essential? service)) =2D (every (lambda (requirements) =2D (not (memq (first (live-service-provision serv= ice)) =2D requirements))) =2D (map live-service-requirement running)))) =2D =2D (define to-unload =2D (filter obsolete? =2D (remove (lambda (service) =2D (memq (first (live-service-provision servi= ce)) =2D (map first '#$target-services))) =2D running))) =2D =2D (define to-start =2D (remove (lambda (service-pair) =2D (memq (first service-pair) =2D (map (compose first live-service-provision) =2D running))) =2D '#$target-services)) =2D =2D ;; Unload obsolete services. =2D (for-each (lambda (service) =2D (false-if-exception =2D (unload-service service))) =2D to-unload) =2D =2D ;; Load the service files for any new services and start them. =2D (load-services/safe (map second to-start)) =2D (for-each start-service (map first to-start)) =2D =2D #t))) =2D =2D (mlet %store-monad ((target-services target-services)) =2D (machine-remote-eval machine (remote-exp target-services)))) =2D (define (machine-boot-parameters machine) "Monadic procedure returning a list of 'boot-parameters' for the generat= ions of MACHINE's system profile, ordered from most recent to oldest." @@ -275,71 +166,67 @@ of MACHINE's system profile, ordered from most recent= to oldest." (boot-parameters-kernel-arguments params)))))))) generations)))) =20 =2D(define (install-bootloader machine) =2D "Create a bootloader entry for the new system generation on MACHINE, a= nd =2Dconfigure the bootloader to boot that generation by default." =2D (define bootloader-installer-script =2D (@@ (guix scripts system) bootloader-installer-script)) =2D =2D (define (remote-exp installer bootcfg bootcfg-file) =2D (with-extensions (list guile-gcrypt) =2D (with-imported-modules (source-module-closure '((gnu build install) =2D (guix store) =2D (guix utils))) =2D #~(begin =2D (use-modules (gnu build install) =2D (guix store) =2D (guix utils)) =2D (let* ((gc-root (string-append "/" %gc-roots-directory "/boo= tcfg")) =2D (temp-gc-root (string-append gc-root ".new"))) =2D =2D (switch-symlinks temp-gc-root gc-root) =2D =2D (unless (false-if-exception =2D (begin =2D ;; The implementation of 'guix system reconfigu= re' =2D ;; saves the load path here. This is unnecessar= y here =2D ;; because each invocation of 'remote-eval' run= s in a =2D ;; distinct Guile REPL. =2D (install-boot-config #$bootcfg #$bootcfg-file "= /") =2D ;; The installation script may write to stdout,= which =2D ;; confuses 'remote-eval' when it attempts to r= ead a =2D ;; result from the remote REPL. We work around = this =2D ;; by forcing the output to a string. =2D (with-output-to-string =2D (lambda () =2D (primitive-load #$installer))))) =2D (delete-file temp-gc-root) =2D (error "failed to install bootloader")) =2D =2D (rename-file temp-gc-root gc-root) =2D #t))))) =2D =2D (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine= ))) =2D (let* ((os (machine-system machine)) =2D (bootloader ((compose bootloader-configuration-bootloader =2D operating-system-bootloader) =2D os)) =2D (bootloader-target (bootloader-configuration-target =2D (operating-system-bootloader os))) =2D (installer (bootloader-installer-script =2D (bootloader-installer bootloader) =2D (bootloader-package bootloader) =2D bootloader-target =2D "/")) =2D (menu-entries (map boot-parameters->menu-entry boot-parameter= s)) =2D (bootcfg (operating-system-bootcfg os menu-entries)) =2D (bootcfg-file (bootloader-configuration-file bootloader))) =2D (machine-remote-eval machine (remote-exp installer bootcfg bootcfg= -file))))) =2D (define (deploy-managed-host machine) "Internal implementation of 'deploy-machine' for MACHINE instances with = an environment type of 'managed-host." =2D (maybe-raise-unsupported-configuration-error machine) =2D (mbegin %store-monad =2D (switch-to-system machine) =2D (upgrade-shepherd-services machine) =2D (install-bootloader machine))) + (define target-services + ;; Monadic expression evaluating to a list of + ;; (shepherd-service-canonical-name, shepherd-service-file) pairs for = the + ;; services in MACHINE's operating system configuration. + (mapm %store-monad + (lambda (service) + (mlet %store-monad ((file ((compose lower-object + shepherd-service-file) + service))) + (return (list (shepherd-service-canonical-name service) + (derivation->output-path file))))) + (service-value + (fold-services (operating-system-services (machine-system machi= ne)) + #:target-type shepherd-root-service-type)))) + + (define (run-switch-to-system machine) + "Monadic procedure serializing the items in MACHINE necessary to build= a +G-Expression with 'switch-to-system'." + (let* ((os (machine-system machine)) + (activation-script (operating-system-activation-script os))) + (mlet %store-monad ((osdrv (operating-system-derivation os))) + (machine-remote-eval machine + (switch-to-system osdrv activation-script))))) + + (define (run-upgrade-shepherd-services machine) + "Monadic procedure serializing the items in MACHINE necessary to build= a +G-Expression with 'upgrade-shepherd-services'." + (mlet %store-monad ((target-services target-services)) + (machine-remote-eval machine + (upgrade-shepherd-services target-services)))) + + (define (run-install-bootloader machine) + "Monadic procedure serializing the items in MACHINE necessary to build= a +G-Expression with 'install-bootloader'." + (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine= ))) + (let* ((os (machine-system machine)) + (bootloader ((compose bootloader-configuration-bootloader + operating-system-bootloader) + os)) + (target (bootloader-configuration-target + (operating-system-bootloader os))) + (installer (bootloader-installer-script + (bootloader-installer bootloader) + (bootloader-package bootloader) + target + "/")) + (menu-entries (map boot-parameters->menu-entry boot-parameter= s)) + (bootcfg (operating-system-bootcfg os menu-entries)) + (bootcfg-file (bootloader-configuration-file bootloader))) + (machine-remote-eval machine + (install-bootloader installer bootcfg + bootcfg-file "/"))))) + + (maybe-raise-missing-configuration-error machine) + (mapm %store-monad (cut <> machine) + (list run-switch-to-system + run-upgrade-shepherd-services + run-install-bootloader))) =20 ;;; =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0f4aYACgkQ9Qb9Fp2P 2VrYWA//afYfK0/aUJtB4LRbWzlK9a2I8/h0Lr77OumWev39qWNf3dgrI53DfoJ8 k1rnOshpaZPzKXhGg5XYQRxUhQiUOrB3WVwsdiuZlKxIqyldOd2+w13ndAhaBVky 2WopOjJ9Poh54G1ccmWfwucKx2oMwKq5JdfkXcOcswhKteo6yZRY7TZV7zNpf3n7 uulHzq0fdZs1BRJIwo/VdtCe4N+ngzLmhLyTAjX2ef7BcpcKTXraI2IcHtLpcf2p vqODAenlJg8GkODQ0gphSdjmkP6Hf4u8+fDY92dI+9XUAQGLVnjb74vOwI0X/YH7 7gt8Csldn+gAYoCDpJqVa8Ug6Vb0O6GFp7aVpgKoUYtvjH92D296tjsvPScPeOv2 gDxDqXp+WnRb5TkVZqn6rKJyLQEHQ6vDzr0sfhNAUbEaJghasNgjyIxljlYCGHoS b0YG5shD0f+sll9tR+h5Nvok2FbTrJgwm3yxzANT3PlQDrZBm8on9Occl2DGmFoF 1TlGcuh5VLprCDXWXEV0K8eBFyG+aifsOdMNVubRa+GjFMUOnVZ7JCY2WZe2YH3H d1vFBlQtRgCujoUM5lChFFSsxixcsz9mjDJzuwj0IoZ8nuY8vuO7URd1K0LHMQFB V8yEJZkDdhZLtL/SCBF04kA/ZKnOVWnvD9rR+8zNEZclWwuHAOs= =/LPw -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 05 19:48:47 2019 Received: (at 36404) by debbugs.gnu.org; 5 Jul 2019 23:48:47 +0000 Received: from localhost ([127.0.0.1]:53993 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXwc-0007fD-UO for submit@debbugs.gnu.org; Fri, 05 Jul 2019 19:48:47 -0400 Received: from mx.sdf.org ([205.166.94.20]:50745) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjXwa-0007f5-Mz for 36404@debbugs.gnu.org; Fri, 05 Jul 2019 19:48:45 -0400 Received: from Upsilon (mobile-166-171-185-104.mycingular.net [166.171.185.104]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x65NmcvP003834 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Fri, 5 Jul 2019 23:48:39 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 3/3] guix system: Reimplement 'reconfigure'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> <877e8wf3iz.fsf_-_@sdf.lonestar.org> <8736jkf3h5.fsf_-_@sdf.lonestar.org> Date: Fri, 05 Jul 2019 19:48:36 -0400 In-Reply-To: <8736jkf3h5.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 19:47:50 -0400") Message-ID: <87y31cdovf.fsf_-_@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * guix/scripts/system.scm (switch-to-system) (upgrade-shepherd-services, install-bootloader): Delete variable. * guix/scripts/system.scm (%switch-to-system) (%upgrade-shepherd-services, %install-bootloader): New variable. =2D-- guix/scripts/system.scm | 161 +++++++++++++--------------------------- 1 file changed, 50 insertions(+), 111 deletions(-) diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index 21858ee7d..1f7912dcf 100644 =2D-- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -41,6 +41,7 @@ delete-matching-generations) #:use-module (guix graph) #:use-module (guix scripts graph) + #:use-module (guix scripts system reconfigure) #:use-module (guix build utils) #:use-module (guix progress) #:use-module ((guix build syscalls) #:select (terminal-columns)) @@ -179,38 +180,16 @@ TARGET, and register them." =20 (return *unspecified*))) =20 =2D(define* (install-bootloader installer =2D #:key =2D bootcfg bootcfg-file =2D target) +(define (%install-bootloader installer bootcfg bootcfg-file target) "Run INSTALLER, a bootloader installation script, with error handling, in %STORE-MONAD." =2D (mlet %store-monad ((installer-drv (if installer =2D (lower-object installer) =2D (return #f))) =2D (bootcfg (lower-object bootcfg))) =2D (let* ((gc-root (string-append target %gc-roots-directory =2D "/bootcfg")) =2D (temp-gc-root (string-append gc-root ".new")) =2D (install (and installer-drv =2D (derivation->output-path installer-drv))) =2D (bootcfg (derivation->output-path bootcfg))) =2D ;; Prepare the symlink to bootloader config file to make sure that= it's =2D ;; a GC root when 'installer-drv' completes (being a bit paranoid.) =2D (switch-symlinks temp-gc-root bootcfg) =2D =2D (unless (false-if-exception =2D (begin =2D (install-boot-config bootcfg bootcfg-file target) =2D (when install =2D (save-load-path-excursion (primitive-load install))))) =2D (delete-file temp-gc-root) =2D (leave (G_ "failed to install bootloader ~a~%") install)) =2D =2D ;; Register bootloader config file as a GC root so that its depend= encies =2D ;; (background image, font, etc.) are not reclaimed. =2D (rename-file temp-gc-root gc-root) =2D (return #t)))) + (mlet* %store-monad ((file (lower-object + (scheme-file "install-bootloader.scm" + (install-bootloader installer b= ootcfg + bootcfg-file + target)))) + (_ (built-derivations (list file)))) + (primitive-load (derivation->output-path file)))) =20 (define* (install os-drv target #:key (log-port (current-output-port)) @@ -266,10 +245,8 @@ the ownership of '~a' may be incorrect!~%") (populate os-dir target) =20 (mwhen install-bootloader? =2D (install-bootloader bootloader-installer =2D #:bootcfg bootcfg =2D #:bootcfg-file bootcfg-file =2D #:target target)))))) + (%install-bootloader bootloader-installer bootcfg + bootcfg-file target)))))) =20 ;;; @@ -336,81 +313,47 @@ unload." (warning (G_ "failed to obtain list of shepherd services~%")) (return #f))))) =20 =2D(define (upgrade-shepherd-services os) +(define (%upgrade-shepherd-services os) "Upgrade the Shepherd (PID 1) by unloading obsolete services and loading= new services specified in OS and not currently running. =20 This is currently very conservative in that it does not stop or unload any running service. Unloading or stopping the wrong service ('udev', say) co= uld bring the system down." =2D (define new-services =2D (service-value =2D (fold-services (operating-system-services os) =2D #:target-type shepherd-root-service-type))) =2D =2D ;; Arrange to simply emit a warning if the service upgrade fails. =2D (with-shepherd-error-handling =2D (call-with-service-upgrade-info new-services =2D (lambda (to-restart to-unload) =2D (for-each (lambda (unload) =2D (info (G_ "unloading service '~a'...~%") unload) =2D (unload-service unload)) =2D to-unload) =2D =2D (with-monad %store-monad =2D (munless (null? new-services) =2D (let ((new-service-names (map shepherd-service-canonical-na= me new-services)) =2D (to-restart-names (map shepherd-service-canonical-na= me to-restart)) =2D (to-start (filter shepherd-service-auto-star= t? new-services))) =2D (info (G_ "loading new services:~{ ~a~}...~%") new-service= -names) =2D (unless (null? to-restart-names) =2D ;; Listing TO-RESTART-NAMES in the message below wouldn'= t help =2D ;; because many essential services cannot be meaningfully =2D ;; restarted. See . =2D (format #t (G_ "To complete the upgrade, run 'herd resta= rt SERVICE' to stop, =2Dupgrade, and restart each service that was not automatically restarted.\= n"))) =2D (mlet %store-monad ((files (mapm %store-monad =2D (compose lower-object =2D shepherd-service= -file) =2D new-services))) =2D ;; Here we assume that FILES are exactly those that were= computed =2D ;; as part of the derivation that built OS, which is nor= mally the =2D ;; case. =2D (load-services/safe (map derivation->output-path files)) =2D =2D (for-each start-service =2D (map shepherd-service-canonical-name to-start)) =2D (return #t))))))))) =2D =2D(define* (switch-to-system os =2D #:optional (profile %system-profile)) =2D "Make a new generation of PROFILE pointing to the directory of OS, swi= tch to =2Dit atomically, and then run OS's activation script." + (define target-services + ;; Monadic expression evaluating to a list of + ;; (shepherd-service-canonical-name, shepherd-service-file) pairs for = the + ;; services in MACHINE's operating system configuration. + (mapm %store-monad + (lambda (service) + (mlet %store-monad ((file ((compose lower-object + shepherd-service-file) + service))) + (return (list (shepherd-service-canonical-name service) + (derivation->output-path file))))) + (service-value + (fold-services (operating-system-services os) + #:target-type shepherd-root-service-type)))) + + (mlet* %store-monad ((target-services target-services) + (file (lower-object + (scheme-file "upgrade-shepherd-services.scm" + (upgrade-shepherd-services + target-services)))) + (_ (built-derivations (list file)))) + (primitive-load (derivation->output-path file)))) + +(define (%switch-to-system os) + "Make a new generation of PROFILE pointing to the directory of OS, switch +to it atomically, and then run OS's activation script." (mlet* %store-monad ((drv (operating-system-derivation os)) =2D (script (lower-object (operating-system-activatio= n-script os)))) =2D (let* ((system (derivation->output-path drv)) =2D (number (+ 1 (generation-number profile))) =2D (generation (generation-file-name profile number))) =2D (switch-symlinks generation system) =2D (switch-symlinks profile generation) =2D =2D (format #t (G_ "activating system...~%")) =2D =2D ;; The activation script may change $PATH, among others, so protect =2D ;; against that. =2D (save-environment-excursion =2D ;; Tell 'activate-current-system' what the new system is. =2D (setenv "GUIX_NEW_SYSTEM" system) =2D =2D ;; The activation script may modify '%load-path' & co., so protect =2D ;; against that. This is necessary to ensure that =2D ;; 'upgrade-shepherd-services' gets to see the right modules when= it =2D ;; computes derivations with 'gexp->derivation'. =2D (save-load-path-excursion =2D (primitive-load (derivation->output-path script)))) =2D =2D ;; Finally, try to update system services. =2D (upgrade-shepherd-services os)))) + (script (lower-object + (operating-system-activation-script os))) + (file (lower-object + (scheme-file "switch-to-system.scm" + (switch-to-system drv script)))) + (_ (built-derivations (list file)))) + (primitive-load (derivation->output-path file)))) =20 (define-syntax-rule (unless-file-not-found exp) (catch 'system-error @@ -514,10 +457,7 @@ STORE is an open connection to the store." (built-derivations drvs) ;; Only install bootloader configuration file. Thus, no installe= r is ;; provided here. =2D (install-bootloader #f =2D #:bootcfg bootcfg =2D #:bootcfg-file bootcfg-file =2D #:target target)))))) + (%install-bootloader #f bootcfg bootcfg-file target)))))) =20 ;;; @@ -919,12 +859,11 @@ static checks." (case action ((reconfigure) (mbegin %store-monad =2D (switch-to-system os) + (%switch-to-system os) + (%upgrade-shepherd-services os) (mwhen install-bootloader? =2D (install-bootloader bootloader-script =2D #:bootcfg bootcfg =2D #:bootcfg-file bootcfg-file =2D #:target "/")))) + (%install-bootloader bootloader-script bootcfg + bootcfg-file (or target "/"))))) ((init) (newline) (format #t (G_ "initializing operating system under '~a'...~%= ") =2D-=20 2.22.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0f4dQACgkQ9Qb9Fp2P 2VrsKw//dn5TqcKmbV6YwOKjBdXpDfHNigOgRVMaSVacJW+JPvTj/77hacFzzrUs pPL0tR+JgxkvZhlAuqeukEqHs45FcyJ9hzmpo24WFuFw8gu/CFQEuFY9g7XDxATi WFK5Ise+RpcVegsFVr947lpOSGXlGDANFII55EV8MnRB4pXBA193Vmn7ItBUSSwg HJk/k4m2LllEIPScP6da0EN9th8Lx941FPMbqbEhyNF+hnOQNXutVs83vPlx+YOc OlGUKrZ6XE8TnGr3JjFmb0WatM6JfdUaeXT9hrEEZgTDjwi9vc9wX/hP3R8vQ7pp ZlcUBwg5xxeGulc4hhg0H2hLZbLT6pyf4zBFtI0KIDMhsCX1yuBzN//Y7HKg2RY6 RnVOag4hh7tF+aefvYSoua/PZTES83xSvp1yDzM1stZ5EVp/f9LGTvfbqiHaV/KN N5evpVHYKm9FbDlgse0QGBZuzmoBQdABbrAK9NXcJCPwxaULXPvPrDUpddjgN1Fa wrsvXv0PzNzL1djwmBs0iw1x+Ef4OOQJOWBTwn6vRBZCdk6QCqtuOXVRvV8YYyYU WVG5fx+czo0N4alFoxR1NQwyz5HwedewSeYQdTrOqlm467h2Nh4KvBu5367+xYNo 9skmJQJcT+3gwJeKkOE3WbxXKBYSlsueg5A2JgPsksAhBImqQU8= =8uPB -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 06 17:50:36 2019 Received: (at 36404-done) by debbugs.gnu.org; 6 Jul 2019 21:50:36 +0000 Received: from localhost ([127.0.0.1]:55946 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjsZn-0005b1-TC for submit@debbugs.gnu.org; Sat, 06 Jul 2019 17:50:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35557) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjsZl-0005am-R0 for 36404-done@debbugs.gnu.org; Sat, 06 Jul 2019 17:50:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59149) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjsZe-00078G-5P; Sat, 06 Jul 2019 17:50:26 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=55614 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjsZc-0007UG-TO; Sat, 06 Jul 2019 17:50:26 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH v5 4/4] doc: Add section for 'guix deploy'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87imsl9tsx.fsf_-_@sdf.lonestar.org> <87ef399tpu.fsf_-_@sdf.lonestar.org> <87a7dx9tog.fsf_-_@sdf.lonestar.org> <875zol9tn2.fsf_-_@sdf.lonestar.org> <871rz99tl9.fsf_-_@sdf.lonestar.org> <875zoldqah.fsf@kyleam.com> <87muhwtmfp.fsf@sdf.lonestar.org> <871rz874l2.fsf@kyleam.com> <877e90tj7l.fsf_-_@sdf.lonestar.org> <8736jotj5v.fsf_-_@sdf.lonestar.org> <87y31gs4k5.fsf_-_@sdf.lonestar.org> <87y31cnb2t.fsf@gnu.org> <87ftnkgvo8.fsf_-_@sdf.lonestar.org> <878stcgvmf.fsf_-_@sdf.lonestar.org> <874l40gvl2.fsf_-_@sdf.lonestar.org> <87zhlsfgzc.fsf_-_@sdf.lonestar.org> <87v9wgfgws.fsf_-_@sdf.lonestar.org> <87bly7n0ze.fsf@dustycloud.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 18 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 06 Jul 2019 23:50:21 +0200 In-Reply-To: <87bly7n0ze.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sat, 06 Jul 2019 02:14:29 -0400") Message-ID: <875zoehlya.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404-done Cc: "Jakob L. Kreuze" , "Thompson, David" , 36404-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Christopher Lemmer Webber skribis: > Since those changes seemed to reflect everyone's requests, I've pushed > it to git master. Thank you. > Huge congrats to Jakob! I'm stoked about it. Seconded! > Now who can race to the finish line to be the first one using these > tools for their server deployment? :) Heheh, we could put them to good use on the build farm=E2=80=A6 Cheers, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 06 18:02:17 2019 Received: (at 36404) by debbugs.gnu.org; 6 Jul 2019 22:02:17 +0000 Received: from localhost ([127.0.0.1]:55950 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjsl6-0005uJ-0N for submit@debbugs.gnu.org; Sat, 06 Jul 2019 18:02:16 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37935) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjsl2-0005u4-Po for 36404@debbugs.gnu.org; Sat, 06 Jul 2019 18:02:14 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59224) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjskx-0006XQ-5p; Sat, 06 Jul 2019 18:02:07 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=55632 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjsks-0004OE-Pe; Sat, 06 Jul 2019 18:02:04 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH 0/3] Refactor out common behavior for system reconfiguration. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 19 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 07 Jul 2019 00:02:01 +0200 In-Reply-To: <87bly8f3kq.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 19:45:41 -0400") Message-ID: <87wogug6ue.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > Ludovic Court=C3=A8s writes: > >> Note that writing tests could be tricky because it=E2=80=99s about testi= ng the >> effect of these reconfigure actions. At any rate, let us know how it >> goes! > > This is a _very_ preliminary patch series. I'm not nearly done with it > yet; the procedures in guix/scripts/system.scm that I've replaced have > some handling for i.e. installing the bootloader configuration without > running the installer script, which my reimplementations don't yet > support. > > I'm sending this tonight to make sure I'm on the right track: is this > sort of what you meant by extracting the common behavior into scripts? Yes! > Also, I didn't include any tests as part of this series, but > implementing reconfiguration like this does, indeed, make testing for > 'guix deploy' much, much easier. And we'll get some tests for the > behavior of 'guix system reconfigure' out of it, too! As you can imagine we=E2=80=99ll have to be careful with =E2=80=98guix syst= em reconfigure=E2=80=99=E2=80=94let=E2=80=99s not break everyone=E2=80=99s sys= tem. ;-) But yes, it seems like the right thing to me. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 06 18:12:01 2019 Received: (at 36404) by debbugs.gnu.org; 6 Jul 2019 22:12:01 +0000 Received: from localhost ([127.0.0.1]:55955 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjsuU-00069T-N2 for submit@debbugs.gnu.org; Sat, 06 Jul 2019 18:11:58 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39984) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjsuS-00069F-QT for 36404@debbugs.gnu.org; Sat, 06 Jul 2019 18:11:57 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59271) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjsuN-0006CW-Je; Sat, 06 Jul 2019 18:11:51 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=55638 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjsuM-0005vy-MB; Sat, 06 Jul 2019 18:11:51 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH 1/3] guix system: Add 'reconfigure' module. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> <877e8wf3iz.fsf_-_@sdf.lonestar.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 19 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 07 Jul 2019 00:11:49 +0200 In-Reply-To: <877e8wf3iz.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 19:46:44 -0400") Message-ID: <877e8ug6e2.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > * guix/scripts/system/reconfigure.scm: New file. > * Makefile.am (MODULES): Add it. > * guix/scripts/system.scm (bootloader-installer-script): Export variable. > +;;; Copyright =C2=A9 2019 Jakob L. Kreuze Could you preserve the copyright lines of (guix scripts system) that apply to these portions of code, roughly? I think all the procedures in (guix scripts system reconfigure) could return a rather than a gexp. Actually a would even be cleaner than a , as it could better handle transitions like you=E2=80=99re on a Guile 2.2 system reconfiguring towards= a Guile 3 system. Consequently you could rename =E2=80=98switch-to-system=E2=80=99 to =E2=80=98switch-system-program=E2=80=99, and so on. > +(define (switch-to-system system-derivation activation-script) I think it could simply take an record and derive the relevant bits from that. > + (switch-symlinks generation system) > + (switch-symlinks %system-profile generation) > + ;; The implementation of 'guix system reconfigure' saves the > + ;; load path and environment here. This is unnecessary here > + ;; because each invocation of 'remote-eval' runs in a distin= ct > + ;; Guile REPL. > + (setenv "GUIX_NEW_SYSTEM" system) This comment may become irrelevant. > + ;; The activation script may write to stdout, which confuses > + ;; 'remote-eval' when it attempts to read a result from the > + ;; remote REPL. We work around this by forcing the output to= a > + ;; string. > + (with-output-to-string > + (lambda () > + (primitive-load #$activation-script)))))))) Same here? For =E2=80=98guix system reconfigure=E2=80=99, we=E2=80=99d rat= her not lose messages written to stdout by ACTIVATION-SCRIPT. > + (unless (false-if-exception > + (begin > + ;; The implementation of 'guix system reconfigure' > + ;; saves the load path here. This is unnecessary = here > + ;; because each invocation of 'remote-eval' runs = in a > + ;; distinct Guile REPL. > + (install-boot-config #$bootcfg #$bootcfg-file #$t= arget) > + ;; The installation script may write to stdout, w= hich > + ;; confuses 'remote-eval' when it attempts to rea= d a > + ;; result from the remote REPL. We work around th= is > + ;; by forcing the output to a string. > + (with-output-to-string > + (lambda () > + (primitive-load #$installer-script))))) Same as above. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 06 18:14:02 2019 Received: (at 36404) by debbugs.gnu.org; 6 Jul 2019 22:14:02 +0000 Received: from localhost ([127.0.0.1]:55959 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjswT-0006D4-Md for submit@debbugs.gnu.org; Sat, 06 Jul 2019 18:14:02 -0400 Received: from eggs.gnu.org ([209.51.188.92]:40393) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjswR-0006Ce-H7 for 36404@debbugs.gnu.org; Sat, 06 Jul 2019 18:14:00 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59312) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjswG-0007Li-Us; Sat, 06 Jul 2019 18:13:50 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=55642 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjswE-000632-Ks; Sat, 06 Jul 2019 18:13:48 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH 2/3] machine: Reimplement 'managed-host-environment-type' deployment. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> <877e8wf3iz.fsf_-_@sdf.lonestar.org> <8736jkf3h5.fsf_-_@sdf.lonestar.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 19 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 07 Jul 2019 00:13:45 +0200 In-Reply-To: <8736jkf3h5.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 19:47:50 -0400") Message-ID: <87y31aerqe.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > + (define (run-switch-to-system machine) > + "Monadic procedure serializing the items in MACHINE necessary to bui= ld a > +G-Expression with 'switch-to-system'." > + (let* ((os (machine-system machine)) > + (activation-script (operating-system-activation-script os))) > + (mlet %store-monad ((osdrv (operating-system-derivation os))) > + (machine-remote-eval machine > + (switch-to-system osdrv activation-script))= ))) Normally you should never need to call =E2=80=98operating-system-derivation= =E2=80=99 because you can just insert an in a gexp and it=E2=80=99= ll do the right thing: #~(frob #$os) Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 06 18:20:16 2019 Received: (at 36404) by debbugs.gnu.org; 6 Jul 2019 22:20:16 +0000 Received: from localhost ([127.0.0.1]:55963 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjt2W-0006Nq-Hk for submit@debbugs.gnu.org; Sat, 06 Jul 2019 18:20:16 -0400 Received: from eggs.gnu.org ([209.51.188.92]:41695) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjt2V-0006NW-Fr for 36404@debbugs.gnu.org; Sat, 06 Jul 2019 18:20:15 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59454) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hjt2J-0002iF-Cb; Sat, 06 Jul 2019 18:20:05 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=55646 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hjt2I-0006P7-Ed; Sat, 06 Jul 2019 18:20:02 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Re: [bug#36404] [PATCH 3/3] guix system: Reimplement 'reconfigure'. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> <877e8wf3iz.fsf_-_@sdf.lonestar.org> <8736jkf3h5.fsf_-_@sdf.lonestar.org> <87y31cdovf.fsf_-_@sdf.lonestar.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 19 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 07 Jul 2019 00:20:00 +0200 In-Reply-To: <87y31cdovf.fsf_-_@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 05 Jul 2019 19:48:36 -0400") Message-ID: <87imseerfz.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > +(define (%upgrade-shepherd-services os) > "Upgrade the Shepherd (PID 1) by unloading obsolete services and loadi= ng new > services specified in OS and not currently running. >=20=20 > This is currently very conservative in that it does not stop or unload a= ny > running service. Unloading or stopping the wrong service ('udev', say) = could > bring the system down." > - (define new-services > - (service-value > - (fold-services (operating-system-services os) > - #:target-type shepherd-root-service-type))) > - > - ;; Arrange to simply emit a warning if the service upgrade fails. > - (with-shepherd-error-handling > - (call-with-service-upgrade-info new-services > - (lambda (to-restart to-unload) I think you=E2=80=99d need to include the =E2=80=98call-with-service-upgrad= e-info=E2=80=99 call in the service-upgrade program that (guix scripts system reconfigure) produces. It=E2=80=99s an important part of reconfiguration. However, =E2=80=98call-with-service-upgrade-info=E2=80=99 relies on (guix g= raph), which pulls in (guix monads) and many modules that we don=E2=80=99t actually need. It=E2=80=99s probably just an annoyance more than a real problem, but I thi= nk we should eventually change the (guix graph) API so that it no longer relies on the =E2=80=98%store-monad=E2=80=99, which in turn will make it a = better fit in this context. Thanks for quickly hacking on this! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 07 08:02:22 2019 Received: (at 36404) by debbugs.gnu.org; 7 Jul 2019 12:02:22 +0000 Received: from localhost ([127.0.0.1]:56240 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hk5s6-0002Oj-EQ for submit@debbugs.gnu.org; Sun, 07 Jul 2019 08:02:22 -0400 Received: from dustycloud.org ([50.116.34.160]:55930) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hk5s4-0002Ob-32 for 36404@debbugs.gnu.org; Sun, 07 Jul 2019 08:02:20 -0400 Received: from twig (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 05E0F2670D; Sun, 7 Jul 2019 08:02:17 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: "Jakob L. Kreuze" Subject: Re: [bug#36404] [PATCH 0/3] Refactor out common behavior for system reconfiguration. In-reply-to: <87bly8f3kq.fsf_-_@sdf.lonestar.org> Date: Sun, 07 Jul 2019 03:02:17 -0400 Message-ID: <87wogucop2.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.1 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Side note: I closed this issue when the initial set of patches were merged. It seems there's ongoing work... should we reopen it or make a separate issue? I'm unsure. Content analysis details: (1.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record 1.1 DATE_IN_PAST_03_06 Date: is 3 to 6 hours before Received: date -0.0 SPF_HELO_PASS SPF: HELO matches SPF record X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Ludovic =?utf-8?Q?Court=C3=A8s?= , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.1 (/) Side note: I closed this issue when the initial set of patches were merged. It seems there's ongoing work... should we reopen it or make a separate issue? I'm unsure. From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 07 08:13:54 2019 Received: (at 36404) by debbugs.gnu.org; 7 Jul 2019 12:13:54 +0000 Received: from localhost ([127.0.0.1]:56250 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hk63D-0002fH-H8 for submit@debbugs.gnu.org; Sun, 07 Jul 2019 08:13:53 -0400 Received: from dustycloud.org ([50.116.34.160]:55952) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hk63A-0002f8-Vk for 36404@debbugs.gnu.org; Sun, 07 Jul 2019 08:13:49 -0400 Received: from twig (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id C5AC826674; Sun, 7 Jul 2019 08:13:46 -0400 (EDT) References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> <877e8wf3iz.fsf_-_@sdf.lonestar.org> <8736jkf3h5.fsf_-_@sdf.lonestar.org> User-agent: mu4e 1.2.0; emacs 26.2 From: Christopher Lemmer Webber To: "Jakob L. Kreuze" Subject: Re: [bug#36404] [PATCH 2/3] machine: Reimplement 'managed-host-environment-type' deployment. In-reply-to: <8736jkf3h5.fsf_-_@sdf.lonestar.org> Date: Sun, 07 Jul 2019 03:13:46 -0400 Message-ID: <87v9weco5x.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.1 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: In some ways it looks like a portion of the previous patch and a portion of this patch are a "move and modify" of what are sort-of the same chunks of code. But it's a bit weird to me that the code is [...] Content analysis details: (1.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: guix.info] -0.0 SPF_PASS SPF: sender matches SPF record 1.1 DATE_IN_PAST_03_06 Date: is 3 to 6 hours before Received: date -0.0 SPF_HELO_PASS SPF: HELO matches SPF record X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , Ludovic =?utf-8?Q?Court=C3=A8s?= , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.1 (/) In some ways it looks like a portion of the previous patch and a portion of this patch are a "move and modify" of what are sort-of the same chunks of code. But it's a bit weird to me that the code is added in the previous commit and removed in this one? It might be clearer to the reader that this is what is happening if it's in the same commit. Jakob L. Kreuze writes: > * gnu/machine/ssh.scm (switch-to-system, upgrade-shepherd-services) > (install-bootloader): Delete variable. > * gnu/machine/ssh.scm (deploy-managed-host): Rewrite procedure. > --- > gnu/machine/ssh.scm | 235 ++++++++++++-------------------------------- > 1 file changed, 61 insertions(+), 174 deletions(-) > > diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm > index a7d1a967a..72e6407f0 100644 > --- a/gnu/machine/ssh.scm > +++ b/gnu/machine/ssh.scm > @@ -30,10 +30,13 @@ > #:use-module (guix monads) > #:use-module (guix records) > #:use-module (guix remote) > + #:use-module (guix scripts system) > + #:use-module (guix scripts system reconfigure) > #:use-module (guix ssh) > #:use-module (guix store) > #:use-module (ice-9 match) > #:use-module (srfi srfi-19) > + #:use-module (srfi srfi-26) > #:use-module (srfi srfi-35) > #:export (managed-host-environment-type > > @@ -105,118 +108,6 @@ an environment type of 'managed-host." > ;;; System deployment. > ;;; > > -(define (switch-to-system machine) > - "Monadic procedure creating a new generation on MACHINE and execute the > -activation script for the new system configuration." > - (define (remote-exp drv script) > - (with-extensions (list guile-gcrypt) > - (with-imported-modules (source-module-closure '((guix config) > - (guix profiles) > - (guix utils))) > - #~(begin > - (use-modules (guix config) > - (guix profiles) > - (guix utils)) > - > - (define %system-profile > - (string-append %state-directory "/profiles/system")) > - > - (let* ((system #$drv) > - (number (1+ (generation-number %system-profile))) > - (generation (generation-file-name %system-profile number))) > - (switch-symlinks generation system) > - (switch-symlinks %system-profile generation) > - ;; The implementation of 'guix system reconfigure' saves the > - ;; load path and environment here. This is unnecessary here > - ;; because each invocation of 'remote-eval' runs in a distinct > - ;; Guile REPL. > - (setenv "GUIX_NEW_SYSTEM" system) > - ;; The activation script may write to stdout, which confuses > - ;; 'remote-eval' when it attempts to read a result from the > - ;; remote REPL. We work around this by forcing the output to a > - ;; string. > - (with-output-to-string > - (lambda () > - (primitive-load #$script)))))))) > - > - (let* ((os (machine-system machine)) > - (script (operating-system-activation-script os))) > - (mlet* %store-monad ((drv (operating-system-derivation os))) > - (machine-remote-eval machine (remote-exp drv script))))) > - > -;; XXX: Currently, this does NOT attempt to restart running services. This is > -;; also the case with 'guix system reconfigure'. > -;; > -;; See . > -(define (upgrade-shepherd-services machine) > - "Monadic procedure unloading and starting services on the remote as needed > -to realize the MACHINE's system configuration." > - (define target-services > - ;; Monadic expression evaluating to a list of (name output-path) pairs for > - ;; all of MACHINE's services. > - (mapm %store-monad > - (lambda (service) > - (mlet %store-monad ((file ((compose lower-object > - shepherd-service-file) > - service))) > - (return (list (shepherd-service-canonical-name service) > - (derivation->output-path file))))) > - (service-value > - (fold-services (operating-system-services (machine-system machine)) > - #:target-type shepherd-root-service-type)))) > - > - (define (remote-exp target-services) > - (with-imported-modules '((gnu services herd)) > - #~(begin > - (use-modules (gnu services herd) > - (srfi srfi-1)) > - > - (define running > - (filter live-service-running (current-services))) > - > - (define (essential? service) > - ;; Return #t if SERVICE is essential and should not be unloaded > - ;; under any circumstance. > - (memq (first (live-service-provision service)) > - '(root shepherd))) > - > - (define (obsolete? service) > - ;; Return #t if SERVICE can be safely unloaded. > - (and (not (essential? service)) > - (every (lambda (requirements) > - (not (memq (first (live-service-provision service)) > - requirements))) > - (map live-service-requirement running)))) > - > - (define to-unload > - (filter obsolete? > - (remove (lambda (service) > - (memq (first (live-service-provision service)) > - (map first '#$target-services))) > - running))) > - > - (define to-start > - (remove (lambda (service-pair) > - (memq (first service-pair) > - (map (compose first live-service-provision) > - running))) > - '#$target-services)) > - > - ;; Unload obsolete services. > - (for-each (lambda (service) > - (false-if-exception > - (unload-service service))) > - to-unload) > - > - ;; Load the service files for any new services and start them. > - (load-services/safe (map second to-start)) > - (for-each start-service (map first to-start)) > - > - #t))) > - > - (mlet %store-monad ((target-services target-services)) > - (machine-remote-eval machine (remote-exp target-services)))) > - > (define (machine-boot-parameters machine) > "Monadic procedure returning a list of 'boot-parameters' for the generations > of MACHINE's system profile, ordered from most recent to oldest." > @@ -275,71 +166,67 @@ of MACHINE's system profile, ordered from most recent to oldest." > (boot-parameters-kernel-arguments params)))))))) > generations)))) > > -(define (install-bootloader machine) > - "Create a bootloader entry for the new system generation on MACHINE, and > -configure the bootloader to boot that generation by default." > - (define bootloader-installer-script > - (@@ (guix scripts system) bootloader-installer-script)) > - > - (define (remote-exp installer bootcfg bootcfg-file) > - (with-extensions (list guile-gcrypt) > - (with-imported-modules (source-module-closure '((gnu build install) > - (guix store) > - (guix utils))) > - #~(begin > - (use-modules (gnu build install) > - (guix store) > - (guix utils)) > - (let* ((gc-root (string-append "/" %gc-roots-directory "/bootcfg")) > - (temp-gc-root (string-append gc-root ".new"))) > - > - (switch-symlinks temp-gc-root gc-root) > - > - (unless (false-if-exception > - (begin > - ;; The implementation of 'guix system reconfigure' > - ;; saves the load path here. This is unnecessary here > - ;; because each invocation of 'remote-eval' runs in a > - ;; distinct Guile REPL. > - (install-boot-config #$bootcfg #$bootcfg-file "/") > - ;; The installation script may write to stdout, which > - ;; confuses 'remote-eval' when it attempts to read a > - ;; result from the remote REPL. We work around this > - ;; by forcing the output to a string. > - (with-output-to-string > - (lambda () > - (primitive-load #$installer))))) > - (delete-file temp-gc-root) > - (error "failed to install bootloader")) > - > - (rename-file temp-gc-root gc-root) > - #t))))) > - > - (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine))) > - (let* ((os (machine-system machine)) > - (bootloader ((compose bootloader-configuration-bootloader > - operating-system-bootloader) > - os)) > - (bootloader-target (bootloader-configuration-target > - (operating-system-bootloader os))) > - (installer (bootloader-installer-script > - (bootloader-installer bootloader) > - (bootloader-package bootloader) > - bootloader-target > - "/")) > - (menu-entries (map boot-parameters->menu-entry boot-parameters)) > - (bootcfg (operating-system-bootcfg os menu-entries)) > - (bootcfg-file (bootloader-configuration-file bootloader))) > - (machine-remote-eval machine (remote-exp installer bootcfg bootcfg-file))))) > - > (define (deploy-managed-host machine) > "Internal implementation of 'deploy-machine' for MACHINE instances with an > environment type of 'managed-host." > - (maybe-raise-unsupported-configuration-error machine) > - (mbegin %store-monad > - (switch-to-system machine) > - (upgrade-shepherd-services machine) > - (install-bootloader machine))) > + (define target-services > + ;; Monadic expression evaluating to a list of > + ;; (shepherd-service-canonical-name, shepherd-service-file) pairs for the > + ;; services in MACHINE's operating system configuration. > + (mapm %store-monad > + (lambda (service) > + (mlet %store-monad ((file ((compose lower-object > + shepherd-service-file) > + service))) > + (return (list (shepherd-service-canonical-name service) > + (derivation->output-path file))))) > + (service-value > + (fold-services (operating-system-services (machine-system machine)) > + #:target-type shepherd-root-service-type)))) > + > + (define (run-switch-to-system machine) > + "Monadic procedure serializing the items in MACHINE necessary to build a > +G-Expression with 'switch-to-system'." > + (let* ((os (machine-system machine)) > + (activation-script (operating-system-activation-script os))) > + (mlet %store-monad ((osdrv (operating-system-derivation os))) > + (machine-remote-eval machine > + (switch-to-system osdrv activation-script))))) > + > + (define (run-upgrade-shepherd-services machine) > + "Monadic procedure serializing the items in MACHINE necessary to build a > +G-Expression with 'upgrade-shepherd-services'." > + (mlet %store-monad ((target-services target-services)) > + (machine-remote-eval machine > + (upgrade-shepherd-services target-services)))) > + > + (define (run-install-bootloader machine) > + "Monadic procedure serializing the items in MACHINE necessary to build a > +G-Expression with 'install-bootloader'." > + (mlet* %store-monad ((boot-parameters (machine-boot-parameters machine))) > + (let* ((os (machine-system machine)) > + (bootloader ((compose bootloader-configuration-bootloader > + operating-system-bootloader) > + os)) > + (target (bootloader-configuration-target > + (operating-system-bootloader os))) > + (installer (bootloader-installer-script > + (bootloader-installer bootloader) > + (bootloader-package bootloader) > + target > + "/")) > + (menu-entries (map boot-parameters->menu-entry boot-parameters)) > + (bootcfg (operating-system-bootcfg os menu-entries)) > + (bootcfg-file (bootloader-configuration-file bootloader))) > + (machine-remote-eval machine > + (install-bootloader installer bootcfg > + bootcfg-file "/"))))) > + > + (maybe-raise-missing-configuration-error machine) > + (mapm %store-monad (cut <> machine) > + (list run-switch-to-system > + run-upgrade-shepherd-services > + run-install-bootloader))) > > > ;;; From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 07 09:05:52 2019 Received: (at 36404) by debbugs.gnu.org; 7 Jul 2019 13:05:52 +0000 Received: from localhost ([127.0.0.1]:56279 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hk6rW-0003tb-P7 for submit@debbugs.gnu.org; Sun, 07 Jul 2019 09:05:52 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49475) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hk6rU-0003tM-Q4 for 36404@debbugs.gnu.org; Sun, 07 Jul 2019 09:05:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40829) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hk6rL-0001p0-Lx; Sun, 07 Jul 2019 09:05:40 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=55750 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hk6rH-0007GD-5I; Sun, 07 Jul 2019 09:05:37 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 2/3] machine: Reimplement 'managed-host-environment-type' deployment. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> <877e8wf3iz.fsf_-_@sdf.lonestar.org> <8736jkf3h5.fsf_-_@sdf.lonestar.org> <87v9weco5x.fsf@dustycloud.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 19 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 07 Jul 2019 15:05:31 +0200 In-Reply-To: <87v9weco5x.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sun, 07 Jul 2019 03:13:46 -0400") Message-ID: <87ef32dmg4.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , "Jakob L. Kreuze" , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Christopher Lemmer Webber skribis: > In some ways it looks like a portion of the previous patch and a portion > of this patch are a "move and modify" of what are sort-of the same > chunks of code. But it's a bit weird to me that the code is added in > the previous commit and removed in this one? It might be clearer to the > reader that this is what is happening if it's in the same commit. Yes, good point. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 07 09:06:26 2019 Received: (at 36404) by debbugs.gnu.org; 7 Jul 2019 13:06:26 +0000 Received: from localhost ([127.0.0.1]:56283 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hk6s6-0003ur-Dk for submit@debbugs.gnu.org; Sun, 07 Jul 2019 09:06:26 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49574) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hk6s3-0003ub-Gq for 36404@debbugs.gnu.org; Sun, 07 Jul 2019 09:06:23 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40836) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hk6ry-0002Yj-D1; Sun, 07 Jul 2019 09:06:18 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=55752 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hk6ru-0002zl-Lw; Sun, 07 Jul 2019 09:06:17 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Lemmer Webber Subject: Re: [bug#36404] [PATCH 0/3] Refactor out common behavior for system reconfiguration. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> <87wogucop2.fsf@dustycloud.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 19 Messidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 07 Jul 2019 15:06:12 +0200 In-Reply-To: <87wogucop2.fsf@dustycloud.org> (Christopher Lemmer Webber's message of "Sun, 07 Jul 2019 03:02:17 -0400") Message-ID: <87a7dqdmez.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36404 Cc: Ricardo Wurmus , "Jakob L. Kreuze" , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Christopher Lemmer Webber skribis: > Side note: I closed this issue when the initial set of patches were > merged. It seems there's ongoing work... should we reopen it or make a > separate issue? I'm unsure. We should probably open a new issue, indeed! Jakob, do you want to continue the discussion in a separate issue? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 08 15:22:42 2019 Received: (at 36404) by debbugs.gnu.org; 8 Jul 2019 19:22:42 +0000 Received: from localhost ([127.0.0.1]:59559 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hkZDl-00035Q-Qt for submit@debbugs.gnu.org; Mon, 08 Jul 2019 15:22:42 -0400 Received: from ol.sdf.org ([205.166.94.20]:50603 helo=mx.sdf.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hkZDh-00034x-Je for 36404@debbugs.gnu.org; Mon, 08 Jul 2019 15:22:40 -0400 Received: from Upsilon (mobile-166-171-186-40.mycingular.net [166.171.186.40]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x68JMKWC027924 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO); Mon, 8 Jul 2019 19:22:33 GMT From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#36404] [PATCH 0/3] Refactor out common behavior for system reconfiguration. References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> <87d0isrsmk.fsf@sdf.lonestar.org> <878std3fw0.fsf@sdf.lonestar.org> <87wogwoqrg.fsf@gnu.org> <87bly8f3kq.fsf_-_@sdf.lonestar.org> <87wogucop2.fsf@dustycloud.org> <87a7dqdmez.fsf@gnu.org> Date: Mon, 08 Jul 2019 15:22:12 -0400 In-Reply-To: <87a7dqdmez.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sun, 07 Jul 2019 15:06:12 +0200") Message-ID: <87muhoib6j.fsf@sdf.lonestar.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36404 Cc: Christopher Lemmer Webber , 36404@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Chris and Ludovic! Ludovic Court=C3=A8s writes: > zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: >=20 > > * guix/scripts/system/reconfigure.scm: New file. > > * Makefile.am (MODULES): Add it. > > * guix/scripts/system.scm (bootloader-installer-script): Export variabl= e. >=20 > > +;;; Copyright =C2=A9 2019 Jakob L. Kreuze >=20 > Could you preserve the copyright lines of (guix scripts system) that > apply to these portions of code, roughly? I've copied over all of the copyright lines in the original file. Briefly looking through the log for 'guix/scripts/system.scm', it seems that most of the committers have touched the code for system reconfiguration at one point or another. Let me know if you'd like me to comb through the logs more finely and update the copyright lines accordingly. > I think all the procedures in (guix scripts system reconfigure) could > return a rather than a gexp. Actually a > would even be cleaner than a , as it could better handle > transitions like you=E2=80=99re on a Guile 2.2 system reconfiguring towar= ds a > Guile 3 system. >=20 > Consequently you could rename =E2=80=98switch-to-system=E2=80=99 to > =E2=80=98switch-system-program=E2=80=99, and so on. That'd make writing tests for these procedures a little bit easier, too. > I think it could simply take an record and derive > the relevant bits from that. You're right, and it's an especially easy change to make once I factor in your later comment about not needing to invoke 'operating-system-derivation' directly. > This comment may become irrelevant. Good catch. At this point it's little more than a historical detail. > Same here? For =E2=80=98guix system reconfigure=E2=80=99, we=E2=80=99d ra= ther not lose > messages written to stdout by ACTIVATION-SCRIPT. That's a good point. I've modified 'install-bootloader-program' to similarly return the installer script's output as a string (it was previously being discarded.) > I think you=E2=80=99d need to include the =E2=80=98call-with-service-upgr= ade-info=E2=80=99 > call in the service-upgrade program that (guix scripts system > reconfigure) produces. It=E2=80=99s an important part of reconfiguration. >=20 > However, =E2=80=98call-with-service-upgrade-info=E2=80=99 relies on (guix= graph), > which pulls in (guix monads) and many modules that we don=E2=80=99t actua= lly > need. >=20 > It=E2=80=99s probably just an annoyance more than a real problem, but I t= hink > we should eventually change the (guix graph) API so that it no longer > relies on the =E2=80=98%store-monad=E2=80=99, which in turn will make it = a better fit > in this context. Services are serialized on the host side, so we can make use of 'call-with-service-upgrade-info' without drawing any of its dependencies into the G-Expression. :) It's nicer than my hacky 'target-services' expression, anyway. Christopher Lemmer Webber writes: > In some ways it looks like a portion of the previous patch and a > portion of this patch are a "move and modify" of what are sort-of the > same chunks of code. But it's a bit weird to me that the code is added > in the previous commit and removed in this one? It might be clearer to > the reader that this is what is happening if it's in the same commit. Looking at the diff now, I definitely see what you're talking about. Squashed! Ludovic Court=C3=A8s writes: > Christopher Lemmer Webber skribis: > >> Side note: I closed this issue when the initial set of patches were >> merged. It seems there's ongoing work... should we reopen it or make a >> separate issue? I'm unsure. > > We should probably open a new issue, indeed! Jakob, do you want to > continue the discussion in a separate issue? Yes, please! I'll open a new ticket on guix-patches with my revisions. Regards, Jakob --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0jl+cACgkQ9Qb9Fp2P 2VpSAw//fgbaoyx4yOQIb3Xy1zXOHKU6UERKGOkc9R1nkUX5hik8SiR1DTC01xKt eoOIZpxZm07W7E10HAbnTDaNCJ4A3Nh5ozAstbxy7avRwQXWko7qegbG0Oleg4OL FVBSBk3yUNJVaI5bPNDel4UCw6zXK4z86ghFLzm9DSHixSQShnPFvNLtSo+G0I+c B+X5Hyy22/ISijyTAfGqjaRPE8orAXWTJx0NzwRhXq74z5tswk1iMndM8CkSA8/y Ycf8832sU3zTjIW0L2LYpwOlSfht+Is1yKPyXmg4lGEJBpHpl0wMcMuSBZ3fV/UX K9LSrFhPaU88efn/Ek0vPnL1t17gGWiPCCBCher/3fTBhVy0wm+aYjwef6l8oYnE OQ/ZA4DGAdMEkuhWc2T6BtIlu26ZOAcz7ApA5YaWDJ/jpyNtpHTnxQcugNGw9cp2 myoOez7gfcXX9myLzi/E2WcA3GctBWjyXzHkxU2vpr7+3nCnGyEa9ik01eNtNBeM cTtWltK9hOlvS2LJq8ZS36qSZHvNBW6kzp2WZIm4RUL13WCkTgx7JJjUzxMxTAkq iSET0glvVGelo5A8CX0C0Ni/3WfLJANrDLLQ47B4J0T2yi9AP4P9lnpBPikMT0ZX ngUfSjMLATIXzANfiyiLCDfRQe9Pfc6r9P7fvO6aryqTMeiKLRc= =UiBk -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Aug 15 16:19:37 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 06 Aug 2019 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator