From unknown Tue Aug 19 23:13:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#31285] [PATCH 0/1] guix: Add git-fetch/impure. Resent-From: Chris Marusich Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 27 Apr 2018 08:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 31285 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 31285@debbugs.gnu.org Cc: Chris Marusich X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.152481695131600 (code B ref -1); Fri, 27 Apr 2018 08:16:02 +0000 Received: (at submit) by debbugs.gnu.org; 27 Apr 2018 08:15:51 +0000 Received: from localhost ([127.0.0.1]:40913 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fByXn-0008Dc-Is for submit@debbugs.gnu.org; Fri, 27 Apr 2018 04:15:51 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47449) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fByXl-0008DP-Tn for submit@debbugs.gnu.org; Fri, 27 Apr 2018 04:15:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fByXf-0003qB-Ry for submit@debbugs.gnu.org; Fri, 27 Apr 2018 04:15:44 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_40,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:40711) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fByXf-0003q5-O3 for submit@debbugs.gnu.org; Fri, 27 Apr 2018 04:15:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38410) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fByXe-0006NE-CX for guix-patches@gnu.org; Fri, 27 Apr 2018 04:15:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fByXd-0003pI-C9 for guix-patches@gnu.org; Fri, 27 Apr 2018 04:15:42 -0400 Received: from mail-pg0-x22d.google.com ([2607:f8b0:400e:c05::22d]:36734) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fByXd-0003p7-6y for guix-patches@gnu.org; Fri, 27 Apr 2018 04:15:41 -0400 Received: by mail-pg0-x22d.google.com with SMTP id i6-v6so996810pgv.3 for ; Fri, 27 Apr 2018 01:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Bd9Vbq1sMlAnJog2VWSIvKFBMeCILw8V2eVgCSSNiQM=; b=naEaaefUAlp64u26aAZytaHWtfH/Q8jz2V+U5/DQ3SeeF2Get3vj0xLtDRT1SlqAZh QOsviHZPi6HoqUsW3rij+IJ6zqKjlqb/bTriGqRhe92oRvSdh9bONwmJNHkPsrT2cxeS dOXZH155PMWMrhdahSlrb12Z1VyyEOm5uNqvY11vfXUjMI3k44r9PeZHeBseiHNaSFkV a1lhtftzyqUEcc6GTeYgLaeH4JQidR/mt8va6YKAz1MXtw9//8mNMA6pRBd4smIUY09p +t9H3+hxIeyynHmQNFeIcQv+XtUFaBftRrvtA9dOAp6vlsAJylpGbCyUm6HC/v2JX9C4 tmDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Bd9Vbq1sMlAnJog2VWSIvKFBMeCILw8V2eVgCSSNiQM=; b=tjS/5d43AbHp8Dp+fqTzjyeNnom9Y/RUTq50ZdGWErj1X389qCu3MHnnv+ecU8+e9Z q6Jo7Jur+aTSJBsiPxYjX/2bh/q9fwOpTahX9RRcDE2iBdBbXWQycmzskEeccGDDsOrw 9iPrhqDmGBFy81DoZ3p8BO6cwRstsmanbtrG/nwsZipssWaQYYpZjZHQBApLhLH6fExO A3iBURwttrnlhcbwfHPpW+3AUibScVHX0Ji/IB1deSMu1gjpx4IGnWczxlLZOo+DPPZq hXHi7SJn7aA8il2wZ486LmcLdrW3bmEyek0cSTvtYei1tyZr/H0TRzxy6LUBpKyYgC5K qdSg== X-Gm-Message-State: ALQs6tDv7lVnp/ZgT+ZbAG5SCcYjAK+1IPvOu4spmq8ZNpcFUQcfzjw9 XL+b+TEw8XwyrLiPoEHGd12M9Q== X-Google-Smtp-Source: AB8JxZrEJulG54ysgqP3H1bibMtksDmzm37PnnZytI5UeX29s+KUeNFrf3waCdcHlaMwgrfUpoxmKg== X-Received: by 10.98.214.218 with SMTP id a87mr1346146pfl.200.1524816939888; Fri, 27 Apr 2018 01:15:39 -0700 (PDT) Received: from garuda.local.net ([2601:602:9d02:4725:6495:ba21:1ebe:620a]) by smtp.gmail.com with ESMTPSA id u72-v6sm1569941pgb.16.2018.04.27.01.15.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Apr 2018 01:15:38 -0700 (PDT) From: Chris Marusich Date: Fri, 27 Apr 2018 01:15:20 -0700 Message-Id: <20180427081520.28645-1-cmmarusich@gmail.com> X-Mailer: git-send-email 2.17.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi Guix! Sometimes, a Git repository may only be available via an authenticated SSH connection. Even in the case of repositories that only contain free software, this situation can arise for administrative or compliance-related reasons. How can one define a package in such a situation? This patch adds a new origin method, git-fetch/impure, which solves that problem. Specifically, git-fetch/impure creates a fixed-output derivation that fetches the Git repository outside of a derivation, in the environment of the invoking user. In particular, this enables SSH to communicate with the user's SSH agent, which in turn allows Git to fetch the repository over an authenticated SSH connection. In addition, because it is a fixed-output derivation, the output of a successful git-fetch/impure is guaranteed to be identical to the output of a pure git-fetch for any given commit. Here's a simple example: (define-public guix-over-ssh (package (inherit guix) (name "guix-over-ssh") (source (origin (inherit (package-source guix)) (method git-fetch/impure) (uri (git-reference (inherit (origin-uri (package-source guix))) (url "ssh://marusich@git.sv.gnu.org:/srv/git/guix.git"))))))) In this particular example, my username appears in the package definition, but there is no reason why that has to be so. In many systems, it is possible to grant access to multiple users with different SSH keys under a single shared user name. And in other systems, an automated build system might need to fetch sources using its own unique system user name and SSH key. All in all, I think this is pretty useful. It enables developers to define packages in environments where authenticated access to Git repositories is required. Please let me know what you think! Chris Marusich (1): guix: Add git-fetch/impure. doc/guix.texi | 24 +++++++ guix/git-download.scm | 150 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 174 insertions(+) -- 2.17.0 From unknown Tue Aug 19 23:13:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#31285] [PATCH 1/1] guix: Add git-fetch/impure. Resent-From: Chris Marusich Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 27 Apr 2018 08:28:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31285 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 31285@debbugs.gnu.org Cc: Chris Marusich Received: via spool by 31285-submit@debbugs.gnu.org id=B31285.1524817630353 (code B ref 31285); Fri, 27 Apr 2018 08:28:02 +0000 Received: (at 31285) by debbugs.gnu.org; 27 Apr 2018 08:27:10 +0000 Received: from localhost ([127.0.0.1]:40930 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fByik-00005d-BC for submit@debbugs.gnu.org; Fri, 27 Apr 2018 04:27:10 -0400 Received: from mail-pg0-f52.google.com ([74.125.83.52]:34759) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fByii-00005N-EW for 31285@debbugs.gnu.org; Fri, 27 Apr 2018 04:27:09 -0400 Received: by mail-pg0-f52.google.com with SMTP id p10-v6so1020089pgn.1 for <31285@debbugs.gnu.org>; Fri, 27 Apr 2018 01:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fR8TCw+I8E7nlJHlezSKlkFdlAsfHkz1amRtEUCb3Ok=; b=Oh+aPAv3TOeqxS8Z0PDIzZofzOD8wkiR3yQ9F3E6AEdtxVntpVPfxblOiGvlfquJsS 1xAynhQv1WEs3ni4yS926KPoMIPOkuTS9SNlr2vvZ9mxh6bwYxI6KaPM3q3rcxx6kbf8 2JSfpSnml2l3IRcjqqiC/7R2HJzMjncQxo60HxEU1WYrCloytXxzj2mmKSRTs0v5XMJl jy4m7AHiG4Co33C4LFAwcQYwVoWsaacRRWZXHdMRVwG95QEcYedJBk2PCDHpO+SO56m6 a4FcT2JWR43/9BIJOVfTguB4lpksFZELMAHlp+iuO8TCKPPwrJSto9cSE4RuqhjHKawH dwsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fR8TCw+I8E7nlJHlezSKlkFdlAsfHkz1amRtEUCb3Ok=; b=P2CRNXvlMp0R5mrYKeFRnOXcSwVd2JdseoAqPXogvZIwWdCjRiUZ6jdkhJdsYk9loH iVwNQ4WJS7BfLk4wqwvzdGr3r610CQalksTikOlGj1SMJXSGWvaVMGZSYdiEtT87cqIw BBdniJVJSDFafzbnczE/P5+fVLYlt1OekOS2NlAijnqPlXmMM6Lq1xV3Y6yU91cylZZA iOZEKD5yonZ0s69crSFebZaJlN7TtIH4EY52aB39iWZQpqApyJufCKnXAYDtEbK1qEQM 0mPcorSjrdvwp+dmY5GkPOPJ9+t3LPlp4Kfkg5DQtkeVGvEuaCaK9sVNq036KDLi92tb 618g== X-Gm-Message-State: ALQs6tBUbZDh5R2ZcoRFuOx6nJ2TyETkeG9cpmnvGTDFXNduishDCUWx qfSo5rTmRWsaOXhnCV2NkjFxCQ== X-Google-Smtp-Source: AB8JxZrbLrm86/TsBiVgFmjo7UEysF9WitKysIByMp0Ii4F21/bxOlfWEgXILkzweormBTI09PxK0w== X-Received: by 2002:a17:902:70c4:: with SMTP id l4-v6mr1413897plt.382.1524817621948; Fri, 27 Apr 2018 01:27:01 -0700 (PDT) Received: from garuda.local.net ([2601:602:9d02:4725:6495:ba21:1ebe:620a]) by smtp.gmail.com with ESMTPSA id 76sm2458643pfm.178.2018.04.27.01.27.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Apr 2018 01:27:00 -0700 (PDT) From: Chris Marusich Date: Fri, 27 Apr 2018 01:26:42 -0700 Message-Id: <20180427082642.28760-1-cmmarusich@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180427081520.28645-1-cmmarusich@gmail.com> References: <20180427081520.28645-1-cmmarusich@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * guix/git-download.scm (clone-to-store, clone-to-store*) (git-reference->name, git-fetch/impure): New procedures. Export git-fetch/impure. * doc/guix.texi (origin Reference): Document it. --- doc/guix.texi | 24 +++++++ guix/git-download.scm | 150 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 174 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index 75886e94b..182e15428 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -3553,6 +3553,30 @@ specified in the @code{uri} field as a @code{git-reference} object; a (url "git://git.debian.org/git/pkg-shadow/shadow") (commit "v4.1.5.1")) @end example + +@vindex git-fetch/impure +@item @var{git-fetch/impure} from @code{(guix git-download)} +This procedure is the same as @code{git-fetch} in spirit; however, it +explicitly allows impurities from the environment in which it is +invoked: the @code{ssh} client program currently available via the +@code{PATH} environment variable, its SSH configuration file (usually +found at @file{~/.ssh/config}), and any SSH agent that is currently +running (usually made available via environment variables such as +@code{SSH_AUTH_SOCK}). Such impurities may seem concerning at first +blush; however, because this method will fail unless its content hash +matches the expected value, a successful git-fetch/impure is guaranteed +to produce the exact same output as a successful git-fetch for the same +commit. + +This procedure is useful if for example you need to fetch a Git +repository that is only available via an authenticated SSH connection. +In this case, an example @code{git-reference} might look like this: + +@example +(git-reference + (url "ssh://username@@git.sv.gnu.org:/srv/git/guix.git") + (commit "486de7377f25438b0f44fd93f97e9ef822d558b8")) +@end example @end table @item @code{sha256} diff --git a/guix/git-download.scm b/guix/git-download.scm index 33f102bc6..04c90e448 100644 --- a/guix/git-download.scm +++ b/guix/git-download.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2014, 2015, 2016, 2017 Ludovic Courtès ;;; Copyright © 2017 Mathieu Lirzin ;;; Copyright © 2017 Christopher Baines +;;; Copyright © 2018 Chris Marusich ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,14 +25,19 @@ #:use-module (guix store) #:use-module (guix monads) #:use-module (guix records) + #:use-module (guix derivations) #:use-module (guix packages) #:use-module (guix modules) + #:use-module (guix ui) + #:use-module ((guix build git) + #:select ((git-fetch . build:git-fetch))) #:autoload (guix build-system gnu) (standard-packages) #:use-module (ice-9 match) #:use-module (ice-9 popen) #:use-module (ice-9 rdelim) #:use-module (ice-9 vlist) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) #:export (git-reference git-reference? git-reference-url @@ -39,6 +45,7 @@ git-reference-recursive? git-fetch + git-fetch/impure git-version git-file-name git-predicate)) @@ -140,6 +147,149 @@ HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." #:recursive? #t #:guile-for-build guile))) +(define (clone-to-store store name git-reference hash runtime-dependencies) + "Clone a Git repository and add it to the store. STORE is an open +connection to the store. NAME will be used as the file name. GIT-REFERENCE +is a describing the Git repository to clone. HASH is the +recursive SHA256 hash value of the Git repository, as produced by \"guix hash +--recursive\" after the .git directories have been removed; if a fixed output +derivation has already added content to the store with this HASH, then this +procedure returns immediately. RUNTIME-DEPENDENCIES is a list of store paths; +the \"bin\" directory of the RUNTIME-DEPENDENCIES will be added to the PATH +environment variable before running the \"git\" program." + (define (is-source? name stat) + ;; It's source if and only if it isn't a .git directory. + (not (and (eq? (stat:type stat) 'directory) + (equal? name ".git")))) + + (define (clean staging-directory) + (when (file-exists? staging-directory) + (info (G_ "Removing staging directory `~a'~%") staging-directory) + (delete-file-recursively staging-directory))) + + (define (fetch staging-directory) + (info + (G_ "Downloading Git repository `~a' to staging directory `~a'~%") + (git-reference-url git-reference) + staging-directory) + (mkdir-p staging-directory) + ;; TODO: Make Git print to stderr instead of stdout. + (build:git-fetch + (git-reference-url git-reference) + (git-reference-commit git-reference) + staging-directory + #:recursive? (git-reference-recursive? git-reference)) + (info (G_ "Adding `~a' to the store~%") staging-directory) + ;; Even when the git fetch was not done recursively, we want to + ;; recursively add to the store the results of the git fetch. + (add-to-store store name #t "sha256" staging-directory + #:select? is-source?)) + + ;; To avoid fetching the repository when it has already been added to the + ;; store previously, the name passed to fixed-output-path must be the same + ;; as the name used when calling gexp->derivation in git-fetch/ssh. + (let* ((already-fetched? (false-if-exception + (valid-path? store (fixed-output-path name hash)))) + (tmpdir (or (getenv "TMPDIR") "/tmp")) + (checkouts-directory (string-append tmpdir "/guix-git-ssh-checkouts")) + (staging-directory (string-append checkouts-directory "/" name)) + (original-path (getenv "PATH"))) + ;; We might need to clean up before starting. For example, we would need + ;; to do that if Guile crashed during a previous fetch. + (clean staging-directory) + (unless already-fetched? + ;; Put our Guix-managed runtime dependencies at the front of the PATH so + ;; they will be used in favor of whatever happens to be in the user's + ;; environment (except for SSH, of course). Redirect stdout to stderr + ;; to keep set-path-environment-variable from printing a misleading + ;; message about PATH's value, since we immediately change it. + (parameterize ((current-output-port (%make-void-port "w"))) + (set-path-environment-variable "PATH" '("bin") runtime-dependencies)) + (let ((new-path (if original-path + (string-append (getenv "PATH") ":" original-path) + (getenv "PATH")))) + (setenv "PATH" new-path) + (info (G_ "Set environment variable PATH to `~a'~%") new-path) + (let ((result (fetch staging-directory))) + (clean staging-directory) + result))))) + +(define clone-to-store* (store-lift clone-to-store)) + +(define (git-reference->name git-reference) + (let ((repository-name (basename (git-reference-url git-reference) ".git")) + (short-commit (string-take (git-reference-commit git-reference) 9))) + (string-append repository-name "-" short-commit "-checkout"))) + +(define* (git-fetch/impure ref hash-algo hash + #:optional name + #:key + (system (%current-system)) + (guile (default-guile))) + "Return a fixed-output derivation that fetches REF, a +object. The output is expected to have recursive hash HASH of type +HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f. + +This procedure is the same as git-fetch in spirit; however, it explicitly +allows impurities from the environment in which it is invoked: the \"ssh\" +client program currently available via the PATH environment variable, its SSH +configuration file (usually found at ~/.ssh/config), and any SSH agent that is +currently running (usually made available via environment variables such as +SSH_AUTH_SOCK). Such impurities may seem concerning at first blush; however, +because a fixed-output derivation will fail unless its content hash is +correct, a successful git-fetch/impure is guaranteed to produce the exact same +output as a successful git-fetch for the same commit. + +This procedure is useful if for example you need to fetch a Git repository +that is only available via an authenticated SSH connection." + ;; Do the Git fetch in the host environment so that it has access to the + ;; user's SSH agent, SSH config, and other tools. This will only work if we + ;; are running in an environment with a properly installed and configured + ;; SSH. It is impure because it happens outside of a derivation, but it + ;; allows us to fetch a Git repository that is only available over SSH. + (mlet* %store-monad + ((name -> (or name (git-reference->name ref))) + (guile (package->derivation guile system)) + (git -> `("git" ,(git-package))) + ;; When doing 'git clone --recursive', we need sed, grep, etc. to be + ;; available so that 'git submodule' works. We do not add an SSH + ;; client to the inputs here, since we explicltly want to use the SSH + ;; client, SSH agent, and SSH config from the user's environment. + (inputs -> `(,git ,@(if (git-reference-recursive? ref) + (standard-packages) + '()))) + (input-packages -> (match inputs (((names packages outputs ...) ...) + packages))) + (input-derivations (sequence %store-monad + (map (cut package->derivation <> system) + input-packages))) + ;; The tools that clone-to-store requires (e.g., Git) must be built + ;; before we invoke clone-to-store. + (ignored (built-derivations input-derivations)) + (input-paths -> (map derivation->output-path input-derivations)) + (checkout (clone-to-store* name ref hash input-paths))) + (gexp->derivation + ;; To avoid fetching the repository when it's already been added to the + ;; store previously, the name used here must be the same as the name used + ;; when calling fixed-output-path in clone-to-store. + name + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils)) + (copy-recursively #$checkout #$output))) + ;; Slashes are not allowed in file names. + #:script-name "git-download-ssh" + #:system system + ;; Fetching a Git repository is usually a network-bound operation, so + ;; offloading is unlikely to speed things up. + #:local-build? #t + #:hash-algo hash-algo + #:hash hash + ;; Even when the git fetch will not be done recursively, we want to + ;; recursively add to the store the results of the git fetch. + #:recursive? #t + #:guile-for-build guile))) + (define (git-version version revision commit) "Return the version string for packages using git-download." (string-append version "-" revision "." (string-take commit 7))) -- 2.17.0 From unknown Tue Aug 19 23:13:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#31285] [PATCH 1/1] guix: Add git-fetch/impure. Resent-From: Chris Marusich Original-Sender: "Debbugs-submit" Resent-CC: mhw@netris.org, dthompson2@worcester.edu, ludo@gnu.org, guix-patches@gnu.org Resent-Date: Mon, 30 Apr 2018 02:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31285 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 31285@debbugs.gnu.org Cc: Mark H Weaver , "Thompson, David" , ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) X-Debbugs-Original-Xcc: Mark H Weaver , "Thompson, David" , ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Received: via spool by 31285-submit@debbugs.gnu.org id=B31285.15250565792062 (code B ref 31285); Mon, 30 Apr 2018 02:50:01 +0000 Received: (at 31285) by debbugs.gnu.org; 30 Apr 2018 02:49:39 +0000 Received: from localhost ([127.0.0.1]:43326 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fCysl-0000XC-7L for submit@debbugs.gnu.org; Sun, 29 Apr 2018 22:49:39 -0400 Received: from mail-pg0-f44.google.com ([74.125.83.44]:33310) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fCysf-0000Wv-7M for 31285@debbugs.gnu.org; Sun, 29 Apr 2018 22:49:36 -0400 Received: by mail-pg0-f44.google.com with SMTP id i194-v6so5366101pgd.0 for <31285@debbugs.gnu.org>; Sun, 29 Apr 2018 19:49:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:references:date:in-reply-to:message-id:user-agent :mime-version; bh=pCD1qNQOx5evEgcntHhBed57VmpZj+MTQFgpDNlVMGY=; b=JbaAPu8UD7dMTusryTkKt59FIwolcZS+1qiGbmBnPWDXXoXZ4dm0bx7jLuWFyoziKA uWfJzpZ8EgF8oUsRis3xIGpk6utEuYhK54cTWYkj+WRzEcBThnDWxgpum0U0NCIRLBO+ QXnD7tqhztdeMdsrhQBX8orSz6AcCqpdDUZ9LLLv82qp7BDSd/XPUUugtQV32/x7XzZe cEKLjLirf4IRwqRI0V6RJwfCZK3v11C5O8Iw9kLhNyfUs5XjDp6F0nOJjd5XujF/KMhp Y1CpsZV3bwGZVjMRUjdZwp2sdp2xdqkRaYNkczu84l3cmo6rn4Kn8tRleWFWYpB0LSxL ievg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=pCD1qNQOx5evEgcntHhBed57VmpZj+MTQFgpDNlVMGY=; b=m4q5eJHVdyyBJ8b7k31lzMrJD2zHpN+XJm9SeyWvEOriFhAVqCuLtqm2AzxxjZgdnG ELAuIrtGH72+0/FRtcLL9Y+1xwwH2MlmabyHlAxXbVQo8Ot+AK4oJIbsQ339xLCYqHBL wKVMDjJm048CAl+9NzHmQW0T+WEOhH64sviJ0TAyWwKrGmxGEj+R/TAT3HVr5lhzultz ECCLf0BTp54uvHdAW8eZKteyXe+bzl8LOXx4CNfjBhUn3WW9oM9rxZeOTRS6EvsFgehQ x+/z0r8XbMkTTSKd2KpPj0isnLl8BHAj3Nz3QVAaX4ErzBUi3uOaQl68wQ7XzWy8NWs0 SrHw== X-Gm-Message-State: ALQs6tCuwKAan2cko8YK5rWV3qvzi13g+eb6s5xOPJOJvrID314Eg3nM +bkO7j+fCJTs9SNECvM2ejlRig== X-Google-Smtp-Source: AB8JxZpqlAi0dqQAak2Mud7DYKLamL2ge6vflpW6+Cw2CwefXyOAoK4+wJXzWi2TRmacl4P24uZwJA== X-Received: by 10.98.178.4 with SMTP id x4mr10470776pfe.21.1525056566522; Sun, 29 Apr 2018 19:49:26 -0700 (PDT) Received: from garuda.local ([2601:602:9d02:4725:6495:ba21:1ebe:620a]) by smtp.gmail.com with ESMTPSA id t1-v6sm10564400pgs.47.2018.04.29.19.49.24 for <31285@debbugs.gnu.org> (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 29 Apr 2018 19:49:25 -0700 (PDT) From: Chris Marusich References: <20180427081520.28645-1-cmmarusich@gmail.com> <20180427082642.28760-1-cmmarusich@gmail.com> Date: Sun, 29 Apr 2018 19:49:23 -0700 In-Reply-To: <20180427082642.28760-1-cmmarusich@gmail.com> (Chris Marusich's message of "Fri, 27 Apr 2018 01:26:42 -0700") Message-ID: <87sh7dcsss.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Mark, Ludo, and David, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Hello, > > Chris Marusich skribis: > >> You've both said that you would prefer not to add git-fetch/impure to >> Guix. Can you help me to understand why you feel that way? I really >> think it would be nice if Guix could fetch Git repositories over SSH >> using public key authentication, so I'm hoping that we can talk about it >> and figure out an acceptable way to implement it. > > One argument against it would be that it encourages people (or at least > makes it very easy) to write origins that depend on external state, and > thus may be non-reproducible by others, and that Guix itself should > provide tools for writing reproducible build definitions. The impurity bothers me, too. If you don't have the right SSH key available or your SSH installation isn't configured in just the right way, then an origin defined using git-fetch/impure won't work. Could we eliminate the impurity by adding a feature to the guix-daemon that allows an administrator (i.e., root) to configure an SSH key for guix-daemon to use when fetching Git repositories over SSH? If it's possible, I think that would be preferable. What do you think of that idea? Also, here's a new version of the patch, which fixes/improves some random things I noticed. =2D-=20 Chris --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-guix-Add-git-fetch-impure.patch Content-Transfer-Encoding: quoted-printable From=20650fb09fc25f78cea23f4db6504a40fd6cb9a10b Mon Sep 17 00:00:00 2001 From: Chris Marusich Date: Fri, 27 Apr 2018 00:42:45 -0700 Subject: [PATCH] guix: Add git-fetch/impure. * guix/git-download.scm (clone-to-store, clone-to-store*) (git-reference->name, git-fetch/impure): New procedures. Export git-fetch/impure. * doc/guix.texi (origin Reference): Document it. =2D-- doc/guix.texi | 25 +++++++ guix/git-download.scm | 166 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 191 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index 75886e94b..68b20e84d 100644 =2D-- a/doc/guix.texi +++ b/doc/guix.texi @@ -3553,6 +3553,31 @@ specified in the @code{uri} field as a @code{git-ref= erence} object; a (url "git://git.debian.org/git/pkg-shadow/shadow") (commit "v4.1.5.1")) @end example + +@vindex git-fetch/impure +@item @var{git-fetch/impure} from @code{(guix git-download)} +This procedure yields the same result as @code{git-fetch}; however, it +explicitly allows impurities from the environment in which it is +invoked: the @code{ssh} client program currently available via the +@code{PATH} environment variable, its SSH configuration file (usually +found at @file{~/.ssh/config}), and any SSH agent that is currently +running (usually made available via environment variables such as +@code{SSH_AUTH_SOCK}). + +The @code{git-fetch/impure} fetch method should not be used in package +origins in the official Guix distribution. Due to its impurity, if two +people have configured SSH differently, it is possible that the origin +will work for one person but not for the other. This fetch method is +intended as a convenience for cases where, due to the circumstances of +your situation, the Git repository is only available over an +authenticated SSH connection. In this case, an example +@code{git-reference} might look like this: + +@example +(git-reference + (url "ssh://username@@git.sv.gnu.org:/srv/git/guix.git") + (commit "486de7377f25438b0f44fd93f97e9ef822d558b8")) +@end example @end table =20 @item @code{sha256} diff --git a/guix/git-download.scm b/guix/git-download.scm index 33f102bc6..68947cf9b 100644 =2D-- a/guix/git-download.scm +++ b/guix/git-download.scm @@ -2,6 +2,7 @@ ;;; Copyright =C2=A9 2014, 2015, 2016, 2017 Ludovic Court=C3=A8s ;;; Copyright =C2=A9 2017 Mathieu Lirzin ;;; Copyright =C2=A9 2017 Christopher Baines +;;; Copyright =C2=A9 2018 Chris Marusich ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,14 +25,19 @@ #:use-module (guix store) #:use-module (guix monads) #:use-module (guix records) + #:use-module (guix derivations) #:use-module (guix packages) #:use-module (guix modules) + #:use-module (guix ui) + #:use-module ((guix build git) + #:select ((git-fetch . build:git-fetch))) #:autoload (guix build-system gnu) (standard-packages) #:use-module (ice-9 match) #:use-module (ice-9 popen) #:use-module (ice-9 rdelim) #:use-module (ice-9 vlist) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) #:export (git-reference git-reference? git-reference-url @@ -39,6 +45,7 @@ git-reference-recursive? =20 git-fetch + git-fetch/impure git-version git-file-name git-predicate)) @@ -140,6 +147,165 @@ HASH-ALGO (a symbol). Use NAME as the file name, or = a generic name if #f." #:recursive? #t #:guile-for-build guile))) =20 +(define (clone-to-store store name git-reference hash runtime-dependencies) + "Clone a Git repository, add it to the store, and return its store path. +STORE is an open connection to the store. NAME will be used as the file n= ame. +GIT-REFERENCE is a describing the Git repository to clone. +HASH is the recursive SHA256 hash value of the Git repository, as produced= by +\"guix hash --recursive\" after the .git directories have been removed; if= a +fixed output derivation has already added content to the store with this H= ASH, +then this procedure returns immediately. RUNTIME-DEPENDENCIES is a list of +store paths; the \"bin\" directory of the RUNTIME-DEPENDENCIES will be add= ed +to the PATH environment variable before running the \"git\" program." + (define (is-source? name stat) + ;; It's source if and only if it isn't a .git directory. + (not (and (eq? (stat:type stat) 'directory) + (equal? name ".git")))) + + (define (clean staging-directory) + (when (file-exists? staging-directory) + (info (G_ "Removing staging directory `~a'~%") staging-directory) + (delete-file-recursively staging-directory))) + + (define (fetch staging-directory) + (info + (G_ "Downloading Git repository `~a' to staging directory `~a'~%") + (git-reference-url git-reference) + staging-directory) + (mkdir-p staging-directory) + ;; Git prints some messages to stdout, which is a minor blemish becaus= e it + ;; interferes with convenient shell idioms like "ls $(guix build + ;; my-package)". However, if we try to redirect stdout to stderr using + ;; with-output-to-port, and if Git fails because SSH is not available, + ;; then mysteriously Git's helpful error messages do not get printed. = It + ;; seems better to surface useful error messages here than to hide the= m. + (build:git-fetch + (git-reference-url git-reference) + (git-reference-commit git-reference) + staging-directory + #:recursive? (git-reference-recursive? git-reference)) + (info (G_ "Adding `~a' to the store~%") staging-directory) + ;; Even when the git fetch was not done recursively, we want to + ;; recursively add to the store the results of the git fetch. + (add-to-store store name #t "sha256" staging-directory + #:select? is-source?)) + + ;; To ensure the derivation produced by git-fetch/impure does not need t= o be + ;; run, the name passed to fixed-output-path must be the same as the name + ;; used when calling gexp->derivation in git-fetch/impure. + (let* ((output (fixed-output-path name hash)) + (already-fetched? (false-if-exception (valid-path? store output))) + (tmpdir (or (getenv "TMPDIR") "/tmp")) + (checkouts-directory (string-append + tmpdir "/guix-git-ssh-checkouts")) + (staging-directory (string-append checkouts-directory "/" name)) + (original-path (getenv "PATH"))) + ;; We might need to clean up before starting. For example, we would n= eed + ;; to do that if Guile crashed during a previous fetch. + (clean staging-directory) + (if already-fetched? + output + (begin + ;; Put our Guix-managed runtime dependencies at the front of the + ;; PATH so they will be used in favor of whatever happens to be = in + ;; the user's environment (except for SSH, of course). Redirect + ;; stdout to stderr to keep set-path-environment-variable from + ;; printing a misleading message about PATH's value, since we + ;; immediately change it. + (with-output-to-port (%make-void-port "w") + (lambda () + (set-path-environment-variable + "PATH" '("bin") runtime-dependencies))) + (let ((new-path (if original-path + (string-append + (getenv "PATH") ":" original-path) + (getenv "PATH")))) + (setenv "PATH" new-path) + (info (G_ "Set environment variable PATH to `~a'~%") new-path) + (let ((result (fetch staging-directory))) + (clean staging-directory) + result)))))) + +(define clone-to-store* (store-lift clone-to-store)) + +(define (git-reference->name git-reference) + (let ((repository-name (basename (git-reference-url git-reference) ".git= ")) + (short-commit (string-take (git-reference-commit git-reference) 9)= )) + (string-append repository-name "-" short-commit "-checkout"))) + +(define* (git-fetch/impure ref hash-algo hash + #:optional name + #:key + (system (%current-system)) + (guile (default-guile))) + "Return a fixed-output derivation that fetches REF, a +object. The output is expected to have recursive hash HASH of type +HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f. + +This procedure yields the same result as git-fetch; however, it explicitly +allows impurities from the environment in which it is invoked: the \"ssh\" +client program currently available via the PATH environment variable, its = SSH +configuration file (usually found at ~/.ssh/config), and any SSH agent tha= t is +currently running (usually made available via environment variables such as +SSH_AUTH_SOCK). + +This procedure should not be used in package origins in the official Guix +distribution. Due to its impurity, if two people have configured SSH +differently, it is possible that the origin will work for one person but n= ot +for the other. This fetch method is intended as a convenience for cases +where, due to the circumstances of your situation, the Git repository is o= nly +available over an authenticated SSH connection." + (mlet* %store-monad + ((name -> (or name (git-reference->name ref))) + (guile (package->derivation guile system)) + (git -> `("git" ,(git-package))) + ;; When doing 'git clone --recursive', we need sed, grep, etc. to be + ;; available so that 'git submodule' works. We do not add an SSH + ;; client to the inputs here, since we explicitly want to use the S= SH + ;; client, SSH agent, and SSH config from the current environment. + (inputs -> `(,git ,@(if (git-reference-recursive? ref) + (standard-packages) + '()))) + (input-packages -> (match inputs (((names packages outputs ...) ...) + packages))) + (input-derivations (sequence %store-monad + (map (cut package->derivation <> syste= m) + input-packages))) + ;; The tools that clone-to-store requires (e.g., Git) must be built + ;; before we invoke clone-to-store. + (ignored (built-derivations input-derivations)) + (input-paths -> (map derivation->output-path input-derivations)) + (checkout (clone-to-store* name ref hash input-paths))) + ;; To ensure that commands like "guix build --source my-package" don't + ;; fail, return (as a monadic value) a derivation here. We could just + ;; tail-call clone-to-store* instead of going through the effort of + ;; returning a derivation here, but then the aforementioned command wo= uld + ;; fail for the same reason that it fails when the origin is defined w= ith + ;; "local-file". This is the ONLY reason why we call gexp->derivation + ;; here. In fact, this derivation will never actually be run, since we + ;; always fetch its contents via clone-to-store* first. + (gexp->derivation + ;; To ensure this derivation does not need to be run, the name used h= ere + ;; must be the same as the name used when calling fixed-output-path in + ;; clone-to-store. + name + ;; This builder never runs, so the actual builder code doesn't matter. + ;; However, we must ungexp the output variable, or the derivation will + ;; produce no output path. + #~(ungexp output) + ;; Slashes are not allowed in file names. + #:script-name "git-download-impure" + #:system system + ;; Fetching a Git repository is usually a network-bound operation, so + ;; offloading is unlikely to speed things up. + #:local-build? #t + #:hash-algo hash-algo + #:hash hash + ;; Even when the git fetch will not be done recursively, we want to + ;; recursively add to the store the results of the git fetch. + #:recursive? #t + #:guile-for-build guile))) + (define (git-version version revision commit) "Return the version string for packages using git-download." (string-append version "-" revision "." (string-take commit 7))) =2D-=20 2.17.0 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlrmhDMACgkQ3UCaFdgi Rp2LFhAAkAptSz6erC9kTA4UKSw18399eW7+nS2xVzKsB5GRBORywLMFO+JYiHpW wVA0XFO5esoYQYFj09oU4heNLVSWU2F4XItXdAgh4mVc5BOCHYnsUtr7J94R9rpH FN+3cJiW+dDl2EOGa5f5LDNlAzpOOXJw+312rRs5gxUIzS7aM9inwTd6OsuUaVM9 0a7GvVcrs79aVJ+jCmyIYy4FXB/1oV/Boe0diuss857peZW1GwEUbFjBXpy2Joo1 SCP51RaENVlpTZdeHC8jjTUWkrhq3bP53ffMRh+gZFkdsmcs/vlWxZ0jHiWHwW1x Ypb+ybGz/cc719GRJm6Y88nYIjsiHZZ9Q/cTqoK36LW5MjOwzmJOJVMyrwyq/Eg5 TOd2Zuvsow7ofipkZoobt7knezH04FSQeTlzl/tS/l1vJrfoHKqA9S/JdzZBeAnz B8PHj7WHqYnm6v/Fw3KKzKrdcTOFLNCtncUpIC9jSmUUwknHP7AfUaBROXEH007X mpvCeJV5e67zCCwY7UrkPPZQK/geqDDoJUPKs9VQki8Qwr8K5UlknvcGenMK5dke r2BBaAI/i0jjQUTIOP6H46LHvRSPn6NPWEvk8garSzNmFo8YpVoIkt9gHjY1RMNE Gs2zYXwjmGclWcBc/Ot2cZgiPj7F/Bmm9l7dMHIMNQy4NDocZtw= =GC4l -----END PGP SIGNATURE----- --==-=-=-- From unknown Tue Aug 19 23:13:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#31285] [PATCH 0/1] guix: Add git-fetch/impure. References: <20180427081520.28645-1-cmmarusich@gmail.com> Resent-From: sirgazil Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 18 Apr 2020 15:55:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31285 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: "31285" <31285@debbugs.gnu.org> Received: via spool by 31285-submit@debbugs.gnu.org id=B31285.158722525618061 (code B ref 31285); Sat, 18 Apr 2020 15:55:02 +0000 Received: (at 31285) by debbugs.gnu.org; 18 Apr 2020 15:54:16 +0000 Received: from localhost ([127.0.0.1]:43449 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jPpnM-0004hE-IE for submit@debbugs.gnu.org; Sat, 18 Apr 2020 11:54:16 -0400 Received: from sender4-pp-o93.zoho.com ([136.143.188.93]:25336) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jPpnK-0004h5-CZ for 31285@debbugs.gnu.org; Sat, 18 Apr 2020 11:54:14 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1587225251; cv=none; d=zohomail.com; s=zohoarc; b=cbNalw+Reu9UTuem2wrmSimNhP024f/30f/FrP1fZ5EkoHVL6+HmQ6SJeUv4hqigWJz31vYs1Z/QgeqFub073cXA2m4oaSWVNS9neqCSn9YY1QbQHgaEudCuVVeDZWf31txc51WwL6IHZKfRPp+htWHQR4YSSVyFB28dwatd78E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587225251; h=Content-Type:Content-Transfer-Encoding:Date:From:MIME-Version:Message-ID:Subject:To; bh=kBgsK1aVWEWBG70ovL3czTV27YFJFNQ3Q2Qj9EADOog=; b=g2ZwN9rDz4q2InOgJ5nFDyq2iiCihe4Jbp432h60fEk/QhlSrgfvEYp0n9lYDYNk4rlYPsKJpJoO27T4MgpG5vm5dL3okeDPmlF3n465vEDGwz3qTcFN7iaEbtkj8rJqdc0xp/Ys9U9ohL+feTM5gVCTd7yxSYOiOQUbbq/2+0g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zoho.com; spf=pass smtp.mailfrom=sirgazil@zoho.com; dmarc=pass header.from= header.from= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=zapps768; d=zoho.com; h=date:from:to:message-id:in-reply-to:subject:mime-version:content-type:user-agent; b=kfRwLhEd1TPOxTwLQ2Do2iUjjnGK20ZWfScu4fAX3D7V/XDHIUiMpXpPZjtAYNiMnyueO4S23426 4szqbU5cFITFn5bLJJXRMIqFAE61OujwWaedPHYymU/iGytWpGRk DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1587225251; s=zm2020; d=zoho.com; i=sirgazil@zoho.com; h=Date:From:To:Message-ID:In-Reply-To:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=kBgsK1aVWEWBG70ovL3czTV27YFJFNQ3Q2Qj9EADOog=; b=V9Km6AR2FXB0+OPoBvkk3sbvR1Cvv8Ziyg58HxD97uWME1gEsH5U/BkBSzORB5SR UYVe31gT1DHVQISOogLZYx2af5L65j9X/KkKWina5XE0mQPFkF4LZpdGPoNQoeTZkwq 4kZBslZa0kueWgIarVxHPMFRLmjUCv3VdvKPPe1c= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1587225244846429.72523509264533; Sat, 18 Apr 2020 08:54:04 -0700 (PDT) Received: from [179.15.13.185] by mail.zoho.com with HTTP;Sat, 18 Apr 2020 08:54:04 -0700 (PDT) Date: Sat, 18 Apr 2020 15:54:04 +0000 From: sirgazil Message-ID: <1718dff34ac.cc32d28414025.6904503667509437602@zoho.com> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, I feel the same as Chris. I started doing some packaging this year, and really felt downhearted when I found there was no support for package definitions with SSH authenticated git repositories (for private use, of course). In my case, I need this for two reasons: * I want to use Guix channels for experimental packages, prototypes and pre-alpha software that should be available for some people only. * I want to use Guix channels for production-ready packages of in-house tools that are only useful for private businesses. In both cases, the channels and software sources would be in Git repositories hosted by third-parties like GitLab, BitBucket, etc., which provide SSH authentication. There are some comments already about Chris' patch in another bug report (issues.guix.gnu.org/issue/31284). I agree that "git-fetch/impure" must not be used in Guix's official channel(s), but I'd like Guix to include it in its API for use in private channels. I think having this functionality would make it even easier to adopt the GNU Guix in mainstream culture. --- https://sirgazil.bitbucket.io/ From unknown Tue Aug 19 23:13:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#31285] [PATCH 0/1] guix: Add git-fetch/impure. References: <20180427081520.28645-1-cmmarusich@gmail.com> In-Reply-To: <20180427081520.28645-1-cmmarusich@gmail.com> Resent-From: Luis Felipe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 22 Oct 2020 00:45:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31285 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: "31285@debbugs.gnu.org" <31285@debbugs.gnu.org> Reply-To: Luis Felipe Received: via spool by 31285-submit@debbugs.gnu.org id=B31285.16033274928534 (code B ref 31285); Thu, 22 Oct 2020 00:45:02 +0000 Received: (at 31285) by debbugs.gnu.org; 22 Oct 2020 00:44:52 +0000 Received: from localhost ([127.0.0.1]:50903 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kVOiq-0002DZ-Bh for submit@debbugs.gnu.org; Wed, 21 Oct 2020 20:44:52 -0400 Received: from mail4.protonmail.ch ([185.70.40.27]:57447) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kVOin-0002DM-Q3 for 31285@debbugs.gnu.org; Wed, 21 Oct 2020 20:44:50 -0400 Date: Thu, 22 Oct 2020 00:44:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1603327483; bh=weA3LBmE/aby00QjENlHquN09g2Sfy3eGPxHWqTkcZY=; h=Date:To:From:Reply-To:Subject:From; b=S9k211F2dJXn15utRCHgyluf10TdyTDPpUlY8BYCE9Hwer8PBOxWTBca9RvJ4t5v8 kBUSok+ypFvtcC214dQfVSt9XKuOUs+psNlaA72JusvOARar7O5/wYt9zNqPZUKICs w47cLEYhPrK8enQ2EBlvpPjf4DM5OaLhaQVB9mgI= From: Luis Felipe Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: > Sometimes, a Git repository may only be available via an authenticatedSSH connection. Even in the case of repositories that only containfree software, this situation can arise for administrative orc [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [185.70.40.27 listed in wl.mailspike.net] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (luis.felipe.la[at]protonmail.com) 2.1 AC_FROM_MANY_DOTS Multiple periods in From user name X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Sometimes, a Git repository may only be available via an authenticatedSSH= connection. Even in the case of repositories that only containfree softwar= e, this situation can arise for administrative orcompliance-related reasons= . How can one define a package in such asituation? Correct me if I'm wrong, but I think this is possible now. All you have to = do is pass a git-checkout record to the package source field instead of an = origin (see the (guix git) module). For example: (source (git-checkout (url "git@gitlab.com:luis-felipe/guile-lab.git") (commit (string-append "v" version)))) I'm using this for my private packages, and it seems to work. From unknown Tue Aug 19 23:13:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#31285] [PATCH 1/1] guix: Add git-fetch/impure. Resent-From: zimoun Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 01 Dec 2020 18:14:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31285 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Chris Marusich Cc: 31285@debbugs.gnu.org, Mark H Weaver , Ludovic =?UTF-8?Q?Court=C3=A8s?= , "Thompson, David" Received: via spool by 31285-submit@debbugs.gnu.org id=B31285.160684639423391 (code B ref 31285); Tue, 01 Dec 2020 18:14:01 +0000 Received: (at 31285) by debbugs.gnu.org; 1 Dec 2020 18:13:14 +0000 Received: from localhost ([127.0.0.1]:33248 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kkA9J-00065C-JE for submit@debbugs.gnu.org; Tue, 01 Dec 2020 13:13:13 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:35404) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kkA9H-00064x-2E for 31285@debbugs.gnu.org; Tue, 01 Dec 2020 13:13:12 -0500 Received: by mail-wr1-f65.google.com with SMTP id r3so4129212wrt.2 for <31285@debbugs.gnu.org>; Tue, 01 Dec 2020 10:13:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=kDpkySuQZWTCvLQ4fEG1/U0JgBUq4kbyH83NHsit33M=; b=BDBWvH0U4spNHf4In+bOqKjXlEbfeSwLJfcyM8LrR/Q/ZuhuYIr15SvN7GQu8cwLQh w9y/TVl4X2MU+UZzLvRb2Fj3DE5OkumJj9D74GyvCIMOjGTCq7PaA/AyXqLZ+g65383l 3LCWxsGn1zWDwxkdtImauIRZx3t3IqLynQa+weFOdA9lVkM/HKvZDyz8SkdZU5KZLk2v JF5e5ufCWwSZSBesiA3DDUGz1gw5VDo1D0ryy4jtJpcc60abNlSddqdEW+U+S0KccbRs P/wI7dp4Aa/mkLGbLSAZfvfatwb+M0QdQYs3bKri7cxP8ru0B/l5OIurfnFYFcEINtgQ Fg0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=kDpkySuQZWTCvLQ4fEG1/U0JgBUq4kbyH83NHsit33M=; b=Ggp8J7RbmVcsdShykJWiTafVzOJ+TG7X8+faLygN+iEIfnPyNlazq2y+tGc4eT0kUl x/ZB/g8Ydh+W6OY0h1DroIOW19epzYedWefHhnrh0bqmG8eq5nFmxDM6oE0J56FJqvM/ 24x9X3IPDvY/DRO8whP9rAdHG4IzzK99u555+c9d0xQUG/7G245RZT2U9NQ/Rk9zmfLJ 1KQBPEHmO9jPyBimMv1OgSkpO46UjkUZpLeh0gUO7QVscer+uRGWpjsAXVigtAU8C3bV +iZ9qKrLCZhrSdxaNi7mBj+FniXSZe4BPpz1cuTfEmDcuIHPK3qLlOvFn/+UzAUHijYW c6DQ== X-Gm-Message-State: AOAM5332MFKgAUs1M/VUi+NylVZmkXYqHpWe2osNPU1x/TbRH7pfq5Xx c/CP3BH9CO8qfNUqMCCUJvg= X-Google-Smtp-Source: ABdhPJw/ZguKAsAWdcKGbee981Dg2tRgK1f2Ui5GgehFt3Q0bguvQGbWwBpFbg7lGiQwbaDJyDGJ5w== X-Received: by 2002:adf:e3cf:: with SMTP id k15mr5456126wrm.259.1606846385415; Tue, 01 Dec 2020 10:13:05 -0800 (PST) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id a9sm633118wrp.21.2020.12.01.10.13.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Dec 2020 10:13:04 -0800 (PST) From: zimoun References: <20180427081520.28645-1-cmmarusich@gmail.com> <20180427082642.28760-1-cmmarusich@gmail.com> <87sh7dcsss.fsf@gmail.com> Date: Tue, 01 Dec 2020 19:06:38 +0100 In-Reply-To: <87sh7dcsss.fsf@gmail.com> (Chris Marusich's message of "Sun, 29 Apr 2018 19:49:23 -0700") Message-ID: <86im9lv1sh.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, The bug #31285 is mainly about allow Git over SSH, see: and I do not know where the discussion below happened=E2=80=A6 On Sun, 29 Apr 2018 at 19:49, Chris Marusich wrote: > Hi Mark, Ludo, and David, > > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> Hello, >> >> Chris Marusich skribis: >> >>> You've both said that you would prefer not to add git-fetch/impure to >>> Guix. Can you help me to understand why you feel that way? I really >>> think it would be nice if Guix could fetch Git repositories over SSH >>> using public key authentication, so I'm hoping that we can talk about it >>> and figure out an acceptable way to implement it. >> >> One argument against it would be that it encourages people (or at least >> makes it very easy) to write origins that depend on external state, and >> thus may be non-reproducible by others, and that Guix itself should >> provide tools for writing reproducible build definitions. > > The impurity bothers me, too. If you don't have the right SSH key > available or your SSH installation isn't configured in just the right > way, then an origin defined using git-fetch/impure won't work. > > Could we eliminate the impurity by adding a feature to the guix-daemon > that allows an administrator (i.e., root) to configure an SSH key for > guix-daemon to use when fetching Git repositories over SSH? If it's > possible, I think that would be preferable. What do you think of that > idea? > > Also, here's a new version of the patch, which fixes/improves some > random things I noticed. =E2=80=A6and the question is: is it still relevant? I am not sure to get i= f the use-case of the initial motivation is not covered by the current =E2=80=99git-fetch=E2=80=99. If not, what is the status of this patch: rej= ected with which reason or merged? Thanks, simon From unknown Tue Aug 19 23:13:42 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Chris Marusich Subject: bug#31285: closed (Re: bug#31285: [PATCH 0/1] guix: Add git-fetch/impure.) Message-ID: References: <87sg0hz1sv.fsf_-_@gmail.com> <20180427081520.28645-1-cmmarusich@gmail.com> X-Gnu-PR-Message: they-closed 31285 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 31285@debbugs.gnu.org Date: Wed, 14 Jul 2021 09:24:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1626254642-16806-1" This is a multi-part message in MIME format... ------------=_1626254642-16806-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #31285: [PATCH 0/1] guix: Add git-fetch/impure. which was filed against the guix-patches package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 31285@debbugs.gnu.org. --=20 31285: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D31285 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1626254642-16806-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 31285-done) by debbugs.gnu.org; 14 Jul 2021 09:23:56 +0000 Received: from localhost ([127.0.0.1]:42899 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m3b7U-0004MU-BW for submit@debbugs.gnu.org; Wed, 14 Jul 2021 05:23:56 -0400 Received: from mail-pf1-f170.google.com ([209.85.210.170]:38619) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m3b7Q-0004MD-Vs for 31285-done@debbugs.gnu.org; Wed, 14 Jul 2021 05:23:55 -0400 Received: by mail-pf1-f170.google.com with SMTP id j9so1490659pfc.5 for <31285-done@debbugs.gnu.org>; Wed, 14 Jul 2021 02:23:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=ElPKYxh1ATgQK3dS34bdFLQl7Qo+gDsZRKefAcez2ig=; b=OsiwVkq0UrNC74O1h0atbMnr5DUb2VdPqh70jN/ZEdmdnNSs8bY6UrTyHAB5Z9RRef A7jKg0wWgjkoMyBxnCFc/XgpYl0qyEl2Ww5Ua86/e7pd83k6nySUQKbMmoA+VjvRHm2u HF5QzPU+zdCekQ+l7XMWayS/kZKrKlLfEdBly9fIGhVO8KXmM8bpGLOOeRR+TjkfaprE Hlh8jZ1w7SoVzR5rDX4BK1FXSvE6En9FU3o4dGHQX33rdYYdpw3bCXBvHqKzgGYFXiCz KjLhXlO5ZkmTN/kYy3V8P5EyrJP6Nv1Gvsbi5lO82GDwMqo5L3NAdSRj9O5KbYBZd7G3 OEwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=ElPKYxh1ATgQK3dS34bdFLQl7Qo+gDsZRKefAcez2ig=; b=f8yjaZcMJHVrazfBV7MgYBU+xZeQlo3BIn2QLf2zYvCHxaXGvRnp4BKM+h/bUN+hnn fX8X00bec15J7VBKnXu8WkE7xQRCAJTm3PjXC0eyKtusPX+JNdF/cuhONiSfilOzDhv8 LOSidkJc3sdTuWCGQv/FJvk3BzPMxy3Rewl1uBEs+0c+tY1jL1D5vOEf6sGNCRh+XxFQ TW+TgAs4zY9E4cOTjsVomQh1B6Mu2V76gXVPCGZmNRgH1zy2KDMXEJQZsWtqg4cYte/N Ep9gCrBxCMo/3KPzhE9KReztDpZvF6XvSWsmIEDYcBut594DpD8d3jQwM6gTt6yr+Iy2 dumg== X-Gm-Message-State: AOAM532CP98EP89WcxScTF2KTlb6Xh5B2Ji7F0I76/FxiTL5pS69BRJq PbMRj5QhGdR4mTi6AMCtfUdLmMDpe65p9w== X-Google-Smtp-Source: ABdhPJz96bBwD81fXTgEnopwtNNriKMR5cgq1UXIzpsTbumr+ZSlTgRHJeP8YU62tvx4ZB6ijTAZvQ== X-Received: by 2002:a63:e118:: with SMTP id z24mr8682760pgh.212.1626254626492; Wed, 14 Jul 2021 02:23:46 -0700 (PDT) Received: from garuda-lan ([2601:601:9d01:3f90::b00d]) by smtp.gmail.com with ESMTPSA id s36sm2184335pgk.64.2021.07.14.02.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jul 2021 02:23:45 -0700 (PDT) From: Chris Marusich To: Luis Felipe Subject: Re: bug#31285: [PATCH 0/1] guix: Add git-fetch/impure. References: <20180427081520.28645-1-cmmarusich@gmail.com> Date: Wed, 14 Jul 2021 02:23:44 -0700 In-Reply-To: (Luis Felipe's message of "Thu, 22 Oct 2020 00:44:37 +0000") Message-ID: <87sg0hz1sv.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 31285-done Cc: 31285-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Luis Felipe writes: >> Sometimes, a Git repository may only be available via an authenticatedSS= H connection. Even in the case of repositories that only containfree softwa= re, this situation can arise for administrative orcompliance-related reason= s. How can one define a package in such asituation? > > > Correct me if I'm wrong, but I think this is possible now. All you have t= o do is pass a git-checkout record to the package source field instead of a= n origin (see the (guix git) module). For example: > > (source > (git-checkout > (url "git@gitlab.com:luis-felipe/guile-lab.git") > (commit (string-append "v" version)))) > > I'm using this for my private packages, and it seems to work. Yes, this does work. Combined with the fact that it is now possible to "guix pull" channels over SSH, there is no need for this patch any more. The "git-checkout" gexp-compiler basically does the same thing that I was trying to do (it is still "impure" in that the fetching happens outside the store), but it does it more elegantly. I'm closing this report. =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJJBAEBCAAzFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAmDurSAVHGNtbWFydXNp Y2hAZ21haWwuY29tAAoJEN1AmhXYIkadPxQP/1LNLTb64wzSUwJGapQxF2KCV0TA 1DjMUpkKFXS3GgYelLFJWAJECmmjwiSaxrp3/3ujYA4h68WHYOp66IdpRcCRcteH Q/5ealPro8+vTWIzMWf8we7GAFKvVgOiwwiHX3MNT/QQMZiGLzVFBYzX/ktvnn91 4yMi7QDZjC0Jpx79RHVh18XjF5FhOcD5F1/H7dH6aJzXb202XVuvU3YgTB8kp3q+ SvNMXdnTCUKCBALMlimlgGCDtoaC5Lh7AdjT/TIDX3SX+3ea8935N7qX1FAsC3EY tFatUZy5aWB+46NR6TaIrTpGQVOSbq66oS8WmtcfOX8Jk4u0j09huictdAGVeeAF nPInIqn/OT07DXLaW+9r+7OWc53m+fDnw++P3fEdsjWWoAoHbVdOgK/Zkm3IX/tw fv6NZuppLW5WcetibvdCKd8L4yXu+qZuUk3QqLe9qeNJ4aZsvUhi94P6pSKJ6pMq YYKRtxDVb+5g7sJ51dRUMj3pr8S+tYJ8E+3aJrzSBuXWUK04pNeeMnZmuwkN6XQQ xqld7FRMVhYxJi5pyd4Uw0xWd+z2e7O3+/PF+Gom3N0kqoUUMSR9Dledz4ZKNssk 012iftCIK3dvbPQjf2Tv5fih2de55pTT2BVBNG0XK0eDItYqg4yIo1YxN1+J1VdD 90I1FSwMoAeMYHp4 =/XPX -----END PGP SIGNATURE----- --=-=-=-- ------------=_1626254642-16806-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 27 Apr 2018 08:15:51 +0000 Received: from localhost ([127.0.0.1]:40913 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fByXn-0008Dc-Is for submit@debbugs.gnu.org; Fri, 27 Apr 2018 04:15:51 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47449) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fByXl-0008DP-Tn for submit@debbugs.gnu.org; Fri, 27 Apr 2018 04:15:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fByXf-0003qB-Ry for submit@debbugs.gnu.org; Fri, 27 Apr 2018 04:15:44 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_40,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:40711) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fByXf-0003q5-O3 for submit@debbugs.gnu.org; Fri, 27 Apr 2018 04:15:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38410) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fByXe-0006NE-CX for guix-patches@gnu.org; Fri, 27 Apr 2018 04:15:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fByXd-0003pI-C9 for guix-patches@gnu.org; Fri, 27 Apr 2018 04:15:42 -0400 Received: from mail-pg0-x22d.google.com ([2607:f8b0:400e:c05::22d]:36734) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fByXd-0003p7-6y for guix-patches@gnu.org; Fri, 27 Apr 2018 04:15:41 -0400 Received: by mail-pg0-x22d.google.com with SMTP id i6-v6so996810pgv.3 for ; Fri, 27 Apr 2018 01:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Bd9Vbq1sMlAnJog2VWSIvKFBMeCILw8V2eVgCSSNiQM=; b=naEaaefUAlp64u26aAZytaHWtfH/Q8jz2V+U5/DQ3SeeF2Get3vj0xLtDRT1SlqAZh QOsviHZPi6HoqUsW3rij+IJ6zqKjlqb/bTriGqRhe92oRvSdh9bONwmJNHkPsrT2cxeS dOXZH155PMWMrhdahSlrb12Z1VyyEOm5uNqvY11vfXUjMI3k44r9PeZHeBseiHNaSFkV a1lhtftzyqUEcc6GTeYgLaeH4JQidR/mt8va6YKAz1MXtw9//8mNMA6pRBd4smIUY09p +t9H3+hxIeyynHmQNFeIcQv+XtUFaBftRrvtA9dOAp6vlsAJylpGbCyUm6HC/v2JX9C4 tmDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Bd9Vbq1sMlAnJog2VWSIvKFBMeCILw8V2eVgCSSNiQM=; b=tjS/5d43AbHp8Dp+fqTzjyeNnom9Y/RUTq50ZdGWErj1X389qCu3MHnnv+ecU8+e9Z q6Jo7Jur+aTSJBsiPxYjX/2bh/q9fwOpTahX9RRcDE2iBdBbXWQycmzskEeccGDDsOrw 9iPrhqDmGBFy81DoZ3p8BO6cwRstsmanbtrG/nwsZipssWaQYYpZjZHQBApLhLH6fExO A3iBURwttrnlhcbwfHPpW+3AUibScVHX0Ji/IB1deSMu1gjpx4IGnWczxlLZOo+DPPZq hXHi7SJn7aA8il2wZ486LmcLdrW3bmEyek0cSTvtYei1tyZr/H0TRzxy6LUBpKyYgC5K qdSg== X-Gm-Message-State: ALQs6tDv7lVnp/ZgT+ZbAG5SCcYjAK+1IPvOu4spmq8ZNpcFUQcfzjw9 XL+b+TEw8XwyrLiPoEHGd12M9Q== X-Google-Smtp-Source: AB8JxZrEJulG54ysgqP3H1bibMtksDmzm37PnnZytI5UeX29s+KUeNFrf3waCdcHlaMwgrfUpoxmKg== X-Received: by 10.98.214.218 with SMTP id a87mr1346146pfl.200.1524816939888; Fri, 27 Apr 2018 01:15:39 -0700 (PDT) Received: from garuda.local.net ([2601:602:9d02:4725:6495:ba21:1ebe:620a]) by smtp.gmail.com with ESMTPSA id u72-v6sm1569941pgb.16.2018.04.27.01.15.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Apr 2018 01:15:38 -0700 (PDT) From: Chris Marusich To: guix-patches@gnu.org Subject: [PATCH 0/1] guix: Add git-fetch/impure. Date: Fri, 27 Apr 2018 01:15:20 -0700 Message-Id: <20180427081520.28645-1-cmmarusich@gmail.com> X-Mailer: git-send-email 2.17.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit Cc: Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi Guix! Sometimes, a Git repository may only be available via an authenticated SSH connection. Even in the case of repositories that only contain free software, this situation can arise for administrative or compliance-related reasons. How can one define a package in such a situation? This patch adds a new origin method, git-fetch/impure, which solves that problem. Specifically, git-fetch/impure creates a fixed-output derivation that fetches the Git repository outside of a derivation, in the environment of the invoking user. In particular, this enables SSH to communicate with the user's SSH agent, which in turn allows Git to fetch the repository over an authenticated SSH connection. In addition, because it is a fixed-output derivation, the output of a successful git-fetch/impure is guaranteed to be identical to the output of a pure git-fetch for any given commit. Here's a simple example: (define-public guix-over-ssh (package (inherit guix) (name "guix-over-ssh") (source (origin (inherit (package-source guix)) (method git-fetch/impure) (uri (git-reference (inherit (origin-uri (package-source guix))) (url "ssh://marusich@git.sv.gnu.org:/srv/git/guix.git"))))))) In this particular example, my username appears in the package definition, but there is no reason why that has to be so. In many systems, it is possible to grant access to multiple users with different SSH keys under a single shared user name. And in other systems, an automated build system might need to fetch sources using its own unique system user name and SSH key. All in all, I think this is pretty useful. It enables developers to define packages in environments where authenticated access to Git repositories is required. Please let me know what you think! Chris Marusich (1): guix: Add git-fetch/impure. doc/guix.texi | 24 +++++++ guix/git-download.scm | 150 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 174 insertions(+) -- 2.17.0 ------------=_1626254642-16806-1--