From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 05 03:07:48 2019 Received: (at submit) by debbugs.gnu.org; 5 Apr 2019 07:07:48 +0000 Received: from localhost ([127.0.0.1]:45021 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCIx2-0007Cc-IW for submit@debbugs.gnu.org; Fri, 05 Apr 2019 03:07:48 -0400 Received: from eggs.gnu.org ([209.51.188.92]:48452) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCIx0-0007CP-58 for submit@debbugs.gnu.org; Fri, 05 Apr 2019 03:07:46 -0400 Received: from lists.gnu.org ([209.51.188.17]:36838) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hCIwu-0002Ux-Ci for submit@debbugs.gnu.org; Fri, 05 Apr 2019 03:07:40 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34579) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hCIwt-0005eH-13 for guix-patches@gnu.org; Fri, 05 Apr 2019 03:07:40 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hCIwp-0002Qh-77 for guix-patches@gnu.org; Fri, 05 Apr 2019 03:07:37 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]:45859) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hCIwn-0002OG-OW for guix-patches@gnu.org; Fri, 05 Apr 2019 03:07:34 -0400 Received: by mail-pf1-x42d.google.com with SMTP id e24so2746892pfi.12 for ; Fri, 05 Apr 2019 00:07:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:mime-version:date:subject:cc:to:message-id; bh=cBm637cY30uCX8EyLPsK2czaWe8fUnnYEqyA4XvlM2M=; b=Kte4ze3VWUzm4JaXcH0fpbmivP9d7k/Oh24TAOM3t1i4Dl2t4MyE6rcjmnJFtzeunT 2q1XUmg+7zQepm3jhPycQQR3JOqCKD9ZOpO9y9yCaJoZVu37NRsUiHPSZXoMlGz+MXz/ L1WRbufifVwNGmkVr6p+S8ZJcfZvro953hiI39zt+7QXfUwm631kFd/WL+r1F63a84MB PtwKv1fPncfkzfHXMFkpDOHqboLJYUUcyW6l17XdQdUsy1y97relRhY3XZdj/Ir8ksM9 g08A67h4LzEFZdV0IgoRBMVeGjpJVCScLXUXnPOUGZFm7K7bfQw1Ix6TEFebxa32hSk7 bBaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:date:subject:cc:to:message-id; bh=cBm637cY30uCX8EyLPsK2czaWe8fUnnYEqyA4XvlM2M=; b=VphZnzKybNKHGSCFkBKhCSpo0BcXjqD+XBOah09hVcAGqSxQRB+1N8L+09KKAIzUvs d3ZoKCwlFkkgaXeqMdwfHlbtL/iS3ATMJcivgSTgjXhs9OHLmfbxTjPQX4a34ylytkMl eHmf33O+OGFszVj8049Qyu0YP/hXC8lk3k9CdLw8Bqyx8tmkBQi+afZINjlTeotJUUZH 45STwI94l32DUFzweXBUjMxViw0XcrHu66LfoWbqmDBtYj5vY8HPzKCkgp3ITdxXJo4J yVFScmxeBza4zv7ED+2OGpda+Ww5DLG+Rk++b7dyiXaRn17k19GZz888R3Js1C4Bdoul 3huw== X-Gm-Message-State: APjAAAXx8WQ3CPWoNIG/DmiAdr/aGcXkvrUhYlxldgxf1ukO9DIKCM6o eSQHDlpe/kozmdazsRECm+lHY9eJWrc= X-Google-Smtp-Source: APXvYqwpiKDwIx/LcSFOhJ9qT3lXkV2rFbpeITgtAVEHIBBsxiIjrxo8JnErlowxgXT/EepSefmlCQ== X-Received: by 2002:a62:4852:: with SMTP id v79mr10691316pfa.72.1554448052325; Fri, 05 Apr 2019 00:07:32 -0700 (PDT) Received: from ?IPv6:2601:602:9a00:1784:6544:9ffe:4d1:80e3? ([2601:602:9a00:1784:6544:9ffe:4d1:80e3]) by smtp.gmail.com with ESMTPSA id e23sm30331646pfd.11.2019.04.05.00.07.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Apr 2019 00:07:31 -0700 (PDT) From: Ivan Petkov Content-Type: multipart/mixed; boundary="Apple-Mail=_39CC20B5-321B-4A48-856D-4075DA53EF76" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Fri, 5 Apr 2019 00:07:27 -0700 Subject: [PATCH] build-system/cargo: refactor phases to successfully build To: guix-patches@gnu.org Message-Id: X-Mailer: Apple Mail (2.3445.9.1) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::42d X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: submit Cc: Chris Marusich X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --Apple-Mail=_39CC20B5-321B-4A48-856D-4075DA53EF76 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 This patch refactors the cargo-build-system builder code to correctly = build imported Rust crates. Specifically the changes: * Do not rely on a Cargo.lock presence to determine installation. Cargo = will automatically create the file at its first invocation, so instead we = check the manifest for any executable targets before attempting an = installation * Do not attempt to modify the Cargo.toml file. There are many system = specific package in crates.io (e.g. for Windows, Redox, Fuschia, WASM, etc.) and attempting to keep up with what crates must be patched out is futile. * The build phases will honor a skip-build? flag which allows for short-circuiting for optional packages which require nightly features or = cannot be built for the current platform. Changes which still need to be done: * Update the host-side code to expand transitive inputs: cargo requires = that all transitive crate dependencies are present in its (vendored) index, = but doing so by hand in the package definitions will become unwieldy. * Update the host-side code to detect any "circular" dependencies which = can result from a naive import Unfortunately there isn't a good way to test this patch at the moment. Importing a non-trivial crate requires a lot of manual resolution, = especially with the points above remaining unimplemented. If someone would really like to see the input package definitions I was = using to test, I'd be happy to share, though I'd advise that it's pretty = hacked up for my own convoluted testing at the moment. =E2=80=94Ivan --Apple-Mail=_39CC20B5-321B-4A48-856D-4075DA53EF76 Content-Disposition: attachment; filename=0001-build-system-cargo-refactor-phases-to-successfully-b.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0001-build-system-cargo-refactor-phases-to-successfully-b.patch" Content-Transfer-Encoding: quoted-printable =46rom=2009c2c9e76e8c9d12bd230d953f9d197efbfb1079=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Ivan=20Petkov=20=0A= Date:=20Tue,=202=20Apr=202019=2003:02:51=20-0700=0ASubject:=20[PATCH]=20= build-system/cargo:=20refactor=20phases=20to=20successfully=20build=0A=0A= *=20guix/build-system/cargo.scm=20(%cargo-build-system-modules):=0AAdd=20= (json=20parser)=0A(cargo-build):=0A[vendor-dir]:=20Define=20flag=20and=20= pass=20it=20to=20builder=20code.=0A[cargo-test-flags]:=20Likewise.=0A= [skip-build?]:=20Likewise.=0A*=20guix/build/cargo-build/system.scm=20= (#:use-module):=20use=20(json=20parser)=0A(package-name->crate-name):=20= Delete=20it=0A(manifest-targets):=20Add=20it=0A(has-executable-target?):=20= Add=20it=0A(configure):=20Add=20#:vendor-dir=20name=20and=20use=20it.=0A= Don't=20touch=20Cargo.toml.=0ADon't=20symlink=20to=20duplicate=20inputs.=0A= Remove=20useless=20registry=20line=20from=20cargo=20config.=0ADefine=20= RUSTFLAGS=20to=20lift=20lint=20restrictions.=0A(build):=20Add=20= #:skip-build?=20flag=20and=20use=20it.=0A(check):=20Likewise.=0AAdd=20= #:cargo-tset-flags=20and=20pass=20it=20to=20cargo=0A(install):=20Factor=20= source=20logic=20to=20install-source.=0ADefine=20#:skip-build?=20flag=20= and=20use=20it.=0AOnly=20install=20if=20executable=20targets=20are=20= present.=0A(install-source):=20Copy=20entire=20crate=20directory=20not=20= just=20src.=0A[generate-checksums]=20pass=20dummy=20file=20for=20unused=20= second=20argument=0A(%standard-phases):=20Add=20install-source=20phase=0A= ---=0A=20guix/build-system/cargo.scm=20=20=20=20=20=20=20|=20=20=207=20= ++=0A=20guix/build/cargo-build-system.scm=20|=20151=20= +++++++++++++++++-------------=0A=202=20files=20changed,=2093=20= insertions(+),=2065=20deletions(-)=0A=0Adiff=20--git=20= a/guix/build-system/cargo.scm=20b/guix/build-system/cargo.scm=0Aindex=20= 7ff4e90f71..38290597ee=20100644=0A---=20a/guix/build-system/cargo.scm=0A= +++=20b/guix/build-system/cargo.scm=0A@@=20-59,13=20+59,17=20@@=20to=20= NAME=20and=20VERSION."=0A=20(define=20%cargo-build-system-modules=0A=20=20= =20;;=20Build-side=20modules=20imported=20by=20default.=0A=20=20=20= `((guix=20build=20cargo-build-system)=0A+=20=20=20=20(json=20parser)=0A=20= =20=20=20=20,@%cargo-utils-modules))=0A=20=0A=20(define*=20(cargo-build=20= store=20name=20inputs=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20#:key=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20(tests?=20#t)=0A=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20(test-target=20#f)=0A+=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(vendor-dir=20= "guix-vendor")=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20(cargo-build-flags=20''("--release"))=0A+=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(cargo-test-flags=20= ''("--release"))=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20(skip-build?=20#f)=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20(phases=20'(@=20(guix=20build=20= cargo-build-system)=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20%standard-phases))=0A= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= (outputs=20'("out"))=0A@@=20-90,7=20+94,10=20@@=20to=20NAME=20and=20= VERSION."=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20source))=0A=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20#:system=20,system=0A=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#:test-target=20= ,test-target=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20#:vendor-dir=20,vendor-dir=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20#:cargo-build-flags=20,cargo-build-flags=0A+=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#:cargo-test-flags=20= ,cargo-test-flags=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20#:skip-build?=20,skip-build?=0A=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20#:tests?=20,tests?=0A=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20#:phases=20,phases=0A=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#:outputs=20%outputs=0Adiff=20= --git=20a/guix/build/cargo-build-system.scm=20= b/guix/build/cargo-build-system.scm=0Aindex=2020087fa6c4..8616873ce8=20= 100644=0A---=20a/guix/build/cargo-build-system.scm=0A+++=20= b/guix/build/cargo-build-system.scm=0A@@=20-1,6=20+1,7=20@@=0A=20;;;=20= GNU=20Guix=20---=20Functional=20package=20management=20for=20GNU=0A=20= ;;;=20Copyright=20=C2=A9=202016=20David=20Craven=20=0A=20= ;;;=20Copyright=20=C2=A9=202017=20Mathieu=20Othacehe=20= =0A+;;;=20Copyright=20=C2=A9=202019=20Ivan=20= Petkov=20=0A=20;;;=0A=20;;;=20This=20file=20is=20= part=20of=20GNU=20Guix.=0A=20;;;=0A@@=20-26,6=20+27,7=20@@=0A=20=20=20= #:use-module=20(ice-9=20ftw)=0A=20=20=20#:use-module=20(ice-9=20format)=0A= =20=20=20#:use-module=20(ice-9=20match)=0A+=20=20#:use-module=20(json=20= parser)=0A=20=20=20#:use-module=20(srfi=20srfi-1)=0A=20=20=20= #:use-module=20(srfi=20srfi-26)=0A=20=20=20#:export=20(%standard-phases=0A= @@=20-37,81=20+39,88=20@@=0A=20;;=0A=20;;=20Code:=0A=20=0A-;;=20FIXME:=20= Needs=20to=20be=20parsed=20from=20url=20not=20package=20name.=0A-(define=20= (package-name->crate-name=20name)=0A-=20=20"Return=20the=20crate=20name=20= of=20NAME."=0A-=20=20(match=20(string-split=20name=20#\-)=0A-=20=20=20=20= (("rust"=20rest=20...)=0A-=20=20=20=20=20(string-join=20rest=20"-"))=0A-=20= =20=20=20(_=20#f)))=0A+(define=20(manifest-targets)=0A+=20=20"Extract=20= all=20targets=20from=20the=20Cargo.toml=20manifest"=0A+=20=20(let*=20= ((port=20(open-input-pipe=20"cargo=20read-manifest"))=0A+=20=20=20=20=20=20= =20=20=20(data=20(json->scm=20port))=0A+=20=20=20=20=20=20=20=20=20= (targets=20(hash-ref=20data=20"targets"=20'())))=0A+=20=20=20=20= (close-port=20port)=0A+=20=20=20=20targets))=0A=20=0A-(define*=20= (configure=20#:key=20inputs=20#:allow-other-keys)=0A+(define=20= (has-executable-target?)=0A+=20=20"Check=20if=20the=20current=20cargo=20= project=20declares=20any=20binary=20targets."=0A+=20=20(let*=20((bin?=20= (lambda=20(kind)=20(string=3D?=20kind=20"bin")))=0A+=20=20=20=20=20=20=20= =20=20(get-kinds=20(lambda=20(dep)=20(hash-ref=20dep=20"kind")))=0A+=20=20= =20=20=20=20=20=20=20(bin-dep?=20(lambda=20(dep)=20(find=20bin?=20= (get-kinds=20dep)))))=0A+=20=20=20=20(find=20bin-dep?=20= (manifest-targets))))=0A+=0A+(define*=20(configure=20#:key=20inputs=0A+=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(vendor-dir=20= "guix-vendor")=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20#:allow-other-keys)=0A=20=20=20"Replace=20Cargo.toml=20[dependencies]=20= section=20with=20guix=20inputs."=0A-=20=20;;=20Make=20sure=20Cargo.toml=20= is=20writeable=20when=20the=20crate=20uses=20git-fetch.=0A-=20=20(chmod=20= "Cargo.toml"=20#o644)=0A=20=20=20(chmod=20"."=20#o755)=0A-=20=20(if=20= (not=20(file-exists?=20"vendor"))=0A-=20=20=20=20(if=20(not=20= (file-exists?=20"Cargo.lock"))=0A-=20=20=20=20=20=20(begin=0A-=20=20=20=20= =20=20=20=20(substitute*=20"Cargo.toml"=0A-=20=20=20=20=20=20=20=20=20=20= ((".*32-sys.*")=20"=0A-")=0A-=20=20=20=20=20=20=20=20=20=20= ((".*winapi.*")=20"=0A-")=0A-=20=20=20=20=20=20=20=20=20=20= ((".*core-foundation.*")=20"=0A-"))=0A-=20=20=20=20=20=20=20=20;;=20= Prepare=20one=20new=20directory=20with=20all=20the=20required=20= dependencies.=0A-=20=20=20=20=20=20=20=20;;=20It's=20necessary=20to=20do=20= this=20(instead=20of=20just=20using=20/gnu/store=20as=20the=0A-=20=20=20=20= =20=20=20=20;;=20directory)=20because=20we=20want=20to=20hide=20the=20= libraries=20in=20subdirectories=0A-=20=20=20=20=20=20=20=20;;=20=20=20= share/rust-source/...=20instead=20of=20polluting=20the=20user's=20= profile=20root.=0A-=20=20=20=20=20=20=20=20(mkdir=20"vendor")=0A-=20=20=20= =20=20=20=20=20(for-each=0A-=20=20=20=20=20=20=20=20=20=20(match-lambda=0A= -=20=20=20=20=20=20=20=20=20=20=20=20((name=20.=20path)=0A-=20=20=20=20=20= =20=20=20=20=20=20=20=20(let=20((crate=20(package-name->crate-name=20= name)))=0A-=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(when=20(and=20= crate=20path)=0A-=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= (match=20(string-split=20(basename=20path)=20#\-)=0A-=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20((_=20...=20version)=0A-=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(symlink=20(string-append=20= path=20"/share/rust-source")=0A-=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(string-append=20"vendor/"=20= (basename=20path)))))))))=0A-=20=20=20=20=20=20=20=20=20=20inputs)=0A-=20= =20=20=20=20=20=20=20;;=20Configure=20cargo=20to=20actually=20use=20this=20= new=20directory.=0A-=20=20=20=20=20=20=20=20(mkdir-p=20".cargo")=0A-=20=20= =20=20=20=20=20=20(let=20((port=20(open-file=20".cargo/config"=20"w"=20= #:encoding=20"utf-8")))=0A-=20=20=20=20=20=20=20=20=20=20(display=20"=0A= +=20=20;;=20Prepare=20one=20new=20directory=20with=20all=20the=20= required=20dependencies.=0A+=20=20;;=20It's=20necessary=20to=20do=20this=20= (instead=20of=20just=20using=20/gnu/store=20as=20the=0A+=20=20;;=20= directory)=20because=20we=20want=20to=20hide=20the=20libraries=20in=20= subdirectories=0A+=20=20;;=20share/rust-source/...=20instead=20of=20= polluting=20the=20user's=20profile=20root.=0A+=20=20(mkdir-p=20= vendor-dir)=0A+=20=20(for-each=0A+=20=20=20=20(match-lambda=0A+=20=20=20=20= =20=20((name=20.=20path)=0A+=20=20=20=20=20=20=20(let*=20((rust-share=20= (string-append=20path=20"/share/rust-source"))=0A+=20=20=20=20=20=20=20=20= =20=20=20=20=20=20(basepath=20(basename=20path))=0A+=20=20=20=20=20=20=20= =20=20=20=20=20=20=20(link-dir=20(string-append=20vendor-dir=20"/"=20= basepath)))=0A+=20=20=20=20=20=20=20=20=20(and=20(file-exists?=20= rust-share)=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20;;=20Gracefully=20= handle=20duplicate=20inputs=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20= (not=20(file-exists?=20link-dir))=0A+=20=20=20=20=20=20=20=20=20=20=20=20= =20=20(symlink=20rust-share=20link-dir)))))=0A+=20=20=20=20inputs)=0A+=20= =20;;=20Configure=20cargo=20to=20actually=20use=20this=20new=20= directory.=0A+=20=20(mkdir-p=20".cargo")=0A+=20=20(let=20((port=20= (open-file=20".cargo/config"=20"w"=20#:encoding=20"utf-8")))=0A+=20=20=20= =20(display=20"=0A=20[source.crates-io]=0A-registry=20=3D=20= 'https://github.com/rust-lang/crates.io-index'=0A=20replace-with=20=3D=20= 'vendored-sources'=0A=20=0A=20[source.vendored-sources]=0A=20directory=20= =3D=20'"=20port)=0A-=20=20=20=20=20=20=20=20=20=20(display=20(getcwd)=20= port)=0A-=20=20=20=20=20=20=20=20=20=20(display=20"/vendor"=20port)=0A-=20= =20=20=20=20=20=20=20=20=20(display=20"'=0A+=20=20=20=20(display=20= (string-append=20(getcwd)=20"/"=20vendor-dir)=20port)=0A+=20=20=20=20= (display=20"'=0A=20"=20port)=0A-=20=20=20=20=20=20=20=20=20=20= (close-port=20port)))))=0A-=20=20=20=20(setenv=20"CC"=20(string-append=20= (assoc-ref=20inputs=20"gcc")=20"/bin/gcc"))=0A+=20=20=20=20(close-port=20= port))=0A=20=0A-=20=20=20=20;(setenv=20"CARGO_HOME"=20"/gnu/store")=0A-=20= =20=20=20;=20(setenv=20"CMAKE_C_COMPILER"=20cc)=0A+=20=20;;=20Lift=20= restriction=20on=20any=20lints:=20a=20crate=20author=20may=20have=20= decided=20to=20opt=0A+=20=20;;=20into=20stricter=20lints=20(e.g.=20= #![deny(warnings)])=20during=20their=20own=20builds=0A+=20=20;;=20but=20= we=20don't=20want=20any=20build=20failures=20that=20could=20be=20caused=20= later=20by=0A+=20=20;;=20upgrading=20the=20compiler=20for=20example.=0A+=20= =20(setenv=20"RUSTFLAGS"=20"--cap-lints=20allow")=0A+=20=20(setenv=20= "CC"=20(string-append=20(assoc-ref=20inputs=20"gcc")=20"/bin/gcc"))=0A=20= =20=20#t)=0A=20=0A-(define*=20(build=20#:key=20(cargo-build-flags=20= '("--release"))=0A+(define*=20(build=20#:key=0A+=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20skip-build?=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20(cargo-build-flags=20'("--release"))=0A=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20#:allow-other-keys)=0A=20=20=20"Build=20a=20= given=20Cargo=20package."=0A-=20=20(zero?=20(apply=20system*=20`("cargo"=20= "build"=20,@cargo-build-flags))))=0A+=20=20(or=20skip-build?=0A+=20=20=20= =20=20=20(zero?=20(apply=20system*=20`("cargo"=20"build"=20= ,@cargo-build-flags)))))=0A=20=0A-(define*=20(check=20#:key=20tests?=20= #:allow-other-keys)=0A+(define*=20(check=20#:key=0A+=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20skip-build?=0A+=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20tests?=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= (cargo-test-flags=20'("--release"))=0A+=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20#:allow-other-keys)=0A=20=20=20"Run=20tests=20for=20a=20= given=20Cargo=20package."=0A-=20=20(if=20(and=20tests?=20(file-exists?=20= "Cargo.lock"))=0A-=20=20=20=20=20=20(zero?=20(system*=20"cargo"=20= "test"))=0A-=20=20=20=20=20=20#t))=0A+=20=20(or=20skip-build?=0A+=20=20=20= =20=20=20(and=0A+=20=20=20=20=20=20=20=20tests?=0A+=20=20=20=20=20=20=20=20= (zero?=20(apply=20system*=20`("cargo"=20"test"=20= ,@cargo-test-flags))))))=0A=20=0A=20(define=20(touch=20file-name)=0A=20=20= =20(call-with-output-file=20file-name=20(const=20#t)))=0A=20=0A-(define*=20= (install=20#:key=20inputs=20outputs=20#:allow-other-keys)=0A-=20=20= "Install=20a=20given=20Cargo=20package."=0A+(define*=20(install-source=20= #:key=20inputs=20outputs=20#:allow-other-keys)=0A+=20=20"Install=20the=20= source=20for=20a=20given=20Cargo=20package."=0A=20=20=20(let*=20((out=20= (assoc-ref=20outputs=20"out"))=0A=20=20=20=20=20=20=20=20=20=20(src=20= (assoc-ref=20inputs=20"source"))=0A=20=20=20=20=20=20=20=20=20=20(rsrc=20= (string-append=20(assoc-ref=20outputs=20"src")=0A@@=20-122,22=20+131,34=20= @@=20directory=20=3D=20'"=20port)=0A=20=20=20=20=20;;=20Until=20this=20= changes=20we=20are=20working=20around=20this=20by=0A=20=20=20=20=20;;=20= distributing=20crates=20as=20source=20and=20replacing=0A=20=20=20=20=20= ;;=20references=20in=20Cargo.toml=20with=20store=20paths.=0A-=20=20=20=20= (copy-recursively=20"src"=20(string-append=20rsrc=20"/src"))=0A+=20=20=20= =20(copy-recursively=20"."=20rsrc)=0A=20=20=20=20=20(touch=20= (string-append=20rsrc=20"/.cargo-ok"))=0A-=20=20=20=20= (generate-checksums=20rsrc=20src)=0A+=20=20=20=20(generate-checksums=20= rsrc=20"/dev/null")=0A=20=20=20=20=20(install-file=20"Cargo.toml"=20= rsrc)=0A-=20=20=20=20;;=20When=20the=20package=20includes=20executables=20= we=20install=0A-=20=20=20=20;;=20it=20using=20cargo=20install.=20This=20= fails=20when=20the=20crate=0A-=20=20=20=20;;=20doesn't=20contain=20an=20= executable.=0A-=20=20=20=20(if=20(file-exists?=20"Cargo.lock")=0A-=20=20=20= =20=20=20=20=20(zero?=20(system*=20"cargo"=20"install"=20"--root"=20= out))=0A-=20=20=20=20=20=20=20=20(begin=0A-=20=20=20=20=20=20=20=20=20=20= (mkdir=20out)=0A-=20=20=20=20=20=20=20=20=20=20#t))))=0A+=20=20=20=20= #t))=0A+=0A+(define*=20(install=20#:key=20inputs=20outputs=20skip-build?=20= #:allow-other-keys)=0A+=20=20"Install=20a=20given=20Cargo=20package."=0A= +=20=20(let*=20((out=20(assoc-ref=20outputs=20"out")))=0A+=20=20=20=20= (mkdir-p=20out)=0A+=0A+=20=20=20=20;;=20Make=20cargo=20reuse=20all=20the=20= artifacts=20we=20just=20built=20instead=0A+=20=20=20=20;;=20of=20= defaulting=20to=20making=20a=20new=20temp=20directory=0A+=20=20=20=20= (setenv=20"CARGO_TARGET_DIR"=20"./target")=0A+=20=20=20=20;;=20Force=20= cargo=20to=20honor=20our=20.cargo/config=20definitions=0A+=20=20=20=20;;=20= https://github.com/rust-lang/cargo/issues/6397=0A+=20=20=20=20(setenv=20= "CARGO_HOME"=20".")=0A+=0A+=20=20=20=20;;=20Only=20install=20crates=20= which=20include=20binary=20targets,=0A+=20=20=20=20;;=20otherwise=20= cargo=20will=20raise=20an=20error.=0A+=20=20=20=20(or=20skip-build?=0A+=20= =20=20=20=20=20=20=20(not=20(has-executable-target?))=0A+=20=20=20=20=20=20= =20=20(zero?=20(system*=20"cargo"=20"install"=20"--path"=20"."=20= "--root"=20out)))))=0A=20=0A=20(define=20%standard-phases=0A=20=20=20= (modify-phases=20gnu:%standard-phases=0A=20=20=20=20=20(delete=20= 'bootstrap)=0A+=20=20=20=20(add-before=20'configure=20'install-source=20= install-source)=0A=20=20=20=20=20(replace=20'configure=20configure)=0A=20= =20=20=20=20(replace=20'build=20build)=0A=20=20=20=20=20(replace=20= 'check=20check)=0A--=20=0A2.21.0=0A=0A= --Apple-Mail=_39CC20B5-321B-4A48-856D-4075DA53EF76-- From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 06:33:44 2019 Received: (at 35155) by debbugs.gnu.org; 6 Apr 2019 10:33:44 +0000 Received: from localhost ([127.0.0.1]:46598 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCids-0008UF-5t for submit@debbugs.gnu.org; Sat, 06 Apr 2019 06:33:44 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:46146) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCidp-0008U7-R1 for 35155@debbugs.gnu.org; Sat, 06 Apr 2019 06:33:42 -0400 Received: from localhost (178.113.210.35.wireless.dyn.drei.com [178.113.210.35]) by dd26836.kasserver.com (Postfix) with ESMTPSA id 58ED13363850; Sat, 6 Apr 2019 12:33:29 +0200 (CEST) Date: Sat, 6 Apr 2019 12:32:34 +0200 From: Danny Milosavljevic To: Ivan Petkov Subject: Re: [bug#35155] [PATCH] build-system/cargo: refactor phases to successfully build Message-ID: <20190406123234.3b687936@scratchpost.org> In-Reply-To: References: X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/CAVV4EuF7Uk6rpqxRkfCot_"; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 35155 Cc: Chris Marusich , 35155@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --Sig_/CAVV4EuF7Uk6rpqxRkfCot_ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi Ivan, good idea. (Finally the hack with the Cargo.lock is gone :) ) >There are many system specific > package in crates.io (e.g. for Windows, Redox, Fuschia, WASM, etc.) and > attempting to keep up with what crates must be patched out is futile. I agree. > * The build phases will honor a skip-build? flag which allows for > short-circuiting for optional packages which require nightly features or = cannot > be built for the current platform. Ok, I guess. > Changes which still need to be done: > * Update the host-side code to expand transitive inputs: cargo requires t= hat > all transitive crate dependencies are present in its (vendored) index, but > doing so by hand in the package definitions will become unwieldy. Yeah. Let's do that in an extra patch. > * Update the host-side code to detect any "circular" dependencies which c= an > result from a naive import Yeah. > Unfortunately there isn't a good way to test this patch at the moment. > Importing a non-trivial crate requires a lot of manual resolution, especi= ally > with the points above remaining unimplemented. > If someone would really like to see the input package definitions I was u= sing > to test, I'd be happy to share, though I'd advise that it's pretty hacked= up > for my own convoluted testing at the moment. No, I can use my own set of hacked-together package definition to test it. (The state of my hacked-together package definitions is not good enough to = merge :) ) --Sig_/CAVV4EuF7Uk6rpqxRkfCot_ Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlyogEIACgkQ5xo1VCww uqVorQf+K7YVMFDpuezp75tb9LavtRC0oBAbPWWUqCzfH+gTeK5+rWKx1jdN6Qv2 a996QlYSkCaSMPr1MoVHe01GCWaHPqjx8APhOo4LogT79Brm/7YILzR8ViwdDyJl nozeiswRq1cj00iv85Blx1FBMz3Lq0Ha+bxtP53UdxlrQSwZ6CASYPWQdNMZFNSA K97Dc2FBYpdFkJ8Nq3NXovhx8onPZmbMidX5qsOBJJBUleWQna5NHFAFO85Fx1uw QnrxUTtSh5TGPnn87+XGhUIM03Z1svYqV9iEZk7Gw9lH3hE7EjJIB01xzAPWLUkm WiBRK1BBR6sCerUxbxw2dNqSPSqbMA== =W4mg -----END PGP SIGNATURE----- --Sig_/CAVV4EuF7Uk6rpqxRkfCot_-- From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 06:41:33 2019 Received: (at 35155) by debbugs.gnu.org; 6 Apr 2019 10:41:33 +0000 Received: from localhost ([127.0.0.1]:46602 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCilR-0000EP-2K for submit@debbugs.gnu.org; Sat, 06 Apr 2019 06:41:33 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:46692) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCilQ-0000EI-7J for 35155@debbugs.gnu.org; Sat, 06 Apr 2019 06:41:32 -0400 Received: from localhost (178.113.210.35.wireless.dyn.drei.com [178.113.210.35]) by dd26836.kasserver.com (Postfix) with ESMTPSA id C6A3B336288D; Sat, 6 Apr 2019 12:41:30 +0200 (CEST) Date: Sat, 6 Apr 2019 12:41:25 +0200 From: Danny Milosavljevic To: Ivan Petkov Subject: Re: [bug#35155] [PATCH] build-system/cargo: refactor phases to successfully build Message-ID: <20190406124125.24f6ff7c@scratchpost.org> In-Reply-To: References: X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/5GyKRLtogAnj1YrhEbrowe9"; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 35155 Cc: Chris Marusich , 35155@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --Sig_/5GyKRLtogAnj1YrhEbrowe9 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable >+ (generate-checksums rsrc "/dev/null") Hmm, is the "package" entry in the final file ".cargo-checksums.json" usele= ss? Can it be left off completely? --Sig_/5GyKRLtogAnj1YrhEbrowe9 Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlyoglUACgkQ5xo1VCww uqXK1gf/Tp835djTXUkxdgDqoe7HrztGjwKuqxsdGU2x0v3rPBO530HXlGvxeGXx 9D5jYfHVNSOTBnSD2bFK5Un1/au0KGKAD7FvFQCh0mcFkMzrRxgL9kg2kfmZ+7lo 2tOQDFwHPwAxrkFLLGayusy7zqroB1RdZJuZuyjO0haudg9Phfa8Xp/EIn84+rWI BMp1Z0NJnxCsrAMfyAMMu7mXXmMTOktfGc9cHgaAVn4joQg0+tLd5NoG2nd69a1u Qi1wgAMXM+DK9gT5hCCnHELUgXGEdYl8/have3fS8YusuMf6n9saoFPYR6nT38sv 7deDqybHZKBnhJ40z3VEuwp0yuGZwg== =MFSF -----END PGP SIGNATURE----- --Sig_/5GyKRLtogAnj1YrhEbrowe9-- From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 12:12:49 2019 Received: (at 35155) by debbugs.gnu.org; 6 Apr 2019 16:12:49 +0000 Received: from localhost ([127.0.0.1]:47418 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCnw1-0008Dd-82 for submit@debbugs.gnu.org; Sat, 06 Apr 2019 12:12:49 -0400 Received: from mail-pg1-f175.google.com ([209.85.215.175]:39000) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCnvz-0008DR-Ox for 35155@debbugs.gnu.org; Sat, 06 Apr 2019 12:12:48 -0400 Received: by mail-pg1-f175.google.com with SMTP id k3so4728539pga.6 for <35155@debbugs.gnu.org>; Sat, 06 Apr 2019 09:12:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=eTgRldYAQVCPMQYOl3yPgYzZ/iJFNvk/985EyJ1Md+k=; b=uS4z80ShkV3AguqCZaF7lJ6j+eoS9E7kK/KSUVfuYuYZoJCNm3rwrLFIaL9dr33Ly9 CefaTdYRXe3S0B0kkC+r6XUeWOwz15e6eWZOVxJu+ZvmQiMb139dfZTPmnk+DTrcUpX5 G1cvt4P1qiyrw4t7N51Ak/v+IX7XZFAKame33DnZxbgJ+KmtrmxQaICDyNw4FXlATDSg hHa4XiKPbnopK5OAK7I85856fLjI5JLCIu+RRugnOTs5GcSzy+HhiQvM12de2H0JmXUt uwj+6ljosX2NgaQRaHVQK1kQzcF3X2sJeGpNptbF80grWv9VMv3nu3qA0tWQsh3lIX4U 4JEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=eTgRldYAQVCPMQYOl3yPgYzZ/iJFNvk/985EyJ1Md+k=; b=XbcKTZWUuqNoLExux+zECr50simyH9GSA7pASwrtFzkR6iAGSqxNNxr28TeqplGAWi c7fmpiymHEi25Y7AxtZd7QZCFQVF5RXjj9X/X3zxlhgJjHNgOzKNIOuVdD6nlGYe9uyE +nMu1m5U7tqXW4n584ckcxfLxG5QanKsE1wcMBnDHtdGdY9ojBATa3z/f7/YxUiUU4gK IkZuJPColJJxkA/TzKs2z01ZQgdLazmdKlAaXshqS+SDebw/l72vcrwM0+z/tQU0iKvX Ee6xI/rfKIVpGiBHJxHts/eOgiPfRgTRhAcMqxjT6Tcckz00RgUWckSjpojFF9z4LVEw Inig== X-Gm-Message-State: APjAAAUBZH6l3In9czDDxMMsQPfQJRtu6JyVbl+esqG6ST69OSa5UXuF yP4oNqkqbfZ4tbFeuCNITLw= X-Google-Smtp-Source: APXvYqxm81kvC9WIYU95GSEg4OmI0XMdBSRX7OXZeoAs+eRzBH1ZEd1c1eWQEO7Wlet7Ab8Ys5nuCQ== X-Received: by 2002:a63:5466:: with SMTP id e38mr17925357pgm.340.1554567162123; Sat, 06 Apr 2019 09:12:42 -0700 (PDT) Received: from ?IPv6:2601:602:9a00:1784:25c5:82ed:a22:9ca0? ([2601:602:9a00:1784:25c5:82ed:a22:9ca0]) by smtp.gmail.com with ESMTPSA id y20sm16874215pfe.188.2019.04.06.09.12.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 06 Apr 2019 09:12:41 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: [bug#35155] [PATCH] build-system/cargo: refactor phases to successfully build From: Ivan Petkov In-Reply-To: <20190406124125.24f6ff7c@scratchpost.org> Date: Sat, 6 Apr 2019 09:12:37 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20190406124125.24f6ff7c@scratchpost.org> To: Danny Milosavljevic X-Mailer: Apple Mail (2.3445.9.1) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 35155 Cc: Chris Marusich , 35155@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Danny, > On Apr 6, 2019, at 3:41 AM, Danny Milosavljevic = wrote: >=20 >> + (generate-checksums rsrc "/dev/null") >=20 > Hmm, is the "package" entry in the final file ".cargo-checksums.json" = useless? > Can it be left off completely? Yes, it appears that cargo expects there to be a =E2=80=9Cpackage=E2=80=9D= hash definition in .cargo-checksum.json, but it doesn=E2=80=99t verify the contents (it = only checks if they=E2=80=99ve changed since the last time it updated its Cargo.lock file). Since this procedure is directly used when building rust-1.20, I = didn=E2=80=99t want to change its definition and get blocked on bootstrapping all of rustc = again. Though I agree, we should update the procedure at some point. =E2=80=94Ivan= From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 19:27:30 2019 Received: (at 35155) by debbugs.gnu.org; 6 Apr 2019 23:27:30 +0000 Received: from localhost ([127.0.0.1]:47575 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCuif-0001e1-Q4 for submit@debbugs.gnu.org; Sat, 06 Apr 2019 19:27:30 -0400 Received: from mail-pl1-f193.google.com ([209.85.214.193]:35762) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCuid-0001dj-6D for 35155@debbugs.gnu.org; Sat, 06 Apr 2019 19:27:27 -0400 Received: by mail-pl1-f193.google.com with SMTP id w24so5095915plp.2 for <35155@debbugs.gnu.org>; Sat, 06 Apr 2019 16:27:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:message-id:user-agent :mime-version; bh=Wgg76BjHjR9BQB/XRx2D3zoT59j9cCW30L9G3cv7r2E=; b=N9ekLVYp+dmUOxaeB7gfDs2cFWHz/2mnOeL1d6R6k/L6jsber12FJ6EF9LsORVcN7D +mBr7F6FSd+TwvfhFotRY+0hOOg/Etl6OPGxCnXGMvN/vjHTqfrQc30xgFq7LG0ctKL3 llLBYnfVAptNGIUSDmkMUMLvpTQZvDCgqY/kbHsLcSIYcaU33aHm91awHm0PbEnuP5t+ TycV011m407FsPzyGR/QKAmd2mNSG3RwV2ig5eZ3rn0ywsj13RxQL4aS6SpyEMo9mKKw oX6qgEe5vK9PZvYQZiIIRT/k/NigKRzm1eAgXJiNqpjcsPW2N8qBmOcZE2G7sgnzE5wz qw6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:message-id :user-agent:mime-version; bh=Wgg76BjHjR9BQB/XRx2D3zoT59j9cCW30L9G3cv7r2E=; b=X9oafX9GjMCkDl24n2th8sseLuwnOJtbs3ZS2f5aPZcSMvJydRSNabbkOozMASjlQm QvCxQQCNwVQjaJ7wWCDpkdF/Z6dL+y28MWiaaw+cw0ZrHz9KKabA+SfiS7iKJUA80ZSu N+8qyIoJ3d0gEaZpwlTrFMSK0vTkEf+OXOHlA1vb2gUFHN/AXuppfP+dvF/sxmr1jbql W9bvRPVGM7nfJ9liEwnlDJd4CdI/Gt61eVpFE069w/7vfcFIOEr+lvYzjuJaTBTRs5AT 7KDvtXbPtMv7WMFnv1Rup5YUPBVTZl4iR9KFrKg8Mi5wWMNijlOiVYIo5R7YPeERbplC xAUQ== X-Gm-Message-State: APjAAAWFULxI7zNd6yNKE7C1NK7OVLdlZuEJYq1qkdlMJUVVS1ZScL5h Ah9NpSFUJ7nK7b5R3MBAXNQ= X-Google-Smtp-Source: APXvYqwkjNFcMXuouGBH34XocPDVRo4vqpCA9IXOE7Hde5T7QIyMj7EfqaydSOzjjo8JOhovHmuOwA== X-Received: by 2002:a17:902:280a:: with SMTP id e10mr2683875plb.244.1554593240783; Sat, 06 Apr 2019 16:27:20 -0700 (PDT) Received: from garuda.local ([2607:fb90:f30:4772:c4c4:8f46:ccaf:3424]) by smtp.gmail.com with ESMTPSA id x66sm7931247pfb.78.2019.04.06.16.27.18 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 06 Apr 2019 16:27:19 -0700 (PDT) From: Chris Marusich To: Ivan Petkov Subject: Re: [PATCH] build-system/cargo: refactor phases to successfully build References: Date: Sat, 06 Apr 2019 16:27:14 -0700 Message-ID: <87k1g6zpd9.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 35155 Cc: Danny Milosavljevic , 35155@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi Ivan! Ivan Petkov writes: > This patch refactors the cargo-build-system builder code to correctly bui= ld > imported Rust crates. This is great! Thank you for taking the time to work on it. I have only some high level questions and minor comments. It seems fine to me in general. Since no packages currently use the cargo-build-system, I think we can basically just commit this on master and keep moving forward. Danny, do you agree? > * Do not rely on a Cargo.lock presence to determine installation. Cargo w= ill > automatically create the file at its first invocation, so instead we check > the manifest for any executable targets before attempting an installation Sounds good. > * Do not attempt to modify the Cargo.toml file. There are many system > specific package in crates.io (e.g. for Windows, Redox, Fuschia, WASM, > etc.) and attempting to keep up with what crates must be patched out > is futile. That seems reasonable, too. > * The build phases will honor a skip-build? flag which allows for > short-circuiting for optional packages which require nightly features > or cannot be built for the current platform. Can you elaborate on the motivation for this? Are there situations in which we need to build an optional package, but we aren't actually going to use its build output anywhere? If I'm understanding this correctly, it seems like this new flag would allow us to write a package definition that doesn't actually build a package. If that's the case, I'm not sure why we would bother writing the package definition in the first place. > Changes which still need to be done: > * Update the host-side code to expand transitive inputs: cargo requires t= hat > all transitive crate dependencies are present in its (vendored) index, but > doing so by hand in the package definitions will become unwieldy. > * Update the host-side code to detect any "circular" dependencies which c= an > result from a naive import I agree with that plan. If I've been following along correctly, once we figure out how to properly make all the transitive crate dependencies (in source form) available in the build environment (and resolve the circular dependencies), it should open the door to importing many Rust packages. > * guix/build-system/cargo.scm (%cargo-build-system-modules): > Add (json parser) Nitpick: You're missing a period here, and in a few more sentences in the ChangeLog entry. > Add #:cargo-tset-flags and pass it to cargo Nitpick: The word "tset" should be "test". > (install): Factor source logic to install-source. > Define #:skip-build? flag and use it. > Only install if executable targets are present. > (install-source): Copy entire crate directory not just src. I'm not as familiar with Rust packaging as you are, so the correctness of this part is not as clear to me. My understanding is that a Cargo package that is a library needs to install its source (so that other Cargo libraries/applications can use it) but not any executables. On the other hand, a Cargo package that is an application needs to install executables (so that a user can run it), but not its source. Is that right? What about Cargo packages that are both libraries and applications? Do those even exist? If you could help me understand (or point me to docs that will help me understand) the model that Rust/Cargo follows here, it would be helpful. > +(define* (configure #:key inputs > + (vendor-dir "guix-vendor") > + #:allow-other-keys) > "Replace Cargo.toml [dependencies] section with guix inputs." Is this docstring still accurate after these changes? > + ;; Lift restriction on any lints: a crate author may have decided to o= pt > + ;; into stricter lints (e.g. #![deny(warnings)]) during their own buil= ds > + ;; but we don't want any build failures that could be caused later by > + ;; upgrading the compiler for example. > + (setenv "RUSTFLAGS" "--cap-lints allow") Is this necessary? The docs seem to suggest that Cargo always sets it to "allow" anyway: https://doc.rust-lang.org/rustc/lints/levels.html "This feature is used heavily by Cargo; it will pass --cap-lints allow when compiling your dependencies, so that if they have any warnings, they do not pollute the output of your build." > @@ -122,22 +131,34 @@ directory =3D '" port) > ;; Until this changes we are working around this by > ;; distributing crates as source and replacing > ;; references in Cargo.toml with store paths. Is this comment still accurate? > - (generate-checksums rsrc src) > + (generate-checksums rsrc "/dev/null") This probably deserves a short comment to clarify the intent. Really nice stuff - thank you for sharing it! =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlypNdIACgkQ3UCaFdgi Rp0rEw//ajG11XXdK0RIUqouPzge4MDgVLhWQS2batuokR5ScCkSyugNrfj4qv64 bt5OC4RH75etQEmJgZAedu2ZApUXVdgxrO5pvnENkuo/qY6f/k7D+JMgHfdaY3T7 X1ydiTUOp4hLixM9CxM7hcYwcbj6UDLmeX5fCynLy7rU4keXR0sqsaCfGmxMpXzg RUneIsfLIfKD2U6aPOeGN3CTtMKpDKx3jUxEE24VedIYLNE7KxbKBjbAg0UN/20q dehiPDkV1Je/5GWKrKGceji3IAT60MksL3ey42E3EalzXaaHCxYhBD1GDb0Zwwud F8HaMD8a+v55KUFD7+hpSZJygpYq3z3RDsxBZYLoHTdu6Y5XTCHjNhLNc0FvXA9l fUP9P0brkvgL0+XKmsQKZEaRYF3+S63Ax5TrAr0ePqbi1bqCH9zxT89nwBmd9uCC 3hz6z4GOrODstRxpUHHDi50/1sI1mZh6R0poN6EY3Y2SyzC84ZYhaldusIeUwmsC VystaFXDtl9wDuwTiS/spBXCcOGzBvis77sT0RWiWxV93jSWfI7MA/qqW0CFr5bf +CmkWh90+rdfEQDIbWSbGHcij3jY8lD3qZHPmj7HUnhUkek7lJYqierFK+sCZC8J NZO2IAk+TAl+kZZSOp6+WewQ6YgtOJ/8tjDjNTthc80Uudgqb9Q= =oCMR -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 22:02:50 2019 Received: (at 35155) by debbugs.gnu.org; 7 Apr 2019 02:02:50 +0000 Received: from localhost ([127.0.0.1]:47614 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCx8z-0005NG-Rd for submit@debbugs.gnu.org; Sat, 06 Apr 2019 22:02:50 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:35157) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCx8x-0005N0-DQ for 35155@debbugs.gnu.org; Sat, 06 Apr 2019 22:02:48 -0400 Received: by mail-pl1-f196.google.com with SMTP id w24so5224543plp.2 for <35155@debbugs.gnu.org>; Sat, 06 Apr 2019 19:02:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=RT15/4rikuzp3Dqm+UNzpxRIymjtiiur32xVyaUZtSc=; b=bVgQXdMyNz50CBuS9gUZ1CEfJ9IBJJfuGwe2Vzgy6zNiPNrjJRdTtUvPppC5oNx6Dh u1GMT9LN5Llo4KPGV3Wj2WyikqIEsh+UifsAIJTxB1RUwQV9ZLG3XfiPmnU7x6/JIDlS kZTjQ88O2pYhO3RLvX8Q+6qeyz+KJWlRobHLx+BvOx/+8z93qm9h7HpQ39a6CIvS059o VqecxT1JH85/kz0oyB+frbAtBcLxZ9YA6catuYtHmk4pR0bawfjKiisFrGrS1nIRuC2b DE/3qivWdwFTmirPVWsSvK3ied4bA7LHFvJsFNURAvbs2vUdqcCgdX8WPYXX86vgz5eK 3p3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=RT15/4rikuzp3Dqm+UNzpxRIymjtiiur32xVyaUZtSc=; b=KCShTDK2l41jowzqgER8GkUAxP4uuzFY01egBIozjwmcBLvS3B083qMiGFE1QpgHiQ k/5n3TBv7o6HbG8+A7kBqxjxvmvJAdRbn+C4uf1l2h5jzGEvvFOAo+khybNjVwDq5pBV kg1XzxP+1T4BamjbEaWUpGM1Bqx0VzlCogGfHvJMnS6Z8xe5HEbCMyTWKSfXyFiksRXp eqTgReHjXFJhYrXy6i+68kMhKy6uqV3Ol08KlcBiSPNTKhCw+lfseQ1BhnQ33iJD3EN5 79NPCPIfPlA7lunt5ap7tH51J9vRafZ1kYBptVuCUDMwsMnyjKGbuXBrOPyZWzMaZsWm qRQQ== X-Gm-Message-State: APjAAAVjF2AmhqRVOhLor6CeKTbaWH/9K3Fpe6Al+Zdq/auJkrY+h0Gn JiLo5Pg20npseE2Ou+z6gok= X-Google-Smtp-Source: APXvYqytQJOQH0YYgfMdZODIFMRWI6+04A+UgjStsNke4Wjl3WgSGK0/QQ+sbWKL3nuapXQ0VU3C3A== X-Received: by 2002:a17:902:3e5:: with SMTP id d92mr21057847pld.11.1554602561303; Sat, 06 Apr 2019 19:02:41 -0700 (PDT) Received: from ?IPv6:2601:602:9a00:1784:60cd:908d:a4f8:6a8e? ([2601:602:9a00:1784:60cd:908d:a4f8:6a8e]) by smtp.gmail.com with ESMTPSA id f1sm32147429pgl.35.2019.04.06.19.02.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 06 Apr 2019 19:02:40 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: [PATCH] build-system/cargo: refactor phases to successfully build From: Ivan Petkov In-Reply-To: <87k1g6zpd9.fsf@gmail.com> Date: Sat, 6 Apr 2019 19:02:35 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <51B2A728-850F-4A13-8A6B-D12FD0B0C119@gmail.com> References: <87k1g6zpd9.fsf@gmail.com> To: Chris Marusich X-Mailer: Apple Mail (2.3445.9.1) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 35155 Cc: Danny Milosavljevic , 35155@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Chris! Thank you for the detailed review! Happy to elaborate on any of your = questions, please let me know if you think any of my responses warrant additional = code comments. > On Apr 6, 2019, at 4:27 PM, Chris Marusich = wrote: >=20 >> * The build phases will honor a skip-build? flag which allows for >> short-circuiting for optional packages which require nightly features >> or cannot be built for the current platform. >=20 > Can you elaborate on the motivation for this? Are there situations in > which we need to build an optional package, but we aren't actually = going > to use its build output anywhere? This is meant to be an escape hatch for to skip builds when necessary. Nothing is setting this flag right now, but eventually the host-side = code may need to set this for certain situations. > If I'm understanding this correctly, it seems like this new flag would > allow us to write a package definition that doesn't actually build a > package. If that's the case, I'm not sure why we would bother writing > the package definition in the first place. That=E2=80=99s an accurate observation. Some context: Cargo requires that all possible transitive dependencies are present in its index/vendor directory. This is so it can deterministically build = Cargo.lock files independently of the current platform or what conditional features = are enabled. There are several ways to package crates within guix with respect to = dependent crates: a) Don=E2=80=99t pull in optional dependencies, or those for unsupported = systems, patch out the Cargo.toml file so they=E2=80=99re outright ignored by = cargo b) Create stubs for any unsupported/unpacked crates, basically a = Cargo.toml definition with the expected crate name/version and no code c) Package all crates depended upon by a crate we=E2=80=99re interested = in, possibly annotating it in ways that it isn=E2=80=99t actually built by = the CI (e.g. system specific packages which there is no CI support for). As mentioned in my earlier email, I believe option a to be a non-starter since it will be a never-ending uphill battle. I personally believe = option c is the best long term approach (we may reserve option b for dire = situations). This way if guix is ever ported to other systems (e.g. Linux subsystem = for Windows) packages can still be used by consumers without them having to backfill half the crates ecosystem. Given this information, here=E2=80=99s how I anticipate we=E2=80=99ll = want to skip doing actual package builds: * If the package is annotated as for a specific platform (e.g. = Windows/Redox/Fuscia) the host-side build code can populate (#:skip-build? #t) so it doesn=E2=80= =99t fail during CI builds but its still accessible for consuming crates. * If we have a =E2=80=9Ccircular=E2=80=9D dependency as part of = dev-dependencies (e.g. one crate pulls in an upstream crate during testing to ensure it doesn=E2=80=99t = break anything) we=E2=80=99ll need to detangle the dependency graph by rewriting = duplicate packages to include the #:skip-build? flag. I can elaborate more on this in a = separate email. >> Changes which still need to be done: >> * Update the host-side code to expand transitive inputs: cargo = requires that >> all transitive crate dependencies are present in its (vendored) = index, but >> doing so by hand in the package definitions will become unwieldy. >> * Update the host-side code to detect any "circular" dependencies = which can >> result from a naive import >=20 > I agree with that plan. If I've been following along correctly, once = we > figure out how to properly make all the transitive crate dependencies > (in source form) available in the build environment (and resolve the > circular dependencies), it should open the door to importing many Rust > packages. Yes, precisely, it will be much easier to spot an resolve any other bugs or feature gaps once that=E2=80=99s in place. >=20 >> * guix/build-system/cargo.scm (%cargo-build-system-modules): >> Add (json parser) >=20 > Nitpick: You're missing a period here, and in a few more sentences in > the ChangeLog entry. Happy to update. After skimming the contributing guidelines I was left with the impression that commit messages are to only include the = concrete changes that were made, and any additional elaboration should be made in code comments. What=E2=80=99s the right process for fixing this? Just send an updated = patch to this thread? >=20 >> Add #:cargo-tset-flags and pass it to cargo >=20 > Nitpick: The word "tset" should be "test=E2=80=9D. Whoops, that=E2=80=99s what I get for writing commit messages late at = night :) >=20 >> (install): Factor source logic to install-source. >> Define #:skip-build? flag and use it. >> Only install if executable targets are present. >> (install-source): Copy entire crate directory not just src. >=20 > I'm not as familiar with Rust packaging as you are, so the correctness > of this part is not as clear to me. >=20 > My understanding is that a Cargo package that is a library needs to > install its source (so that other Cargo libraries/applications can use > it) but not any executables. On the other hand, a Cargo package that = is > an application needs to install executables (so that a user can run = it), > but not its source. Is that right? You are correct. When building a crate, cargo needs access to the source of all transitive dependencies, but it=E2=80=99s no longer needed after = the build. The reason we now copy the entire crate directory (rather than just the = src directory) is that some crates have build scripts which usually live = outside of `src` and are needed to successfully build. Although the Cargo.toml has the path to the root build script, there are crates (like serde) = which have auxiliary build-script modules which aren=E2=80=99t shown in the = manifest contents. Rather than muck around and try to guess where the build = script code is, we can copy it all and let cargo sort it out. > What about Cargo packages that are > both libraries and applications? Do those even exist? Yes these are a bit rare but they do exist. I don=E2=80=99t have any = examples on hand, but you can have something akin to curl which can be used as a binary, as well as imported as a library to other projects. >=20 >> +(define* (configure #:key inputs >> + (vendor-dir "guix-vendor") >> + #:allow-other-keys) >> "Replace Cargo.toml [dependencies] section with guix inputs." >=20 > Is this docstring still accurate after these changes? I think the intent is still accurate, though I=E2=80=99ll tweak this to = note vendoring dependencies instead of updating the Cargo.toml. >=20 >> + ;; Lift restriction on any lints: a crate author may have decided = to opt >> + ;; into stricter lints (e.g. #![deny(warnings)]) during their own = builds >> + ;; but we don't want any build failures that could be caused later = by >> + ;; upgrading the compiler for example. >> + (setenv "RUSTFLAGS" "--cap-lints allow") >=20 > Is this necessary? The docs seem to suggest that Cargo always sets it > to "allow" anyway: >=20 > https://doc.rust-lang.org/rustc/lints/levels.html >=20 > "This feature is used heavily by Cargo; it will pass --cap-lints allow > when compiling your dependencies, so that if they have any warnings, > they do not pollute the output of your build.=E2=80=9D It=E2=80=99s true that cargo applies this to dependencies, but it = doesn=E2=80=99t do this for the top level package that=E2=80=99s currently being built (e.g. if = the CI is building some library crate in isolation). As Guix maintainers, we wouldn=E2=80=99t want jobs to start failing because of a new lint = cropping up somewhere in between versions. >=20 >> @@ -122,22 +131,34 @@ directory =3D '" port) >> ;; Until this changes we are working around this by >> ;; distributing crates as source and replacing >> ;; references in Cargo.toml with store paths. >=20 > Is this comment still accurate? I think this is still accurate (modulo Cargo.toml/vendoring word choice which I can tweak). >=20 >> - (generate-checksums rsrc src) >> + (generate-checksums rsrc "/dev/null") >=20 > This probably deserves a short comment to clarify the intent. Do you mean commenting on the intent of `generate-checksums` or the intent of the /dev/null parameter? =E2=80=94Ivan From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 23:05:19 2019 Received: (at 35155) by debbugs.gnu.org; 7 Apr 2019 03:05:19 +0000 Received: from localhost ([127.0.0.1]:47655 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCy7T-0006y3-Jb for submit@debbugs.gnu.org; Sat, 06 Apr 2019 23:05:19 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:36377) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCy7R-0006xo-Lv for 35155@debbugs.gnu.org; Sat, 06 Apr 2019 23:05:18 -0400 Received: by mail-pf1-f193.google.com with SMTP id z5so5546656pfn.3 for <35155@debbugs.gnu.org>; Sat, 06 Apr 2019 20:05:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=G86jY8Id4xS/TV1ObX5PMgGIOvWmZuQcrxzrOG4vuGA=; b=EHRSwGl0X5zIjQpKVKi1hGxlTNxEn439PQwovCBsyyLLLUKWbaOwXgYjdVraN+CHBD 58itlpRVASdk8QIE3/ZA1bM7ulvZuika+SnVYEQMMKBeEHKVnKSvQANMPtdEhzU09zvp TTA3xaOhGnsl8NVpoi1Ha8/FiaY3kgDuChyd/02Mgif2YcARlAyZNx6ks6ckqGSLGVfK wxALtKqaHgyPnhuf6kb2qGp15x3WAEGtKB0EoQ7bP+DDSj3FkLNZUYbOrRk4nCwT+vKC kA0uzvRSD31IwkDSqinVTyVHs587HAGq/fi+oVRBlnfjz4vfsWL4cKIY2rnkSvFSbb37 2A4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=G86jY8Id4xS/TV1ObX5PMgGIOvWmZuQcrxzrOG4vuGA=; b=QszEZxa3ejKborjiaiPbFmHFqMzi462iq092BFt2Y45EX1sAF8T6q8hiYOx4KzWtKN ni8Mktw6/rkJC/aaEN0KGpDs1MSEj3YPRd7cd25R8JEobLBOhE8/YKm5UDLl+3UALvh5 GlNWPgqSF9KE1glc7cF2eCfZuwslHayOwnENiaWEw0zxESMpdnmLlUvO2J981pVok5ho mdT9t/V5x08UbDhJ4R3T0hJHTDPg0Bx5N5du2nC5jpRwM/bxG/ccczU7oWBUxLJdupUb /QIOe88yUudH7PmUQxb0ZEjQEGp//C08pG3YhtQHXqWQ0Jjg5NDthh8FWDFwSZqNAtpi UeOg== X-Gm-Message-State: APjAAAUd2RM+TgwB9NyU17IPUbJ3am9hyCwpxZN4Q835mVCzjLEa8yjj tDFhtuEorIahTJ0F0UNtGqo= X-Google-Smtp-Source: APXvYqzdfZYFWH3UGyTgkL4hLFjlXiAVk6GCuAyYvkpFCAun0b7GMpqduhFy73XgPDD00KRf5kLuBw== X-Received: by 2002:aa7:8289:: with SMTP id s9mr21755176pfm.208.1554606311468; Sat, 06 Apr 2019 20:05:11 -0700 (PDT) Received: from ?IPv6:2601:602:9a00:1784:91ee:3893:2f1f:95f1? ([2601:602:9a00:1784:91ee:3893:2f1f:95f1]) by smtp.gmail.com with ESMTPSA id p66sm20708000pfb.4.2019.04.06.20.05.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 06 Apr 2019 20:05:10 -0700 (PDT) Content-Type: multipart/mixed; boundary="Apple-Mail=_FD053A41-50DA-4847-AEE8-A8D727617FC7" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: [PATCH] build-system/cargo: refactor phases to successfully build From: Ivan Petkov In-Reply-To: <51B2A728-850F-4A13-8A6B-D12FD0B0C119@gmail.com> Date: Sat, 6 Apr 2019 20:05:09 -0700 Message-Id: <228707C3-2A18-4A09-ACDA-373A5584B4C7@gmail.com> References: <87k1g6zpd9.fsf@gmail.com> <51B2A728-850F-4A13-8A6B-D12FD0B0C119@gmail.com> To: Chris Marusich X-Mailer: Apple Mail (2.3445.9.1) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 35155 Cc: Danny Milosavljevic , 35155@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --Apple-Mail=_FD053A41-50DA-4847-AEE8-A8D727617FC7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Chris! > On Apr 6, 2019, at 7:02 PM, Ivan Petkov wrote: >=20 >>=20 >>> * guix/build-system/cargo.scm (%cargo-build-system-modules): >>> Add (json parser) >>=20 >> Nitpick: You're missing a period here, and in a few more sentences in >> the ChangeLog entry. >=20 > Happy to update. After skimming the contributing guidelines I was left > with the impression that commit messages are to only include the = concrete > changes that were made, and any additional elaboration should be made > in code comments. Please disregard this comment, I had misread your original email. I=E2=80=99ve attached an updated version of my original patch. =E2=80=94Ivan --Apple-Mail=_FD053A41-50DA-4847-AEE8-A8D727617FC7 Content-Disposition: attachment; filename=0001-build-system-cargo-refactor-phases-to-successfully-b.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0001-build-system-cargo-refactor-phases-to-successfully-b.patch" Content-Transfer-Encoding: quoted-printable =46rom=2074c59e18e339a71372367291446c93e33a526156=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Ivan=20Petkov=20=0A= Date:=20Tue,=202=20Apr=202019=2003:02:51=20-0700=0ASubject:=20[PATCH]=20= build-system/cargo:=20refactor=20phases=20to=20successfully=20build=0A=0A= *=20guix/build-system/cargo.scm=20(%cargo-build-system-modules):=0AAdd=20= (json=20parser).=0A(cargo-build):=0A[vendor-dir]:=20Define=20flag=20and=20= pass=20it=20to=20builder=20code.=0A[cargo-test-flags]:=20Likewise.=0A= [skip-build?]:=20Likewise.=0A*=20guix/build/cargo-build/system.scm=20= (#:use-module):=20use=20(json=20parser).=0A(package-name->crate-name):=20= Delete=20it.=0A(manifest-targets):=20Add=20it.=0A= (has-executable-target?):=20Add=20it.=0A(configure):=20Add=20= #:vendor-dir=20name=20and=20use=20it.=0ADon't=20touch=20Cargo.toml.=0A= Don't=20symlink=20to=20duplicate=20inputs.=0ARemove=20useless=20registry=20= line=20from=20cargo=20config.=0ADefine=20RUSTFLAGS=20to=20lift=20lint=20= restrictions.=0A(build):=20Add=20#:skip-build?=20flag=20and=20use=20it.=0A= (check):=20Likewise.=0AAdd=20#:cargo-test-flags=20and=20pass=20it=20to=20= cargo.=0A(install):=20Factor=20source=20logic=20to=20install-source.=0A= Define=20#:skip-build?=20flag=20and=20use=20it.=0AOnly=20install=20if=20= executable=20targets=20are=20present.=0A(install-source):=20Copy=20= entire=20crate=20directory=20not=20just=20src.=0A[generate-checksums]=20= pass=20dummy=20file=20for=20unused=20second=20argument.=0A= (%standard-phases):=20Add=20install-source=20phase.=0A---=0A=20= guix/build-system/cargo.scm=20=20=20=20=20=20=20|=20=20=209=20+-=0A=20= guix/build/cargo-build-system.scm=20|=20155=20= +++++++++++++++++-------------=0A=202=20files=20changed,=2095=20= insertions(+),=2069=20deletions(-)=0A=0Adiff=20--git=20= a/guix/build-system/cargo.scm=20b/guix/build-system/cargo.scm=0Aindex=20= 7ff4e90f71..dc137421e9=20100644=0A---=20a/guix/build-system/cargo.scm=0A= +++=20b/guix/build-system/cargo.scm=0A@@=20-59,13=20+59,17=20@@=20to=20= NAME=20and=20VERSION."=0A=20(define=20%cargo-build-system-modules=0A=20=20= =20;;=20Build-side=20modules=20imported=20by=20default.=0A=20=20=20= `((guix=20build=20cargo-build-system)=0A+=20=20=20=20(json=20parser)=0A=20= =20=20=20=20,@%cargo-utils-modules))=0A=20=0A=20(define*=20(cargo-build=20= store=20name=20inputs=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20#:key=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20(tests?=20#t)=0A=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20(test-target=20#f)=0A+=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(vendor-dir=20= "guix-vendor")=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20(cargo-build-flags=20''("--release"))=0A+=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(cargo-test-flags=20= ''("--release"))=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20(skip-build?=20#f)=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20(phases=20'(@=20(guix=20build=20= cargo-build-system)=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20%standard-phases))=0A= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= (outputs=20'("out"))=0A@@=20-90,8=20+94,11=20@@=20to=20NAME=20and=20= VERSION."=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20source))=0A=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20#:system=20,system=0A=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#:test-target=20= ,test-target=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20#:vendor-dir=20,vendor-dir=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20#:cargo-build-flags=20,cargo-build-flags=0A-=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#:tests?=20,tests?=0A= +=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= #:cargo-test-flags=20,cargo-test-flags=0A+=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20#:skip-build?=20,skip-build?=0A+=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#:tests?=20,(and=20tests?=20= (not=20skip-build?))=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20#:phases=20,phases=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20#:outputs=20%outputs=0A=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20#:search-paths=20',(map=20= search-path-specification->sexp=0Adiff=20--git=20= a/guix/build/cargo-build-system.scm=20= b/guix/build/cargo-build-system.scm=0Aindex=2020087fa6c4..b68a1f90d2=20= 100644=0A---=20a/guix/build/cargo-build-system.scm=0A+++=20= b/guix/build/cargo-build-system.scm=0A@@=20-1,6=20+1,7=20@@=0A=20;;;=20= GNU=20Guix=20---=20Functional=20package=20management=20for=20GNU=0A=20= ;;;=20Copyright=20=C2=A9=202016=20David=20Craven=20=0A=20= ;;;=20Copyright=20=C2=A9=202017=20Mathieu=20Othacehe=20= =0A+;;;=20Copyright=20=C2=A9=202019=20Ivan=20= Petkov=20=0A=20;;;=0A=20;;;=20This=20file=20is=20= part=20of=20GNU=20Guix.=0A=20;;;=0A@@=20-26,6=20+27,7=20@@=0A=20=20=20= #:use-module=20(ice-9=20ftw)=0A=20=20=20#:use-module=20(ice-9=20format)=0A= =20=20=20#:use-module=20(ice-9=20match)=0A+=20=20#:use-module=20(json=20= parser)=0A=20=20=20#:use-module=20(srfi=20srfi-1)=0A=20=20=20= #:use-module=20(srfi=20srfi-26)=0A=20=20=20#:export=20(%standard-phases=0A= @@=20-37,81=20+39,86=20@@=0A=20;;=0A=20;;=20Code:=0A=20=0A-;;=20FIXME:=20= Needs=20to=20be=20parsed=20from=20url=20not=20package=20name.=0A-(define=20= (package-name->crate-name=20name)=0A-=20=20"Return=20the=20crate=20name=20= of=20NAME."=0A-=20=20(match=20(string-split=20name=20#\-)=0A-=20=20=20=20= (("rust"=20rest=20...)=0A-=20=20=20=20=20(string-join=20rest=20"-"))=0A-=20= =20=20=20(_=20#f)))=0A-=0A-(define*=20(configure=20#:key=20inputs=20= #:allow-other-keys)=0A-=20=20"Replace=20Cargo.toml=20[dependencies]=20= section=20with=20guix=20inputs."=0A-=20=20;;=20Make=20sure=20Cargo.toml=20= is=20writeable=20when=20the=20crate=20uses=20git-fetch.=0A-=20=20(chmod=20= "Cargo.toml"=20#o644)=0A+(define=20(manifest-targets)=0A+=20=20"Extract=20= all=20targets=20from=20the=20Cargo.toml=20manifest"=0A+=20=20(let*=20= ((port=20(open-input-pipe=20"cargo=20read-manifest"))=0A+=20=20=20=20=20=20= =20=20=20(data=20(json->scm=20port))=0A+=20=20=20=20=20=20=20=20=20= (targets=20(hash-ref=20data=20"targets"=20'())))=0A+=20=20=20=20= (close-port=20port)=0A+=20=20=20=20targets))=0A+=0A+(define=20= (has-executable-target?)=0A+=20=20"Check=20if=20the=20current=20cargo=20= project=20declares=20any=20binary=20targets."=0A+=20=20(let*=20((bin?=20= (lambda=20(kind)=20(string=3D?=20kind=20"bin")))=0A+=20=20=20=20=20=20=20= =20=20(get-kinds=20(lambda=20(dep)=20(hash-ref=20dep=20"kind")))=0A+=20=20= =20=20=20=20=20=20=20(bin-dep?=20(lambda=20(dep)=20(find=20bin?=20= (get-kinds=20dep)))))=0A+=20=20=20=20(find=20bin-dep?=20= (manifest-targets))))=0A+=0A+(define*=20(configure=20#:key=20inputs=0A+=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(vendor-dir=20= "guix-vendor")=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20#:allow-other-keys)=0A+=20=20"Vendor=20Cargo.toml=20dependencies=20as=20= guix=20inputs."=0A=20=20=20(chmod=20"."=20#o755)=0A-=20=20(if=20(not=20= (file-exists?=20"vendor"))=0A-=20=20=20=20(if=20(not=20(file-exists?=20= "Cargo.lock"))=0A-=20=20=20=20=20=20(begin=0A-=20=20=20=20=20=20=20=20= (substitute*=20"Cargo.toml"=0A-=20=20=20=20=20=20=20=20=20=20= ((".*32-sys.*")=20"=0A-")=0A-=20=20=20=20=20=20=20=20=20=20= ((".*winapi.*")=20"=0A-")=0A-=20=20=20=20=20=20=20=20=20=20= ((".*core-foundation.*")=20"=0A-"))=0A-=20=20=20=20=20=20=20=20;;=20= Prepare=20one=20new=20directory=20with=20all=20the=20required=20= dependencies.=0A-=20=20=20=20=20=20=20=20;;=20It's=20necessary=20to=20do=20= this=20(instead=20of=20just=20using=20/gnu/store=20as=20the=0A-=20=20=20=20= =20=20=20=20;;=20directory)=20because=20we=20want=20to=20hide=20the=20= libraries=20in=20subdirectories=0A-=20=20=20=20=20=20=20=20;;=20=20=20= share/rust-source/...=20instead=20of=20polluting=20the=20user's=20= profile=20root.=0A-=20=20=20=20=20=20=20=20(mkdir=20"vendor")=0A-=20=20=20= =20=20=20=20=20(for-each=0A-=20=20=20=20=20=20=20=20=20=20(match-lambda=0A= -=20=20=20=20=20=20=20=20=20=20=20=20((name=20.=20path)=0A-=20=20=20=20=20= =20=20=20=20=20=20=20=20(let=20((crate=20(package-name->crate-name=20= name)))=0A-=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(when=20(and=20= crate=20path)=0A-=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= (match=20(string-split=20(basename=20path)=20#\-)=0A-=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20((_=20...=20version)=0A-=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(symlink=20(string-append=20= path=20"/share/rust-source")=0A-=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20(string-append=20"vendor/"=20= (basename=20path)))))))))=0A-=20=20=20=20=20=20=20=20=20=20inputs)=0A-=20= =20=20=20=20=20=20=20;;=20Configure=20cargo=20to=20actually=20use=20this=20= new=20directory.=0A-=20=20=20=20=20=20=20=20(mkdir-p=20".cargo")=0A-=20=20= =20=20=20=20=20=20(let=20((port=20(open-file=20".cargo/config"=20"w"=20= #:encoding=20"utf-8")))=0A-=20=20=20=20=20=20=20=20=20=20(display=20"=0A= +=20=20;;=20Prepare=20one=20new=20directory=20with=20all=20the=20= required=20dependencies.=0A+=20=20;;=20It's=20necessary=20to=20do=20this=20= (instead=20of=20just=20using=20/gnu/store=20as=20the=0A+=20=20;;=20= directory)=20because=20we=20want=20to=20hide=20the=20libraries=20in=20= subdirectories=0A+=20=20;;=20share/rust-source/...=20instead=20of=20= polluting=20the=20user's=20profile=20root.=0A+=20=20(mkdir-p=20= vendor-dir)=0A+=20=20(for-each=0A+=20=20=20=20(match-lambda=0A+=20=20=20=20= =20=20((name=20.=20path)=0A+=20=20=20=20=20=20=20(let*=20((rust-share=20= (string-append=20path=20"/share/rust-source"))=0A+=20=20=20=20=20=20=20=20= =20=20=20=20=20=20(basepath=20(basename=20path))=0A+=20=20=20=20=20=20=20= =20=20=20=20=20=20=20(link-dir=20(string-append=20vendor-dir=20"/"=20= basepath)))=0A+=20=20=20=20=20=20=20=20=20(and=20(file-exists?=20= rust-share)=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20;;=20Gracefully=20= handle=20duplicate=20inputs=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20= (not=20(file-exists?=20link-dir))=0A+=20=20=20=20=20=20=20=20=20=20=20=20= =20=20(symlink=20rust-share=20link-dir)))))=0A+=20=20=20=20inputs)=0A+=20= =20;;=20Configure=20cargo=20to=20actually=20use=20this=20new=20= directory.=0A+=20=20(mkdir-p=20".cargo")=0A+=20=20(let=20((port=20= (open-file=20".cargo/config"=20"w"=20#:encoding=20"utf-8")))=0A+=20=20=20= =20(display=20"=0A=20[source.crates-io]=0A-registry=20=3D=20= 'https://github.com/rust-lang/crates.io-index'=0A=20replace-with=20=3D=20= 'vendored-sources'=0A=20=0A=20[source.vendored-sources]=0A=20directory=20= =3D=20'"=20port)=0A-=20=20=20=20=20=20=20=20=20=20(display=20(getcwd)=20= port)=0A-=20=20=20=20=20=20=20=20=20=20(display=20"/vendor"=20port)=0A-=20= =20=20=20=20=20=20=20=20=20(display=20"'=0A+=20=20=20=20(display=20= (string-append=20(getcwd)=20"/"=20vendor-dir)=20port)=0A+=20=20=20=20= (display=20"'=0A=20"=20port)=0A-=20=20=20=20=20=20=20=20=20=20= (close-port=20port)))))=0A-=20=20=20=20(setenv=20"CC"=20(string-append=20= (assoc-ref=20inputs=20"gcc")=20"/bin/gcc"))=0A+=20=20=20=20(close-port=20= port))=0A=20=0A-=20=20=20=20;(setenv=20"CARGO_HOME"=20"/gnu/store")=0A-=20= =20=20=20;=20(setenv=20"CMAKE_C_COMPILER"=20cc)=0A+=20=20;;=20Lift=20= restriction=20on=20any=20lints:=20a=20crate=20author=20may=20have=20= decided=20to=20opt=0A+=20=20;;=20into=20stricter=20lints=20(e.g.=20= #![deny(warnings)])=20during=20their=20own=20builds=0A+=20=20;;=20but=20= we=20don't=20want=20any=20build=20failures=20that=20could=20be=20caused=20= later=20by=0A+=20=20;;=20upgrading=20the=20compiler=20for=20example.=0A+=20= =20(setenv=20"RUSTFLAGS"=20"--cap-lints=20allow")=0A+=20=20(setenv=20= "CC"=20(string-append=20(assoc-ref=20inputs=20"gcc")=20"/bin/gcc"))=0A=20= =20=20#t)=0A=20=0A-(define*=20(build=20#:key=20(cargo-build-flags=20= '("--release"))=0A+(define*=20(build=20#:key=0A+=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20skip-build?=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20(cargo-build-flags=20'("--release"))=0A=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20#:allow-other-keys)=0A=20=20=20"Build=20a=20= given=20Cargo=20package."=0A-=20=20(zero?=20(apply=20system*=20`("cargo"=20= "build"=20,@cargo-build-flags))))=0A+=20=20(or=20skip-build?=0A+=20=20=20= =20=20=20(zero?=20(apply=20system*=20`("cargo"=20"build"=20= ,@cargo-build-flags)))))=0A=20=0A-(define*=20(check=20#:key=20tests?=20= #:allow-other-keys)=0A+(define*=20(check=20#:key=0A+=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20tests?=0A+=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20(cargo-test-flags=20'("--release"))=0A+=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20#:allow-other-keys)=0A=20=20=20"Run=20tests=20= for=20a=20given=20Cargo=20package."=0A-=20=20(if=20(and=20tests?=20= (file-exists?=20"Cargo.lock"))=0A-=20=20=20=20=20=20(zero?=20(system*=20= "cargo"=20"test"))=0A+=20=20(if=20tests?=0A+=20=20=20=20=20=20(zero?=20= (apply=20system*=20`("cargo"=20"test"=20,@cargo-test-flags)))=0A=20=20=20= =20=20=20=20#t))=0A=20=0A=20(define=20(touch=20file-name)=0A=20=20=20= (call-with-output-file=20file-name=20(const=20#t)))=0A=20=0A-(define*=20= (install=20#:key=20inputs=20outputs=20#:allow-other-keys)=0A-=20=20= "Install=20a=20given=20Cargo=20package."=0A+(define*=20(install-source=20= #:key=20inputs=20outputs=20#:allow-other-keys)=0A+=20=20"Install=20the=20= source=20for=20a=20given=20Cargo=20package."=0A=20=20=20(let*=20((out=20= (assoc-ref=20outputs=20"out"))=0A=20=20=20=20=20=20=20=20=20=20(src=20= (assoc-ref=20inputs=20"source"))=0A=20=20=20=20=20=20=20=20=20=20(rsrc=20= (string-append=20(assoc-ref=20outputs=20"src")=0A@@=20-120,24=20+127,36=20= @@=20directory=20=3D=20'"=20port)=0A=20=20=20=20=20;;=20Rust=20doesn't=20= have=20a=20stable=20ABI=20yet.=20Because=20of=20this=0A=20=20=20=20=20;;=20= Cargo=20doesn't=20have=20a=20search=20path=20for=20binaries=20yet.=0A=20=20= =20=20=20;;=20Until=20this=20changes=20we=20are=20working=20around=20= this=20by=0A-=20=20=20=20;;=20distributing=20crates=20as=20source=20and=20= replacing=0A-=20=20=20=20;;=20references=20in=20Cargo.toml=20with=20= store=20paths.=0A-=20=20=20=20(copy-recursively=20"src"=20(string-append=20= rsrc=20"/src"))=0A+=20=20=20=20;;=20vendoring=20the=20crates'=20sources=20= by=20symlinking=20them=0A+=20=20=20=20;;=20to=20store=20paths.=0A+=20=20=20= =20(copy-recursively=20"."=20rsrc)=0A=20=20=20=20=20(touch=20= (string-append=20rsrc=20"/.cargo-ok"))=0A-=20=20=20=20= (generate-checksums=20rsrc=20src)=0A+=20=20=20=20(generate-checksums=20= rsrc=20"/dev/null")=0A=20=20=20=20=20(install-file=20"Cargo.toml"=20= rsrc)=0A-=20=20=20=20;;=20When=20the=20package=20includes=20executables=20= we=20install=0A-=20=20=20=20;;=20it=20using=20cargo=20install.=20This=20= fails=20when=20the=20crate=0A-=20=20=20=20;;=20doesn't=20contain=20an=20= executable.=0A-=20=20=20=20(if=20(file-exists?=20"Cargo.lock")=0A-=20=20=20= =20=20=20=20=20(zero?=20(system*=20"cargo"=20"install"=20"--root"=20= out))=0A-=20=20=20=20=20=20=20=20(begin=0A-=20=20=20=20=20=20=20=20=20=20= (mkdir=20out)=0A-=20=20=20=20=20=20=20=20=20=20#t))))=0A+=20=20=20=20= #t))=0A+=0A+(define*=20(install=20#:key=20inputs=20outputs=20skip-build?=20= #:allow-other-keys)=0A+=20=20"Install=20a=20given=20Cargo=20package."=0A= +=20=20(let*=20((out=20(assoc-ref=20outputs=20"out")))=0A+=20=20=20=20= (mkdir-p=20out)=0A+=0A+=20=20=20=20;;=20Make=20cargo=20reuse=20all=20the=20= artifacts=20we=20just=20built=20instead=0A+=20=20=20=20;;=20of=20= defaulting=20to=20making=20a=20new=20temp=20directory=0A+=20=20=20=20= (setenv=20"CARGO_TARGET_DIR"=20"./target")=0A+=20=20=20=20;;=20Force=20= cargo=20to=20honor=20our=20.cargo/config=20definitions=0A+=20=20=20=20;;=20= https://github.com/rust-lang/cargo/issues/6397=0A+=20=20=20=20(setenv=20= "CARGO_HOME"=20".")=0A+=0A+=20=20=20=20;;=20Only=20install=20crates=20= which=20include=20binary=20targets,=0A+=20=20=20=20;;=20otherwise=20= cargo=20will=20raise=20an=20error.=0A+=20=20=20=20(or=20skip-build?=0A+=20= =20=20=20=20=20=20=20(not=20(has-executable-target?))=0A+=20=20=20=20=20=20= =20=20(zero?=20(system*=20"cargo"=20"install"=20"--path"=20"."=20= "--root"=20out)))))=0A=20=0A=20(define=20%standard-phases=0A=20=20=20= (modify-phases=20gnu:%standard-phases=0A=20=20=20=20=20(delete=20= 'bootstrap)=0A+=20=20=20=20(add-before=20'configure=20'install-source=20= install-source)=0A=20=20=20=20=20(replace=20'configure=20configure)=0A=20= =20=20=20=20(replace=20'build=20build)=0A=20=20=20=20=20(replace=20= 'check=20check)=0A--=20=0A2.21.0=0A=0A= --Apple-Mail=_FD053A41-50DA-4847-AEE8-A8D727617FC7-- From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 07 04:40:42 2019 Received: (at 35155) by debbugs.gnu.org; 7 Apr 2019 08:40:42 +0000 Received: from localhost ([127.0.0.1]:47742 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hD3M1-0002da-N2 for submit@debbugs.gnu.org; Sun, 07 Apr 2019 04:40:42 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:37096) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hD3Lz-0002dK-BI for 35155@debbugs.gnu.org; Sun, 07 Apr 2019 04:40:40 -0400 Received: by mail-pg1-f194.google.com with SMTP id e6so5531627pgc.4 for <35155@debbugs.gnu.org>; Sun, 07 Apr 2019 01:40:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=rqjjM8e+PyrKagi2Qquc7IGPPuxbumGPCx2xsH5nkro=; b=iZslr6VmQpOx0A74ifA1f4dXs87XIba3HD4xVRqAFUSVee/47/KJy8r4IzSfvMhezb Wm1c08YQ79RMZeDYc1TLIK0LQ/2SXAF+jTgsJk226V9XqXB7Wrln9uGbWdwc+NcxFZHm jCqertQgiCKykJ1IErs35BNgF5rf2o4/iQrVaNHnIxwnkio71ZD9kAbY6alLUC2BViWh o93JtJx6XfDXrfIwJemZpMuZF+qDCOdvfhnVC6yggVRD8W6/q58pf3HXDbpy21EcPWSp xmn60S/sIYrywYLVFQgo6LV1ojyActDeSxSY4/vrpfzbKtemXGNPQcIA7iW9FGnbvj++ OC4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=rqjjM8e+PyrKagi2Qquc7IGPPuxbumGPCx2xsH5nkro=; b=ZAwdKufOOqSYnnHjrbaaaWGv/VB1Jx+miq1fDHbKQ6vmfeCMXIdu2fDQjwj0v39tYw PaHS1/pe6io1CspY8F/qbzYyScyRV7MtS76cRo/tB6t25BxbRp50oFdn5tGvTiVPWfEA dedu5es/qls7XZ0L09U1OM5BHzy2ALIS/yQQcHWQlKcJIl1m67dN6SHp6glZIIuSFDxD 7y2LQthBaNEHhiHSCcFSVP6PKFNvKFKFK4KKvvVJTfiHnqB5EQbgGmBCafTpp74TMTyT NjIUnNSUql2RLLjAteeYEEkSp2cuo5lOm9jG5WxOIOJ3hTtKNWRnH10m/hFGSGBeVB34 sdMQ== X-Gm-Message-State: APjAAAV94qy7L/Hn5rdMUTs2+N1aTn8RydvUnEwODUOUKhZSXTSsv5IT GZdM570XPDGSjIrkvB6IBh0= X-Google-Smtp-Source: APXvYqxxBatwSx15pkrAZzV7ve3/SFI+7r9Ks6zqAYnBn1dGzgxShKgQnyp1nXlVqjRXm1ns4PO6Mg== X-Received: by 2002:aa7:8c84:: with SMTP id p4mr23292088pfd.164.1554626432907; Sun, 07 Apr 2019 01:40:32 -0700 (PDT) Received: from garuda.local ([2601:601:9d80:25b2::fa8]) by smtp.gmail.com with ESMTPSA id d68sm26343822pfg.16.2019.04.07.01.40.31 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Apr 2019 01:40:31 -0700 (PDT) From: Chris Marusich To: Ivan Petkov Subject: Re: [PATCH] build-system/cargo: refactor phases to successfully build References: <87k1g6zpd9.fsf@gmail.com> <51B2A728-850F-4A13-8A6B-D12FD0B0C119@gmail.com> Date: Sun, 07 Apr 2019 01:40:27 -0700 In-Reply-To: <51B2A728-850F-4A13-8A6B-D12FD0B0C119@gmail.com> (Ivan Petkov's message of "Sat, 6 Apr 2019 19:02:35 -0700") Message-ID: <877ec6jjic.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 35155 Cc: Danny Milosavljevic , 35155@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ivan, Ivan Petkov writes: >> On Apr 6, 2019, at 4:27 PM, Chris Marusich wrote: >>=20 >>> * The build phases will honor a skip-build? flag which allows for >>> short-circuiting for optional packages which require nightly features >>> or cannot be built for the current platform. >>=20 >> Can you elaborate on the motivation for this? Are there situations in >> which we need to build an optional package, but we aren't actually going >> to use its build output anywhere? > > This is meant to be an escape hatch for to skip builds when necessary. > Nothing is setting this flag right now, but eventually the host-side code > may need to set this for certain situations. Understood. >> If I'm understanding this correctly, it seems like this new flag would >> allow us to write a package definition that doesn't actually build a >> package. If that's the case, I'm not sure why we would bother writing >> the package definition in the first place. > > That=E2=80=99s an accurate observation. Some context: > > Cargo requires that all possible transitive dependencies are present in > its index/vendor directory. This is so it can deterministically build Car= go.lock > files independently of the current platform or what conditional features = are > enabled. Do you mean that if a crate X has an optional feature that requires crate Y, then Cargo requires Y to be present when building X even if X is being built with that feature disabled? > * If we have a =E2=80=9Ccircular=E2=80=9D dependency as part of dev-depen= dencies > (e.g. one crate pulls in an upstream crate during testing to ensure it > doesn=E2=80=99t break anything) we=E2=80=99ll need to detangle the depend= ency graph by > rewriting duplicate packages to include the #:skip-build? flag. I can > elaborate more on this in a separate email. I think here, you're talking about the situation in which crate A depends on crate B, which in turn depends on crate A's source, and to break the cycle we will replace B's dependency on A with a dependency on A', where A' is effectively just a source build of A (via #:skip-build? #t). Is that basically right? I agree it would be good to discuss the resolution of circular dependencies in another email thread, but I just wanted to double check with you that my basic understanding of your intent is correct. >>> Add #:cargo-tset-flags and pass it to cargo >>=20 >> Nitpick: The word "tset" should be "test=E2=80=9D. > > Whoops, that=E2=80=99s what I get for writing commit messages late at nig= ht :) It happens to the best of us! :-) > The reason we now copy the entire crate directory (rather than just the s= rc > directory) is that some crates have build scripts which usually live outs= ide > of `src` and are needed to successfully build. Although the Cargo.toml > has the path to the root build script, there are crates (like serde) which > have auxiliary build-script modules which aren=E2=80=99t shown in the man= ifest > contents. Rather than muck around and try to guess where the build script > code is, we can copy it all and let cargo sort it out. OK, that makes sense. I wasn't exactly sure why we changed this part, but now it makes sense. Thank you for explaining it! >> What about Cargo packages that are >> both libraries and applications? Do those even exist? > > Yes these are a bit rare but they do exist. I don=E2=80=99t have any exam= ples on > hand, but you can have something akin to curl which can be used > as a binary, as well as imported as a library to other projects. In those rare cases, your changes are still good to go, right? We don't actually interact with the Cargo.lock file, and if there are any executables, your code will install them. Something else occurred to me. In packages of C libraries, such as the glibc package, we install libraries (e.g., ".so" files go into the "out" output, and ".a" files go into "static" output). However, here we are not installing any libraries like that. Do all Rust developers just use Cargo.toml files to declare their dependencies, and then let Cargo manage all the actual compiling and linking? Do they ever manually install a Rust library (without using Cargo) when hacking on a project? I may be way off base here, since I'm not (yet!) an expert Rustacean. If I'm confused, please help me to understand. >>> + ;; Lift restriction on any lints: a crate author may have decided to= opt >>> + ;; into stricter lints (e.g. #![deny(warnings)]) during their own bu= ilds >>> + ;; but we don't want any build failures that could be caused later by >>> + ;; upgrading the compiler for example. >>> + (setenv "RUSTFLAGS" "--cap-lints allow") >>=20 >> Is this necessary? The docs seem to suggest that Cargo always sets it >> to "allow" anyway: >>=20 >> https://doc.rust-lang.org/rustc/lints/levels.html >>=20 >> "This feature is used heavily by Cargo; it will pass --cap-lints allow >> when compiling your dependencies, so that if they have any warnings, >> they do not pollute the output of your build.=E2=80=9D > > It=E2=80=99s true that cargo applies this to dependencies, but it doesn= =E2=80=99t do this > for the top level package that=E2=80=99s currently being built (e.g. if t= he CI is > building some library crate in isolation). As Guix maintainers, we > wouldn=E2=80=99t want jobs to start failing because of a new lint croppin= g up > somewhere in between versions. Ah, I see. Then yes, I agree: we should set it. >>> - (generate-checksums rsrc src) >>> + (generate-checksums rsrc "/dev/null") >>=20 >> This probably deserves a short comment to clarify the intent. > > Do you mean commenting on the intent of `generate-checksums` > or the intent of the /dev/null parameter? I mean the "/dev/null" argument, mainly. As far as I can tell, it looks like the generate-checksums procedure builds a file with checksums to satisfy some requirement of the cargo tool. That seems reasonable, but I'm not sure why we use "/dev/null" here. Finally, two more minor comments about code style which I don't think you need to change but are good to know going forward: =2D Instead of "system*", we prefer to use "invoke" (defined in (guix build utils)) whenever possible. It throws an exception when an error occurs, so it's harder to accidentally ignore errors. =2D Using "and" and "or" statements for control flow is OK, especially since you're using "system*". In fact, we do this in other parts of Guix, too. However, I personally feel that forms like "if", "when", and "unless" are clearer in some cases and should probably be preferred, especially when using "invoke" instead of "system*". That said, I think it's fine as is. Unless Danny (or someone else) has some more comments, I'll merge your changes in the next few days. =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlypt3sACgkQ3UCaFdgi Rp0Hmw/+IwUNWNTsCfRAIfOWlHHCNIll5uT0e1U1HXU83R/O/BCU1GB55w7cvGAS f00n3P5YtfCfZl1/uPwM7NDTK3eaI3SMuIQa7kaZ/Ax0AlU0tI8KIQq52z5UFi5c lNBFqC3PBox07huwhmiVX7Pc1AsdEzGNvjpHidDnE/NnTvJGiEDCVzbIlEYxAgrf wFPzq4zwK/JJrCdOjvQZUn1GPs8ImKiPrnjcp0nw7csX20Nr7qEYd5fnZLf18QsR LC8kj2dHPNZtxOuv0i+3SVKD+zmFfNUyi8Ufi46Zq+djWPH+uZmqoo4ZTZUNBm+i lTIDYVN4ihZZw3lAxAv4HCIskpP5Dd3WGqGa6Sqyhg0jRyRM6ZJ0JwNhpW57BVfJ tq6QzoZH+EZOTVkjxIRDoCto1m2Fw0L3qfbLFSr+AiqYdBgIreURDie8PKAfYVra k2lbNScBmYokJ/zKjiHOLsvdRA34gQGEodNQb4Hd5bI3UsshlI32EbKDYhv4Atyo 3jIjV6G9tUtoI3vjasXKQDBfdKU7/VSIQJkWBYFJsZnfOGrqEwl0qo43oC9df/tW PuXlXyLilerFGh3wzIHILigc3LRT5XvR36piRLNBNHhjMw6WEY+Oa0T1OWCKCMPS fLcT6F82X7+HzqfLJfGH95Rr7ZIeQzS4hxwDo3KzWIb1b/fS4S0= =oA/H -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 07 11:49:20 2019 Received: (at 35155) by debbugs.gnu.org; 7 Apr 2019 15:49:20 +0000 Received: from localhost ([127.0.0.1]:48611 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hDA2p-0006hj-V0 for submit@debbugs.gnu.org; Sun, 07 Apr 2019 11:49:20 -0400 Received: from mail-pl1-f178.google.com ([209.85.214.178]:39426) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hDA2n-0006hV-My for 35155@debbugs.gnu.org; Sun, 07 Apr 2019 11:49:18 -0400 Received: by mail-pl1-f178.google.com with SMTP id a96so1159463pla.6 for <35155@debbugs.gnu.org>; Sun, 07 Apr 2019 08:49:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=l5ZJCw4CzbtJ6Ba/MJqeItwWDLFkXeptm7V0i2azsjo=; b=RurY63dv9E7C6tHo1XsjrrSgd3VMNB8msDKvnSXL5OjO4hrrlYqWsBv0EDJ2KOGuBG pHmVx5WWlX1WUeEsH06m2AAa8CULCC+J14ABJbA1lilQyWO2aPyfCH766Jja5qTJfyQ+ eClPhcLPI7A3oP6S+O31AF2rb0/DLpRqj86MwImowtxlhw1h/H5ADqXuTiTB/evFAFW/ 5sxIVBrPgw3VT7/3z5eCLpkbHX8cvmPrC278zSGWes70O49HayWdtzJrhyFNS6e6cmyr jbouu3d7vydA/nFwiMpkJ1R9blxMKJ8uS2K4lotSX1kiwnXankq2CmcdOXlI81rYuGUI Jm2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=l5ZJCw4CzbtJ6Ba/MJqeItwWDLFkXeptm7V0i2azsjo=; b=h+EM778aEBhu1Ck91Fr+hHDurMZ86FVGJ1g66bpj02nrfqLFzXY6Keic8H3k3msFO7 kZouG9moSKWaweRq5xLFHsrCbSEKiQ4bifq3U0E6lG2FjjKw+erc+EAmxMcHm34B2bIl TuKhHyT0Ps+ebGLa00e1f6amZK4Dr7c47VrwWP/SQwWOOoTC5VJIIRXocSJBZihuFIbB A7nW+nUvNArOvxLsDVsDlJtpgQOik8QZbfIUjQe2doBb2jeR2Ja/Xl9m4eksOi/k5Bij YpBmA2k7l4Ak/0sqwDTzxHSyY2t//Xgvy04EJs+axZ3DmxyFqWJZCBXpEpXXL7iDOpou Y9/A== X-Gm-Message-State: APjAAAWs0NTgkG3uPaL0dMKJ6xBUbKjJtDHJ6dyMXPdKaomCJyVzVXnN bRkhx3GKzFLNpaY8NT07hM8= X-Google-Smtp-Source: APXvYqwVsWCvIFWj6lW0AWyvMtvVaq3XNQo9wjeeM0tF9nBeri54YyGl/GYfGxbcv5KAdq100Y3Yjg== X-Received: by 2002:a17:902:a98b:: with SMTP id bh11mr25381706plb.63.1554652151704; Sun, 07 Apr 2019 08:49:11 -0700 (PDT) Received: from ?IPv6:2601:602:9a00:1784:cd85:caae:a089:88e7? ([2601:602:9a00:1784:cd85:caae:a089:88e7]) by smtp.gmail.com with ESMTPSA id b14sm39910687pfi.92.2019.04.07.08.49.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 07 Apr 2019 08:49:10 -0700 (PDT) Content-Type: multipart/alternative; boundary="Apple-Mail=_27ECDF33-0D30-49F1-AD48-71AC0A46AEA0" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: [PATCH] build-system/cargo: refactor phases to successfully build From: Ivan Petkov In-Reply-To: <877ec6jjic.fsf@gmail.com> Date: Sun, 7 Apr 2019 08:49:09 -0700 Message-Id: References: <87k1g6zpd9.fsf@gmail.com> <51B2A728-850F-4A13-8A6B-D12FD0B0C119@gmail.com> <877ec6jjic.fsf@gmail.com> To: Chris Marusich X-Mailer: Apple Mail (2.3445.9.1) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 35155 Cc: Danny Milosavljevic , 35155@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --Apple-Mail=_27ECDF33-0D30-49F1-AD48-71AC0A46AEA0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Chris, > On Apr 7, 2019, at 1:40 AM, Chris Marusich = wrote: >=20 > Do you mean that if a crate X has an optional feature that requires > crate Y, then Cargo requires Y to be present when building X even if X > is being built with that feature disabled? Correct, the source to crate Y must be present when crate X is being = built, transitively, or otherwise. >> * If we have a =E2=80=9Ccircular=E2=80=9D dependency as part of = dev-dependencies >> (e.g. one crate pulls in an upstream crate during testing to ensure = it >> doesn=E2=80=99t break anything) we=E2=80=99ll need to detangle the = dependency graph by >> rewriting duplicate packages to include the #:skip-build? flag. I can >> elaborate more on this in a separate email. >=20 > I think here, you're talking about the situation in which crate A > depends on crate B, which in turn depends on crate A's source, and to > break the cycle we will replace B's dependency on A with a dependency = on > A', where A' is effectively just a source build of A (via = #:skip-build? > #t). Is that basically right? >=20 > I agree it would be good to discuss the resolution of circular > dependencies in another email thread, but I just wanted to double = check > with you that my basic understanding of your intent is correct. Yep, your understanding of the situation and potential solution is = correct. I=E2=80=99ll send out a different email thread around this since we = might need some small tweaks to the overall build system to support this. >>> What about Cargo packages that are >>> both libraries and applications? Do those even exist? >>=20 >> Yes these are a bit rare but they do exist. I don=E2=80=99t have any = examples on >> hand, but you can have something akin to curl which can be used >> as a binary, as well as imported as a library to other projects. >=20 > In those rare cases, your changes are still good to go, right? We = don't > actually interact with the Cargo.lock file, and if there are any > executables, your code will install them. Yep, there=E2=80=99s no harm in building the =E2=80=9Csrc=E2=80=9D = output of an application crate (outside of using up some store space) in case something else wants to depend on it. > Something else occurred to me. In packages of C libraries, such as = the > glibc package, we install libraries (e.g., ".so" files go into the = "out" > output, and ".a" files go into "static" output). However, here we are > not installing any libraries like that. Do all Rust developers just = use > Cargo.toml files to declare their dependencies, and then let Cargo > manage all the actual compiling and linking? Do they ever manually > install a Rust library (without using Cargo) when hacking on a = project? In general, cargo is used to perform all building an linking of the = final rust outputs (binaries, shared libraries, and other rust artifacts). Cargo = also supports defining arbitrary build scripts such as building some other non-rust = dependency which is to be linked in the final outputs. https://doc.rust-lang.org/cargo/reference/build-scripts.html = However, cargo does not perform the actual distribution of these = =E2=80=9Cexternal=E2=80=9D dependencies. Crates may vendor their own external sources, or they may expect them to be present in the usual places, or they may support an = environment variable which points them in the right direction. One example is the `jemalloc-sys` crate. If built with an environment = variable pointing to a compiled version of jemalloc, it will build rust bindings = which link to that binary. Otherwise it will build its own vendored version of the = jemalloc source. Handling these packages will need to be done on a case-by-case basis with some additional setup glue in the package definitions, as = necessary. >>>> - (generate-checksums rsrc src) >>>> + (generate-checksums rsrc "/dev/null") >>>=20 >>> This probably deserves a short comment to clarify the intent. >>=20 >> Do you mean commenting on the intent of `generate-checksums` >> or the intent of the /dev/null parameter? >=20 > I mean the "/dev/null" argument, mainly. As far as I can tell, it = looks > like the generate-checksums procedure builds a file with checksums to > satisfy some requirement of the cargo tool. That seems reasonable, = but > I'm not sure why we use "/dev/null" here. Cargo expects the checksum package to include a checksum of each = individual file as well as a checksum for the entire directory. The = `generate-checksums` procedure doesn=E2=80=99t correctly handle the second parameter being a = directory and raises an error. Luckily, cargo doesn=E2=80=99t seem to care about = the contents of this checksum, as long as the declaration is there. I didn=E2=80=99t want to change the `generate-checksums` procedure just = yet since it=E2=80=99s used during building of rust-1.20, and doing so will = require a full bootstrap of the compiler chain, which would have blocked me for a while. > Finally, two more minor comments about code style which I don't think > you need to change but are good to know going forward: >=20 > - Instead of "system*", we prefer to use "invoke" (defined in (guix > build utils)) whenever possible. It throws an exception when an = error > occurs, so it's harder to accidentally ignore errors. >=20 > - Using "and" and "or" statements for control flow is OK, especially > since you're using "system*". In fact, we do this in other parts of > Guix, too. However, I personally feel that forms like "if", "when", > and "unless" are clearer in some cases and should probably be > preferred, especially when using "invoke" instead of "system*=E2=80=9D.= Thank you for the tips, I=E2=80=99ll keep these in mind going forward. Still pretty new to guile, its APIs, and guix=E2=80=99s utilities on top = of that! :) =E2=80=94Ivan --Apple-Mail=_27ECDF33-0D30-49F1-AD48-71AC0A46AEA0 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi = Chris,

On Apr 7, 2019, at 1:40 AM, Chris Marusich = <cmmarusich@gmail.com> wrote:

Do you mean that if a crate X has an optional = feature that requires
crate Y, then Cargo requires Y to be = present when building X even if X
is being built with that = feature disabled?

Correct, the source to crate Y must be present = when crate X is being built,
transitively, or = otherwise.

* If we = have a =E2=80=9Ccircular=E2=80=9D dependency as part of = dev-dependencies
(e.g. one crate pulls in an upstream = crate during testing to ensure it
doesn=E2=80=99t break = anything) we=E2=80=99ll need to detangle the dependency graph by
rewriting duplicate packages to include the #:skip-build? = flag. I can
elaborate more on this in a separate email.

I think here, you're talking = about the situation in which crate A
depends on crate B, = which in turn depends on crate A's source, and to
break = the cycle we will replace B's dependency on A with a dependency on
A', where A' is effectively just a source build of A (via = #:skip-build?
#t).  Is that basically right?

I agree it would be good to discuss the = resolution of circular
dependencies in another email = thread, but I just wanted to double check
with you that my = basic understanding of your intent is correct.

Yep, your = understanding of the situation and potential solution is = correct.
I=E2=80=99ll send out a different email thread around = this since we might need some
small tweaks to the overall = build system to support this.

What about Cargo = packages that are
both libraries and applications? =  Do those even exist?

Yes = these are a bit rare but they do exist. I don=E2=80=99t have any = examples on
hand, but you can have something akin to curl = which can be used
as a binary, as well as imported as a = library to other projects.

In = those rare cases, your changes are still good to go, right?  We = don't
actually interact with the Cargo.lock file, and if = there are any
executables, your code will install them.

Yep, = there=E2=80=99s no harm in building the =E2=80=9Csrc=E2=80=9D output of = an application crate
(outside of using up some store space) in = case something else wants to
depend on it.

Something else occurred to me.  In packages of C = libraries, such as the
glibc package, we install libraries = (e.g., ".so" files go into the "out"
output, and ".a" = files go into "static" output).  However, here we are
not installing any libraries like that.  Do all Rust = developers just use
Cargo.toml files to declare their = dependencies, and then let Cargo
manage all the actual = compiling and linking?  Do they ever manually
install = a Rust library (without using Cargo) when hacking on a = project?

In general, = cargo is used to perform all building an linking of the final = rust
outputs (binaries, shared libraries, and other rust = artifacts). Cargo also supports
defining arbitrary build = scripts such as building some other non-rust dependency
which = is to be linked in the final outputs.





- =    (generate-checksums rsrc src)
+ =    (generate-checksums rsrc "/dev/null")

This probably deserves a short = comment to clarify the intent.

Do you mean commenting on the intent of = `generate-checksums`
or the intent of the /dev/null = parameter?

I mean the = "/dev/null" argument, mainly.  As far as I can tell, it looks
like the generate-checksums procedure builds a file with = checksums to
satisfy some requirement of the cargo tool. =  That seems reasonable, but
I'm not sure why we use = "/dev/null" here.

Cargo expects the checksum package to include a = checksum of each individual
file as well as a checksum for the = entire directory. The `generate-checksums`
procedure doesn=E2=80= =99t correctly handle the second parameter being a = directory
and raises an error. Luckily, cargo doesn=E2=80=99t = seem to care about the contents
of this checksum, as long as = the declaration is there.

I didn=E2=80= =99t want to change the `generate-checksums` procedure just yet = since
it=E2=80=99s used during building of rust-1.20, and = doing so will require a full bootstrap
of the compiler chain, = which would have blocked me for a while.

Finally, two = more minor comments about code style which I don't think
you= need to change but are good to know going forward:

- Instead of "system*", we prefer to use "invoke" (defined in = (guix
 build utils)) whenever possible.  It = throws an exception when an error
 occurs, so it's = harder to accidentally ignore errors.

- = Using "and" and "or" statements for control flow is OK, especially
 since you're using "system*".  In fact, we do = this in other parts of
 Guix, too.  However, I = personally feel that forms like "if", "when",
 and = "unless" are clearer in some cases and should probably be
=  preferred, especially when using "invoke" instead of = "system*=E2=80=9D.

Thank you for the tips, I=E2=80=99ll keep these in mind = going forward.

= --Apple-Mail=_27ECDF33-0D30-49F1-AD48-71AC0A46AEA0-- From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 09 12:01:31 2019 Received: (at 35155-done) by debbugs.gnu.org; 9 Apr 2019 16:01:32 +0000 Received: from localhost ([127.0.0.1]:51904 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hDtBj-0005bq-Kp for submit@debbugs.gnu.org; Tue, 09 Apr 2019 12:01:31 -0400 Received: from mail-pg1-f173.google.com ([209.85.215.173]:34565) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hDtBh-0005bZ-QS for 35155-done@debbugs.gnu.org; Tue, 09 Apr 2019 12:01:30 -0400 Received: by mail-pg1-f173.google.com with SMTP id v12so9553648pgq.1 for <35155-done@debbugs.gnu.org>; Tue, 09 Apr 2019 09:01:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6iRDTLK2Tmk4YCB6uDkewcmWAIc1z9xKeLZ53lBE1gQ=; b=nOPd0Ykzl/GlDNIaYhxCUfbDgnRk+PId08dYbWrdDzBAwL1AIMlRXgJbZjkKG/tz5p F9KhqcvH4i54KSp2AibNSZMKzbn8LA+4SCCmuyqORmYRWL8iuHQkaOlxidTy12gJl71c KcGwZWSC/MX61jSPwJo+Iob0ddk96BbJmw43ltrnOYN6WdCweOGHHsfACqgnRAkBlWGb mBXePtS6rSemfQ1Vi3AyCn2ivJ8kiagjTfb6vvy9SdKAXTU/7UxZgKx/90jEQP+OE3O3 lirtIhfOU6oQm0qwUFLZOle5HR4akNuR1wknRtlHeqpoQSC9UG+YKO54OsRlll57m5PL dm0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6iRDTLK2Tmk4YCB6uDkewcmWAIc1z9xKeLZ53lBE1gQ=; b=DwiMUf9fepPVDkHpwADMar5KILKWA4Vu1i3O4ovsaVCOfO9YkJYRuY6EyyXNHfJop/ AYPJzMIZ93wzrG//GRziRcygWf6MmIqooR0Exo4fYs9ILY6swKJSbSpLL/RfP9lQvkTy Q5Z9rh2hFSFWty6DOqykMPvxifI2SsZXQzwlXK3KMHKNzfl01iI86UvXVOSguQeWRIf6 EZYCk+m6VESNfFZqpGsc8AnTp1HogVHqIcPsCQEidksImRwn3+RvbiF8b0WChmFFqFWy dW0v12WN1mDkNtGlWRb2j418n7A9ofR4nWonzfCwV7MJSjTQEI9K+xfK4b+b/cRNKo7z TUTg== X-Gm-Message-State: APjAAAXaIkkjZ8+Wp1sooVAqcXz16BblcGnnyw2xpG1T20N8hekb4un6 WIvteZjO9kJdKz2fs7LFExc= X-Google-Smtp-Source: APXvYqwtcSJ00LnV24D/l3PNJ2vI1KZx9PKJ0G9YKJ11cSOFaLC9t2srmg5Z3HqXZNHtNY77BCkX3w== X-Received: by 2002:a63:4e5b:: with SMTP id o27mr35734871pgl.204.1554825683485; Tue, 09 Apr 2019 09:01:23 -0700 (PDT) Received: from ?IPv6:2601:602:9a00:1784:90ba:5d2:21bb:17c2? ([2601:602:9a00:1784:90ba:5d2:21bb:17c2]) by smtp.gmail.com with ESMTPSA id h20sm40579735pfj.40.2019.04.09.09.01.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2019 09:01:21 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: [bug#35155] [PATCH] build-system/cargo: refactor phases to successfully build From: Ivan Petkov In-Reply-To: <87mukz8p3g.fsf@garuda.local.i-did-not-set--mail-host-address--so-tickle-me> Date: Tue, 9 Apr 2019 09:01:20 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87k1g6zpd9.fsf@gmail.com> <51B2A728-850F-4A13-8A6B-D12FD0B0C119@gmail.com> <877ec6jjic.fsf@gmail.com> <87mukz8p3g.fsf@garuda.local.i-did-not-set--mail-host-address--so-tickle-me> To: Chris Marusich X-Mailer: Apple Mail (2.3445.9.1) X-Spam-Score: -0.3 (/) X-Debbugs-Envelope-To: 35155-done Cc: 35155-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.3 (-) Thank you Chris! =E2=80=94Ivan > On Apr 9, 2019, at 3:12 AM, Chris Marusich = wrote: >=20 > Hi Ivan, >=20 > I've committed this as 1d3acde5087d50af6a4901fd7614f0940eb7b41d on > master. Thank you! >=20 > --=20 > Chris From unknown Fri Jun 20 07:29:12 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 08 May 2019 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator