From unknown Mon Aug 18 02:05:27 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#41607 <41607@debbugs.gnu.org> To: bug#41607 <41607@debbugs.gnu.org> Subject: Status: Deleted store items are not actually deleted Reply-To: bug#41607 <41607@debbugs.gnu.org> Date: Mon, 18 Aug 2025 09:05:27 +0000 retitle 41607 Deleted store items are not actually deleted reassign 41607 guix submitter 41607 Leo Famulari severity 41607 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Fri May 29 15:09:48 2020 Received: (at submit) by debbugs.gnu.org; 29 May 2020 19:09:48 +0000 Received: from localhost ([127.0.0.1]:56245 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jekO4-0004M9-21 for submit@debbugs.gnu.org; Fri, 29 May 2020 15:09:48 -0400 Received: from lists.gnu.org ([209.51.188.17]:36764) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jekO2-0004M1-Pu for submit@debbugs.gnu.org; Fri, 29 May 2020 15:09:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58916) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jekO2-0000PG-Kr for bug-guix@gnu.org; Fri, 29 May 2020 15:09:46 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:46765) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jekO1-0008UN-8m for bug-guix@gnu.org; Fri, 29 May 2020 15:09:46 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 7EF495C0090; Fri, 29 May 2020 15:09:44 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Fri, 29 May 2020 15:09:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=famulari.name; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=mesmtp; bh=N1yK4amboOKQ9P82+5RvNRH9 MMeWNHQpS0fuHCesbbQ=; b=eQSGHTZ/AfLuckiavrYeDLXlze/3rsaMalTw+a+0 /Ad4g+jMrAsP+c4SnTouh62ppn5aqAwy9s6makPClQiRrFM1bMf7hoilsavpi9Dq bNuw1n2mQXfpSt5CP1YjRzYOXfacKNee8lsmuBgYqBALni10ySzkIeXOYXb2BsRt cAU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=N1yK4a mboOKQ9P82+5RvNRH9MMeWNHQpS0fuHCesbbQ=; b=E+BqpELCmVd/Z8RJ15bvCK f52kYblTQuug/+N2nM89OCCwsDUkcoykb1sIL88HWJO2I9wtmjgN/i9enxy/E6Kv ezDCNd6Tq7FTgexdfAS6JF80s0W3O5nLnYWDxHsSzB1wYbfQnPa/4484p8g5d+1b FEFmMJ3Dr9/FcGJ8hKHGLUppnR+IP2wZg0QuxlqWiKQMt8wh4d3c9mC/5DN9vXvt DDavVq6YE2ICo9hjgM7BJd7YbYWMAKrwavqfd8mLNCDpFr28I6O7sCMI522mQSul +4pR9QgcLZpRjXuhv1MLCXRe4BH3WA7MTlgLL2dQnAv2m3xDzLw7jD/dsspeW3nQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedruddvkedguddtlecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehttd ertddttddvnecuhfhrohhmpefnvghoucfhrghmuhhlrghrihcuoehlvghosehfrghmuhhl rghrihdrnhgrmhgvqeenucggtffrrghtthgvrhhnpedvvddugefffeeitddthfefvdeuhf fgkeeikeegkeevteeghfeftefggeeuudffieenucffohhmrghinhepghhnuhdrohhrghen ucfkphepjeeirdduvdegrddufeekrdeifeenucevlhhushhtvghrufhiiigvpedtnecurf grrhgrmhepmhgrihhlfhhrohhmpehlvghosehfrghmuhhlrghrihdrnhgrmhgv X-ME-Proxy: Received: from localhost (c-76-124-138-63.hsd1.pa.comcast.net [76.124.138.63]) by mail.messagingengine.com (Postfix) with ESMTPA id 10F473060F09; Fri, 29 May 2020 15:09:44 -0400 (EDT) Date: Fri, 29 May 2020 15:09:42 -0400 From: Leo Famulari To: Stephen Scheck Subject: Deleted store items are not actually deleted Message-ID: <20200529190942.GA8440@jasmine.lan> References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Received-SPF: pass client-ip=66.111.4.25; envelope-from=leo@famulari.name; helo=out1-smtp.messagingengine.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/29 15:09:44 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: bug-guix@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) >From the discussion "Guix Docker image inflation" [0] on help-guix: On Fri, May 29, 2020 at 02:29:53PM -0400, Stephen Scheck wrote: > # Now try to delete it... > root@localhost /gnu/store# guix gc --delete > /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules > finding garbage collector roots... > [0 MiB] deleting > '/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' > deleting `/gnu/store/trash' > deleting unused links... > note: currently hard linking saves 1181.36 MiB > > # Still there... > root@localhost /gnu/store# du -hs > /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules > 210M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules Okay, something is definitely not right. [0] https://lists.gnu.org/archive/html/help-guix/2020-05/msg00235.html From debbugs-submit-bounces@debbugs.gnu.org Fri May 29 19:36:25 2020 Received: (at 41607) by debbugs.gnu.org; 29 May 2020 23:36:25 +0000 Received: from localhost ([127.0.0.1]:56498 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jeoY4-0004Xj-N5 for submit@debbugs.gnu.org; Fri, 29 May 2020 19:36:24 -0400 Received: from mail-qk1-f177.google.com ([209.85.222.177]:42134) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jeoY3-0004XX-Kk for 41607@debbugs.gnu.org; Fri, 29 May 2020 19:36:23 -0400 Received: by mail-qk1-f177.google.com with SMTP id s1so3867191qkf.9 for <41607@debbugs.gnu.org>; Fri, 29 May 2020 16:36:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=o9wg4r4NkHRoP9WEfnjZEo/JWPXS42894n5h9Czlfjo=; b=fs1w5ZCFR4QkO1KblZNCBZxveNcXPQuOPFjdtIqW85Ho/pYxulpaTWtzg2SwJXQ5OM B4yQZ1F5ecpDGqBwk2VyBV/K4j28LGcptKTPO7G2rDRHIBiYjBEOdQwYvHnAGICX2XUp XkALDFJQFc+3wB0vK3aa+Ttd0XdSiFQxaC8kleqHOu/vNmoQkg6h5qY1rmc4qyHpGUbS IHacMOyyZQSZIoLTc85kjVTT1f2o3Htg7zrRSNGznYg8wA8H8zXUrt1W2xySx6bdcV8+ JCvhV01yVv8IIMIdGkjL8HCW1ATrZXAb8Sa+inio9T40KfODxgCPIFIaD2+7+kKWW6SA AJzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=o9wg4r4NkHRoP9WEfnjZEo/JWPXS42894n5h9Czlfjo=; b=WXm31C8BZp/Ze2LPp8eqhZNJp6/7eS+VSLhG69Hi4Tt0B6FGSq2FW3VbIbNerGgDTK A1n8D/FhbN+/o4qyPkYAy+3Scc8DcJTnl109c2e7CJ0ETwvYbqw7jhDd9Jg5wK7YLdk+ urJPee9boJ/oeYnTEJFmYzyczqZGpD6qWSwWhwHHRUBdkeyiFywzUFD9I6mKtcNUwtyr gAPign3t8iOhE8XqtVJtF3iipA19eHDZaS06HcC7CiZa6lywTGHb+JeHK/yT6aGO66nn s10EXrvpO3tq6+hPw6nXbO6KLQw1J7HTeH50SYSedSTUFfAXA3riLjayyyvBrChbQ8rA ignw== X-Gm-Message-State: AOAM531h8e+Cm1UDnR+wGaKQh1hvzpedHoTmKHaEicb1PiksBDngIIf2 xtpKUBk+DLCI4MbiehpJafKuGDsZ0r89fOWlkvc= X-Google-Smtp-Source: ABdhPJx4aXZ/FugOguqvK7tIgYVfctCMjjEIs9iRNJURY4NaV3Taw/bbw2OWw21EVeVeld5pctBlryGXk+WaY/gHTBk= X-Received: by 2002:a05:620a:148a:: with SMTP id w10mr9268816qkj.201.1590795378117; Fri, 29 May 2020 16:36:18 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: zimoun Date: Sat, 30 May 2020 01:36:06 +0200 Message-ID: Subject: bug#41607: Deleted store items are not actually deleted To: Stephen Scheck , 41607@debbugs.gnu.org Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: Marius Bakke , Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) -help-guix On Sat, 30 May 2020 at 00:11, Stephen Scheck wrote: >> Do you have '/var/' in your Docker image? Because it looks like the same than: > If you'd like, you can fetch the exact same image and look around yourself: > > docker pull singularsyntax/guix:master-a5374cd # same as singularsyntax/guix:latest Thanks! I have already did with singularsyntax/guix-bootstrap-alpine but then I hit some issues... Well another story. :-) Why there is (in both images) 13 "layers"? > CONTAINER=`docker run --detach --tty --privileged singularsyntax/guix:master-a5374cd` > docker exec --interactive --tty $CONTAINER /run/current-system/profile/bin/bash --login --8<---------------cut here---------------start------------->8--- for f in $(guix gc --list-dead); do du -sh $f; done | sort -h [...] 103M /gnu/store/flcfw8pcs2s0wnm78lp1jwid5lglky3j-guix-packages-base 103M /gnu/store/hl8h9krwiqwn51gydc13jz4846gq55pr-guix-packages-base 106M /gnu/store/mnvkhj7crqcd0d1xa0zqa3hd1f5hmv83-guix-packages-base 107M /gnu/store/6sggbpgg0zkbgxwf3wa2j15dis8z7cr1-guix-packages-base 107M /gnu/store/m0fv2xmfif5pxnfb1bscfvgyfx0x6xdc-guix-packages-base 107M /gnu/store/n339sr8c63f0nzja6yl8zfwy1jklj19j-guix-packages-base 107M /gnu/store/plaay02w581vx9ilyiv93sl1lw54n7h5-guix-packages-base 205M /gnu/store/5mhn1ynxvy7jihsknsnv3yspkkvc0r5s-guix-2e59ae238-modules 205M /gnu/store/8z9qc2bvq8azc08p4miq77yf2agk07aq-guix-843e77205-modules 205M /gnu/store/brbwlbnx56ms50kklyqk9fsf0xkwjjf9-guix-498e2e669-modules 205M /gnu/store/d3h4b7nvnms8d03ddi9b481dlxpykl7l-guix-5e3d16994-modules 205M /gnu/store/ibgjq1ampj8bldrabbsnwik2sr0gg3as-guix-a43fe7acd-modules 205M /gnu/store/l3amdz5xyhflg5wdzlxr2685dq5glic2-guix-527ab3125-modules 210M /gnu/store/0vwg9aqzs5xrk10vcs4dl105s3f42ilf-guix-b1affd477-modules 210M /gnu/store/47aack48aczpzm635axsy4jf2pvmwrv0-guix-ef1d475b0-modules 210M /gnu/store/hz2rn2l0jixg91q4rsdcwc489y71ll29-guix-05e1edf22-modules 210M /gnu/store/mych9fchln22pbhpc5syxyymx4hz496y-guix-8bd0b533b-modules 210M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules --8<---------------cut here---------------end--------------->8--- Well, there is 11 generations which corresponds to --8<---------------cut here---------------start------------->8--- guix describe Generation 12 May 29 2020 22:51:28 (current) guix a5374cd repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: a5374cde918cfeae5c16b43b9f2dd2b24bc3564d --8<---------------cut here---------------end--------------->8--- But "docker pull" downloaded 13 layers, well because these numbers are closed, is it meaningful? Really naive question. Other said, is it not something from the Docker side? Well, it is like Guix deletes something but this something is empty (from the Guix side): --8<---------------cut here---------------start------------->8--- guix gc -D /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules finding garbage collector roots... [0 MiB] deleting '/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' deleting `/gnu/store/trash' deleting unused links... note: currently hard linking saves 1181.36 MiB du -sh /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules 210M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules --8<---------------cut here---------------end--------------->8--- abd the "real" target is still something on the filesytem side (du -sh). Bah, I am not enough familiar with Docker... Good night, simon From debbugs-submit-bounces@debbugs.gnu.org Sun May 31 00:56:52 2020 Received: (at 41607) by debbugs.gnu.org; 31 May 2020 04:56:52 +0000 Received: from localhost ([127.0.0.1]:59550 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfG1k-00057N-El for submit@debbugs.gnu.org; Sun, 31 May 2020 00:56:52 -0400 Received: from mail-pf1-f171.google.com ([209.85.210.171]:32993) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfG1h-000578-RA for 41607@debbugs.gnu.org; Sun, 31 May 2020 00:56:51 -0400 Received: by mail-pf1-f171.google.com with SMTP id n15so1911790pfd.0 for <41607@debbugs.gnu.org>; Sat, 30 May 2020 21:56:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version; bh=FXyPjJoiOr86lhWv0bOtjpLFSsZvqTgS4SMhzSnsYhY=; b=tOSxs12WJnPKgUIc4WmDjI1zeVrLVG6EbHy0PhGTtIsGwxmLYkKE9LSy8ta4CvL+uQ 2yZALjoLoeByk94DFFkXQJsZ3aLIjOH6RaJsA2VrijoPJ1QZza9X60kz7ETJ65CKhEgs jtYHS2w9G38AN0dhY6550TpycOCqgvi6m5RYY35S70BntBWkKIj/U0lwcMbMTFSQIhMG QqRPWoDHbg52njePzD2g38CfckGpQEfs2grYQ1f1ftuUcaDw1Hrg/wxMtW7C2btB6oQz Ox1wcYcfe0LQZxNums2RLUYtNKapyE+EVPQpF3Uw4y5CdSBe4C/LVafVmuxoNviDV4GL oZ6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references :user-agent:date:message-id:mime-version; bh=FXyPjJoiOr86lhWv0bOtjpLFSsZvqTgS4SMhzSnsYhY=; b=fFKMG+BaFqHfgEX+v7JVfePhuLV1QFmaCKa8fsiPkvZbgDkLfMJcoRWYvhhpOJ2RBr HEGFuco0FKRGVhHoWJ/cY2ft0NwiATICvovmgmQN0ncXf8x2lUYaeKK+I3r8kZxtzFpO 6ITbb1iVH/ALCvEehy1+8taYdzQfECAqKv+PhnUQ8iAWiQryofsCq91u30lH7wyDKgRL rgWU+9ebFs3U+tDDdmArkHxWuZKKzkmaQZ+UJxCibwuQr3EYB1bhoOXGxLx7I9+JbE5e UzrZ6rtnnjsTXghPeJ7b5tL8w7A3KngXfNiSib3xHQjT/BA6XRiLrmyplxiEubMtgsRs yotA== X-Gm-Message-State: AOAM531v1XMTDFtngYznybxjzELtk1P5mBlqQTeyYMRfpiKJqCRwyARW H1C/Chtmg/6jZDuLRWjwLwufwuLuybs= X-Google-Smtp-Source: ABdhPJyuRwXbinv9rbZwcP5pr3OTutOV1q3674IBoiBJwIaVweIRRGBZDEtGql247o1syAd0xR/BHg== X-Received: by 2002:a62:1885:: with SMTP id 127mr15352586pfy.258.1590901003201; Sat, 30 May 2020 21:56:43 -0700 (PDT) Received: from garuda-lan (c-73-97-103-127.hsd1.wa.comcast.net. [73.97.103.127]) by smtp.gmail.com with ESMTPSA id p190sm11072267pfp.207.2020.05.30.21.56.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 May 2020 21:56:42 -0700 (PDT) From: Chris Marusich To: Leo Famulari Subject: Re: bug#41607: Deleted store items are not actually deleted In-Reply-To: <20200529190942.GA8440@jasmine.lan> (Leo Famulari's message of "Fri, 29 May 2020 15:09:42 -0400") References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) Date: Sat, 30 May 2020 21:56:38 -0700 Message-ID: <87r1v0k8hl.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: 41607@debbugs.gnu.org, Stephen Scheck X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi Leo, Stephen, and others, I originally wrote a very detailed email describing my investigation of this issue and the results. However, I accidentally deleted it, so please bear with me as I write a rough summary instead. Leo Famulari writes: >>>From the discussion "Guix Docker image inflation" [0] on help-guix: > > On Fri, May 29, 2020 at 02:29:53PM -0400, Stephen Scheck wrote: >> # Now try to delete it... >> root@localhost /gnu/store# guix gc --delete >> /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules >> finding garbage collector roots... >> [0 MiB] deleting >> '/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' >> deleting `/gnu/store/trash' >> deleting unused links... >> note: currently hard linking saves 1181.36 MiB >>=20 >> # Still there... >> root@localhost /gnu/store# du -hs >> /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules >> 210M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules > > Okay, something is definitely not right. > > [0] https://lists.gnu.org/archive/html/help-guix/2020-05/msg00235.html There are two problems. One is that Stephen's images are getting larger every day. The other is that Guix is failing to GC dead store items in the Docker container. The latter does not cause the former. The reason Guix is failing to GC dead items in the Docker container is because those dead items are not on the "top layer", so Docker returns an EXDEV error: https://docs.docker.com/storage/storagedriver/overlayfs-driver/ "Renaming directories: Calling rename(2) for a directory is allowed only when both the source and the destination path are on the top layer. Otherwise, it returns EXDEV error ('cross-device link not permitted'). Your application needs to be designed to handle EXDEV and fall back to a 'copy and unlink' strategy." You can observe this by running guix-daemon with strace in the container, and watching what happens when you try to delete one of the offending store items (make sure it is a directory). For example: =2D-8<---------------cut here---------------start------------->8--- 685 rename("/gnu/store/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database",= "/gnu/store/trash/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database") =3D -= 1 EXDEV (Invalid cross-device link) =2D-8<---------------cut here---------------end--------------->8--- In most cases, when guix-daemon GC's a dead directory, it does this (see: nix/libstore/gc.cc): =2D Create a trash directory (usually /gnu/store/trash) =2D Move dead directories into the trash directory. =2D Delete the trash directory. The trash directory is on the "top layer" because it gets created in the running container. However, in practice many store items from lower layers are made dead when Stephen's script runs "guix pull" and deletes the old profiles. If any of those store items were directories, guix-daemon will fail to GC them because of an XDEV error. If this is confusing to you, I suggest you experiment with Docker a little bit, and look closely at the steps that Stephen's script is running. I outlined this in the email I accidentally deleted, but I'm a little too tired to reproduce it all a second time. I hope you'll understand. Should Guix do anything about this? We could change guix-daemon to take correct action in the face of an XDEV error. We could also improve the logging, since currently it silently swallows the XDEV error. However, even if we make those changes and Guix is able to GC the dead store items, it would not prevent Stephen's images from growing in size without bound. There would still be many store items that came from a prior image (i.e., a lower layer), became dead after running "guix pull" and deleting old profile generations, and still exist in the prior layer, even though they are not visible in the running container. This is due to Docker's design, in which the visible file system is the result of stitching together all the layers with overlayfs. To work around the issue, Stephen can build the images from the same base image, rather than daisy-chaining new images from old ones. That way, they would not accumulate layers without bound. =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAl7TOQYACgkQ3UCaFdgi Rp33Ug/9ECuQ7z0lIJi4RJrDuJynnVbEEXGRN0FLUP55HoWULXGvd/cWxKC1Cr5F NRd598QP+40EYy5SVzNrxXUXTRqmQDSUYbIobUINSwcn+q/HjxwLbTnp6v2fijtM aaSc2YoCFzZaPT6MUjb8CB2Spgf/DF9XbwBmpJZdTK84w0wyDb2KEWRrO1bxq7c2 9tdDv3jylah3+Z7k/VnzDyGvJxqrqSsUWdRvJy4QL9THXaU5EP4zRCt8eHGGgaRs SeyjhbYGGCqh2emQOQptEw3AlFnginulO2P0g4dmRx4SaACMnfVfcopB1jGxPNHv YQXEn2o1/AnFxYhUMzycrXhT2pgjHMVrLBWihaCdZxYP7SSyS1UgHZEUfPknYA7H xhoybpHzVDgECiN3MID/s2mTeFZEofianvDy+Wt1/mSkWWwiRS9Eg75UWvWvrcAi 5VD/WhaOpPtdS+UuqM6geLmRRaA2G+aAU9vxtzdZ9uB12iM4Jio/vVWIbfiHofFg acYkYIwibQAKj3K2mfbl/MGyTLL9zc0TIlmLVDOGx4wpOdNeczSkorf3n0Un61mS 1iuA2aThBVn/bdTQRzNEVy9nqN2nbe5oCXesv1NG76lANNC30c4iFLeqsPRipxAM z+nFAWDAWJzviGglcQnyIy4S97T8n4Kq8RYY9AJrQEZhmT7pY24= =dn7X -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun May 31 04:24:45 2020 Received: (at 41607) by debbugs.gnu.org; 31 May 2020 08:24:45 +0000 Received: from localhost ([127.0.0.1]:59690 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfJGu-0001rk-VO for submit@debbugs.gnu.org; Sun, 31 May 2020 04:24:45 -0400 Received: from mail-qk1-f170.google.com ([209.85.222.170]:41622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfJGs-0001rX-IY for 41607@debbugs.gnu.org; Sun, 31 May 2020 04:24:43 -0400 Received: by mail-qk1-f170.google.com with SMTP id n11so6319035qkn.8 for <41607@debbugs.gnu.org>; Sun, 31 May 2020 01:24:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=daSnxJIw51jCctLVLCCAYUum6szeqTFGwjdzZbh4jZ4=; b=tSKSFfmfjdmCO3ZrDp0vkRc3bDuqgvEs4Eqrml4m8Ussqy8bsvxogt94Q0VuAxnqVI iiiCnWL53dE5DyTTIVo0c4gx0CCUWk5yxwNojIpSAnw3GzpJLQ7rNfEjBPNqZYe9/5zq peK48bmWWsJ9PFb/TW1kLd4YEclUoDe+wkkA0L4+mM/sLeomp3XDlyiJBXvaFq+b/Rqd Daj64w/OUjrc730qI1pXzwMtfIYJO1Dvrlpwo3Gion8UChQz67G4ObohhUby6e1yQvZw wDBWNIH/+9eta3dz2ocBrazBjqw69Zo2PfZfE82URVHesXmzWl0/lTVFmkA6ijzTOWzY dA1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=daSnxJIw51jCctLVLCCAYUum6szeqTFGwjdzZbh4jZ4=; b=GyutaGrjsWDyS+MNo9y7Z4rIdRgEqYHLJTxfKZG/ZYziWpJNCqzzqCZCSBpMy/5TEK aK7DDvj+eI2UPpORBNVla7d/BwcCnsjkaG3Uq36gr+fq9xOhbswHmq3LaQYlt9bcxvdD 06rhFSr1bMDhzncA4B7feZhUrnacRWu2n1RlXbS7P80H8QPh3sJjaFaRLJNQhehkuB9M 4VAs6ITb3IICUGc3ZFPlRCh25WsYKVraKqoqePcxU+D3d0USGgOpVDv6HcGjHFdVX36c JkZOoG7pizFxZCQ2A3/rIrmi93Yrp57+GuPdTpDaSCz2dmaVXLJ3aHDqxDSBA2SHzfj+ tJAQ== X-Gm-Message-State: AOAM530LrkbvZmNYFdvy31zzdYExiHcPmMyFXQfT9e5BApC1Q5kTrYdf /qsdM5CavNOTd5SbckQu2zmgrZhbSqMWjbS490w= X-Google-Smtp-Source: ABdhPJyPXj9CulTaj71OCbMZEBfrazd4oAzTEqLlEOwTZzpI4B8wbU6d1O0mjnFILeH8b7Y5ziPEPpkImgV4XHc6X9w= X-Received: by 2002:ae9:f214:: with SMTP id m20mr14976159qkg.232.1590913476824; Sun, 31 May 2020 01:24:36 -0700 (PDT) MIME-Version: 1.0 References: <87h7vyxqrz.fsf@gmail.com> In-Reply-To: From: zimoun Date: Sun, 31 May 2020 10:24:25 +0200 Message-ID: Subject: Re: Guix Docker image inflation To: Stephen Scheck , 41607@debbugs.gnu.org Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: Chris Marusich , help-guix X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Dear Stephen, Follow ups of https://lists.gnu.org/archive/html/help-guix/2020-05/msg00249.html and bug#41607 CC http://issues.guix.gnu.org/41607 On Sat, 30 May 2020 at 19:02, Stephen Scheck wrote: > You can convince yourself of this by doing something like the following: > > docker run > docker exec dd if=/dev/urandom of=/RANDOM-DATA > bs=1048576 count=1024 > docker commit > docker exec rm /RANDOM-DATA > docker commit It does not convince myself and maybe I am doing wrongly but it is not what I am observing for an example with more than 2 'commits'. Here my session, based on your images rename "fresh" because it will happen on any image. --8<---------------cut here---------------start------------->8--- $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE fresh latest c36cef8306d5 3 weeks ago 1.06GB singularsyntax/guix-bootstrap 1.1.0-alpine-3.11 c36cef8306d5 3 weeks ago 1.06GB $ CONTAINER=`docker run --detach --tty --privileged fresh` $ docker exec --interactive --tty $CONTAINER /bin/sh / # dd if=/dev/urandom of=/DATA bs=1234567 count=1024 dd if=/dev/urandom of=/DATA bs=1234567 count=1024 1024+0 records in 1024+0 records out / # exit exit $ HASH=`docker commit $CONTAINER` && docker tag $HASH add-data $ docker stop $CONTAINER && docker rm $CONTAINER cb89992b76ace2afe5dc6e082c8de121c483dfeeb688d89849713e2cf90b68c7 cb89992b76ace2afe5dc6e082c8de121c483dfeeb688d89849713e2cf90b68c7 $ CONTAINER=`docker run --detach --tty --privileged add-data` $ docker exec --interactive --tty $CONTAINER /bin/sh / # rm /DATA rm /DATA / # dd if=/dev/urandom of=/OTHER bs=1234567 count=1024 dd if=/dev/urandom of=/OTHER bs=1234567 count=1024 1024+0 records in 1024+0 records out / # exit exit $ HASH=`docker commit $CONTAINER` && docker tag $HASH rm-data-add-other $ docker stop $CONTAINER && docker rm $CONTAINER 93e9afe593226ec29669efe8515b47487f455d4ad5e012cc67372c2549ec7256 93e9afe593226ec29669efe8515b47487f455d4ad5e012cc67372c2549ec7256 $ CONTAINER=`docker run --detach --tty --privileged rm-data-add-other` $ docker exec --interactive --tty $CONTAINER /bin/sh / # rm /OTHER rm /OTHER / # exit exit $ HASH=`docker commit $CONTAINER` && docker tag $HASH rm-other $ docker stop $CONTAINER && docker rm $CONTAINER 469b341c2f394ef05f5f43a5d96239853e3552d979028a457a9bdd1096a8fab4 469b341c2f394ef05f5f43a5d96239853e3552d979028a457a9bdd1096a8fab4 $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE rm-other latest b80d300aa755 23 seconds ago 3.59GB rm-data-add-other latest de551eac1d55 About a minute ago 3.59GB add-data latest 6a563daccccd 3 minutes ago 2.32GB fresh latest c36cef8306d5 3 weeks ago 1.06GB singularsyntax/guix-bootstrap 1.1.0-alpine-3.11 c36cef8306d5 3 weeks ago 1.06GB $ CONTAINER=`docker run --detach --tty --privileged rm-other` $ docker exec --interactive --tty $CONTAINER /bin/sh / # ls / ls / bin dev etc gnu home lib media mnt opt proc root run sbin srv sys tmp usr var / # exit exit --8<---------------cut here---------------end--------------->8--- > You'll end up with two new images - the first one should be about 1 GB > larger than the base image, > the second one the same size. As you see, the image 'rm-other' does not container either /DATA or /OTHER and its size is not the same than the initial one 'fresh'. So I do not know if the correct Docker terminology is "layer" because the issue is definitely on the Docker side and not on the Guix side. Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Sun May 31 05:27:55 2020 Received: (at 41607) by debbugs.gnu.org; 31 May 2020 09:27:55 +0000 Received: from localhost ([127.0.0.1]:59730 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfKG2-0003JR-RD for submit@debbugs.gnu.org; Sun, 31 May 2020 05:27:55 -0400 Received: from mail-qk1-f176.google.com ([209.85.222.176]:41723) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfKG1-0003JF-6K for 41607@debbugs.gnu.org; Sun, 31 May 2020 05:27:53 -0400 Received: by mail-qk1-f176.google.com with SMTP id n11so6377805qkn.8 for <41607@debbugs.gnu.org>; Sun, 31 May 2020 02:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=f53ZsjlL7W40Aav5nctiA6CYInhbMfOqkVpCCZ2r4wU=; b=vSLTjaDWL+dXtWWIR9Fjg7hcWXzfai+KGsC9NcPo/oDp3tYZmo+0VxXoBWIb3L6Nf+ dua2qlI8Eb/ruHS20PkeWrSITXRANnaKMSINtTEBno4+RViEAlh6UVgJUZ5dorp5Q/RV Lvxoy58ws7ozENPgFF6jsL1YcpC+UupLPEAKD4NDDO10dwBw9KzL0wrnnkR2JvV/TdsP nRmoe28vu3EiNOhl+4caFCmxXIBBAJL3g2Mdxp1Y2AV9q6V99z5f8QHvOPpQ7SWpldkW UCUo+HmkEhbQ+KQwTlULjxrFccejERMFHMgteoIwGX8xh+fAVypojAv1TJpYIgTcKkEV 80wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=f53ZsjlL7W40Aav5nctiA6CYInhbMfOqkVpCCZ2r4wU=; b=RSEwy1igbEoEMCDbj8S+QY7qqDb5K0xBfOnfHCiSYepuTMY5X7ZLraxeMj9MjSke3P UUzkTgZUGhiC0/lanSj2xtpxX34nI/A/Mcoj88HsmC3CPdTKSY+aonhI7mwJyr5pRhNW gIniDyUPPAVmydp3nJT706lZTYnt1MSpmfqV6bYVPYZT+4SYd91kX8pwu/XQLuGIjaig aqdNTo3jN28ku7VkS8/6WljrdLZx9J58u9aD/An7hg+0kU87IeFCZbwLEboEkTUbU38B 3ASE/Fd773poF7HNSd+SVhJZfaniAHRcrfAF3kROUT5oHOFaCgWgY6rnbk4t8lUkkyJw ljew== X-Gm-Message-State: AOAM53352jrA5Z1/AGM7xd5uCqgSbcJu0UI0Wvn8Fv1hkLMVgji8li2t xOOvoWDf1/GAE274LIBaWeQwRSH74YVqXKZkOtQ= X-Google-Smtp-Source: ABdhPJydBuFFohP7KZzK9hEE7RnuO+5TKsLWwsB40pHxFO8t8UPpF+o5ZLFP+VqqrDPGoXkqUwJruEvBZYIMgwVn1/Y= X-Received: by 2002:ae9:f214:: with SMTP id m20mr15093941qkg.232.1590917267674; Sun, 31 May 2020 02:27:47 -0700 (PDT) MIME-Version: 1.0 References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> In-Reply-To: <87r1v0k8hl.fsf@gmail.com> From: zimoun Date: Sun, 31 May 2020 11:27:36 +0200 Message-ID: Subject: Re: bug#41607: Deleted store items are not actually deleted To: Chris Marusich Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: 41607@debbugs.gnu.org, Stephen Scheck , Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Chris, Thank you for the detailed explanations. On Sun, 31 May 2020 at 06:57, Chris Marusich wrote: > The trash directory is on the "top layer" because it gets created in the > running container. However, in practice many store items from lower > layers are made dead when Stephen's script runs "guix pull" and deletes > the old profiles. If any of those store items were directories, > guix-daemon will fail to GC them because of an XDEV error. If this is > confusing to you, I suggest you experiment with Docker a little bit, and > look closely at the steps that Stephen's script is running. I outlined > this in the email I accidentally deleted, but I'm a little too tired to > reproduce it all a second time. I hope you'll understand. > > Should Guix do anything about this? We could change guix-daemon to take > correct action in the face of an XDEV error. We could also improve the > logging, since currently it silently swallows the XDEV error. What is the correct action? Because somehow these items cannot be removed of the Docker chained images. However, the logging could report the ``error", e.g., "guix gc" lists all the items and their size and currently it says "[0MiB] deleting" so instead it could say "[XDEV error] not valid" with some words about that in the manual, section "Invking guix gc". WDYT? Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Sun May 31 06:50:36 2020 Received: (at 41607) by debbugs.gnu.org; 31 May 2020 10:50:36 +0000 Received: from localhost ([127.0.0.1]:59853 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfLY4-0005T3-9x for submit@debbugs.gnu.org; Sun, 31 May 2020 06:50:36 -0400 Received: from mail-wr1-f50.google.com ([209.85.221.50]:36814) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfLY3-0005Sr-Ep for 41607@debbugs.gnu.org; Sun, 31 May 2020 06:50:35 -0400 Received: by mail-wr1-f50.google.com with SMTP id q11so8596670wrp.3 for <41607@debbugs.gnu.org>; Sun, 31 May 2020 03:50:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=3JcXe06gyEdSadYc6Be3QHbTVy7FvlOzldiYdtV8xCM=; b=cGOqvaWROQdufgevNQaeQqOQYl1jdKDVEp+k47N6IxgBufBwO4JJLlUBpmLRc+iOQK uaqStYePfRh+LfLcNRHyKrgWXfSaNKIUNRZuqkQ5XLEC3ibxSePlJkIjorvHwIMGyIBF wa8IqoxY8lV1KU73aNoGqUR/sTJJFLo9Ebgv1GlOoCs0SA/HH6DrlONeXjuNniYIa7YM s0uy1xL+efvvyxoVY0AOJ4nrpz8JslhRBvCCc8x4qjn7xe+ZciDlC8KGEcSEPTVXBTYy mhla6Ew+EB2ZzQKyoF2uJRduEbOnuJJf6Cl9z19CJAkqxrD6aXGbSa1WqsWOFhdozXaQ vKjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=3JcXe06gyEdSadYc6Be3QHbTVy7FvlOzldiYdtV8xCM=; b=gaZjFIhMfP4u49htmh0L8viPA4CnFq3YI1J3Gi7j5ELGvvpt09QHe+Qo0XxYnbLZTv l/Bi3FUacbNfhIQ+8ZEKL/F/s0V+BwXaMB3oPmZh7d6alj5uAkZnMsbuldzlepnXpDsk 73B5EzlM9so1h40VcEvNFc7plCTMWMpJ/6KoTXqJAhvLjcyI+UjBEqb9Z15Z7uXhLEeT IDJL9eFBin1oaKlASOtLhPGbquxz9VUN7n+b7HWoaGeApjSFV6wzWLnXsh5d8iaz/jd7 bnAKqRuTRNCIeK0h4O/HEqe4Q25eRE3mIJVgLN6AXYkoxvvTuJCC/6r8A3HptgQ8O5N6 IZjA== X-Gm-Message-State: AOAM533mdfvYleD4rSSydJzzvf1Af9dXTA+93Zsnpv5udUa+NkeAKuat f1cNNZCbxaAOhbQy+pCfCF8= X-Google-Smtp-Source: ABdhPJw6ol0lqoLW2NULZFheaWiUcT2zwjdxwqijgLJXZHdIltc5S4qSDDksrKpmXlR7t3779XKFtw== X-Received: by 2002:adf:f2c2:: with SMTP id d2mr10195936wrp.424.1590922229624; Sun, 31 May 2020 03:50:29 -0700 (PDT) Received: from [10.0.0.2] (se167-1-82-242-148-1.fbx.proxad.net. [82.242.148.1]) by smtp.gmail.com with ESMTPSA id v19sm5861803wml.26.2020.05.31.03.50.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 31 May 2020 03:50:28 -0700 (PDT) Subject: Re: Guix Docker image inflation To: zimoun , Stephen Scheck , 41607@debbugs.gnu.org References: <87h7vyxqrz.fsf@gmail.com> From: Vincent Legoll Message-ID: <7a07e6d0-e45d-6c5f-f536-34a065100d8f@gmail.com> Date: Sun, 31 May 2020 12:50:26 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: help-guix X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, maybe you can try: docker export | docker import - img_name This should flatten the layers back to a single one. -- Vincent Legoll From debbugs-submit-bounces@debbugs.gnu.org Sun May 31 13:58:33 2020 Received: (at 41607) by debbugs.gnu.org; 31 May 2020 17:58:33 +0000 Received: from localhost ([127.0.0.1]:33806 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfSED-0003T4-2B for submit@debbugs.gnu.org; Sun, 31 May 2020 13:58:33 -0400 Received: from mail-qk1-f182.google.com ([209.85.222.182]:38086) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfSEB-0003Sq-8B for 41607@debbugs.gnu.org; Sun, 31 May 2020 13:58:31 -0400 Received: by mail-qk1-f182.google.com with SMTP id w1so7066242qkw.5 for <41607@debbugs.gnu.org>; Sun, 31 May 2020 10:58:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7rLMT8lpbjwdwWQYnlktvVR4sGEUZcpx1f1dfD+6Th0=; b=g9qN15yAvBfw/+xmKgJqH4QVt0dd9+ssAY4Mn9DigD5nIa30MRoiTDQrTuk5pzDgqD 8NxZ0fnPcjuRDro3nWUIlDwwdXfX/oC6XFGzBiJ6QBYQKqyGmK1VsTZOoSVfVVbOxhVc 668y4luN+pcCWeD8N6nu7dCrj0bgnmZkHV36IFKj4CegiBXSLXWxVSWq6sBB217ShHPW iI7Y/qEL9aqNRyGueUBX4j1Uhx8U/WqxWxmCRNwVj4BafswegDRa9bLQjXV9T13CYmRm eeXFQN5uFCT+tVBdCIKLJj59egMGTiur+5BbatK73DaZ4AHR0wpd9F9gGyjdC47hMLxX Vfvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7rLMT8lpbjwdwWQYnlktvVR4sGEUZcpx1f1dfD+6Th0=; b=qbzaH3SbusRjvwmmiCmR6NN6wQtvrAXgGgkB8c3b0yvr/5wWpjphNgBEF9CvpRSlNy InA7jvQElBpVKeFR0kkP7uWE5aBmjTvshgnCRukJKcefbuWkDBOGvcB6cqGqouZyLjro Ofb+psnN928hSmU9IorJqDfXt8UaJNr14mD5UJU3/MMkBdet1RiKXoqmLbik5Uaramc9 wW6mACz4jg4CJ6XApkumqmoiTK/YyytmkarAGVc8bI6h8DGdEV0GxymZNlhDX4KkJcsj xYuaHP1VL9ou/pzC4myq3yYE00tx39z7AsogV0fnwxzL95ICxfp8w+mCF63ccnR1D4sd ZkvA== X-Gm-Message-State: AOAM531alErEkl4XbTrS5eJh4xApVhYz+xGwo+X+gGvF5i6wCjQWI21G 0ta5DDDvEohhaSzrb9uUIUv+1ifOh4M8DVEyENA= X-Google-Smtp-Source: ABdhPJxexrb6DXawGDtl4ajibcydR6jGfasDNO9dppGmwuFLdagm8WOru3P42Q0XOL0qGPy2alrXcujSbv/TiU3vfMQ= X-Received: by 2002:a37:4656:: with SMTP id t83mr16839063qka.126.1590947905533; Sun, 31 May 2020 10:58:25 -0700 (PDT) MIME-Version: 1.0 References: <87h7vyxqrz.fsf@gmail.com> <7a07e6d0-e45d-6c5f-f536-34a065100d8f@gmail.com> In-Reply-To: <7a07e6d0-e45d-6c5f-f536-34a065100d8f@gmail.com> From: zimoun Date: Sun, 31 May 2020 19:58:13 +0200 Message-ID: Subject: Re: Guix Docker image inflation To: Vincent Legoll Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: 41607@debbugs.gnu.org, help-guix , Stephen Scheck X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Vincent, On Sun, 31 May 2020 at 12:50, Vincent Legoll wrote: > docker export | docker import - img_name I do not know if it really works here. Maybe I am doing incorrectly... --8<---------------cut here---------------start------------->8--- $ docker images --format "{{.Size}}\t{{.Repository}}" 959MB 4reexport 960MB 3clean 960MB 2remove-hello 959MB 1install-hello 578MB 0new-fresh 1.06GB fresh 1.06GB singularsyntax/guix-bootstrap --8<---------------cut here---------------end--------------->8--- Well, and the interesting part is: --8<---------------cut here---------------start------------->8--- $ CONTAINER=`docker run --detach --tty --privileged 4reexport` $ docker exec --interactive --tty $CONTAINER /bin/sh / # /root/.config/guix/current/bin/guix gc --list-live | grep hello /root/.config/guix/current/bin/guix gc --list-live | grep hello finding garbage collector roots... determining live/dead paths... / # /root/.config/guix/current/bin/guix gc --list-dead | grep hello /root/.config/guix/current/bin/guix gc --list-dead | grep hello finding garbage collector roots... determining live/dead paths... /gnu/store/kg9mirg6xbvzcp0a98v7326n1nvvwgsj-hello-2.10 / # /root/.config/guix/current/bin/guix gc --references /gnu/store/kg9mirg6xbvzcp0a98v7326n1nvvwgsj-hello-2.10 /root/.config/guix/current/bin/guix gc --references /gnu/store/kg9mirg6xbvzcp0a98v7326n1nvvwgsj-he llo-2.10 guix gc: error: path `/gnu/store/kg9mirg6xbvzcp0a98v7326n1nvvwgsj-hello-2.10' is not valid / # exit --8<---------------cut here---------------end--------------->8--- Just for the record, the commands run: --8<---------------cut here---------------start------------->8--- $ CONTAINER=`docker run --detach --tty --privileged fresh` $ CMD='CMD "/root/.config/guix/current/bin/guix-daemon" "--build-users-group=guixbuild"' $ docker export $CONTAINER \ | docker import --change $CMD - 0new-fresh $ CONTAINER=`docker run --detach --tty --privileged 0new-fresh` $ docker exec --interactive --tty $CONTAINER /bin/sh / # /root/.config/guix/current/bin/guix install hello / # exit $ docker stop $CONTAINER $ HASH=`docker commit $CONTAINER` && docker tag $HASH 1install-hello $ CONTAINER=`docker run --detach --tty --privileged 1install-hello` $ docker exec --interactive --tty $CONTAINER /bin/sh / # /root/.config/guix/current/bin/guix remove hello / # exit $ docker stop $CONTAINER $ HASH=`docker commit $CONTAINER` && docker tag $HASH 2remove-hello $ CONTAINER=`docker run --detach --tty --privileged 2remove-hello` $ docker exec --interactive --tty $CONTAINER /bin/sh / # /root/.config/guix/current/bin/guix pull -d / # /root/.config/guix/current/bin/guix package -d / # /root/.config/guix/current/bin/guix gc / # exit $ docker stop $CONTAINER $ HASH=`docker commit $CONTAINER` && docker tag $HASH 3clean $ CONTAINER=`docker run --detach --tty --privileged 3clean` $ docker export $CONTAINER | docker import --change $CMD - 4reexport --8<---------------cut here---------------end--------------->8--- where I cheated with $CMD which does not as is but the full 'CMD...' has to be typed after '--change'. All the best, simon From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 04 07:58:29 2020 Received: (at 41607) by debbugs.gnu.org; 4 Jun 2020 11:58:29 +0000 Received: from localhost ([127.0.0.1]:45255 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgoVx-0000yJ-08 for submit@debbugs.gnu.org; Thu, 04 Jun 2020 07:58:29 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33890) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgoVu-0000y1-EZ for 41607@debbugs.gnu.org; Thu, 04 Jun 2020 07:58:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58882) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgoVp-0007aK-13; Thu, 04 Jun 2020 07:58:21 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45522 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jgoVn-0001mj-Lj; Thu, 04 Jun 2020 07:58:20 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Chris Marusich Subject: Re: bug#41607: Deleted store items are not actually deleted References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 17 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 04 Jun 2020 13:58:17 +0200 In-Reply-To: <87r1v0k8hl.fsf@gmail.com> (Chris Marusich's message of "Sat, 30 May 2020 21:56:38 -0700") Message-ID: <87eeqvcaau.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41607 Cc: 41607@debbugs.gnu.org, Stephen Scheck , Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Chris Marusich skribis: > The reason Guix is failing to GC dead items in the Docker container is > because those dead items are not on the "top layer", so Docker returns > an EXDEV error: > > https://docs.docker.com/storage/storagedriver/overlayfs-driver/ > > "Renaming directories: Calling rename(2) for a directory is allowed only > when both the source and the destination path are on the top > layer. Otherwise, it returns EXDEV error ('cross-device link not > permitted'). Your application needs to be designed to handle EXDEV and > fall back to a 'copy and unlink' strategy." > > You can observe this by running guix-daemon with strace in the > container, and watching what happens when you try to delete one of the > offending store items (make sure it is a directory). For example: > > 685 rename("/gnu/store/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database= ", "/gnu/store/trash/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database") =3D= -1 EXDEV (Invalid cross-device link) > > In most cases, when guix-daemon GC's a dead directory, it does this > (see: nix/libstore/gc.cc): > > - Create a trash directory (usually /gnu/store/trash) > - Move dead directories into the trash directory. > - Delete the trash directory. > > The trash directory is on the "top layer" because it gets created in the > running container. However, in practice many store items from lower > layers are made dead when Stephen's script runs "guix pull" and deletes > the old profiles. If any of those store items were directories, > guix-daemon will fail to GC them because of an XDEV error. If this is > confusing to you, I suggest you experiment with Docker a little bit, and > look closely at the steps that Stephen's script is running. I outlined > this in the email I accidentally deleted, but I'm a little too tired to > reproduce it all a second time. I hope you'll understand. Interesting, thanks for the analysis! > Should Guix do anything about this? We could change guix-daemon to take > correct action in the face of an XDEV error. We could also improve the > logging, since currently it silently swallows the XDEV error. I guess we could delete recursively right away upon EXDEV. It should be just two lines of code, right? > To work around the issue, Stephen can build the images from the same > base image, rather than daisy-chaining new images from old ones. That > way, they would not accumulate layers without bound. Maybe that too. Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 04 14:50:16 2020 Received: (at 41607) by debbugs.gnu.org; 4 Jun 2020 18:50:16 +0000 Received: from localhost ([127.0.0.1]:47452 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jguwS-0007hc-IV for submit@debbugs.gnu.org; Thu, 04 Jun 2020 14:50:16 -0400 Received: from mail-pf1-f177.google.com ([209.85.210.177]:39515) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jguwQ-0007hP-7i for 41607@debbugs.gnu.org; Thu, 04 Jun 2020 14:50:14 -0400 Received: by mail-pf1-f177.google.com with SMTP id d66so3681066pfd.6 for <41607@debbugs.gnu.org>; Thu, 04 Jun 2020 11:50:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=rDc3cdxr3rceez5wE+Nw/lFICw4SSDX8f0RmDERLyyw=; b=CGP7syQm9FVs7wX8+/1EMkiu9bnMm2c8BOhyoe5XW9+SSBX5pYQ2b/YDqYOqEAEaca jo9WCzkshU3sEh+tQHynjeo9PHHeveHGWGjclkMmBjZKvHfJx1UFTH6zEYRDj1rDdAX+ GNIJK1xR5P7Cm9KULkU0U1MFaE7brd8cizYviUUsOD2hcJRcgxjJzflyTQpLFxbDfNPz 1OC9C7rCIAXQ4a22BPs2BN11igZsndFnWIxvqRKlkxKpIhSHFb/9ikrRNoKRH7y6XW3r V/A+JdKUuHbVIgylBohADzTBBeysu1pJkIDRtVbun0qzM0ooLyLjiXAq1TGY/Pb+ZlMN w5kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=rDc3cdxr3rceez5wE+Nw/lFICw4SSDX8f0RmDERLyyw=; b=NBFY3phQmdFGsomxsLnQlvMJr9buL0vCQDDuS9IcZ3b+ALkHjasqPBByxb16HswlCk sphDH++E5ahRnkHjXnD2BjqzxWXTxKn5tGg2+cQfAPFW97ZMNiaIryMuibWV+MgRWL4U 0AvhDus5k78PK8DsQYX+Vldy/kdyE1qS5jytg3T5QUa4mQ2D7DAi8V32J+e2vuBHnedy MIKaYmXeeTQYXfGHvgsKp67o4hOrzqfClS1jY4GiERk34FygEC62ClR6rNDHNcxitK2c 2q+c+suUANOKPdf+bSKSzUtmXRwXh1R+xbXNFRxVBMIA33kvmcvDfdlwID1QbMd6vNL8 QJ8g== X-Gm-Message-State: AOAM532DDUrlZG/r9nDImnY50T9cMN9c7/BhZawBn1m4vrIrMiHLCmds SaNrxYGeT9g9CZmEqqm+zv0= X-Google-Smtp-Source: ABdhPJwq0W6WcKWJv4+1L9OrlJ1Y1agnsUHeBJRNglRXblH6eGh9g+0iosBFcaL+VQ6bv3FL+eIXkA== X-Received: by 2002:a63:c042:: with SMTP id z2mr5681737pgi.231.1591296608096; Thu, 04 Jun 2020 11:50:08 -0700 (PDT) Received: from garuda-lan (c-73-97-103-127.hsd1.wa.comcast.net. [73.97.103.127]) by smtp.gmail.com with ESMTPSA id mu17sm7386577pjb.53.2020.06.04.11.50.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Jun 2020 11:50:06 -0700 (PDT) From: Chris Marusich To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#41607: Deleted store items are not actually deleted References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> Date: Thu, 04 Jun 2020 11:50:05 -0700 In-Reply-To: <87eeqvcaau.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 04 Jun 2020 13:58:17 +0200") Message-ID: <87r1uu1x9e.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: 41607@debbugs.gnu.org, Stephen Scheck , Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: >> Should Guix do anything about this? We could change guix-daemon to take >> correct action in the face of an XDEV error. We could also improve the >> logging, since currently it silently swallows the XDEV error. > > I guess we could delete recursively right away upon EXDEV. It should be > just two lines of code, right? I'll try making the change and report back. Yes, there are other cases where we immediately delete without moving into the trash directory (e.g., when the trash directory fails to be created), so it seems OK. =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAl7ZQl0ACgkQ3UCaFdgi Rp2UYRAAyAuxoPIgmAd6xgOHavBlkMdxo7S6RACQk7AKjChInSutq7wIVqQKV2my 2F1tdcLvti7kgO9my7MjTtzaMsIrgWh0yxZCWUtRuqVCRYX6byZb9mxmyYnZJu1f bigJj/dzfMOC0skLDsN6G2dhy/PfIiT/3uZdsmWeVc4Ka/h/LqtR4AmjDdS3HNIA pX/2J4e+yq4TeurLyfl0ZA+OqR1o7XoFSRBzjsHtr3pyMtqfvPT5csYgKYifenKy 6BQuNMGD/nZgV/S1AMiq9D9LmF60Eb7OBRfZ8tJp5Ho13OPhFiYXb5u4R3Jcrq6E VF6R9FHIS3i1KxEYsvTCSoCRQEy8Dk1Z4uw6lTNtcb3+Q/eFUpeZCoxP4h6V+1vQ 7s2b4IVeD8InZNDINmvT0G3MEKWnxo1LQEQ0RcxHvJn95S6BD+g+H5Tz9QkmtmYw 4KsiuLCh7uSTL6zwOdTMFbMe6uz5eXqSawtELLFO9ClGlno15Pm+bJzekhEkpNw+ k9BaidDHaM+hKVncCObCr1yyMfoA6uQbyX0opsyfcTHpdwtB7SaMI9p+h57iTJ8+ j1xKvyudX3WEAWUp1iUb1hJaHDNMooMmOa86TbigC+6LVKFk+jqXXUN7IiRrkBbX QVInBmJfwuhWyx1ykTSiB4EoXWVuiQDQWTxTv3ixfAh2RU9qK8M= =PXGV -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 05 05:32:36 2020 Received: (at 41607) by debbugs.gnu.org; 5 Jun 2020 09:32:36 +0000 Received: from localhost ([127.0.0.1]:48336 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jh8iK-0004BD-Bu for submit@debbugs.gnu.org; Fri, 05 Jun 2020 05:32:36 -0400 Received: from mail-pg1-f173.google.com ([209.85.215.173]:34524) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jh8iJ-0004Ax-Bj for 41607@debbugs.gnu.org; Fri, 05 Jun 2020 05:32:35 -0400 Received: by mail-pg1-f173.google.com with SMTP id m1so4915504pgk.1 for <41607@debbugs.gnu.org>; Fri, 05 Jun 2020 02:32:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=g6Cz7RI1FbWUBQKEjMCGt4Pkf/fTsuxOy/qCZ2UE62U=; b=KVXKndWZeCb4ktqI0PMtO85gWSSfS2UVaTTwRC1IQL4aHtgWSV15eG+ytjYtIKg9ML UPVn6vjzg0o+AN6dHMP6gt1njAH2yUuwJVfrWEeQHljlpQLr9Gns5HYkdqPJeZ+kxQGW PEBJAMsE4ONDWEwKZzrH0ihGKQ817NS+Z4a7ECchttmm2JAt9rIgb20zA432C1hRkMHC 5gLo7l9PyzLYiFlKvRtu9Lewr8h6BCCOU8r/e62w2LoxCduXdRIttqvGb3qw0t7ODnw4 6lsT2nwYEHcVz++Jvpb2RW7tuOhCuQTcRAfVB7LgQt80FetubDU2eJ8/CXO4mcPc3nmE mBAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=g6Cz7RI1FbWUBQKEjMCGt4Pkf/fTsuxOy/qCZ2UE62U=; b=B0FZ1Nuj8AsCNbpNzU4ScQ7dexWs992auIPmcO+5ujUNEYyQjsG5wgv5ageg47ZgxE 8EcyZMjc/EACVha5mJkye/GoPLE1rUtNCJNCqB1h2rQFHfNJEAXtyUejwaOyjvGV5ady SzdA7cch959WHmVgU8XKiT26fYxdMHKDHOtD0P7sMwQctSrzeyWBGlrApDLXYitBveJm d5ArgQJNMVfIsvMqlzDk6qSCFtH3OeBBFIjnSnjgucF7HpdwuGyFvc3b7pXpFUSOlMi2 tgxcNaOs2Tq+5Mmt4D7RPInLmrU0935RzRSzEDQQHEv1xfNmLcr5xHAOO7y0cTEiRLum g+ow== X-Gm-Message-State: AOAM533ZZkXteHSnWpdHRM4GKGaFmhXJ014RnGOyw/k/fupVtRb/f5v9 3a8xFVFjCRuRfsnUiF96xpA= X-Google-Smtp-Source: ABdhPJw3uv0V+JTEH4tyB1k8cHHSfpMVDImMhT4wIjMOYMSNCoCELUXk62+JHTk5gl2Wh4ZGrcBjaQ== X-Received: by 2002:a65:6795:: with SMTP id e21mr8786912pgr.282.1591349549123; Fri, 05 Jun 2020 02:32:29 -0700 (PDT) Received: from garuda-lan (c-73-97-103-127.hsd1.wa.comcast.net. [73.97.103.127]) by smtp.gmail.com with ESMTPSA id b140sm6731895pfb.119.2020.06.05.02.32.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jun 2020 02:32:27 -0700 (PDT) From: Christopher Marusich X-Google-Original-From: Christopher Marusich To: Chris Marusich Subject: Re: bug#41607: Deleted store items are not actually deleted References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> <87r1uu1x9e.fsf@gmail.com> Date: Fri, 05 Jun 2020 02:32:20 -0700 In-Reply-To: <87r1uu1x9e.fsf@gmail.com> (Chris Marusich's message of "Thu, 04 Jun 2020 11:50:05 -0700") Message-ID: <87pnad6eor.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: 41607@debbugs.gnu.org, Ludovic =?utf-8?Q?Court=C3=A8s?= , Stephen Scheck , Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Chris Marusich writes: > Ludovic Court=C3=A8s writes: > >>> Should Guix do anything about this? We could change guix-daemon to take >>> correct action in the face of an XDEV error. We could also improve the >>> logging, since currently it silently swallows the XDEV error. >> >> I guess we could delete recursively right away upon EXDEV. It should be >> just two lines of code, right? > > I'll try making the change and report back. Yes, there are other cases > where we immediately delete without moving into the trash directory > (e.g., when the trash directory fails to be created), so it seems OK. Here is a patch. Turns out it's was just a one line change! If nobody has any further feedback on it, I'll go ahead and merge it to the master branch in the next couple days. I tested it in one of the Docker containers provided by Stephen which exhibited the problem. I built the new Guix inside the Docker container and verified that a path which was previously unable to be GC'd due to the EXDEV error, was now able to be successfully GC'd. My understanding is that the only reason why the guix-daemon attempts to move dead directories to the trash directory is to save time on deleting, since large directories could take a while to fully delete. If there is any reason why it might be unsafe to delete the directories directly in case of EXDEV (I cannot think of any), please let me know. =2D-=20 Chris --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-daemon-Handle-EXDEV-when-moving-to-trash-directory.patch Content-Transfer-Encoding: quoted-printable From=20505481a6a22819a42320f693988c3f8e13ded080 Mon Sep 17 00:00:00 2001 From: Chris Marusich Date: Thu, 4 Jun 2020 23:26:19 -0700 Subject: [PATCH] daemon: Handle EXDEV when moving to trash directory. Fixes . Reported by Stephen Scheck . * nix/libstore/gc.cc (LocalStore::deletePathRecursive): When we try to move a dead directory into the trashDir using rename(2) but it returns an EXDEV error, just delete the directory instead. This can happen in a Docker container when the directory is not on the "top layer". =2D-- nix/libstore/gc.cc | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/nix/libstore/gc.cc b/nix/libstore/gc.cc index 8bc4e01eb0..845fe278c7 100644 =2D-- a/nix/libstore/gc.cc +++ b/nix/libstore/gc.cc @@ -455,7 +455,10 @@ void LocalStore::deletePathRecursive(GCState & state, = const Path & path) throw SysError(format("unable to rename `%1%' to `%2%'") %= path % tmp); state.bytesInvalidated +=3D size; } catch (SysError & e) { =2D if (e.errNo =3D=3D ENOSPC) { + // In a Docker container, rename(2) returns EXDEV when the sou= rce + // and destination are not both on the "top layer". See: + // https://bugs.gnu.org/41607 + if (e.errNo =3D=3D ENOSPC || e.errNo =3D=3D EXDEV) { printMsg(lvlInfo, format("note: can't create move `%1%': %= 2%") % path % e.msg()); deleteGarbage(state, path); } =2D-=20 2.26.2 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAl7aESQACgkQ3UCaFdgi Rp30yw//Vw6SV8tFac2hCw6CHi8vg8IqjNZdGEq+9sXztSSopCib7KQnoOT9LfqS uGfwQ7yZEeTcQV6H6dOqHB7PmsyljHwIlMeEXDk4v5hCprxdfEftPO0y71BXJdYW B4VgDYuqp2GbLzwV9UCSLLpddOamUZK5Rt/fFsFmx55oheBt5s5I+hExGbb/HxT4 bC9XZhWQpMrJn9DtWvWZ2Fujgy9cEBi7gqXbsJ7XrENU4WXwfgrRgLVxKi1iyJWc LxVFXBY1D1Ob4mvK+mpuWdmvuYMoMHVbxP/so+fbEGcXvvopmYUlarfF+kq8IqLJ +Il7wuuqYKb3u2oDkLZCJ6yrCHeox6/aaK9xB89Lz1DRIHJ29U7T+JlmNFckJ/0+ Vw5BYvLcX1JbJGhyPSEt16gkSRgjT2KA3vqs9Nd9iFZaec5sN+4DqBUGxuJXr718 UCCCe26dKo6sIzzOzkEoD/BAt5k79LusNzNmh7DSsq7ANY1WkdPNHdhM0iMmqdfL 3WOiOtsDq2Cre6BnFJx3wm1d0LZAB/nvvJRDfJ0oiUYo6EzBVydJd+txdmEkq5VG +QDY8ktJ+PHdUMRs2DjqlK5oGoHFfXxIVJ6Qi960aToMWJa+SGxEoe7FJWqI4YpJ mVZGWl46A7eLjjLvTwn8XTK9iFUKjliJJn8ru1C1d8Xftcvq05U= =Jycu -----END PGP SIGNATURE----- --==-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 05 05:36:36 2020 Received: (at 41607) by debbugs.gnu.org; 5 Jun 2020 09:36:36 +0000 Received: from localhost ([127.0.0.1]:48351 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jh8mB-0004HG-O3 for submit@debbugs.gnu.org; Fri, 05 Jun 2020 05:36:35 -0400 Received: from mail-qt1-f174.google.com ([209.85.160.174]:39576) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jh8mA-0004H4-4Z for 41607@debbugs.gnu.org; Fri, 05 Jun 2020 05:36:34 -0400 Received: by mail-qt1-f174.google.com with SMTP id k22so7898089qtm.6 for <41607@debbugs.gnu.org>; Fri, 05 Jun 2020 02:36:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=60EXpBkPYARaZddvvuz6wMXuPTkb/2FqGQxxw7Cne7g=; b=CTLbMuhmidgfTBWdX8K4NRmYgpw42D5Vg/qjcA6fZyaVBG3XQG4Sg8cmUcq2FMFSMA pxDqU/Wmr40WbhV7DE8dUgKPCrA8N7rw6EedY2jQk3DCrXF7/dO+FSiyLql8S9iNveC3 nPhXfcTgfVcT9sftiYDvchTIyLJVikbonRpWdAaNGLUN5K/ZDsK9yYYwwbTEyUDdKxj1 wHJhlwzGbBl3Eav4AblCORlCmraPAUq58NIZyqxaR5hvQQyAg8xkTkLKIwUIN+gBwPic Cjd6iBl/9KoNgjEydUDJupaAiDK3gx90L4r6xpujrAdAh3Wr+w0zPwIBTIIU/uFej0tn Z6Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=60EXpBkPYARaZddvvuz6wMXuPTkb/2FqGQxxw7Cne7g=; b=aEoXr3qx3I8qpZUj8SXykhxAgZjyxgEBJaeUS2aeTn3R+ETP9cN5ukO4Ixn1qbH0Ed tPhE19QKY+8PEBeqydlgmhkA8EY9HVCqHDzZxwtW+mN/CYJ/eOpga+jjRpgWOxsr2K9W xaYEKfL2zan49qswjfhFokCmVJX61Rq7Cm1HPMk5hrJOLqV8ISnSQfY3xl9TiQ6qMf7h LNz/4WluHiulof0kh6ESxiQ53nfQGp1W84VNvLk+8W59jYHbPXLn3NGaETrPCNKb9wOr fJ/uBYVgrMtMlPH5gRqzCJ2Hd68tGub3N4f/ric58RJTjYO7c6Uu+ihzAnY6SWEmUiXE 4ecw== X-Gm-Message-State: AOAM530KUj1cmfM0zi+otp+Biv6a8kmLCeOEC0BFfUxKjVPS0/a2txEt 9ngjMz+STrNH4nE9j7UnCsvVObM2VBecT7zy9wA= X-Google-Smtp-Source: ABdhPJxJh2talf42bU9x4O9W0bMTC9LAtfCD1fdZK8wx+G7T4+rCy7eZBB94vJng/AnrCG3VN/lDgazSWS519BwXjYQ= X-Received: by 2002:ac8:7092:: with SMTP id y18mr9361498qto.354.1591349788543; Fri, 05 Jun 2020 02:36:28 -0700 (PDT) MIME-Version: 1.0 References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> <87r1uu1x9e.fsf@gmail.com> <87pnad6eor.fsf@gmail.com> In-Reply-To: <87pnad6eor.fsf@gmail.com> From: zimoun Date: Fri, 5 Jun 2020 11:36:17 +0200 Message-ID: Subject: Re: bug#41607: Deleted store items are not actually deleted To: Christopher Marusich Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: 41607@debbugs.gnu.org, Stephen Scheck X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Chris, On Fri, 5 Jun 2020 at 11:33, Christopher Marusich wr= ote: > Chris Marusich writes: > > Ludovic Court=C3=A8s writes: > Here is a patch. Turns out it's was just a one line change! If nobody > has any further feedback on it, I'll go ahead and merge it to the master > branch in the next couple days. Really cool! How can I test it? All the best, simon From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 05 12:22:07 2020 Received: (at 41607) by debbugs.gnu.org; 5 Jun 2020 16:22:07 +0000 Received: from localhost ([127.0.0.1]:49700 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhF6c-0001i4-Sq for submit@debbugs.gnu.org; Fri, 05 Jun 2020 12:22:07 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51338) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhF6a-0001hb-VF for 41607@debbugs.gnu.org; Fri, 05 Jun 2020 12:22:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40689) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jhF6V-0003Ie-Ky; Fri, 05 Jun 2020 12:21:59 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=50282 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jhF6V-0007fy-7w; Fri, 05 Jun 2020 12:21:59 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Marusich Subject: Re: bug#41607: Deleted store items are not actually deleted References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> <87r1uu1x9e.fsf@gmail.com> <87pnad6eor.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 18 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 05 Jun 2020 18:21:56 +0200 In-Reply-To: <87pnad6eor.fsf@gmail.com> (Christopher Marusich's message of "Fri, 05 Jun 2020 02:32:20 -0700") Message-ID: <877dwl32l7.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41607 Cc: 41607@debbugs.gnu.org, Stephen Scheck , Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Howdy! Christopher Marusich skribis: > Here is a patch. Turns out it's was just a one line change! If nobody > has any further feedback on it, I'll go ahead and merge it to the master > branch in the next couple days. Yay! > From 505481a6a22819a42320f693988c3f8e13ded080 Mon Sep 17 00:00:00 2001 > From: Chris Marusich > Date: Thu, 4 Jun 2020 23:26:19 -0700 > Subject: [PATCH] daemon: Handle EXDEV when moving to trash directory. > > Fixes . > Reported by Stephen Scheck . > > * nix/libstore/gc.cc (LocalStore::deletePathRecursive): When we try to > move a dead directory into the trashDir using rename(2) but it returns > an EXDEV error, just delete the directory instead. This can happen in a > Docker container when the directory is not on the "top layer". > --- > nix/libstore/gc.cc | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/nix/libstore/gc.cc b/nix/libstore/gc.cc > index 8bc4e01eb0..845fe278c7 100644 > --- a/nix/libstore/gc.cc > +++ b/nix/libstore/gc.cc > @@ -455,7 +455,10 @@ void LocalStore::deletePathRecursive(GCState & state= , const Path & path) > throw SysError(format("unable to rename `%1%' to `%2%'")= % path % tmp); > state.bytesInvalidated +=3D size; > } catch (SysError & e) { > - if (e.errNo =3D=3D ENOSPC) { > + // In a Docker container, rename(2) returns EXDEV when the s= ource > + // and destination are not both on the "top layer". See: > + // https://bugs.gnu.org/41607 > + if (e.errNo =3D=3D ENOSPC || e.errNo =3D=3D EXDEV) { > printMsg(lvlInfo, format("note: can't create move `%1%':= %2%") % path % e.msg()); > deleteGarbage(state, path); > } For consistency with (most) of the code, I=E2=80=99d suggest a /* */ commen= t. Otherwise LGTM, thank you! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 05 13:05:53 2020 Received: (at 41607) by debbugs.gnu.org; 5 Jun 2020 17:05:53 +0000 Received: from localhost ([127.0.0.1]:49757 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhFmy-0002nk-Ms for submit@debbugs.gnu.org; Fri, 05 Jun 2020 13:05:53 -0400 Received: from mail-lj1-f182.google.com ([209.85.208.182]:34544) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhEqk-0001H9-0F for 41607@debbugs.gnu.org; Fri, 05 Jun 2020 12:05:42 -0400 Received: by mail-lj1-f182.google.com with SMTP id b6so12433428ljj.1 for <41607@debbugs.gnu.org>; Fri, 05 Jun 2020 09:05:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Oh9RFZQoHBhwugyE4gUSlVEeSdqnE8ZS/lBEJVsRRr4=; b=bmjF/Ngsf/wVCb6cfz5F/UfpO/iZKBrZ51IDGL2O2noZTgOx9koS+cgUJsR9BnQHyG 9VNQQ+9ktxtZls5qxL/x4BiwbI/hk7/CM6q7W3Hp+czlqSqX/ws5RlLZ5sWEC3jFU5Id 6IyfbASwR/evAE86k6ut1ggZMp4T5++kUI3XrB9T99dhhxMDZhVr8Accd2lHql/eaADK Ibmgws+/UrthPNUh8eJiWdvYE7F2mh7ahkAv0kvD3bIl2FhoayOujGR+d8juKXWM8YAL c822seOA9oWNNB3SmQQj1Dtfq6k7gzwZ7w7Zf16NIWsNaSY4znLjhF4cLhQbSQ8uweZH XK9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Oh9RFZQoHBhwugyE4gUSlVEeSdqnE8ZS/lBEJVsRRr4=; b=rO1hHcJfbRJTy+UN2IveuOw1m61DImNYugWCmPWLym6d8AXcdr7o2rPCfxRWsT/5Xq tUvq5QFu7R5NNHSYg0b4AlBu8JQtuP9aPLcvs5try07/8ZUvlex+J6L18gzDhhnX8c5A p3d6ou6tSTwgXJ2Mxf9qTlaQMjumrc3dgoYQwdIxn64CgmOik0fkNMzMdNYpPicaecsm 64UKTCNysgzoArGBQ/Q1BLeC2j15kBsU2KgS6FPNhvnFc37TXde1kXnPFaFRyNgBR7WT FDqOz9i3t+ULtYKGS4mAlOoNs5KMXy6nK85bx1qBLO2kpKI9qWX2YQ1L8OlfCJw5VIl5 i0/w== X-Gm-Message-State: AOAM531nSKmiyC36VPDgMSaD7Uv06+FUcnUosMpGOgGcp4ZeovO6Cgtj VD3iy0MNI0Uxk4i9QL8LwHZB9pT7yYvCHkjoxYU= X-Google-Smtp-Source: ABdhPJznoS979wYwcbFaBMUHEtIwi/t8pmAeCW9ZoQkylgBzTrDwdyZwxyTE40fEo/O8cHggoJe7WdCHLs3DFrV1zPI= X-Received: by 2002:a05:651c:118f:: with SMTP id w15mr5168823ljo.211.1591373135883; Fri, 05 Jun 2020 09:05:35 -0700 (PDT) MIME-Version: 1.0 References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> <87r1uu1x9e.fsf@gmail.com> <87pnad6eor.fsf@gmail.com> In-Reply-To: <87pnad6eor.fsf@gmail.com> From: Stephen Scheck Date: Fri, 5 Jun 2020 12:05:24 -0400 Message-ID: Subject: Re: bug#41607: Deleted store items are not actually deleted To: Christopher Marusich Content-Type: multipart/alternative; boundary="00000000000078324705a7586e03" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607 X-Mailman-Approved-At: Fri, 05 Jun 2020 13:05:51 -0400 Cc: 41607@debbugs.gnu.org, =?UTF-8?Q?Ludovic_Court=C3=A8s?= , zimoun , Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --00000000000078324705a7586e03 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Very cool - thanks Chris! In the meantime, I've updated my build script to externalize the Guix environment from the Docker container. So far the daily builds are staying nice and small at ~197MB and one layer. The images and updated script are here if anybody is curious: https://hub.docker.com/r/singularsyntax/guix/tags https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-ci.= yml GitLab allows you to cache files between job stages and even full pipeline runs. I first tried to cache /var/guix and /gnu/store and mount them inside a container in which to perform `guix pull` etc. However, it seems that handling hard links on externally mounted file systems from within a container is problematic. I think passing `--disable-deduplication` to guix-daemon might resolve it, but I couldn't figure out where/how to change the Shepherd configuration to do that. So instead, I just copy the directories into and out of the container at start and end of the process. It seems to work. Mounting would probably speed up the process quite a bit if I could make it work. Cheers, -Steve On Fri, Jun 5, 2020 at 5:32 AM Christopher Marusich wrote: > Chris Marusich writes: > > > Ludovic Court=C3=A8s writes: > > > >>> Should Guix do anything about this? We could change guix-daemon to > take > >>> correct action in the face of an XDEV error. We could also improve t= he > >>> logging, since currently it silently swallows the XDEV error. > >> > >> I guess we could delete recursively right away upon EXDEV. It should = be > >> just two lines of code, right? > > > > I'll try making the change and report back. Yes, there are other cases > > where we immediately delete without moving into the trash directory > > (e.g., when the trash directory fails to be created), so it seems OK. > > Here is a patch. Turns out it's was just a one line change! If nobody > has any further feedback on it, I'll go ahead and merge it to the master > branch in the next couple days. > > I tested it in one of the Docker containers provided by Stephen which > exhibited the problem. I built the new Guix inside the Docker container > and verified that a path which was previously unable to be GC'd due to > the EXDEV error, was now able to be successfully GC'd. > > My understanding is that the only reason why the guix-daemon attempts to > move dead directories to the trash directory is to save time on > deleting, since large directories could take a while to fully delete. > If there is any reason why it might be unsafe to delete the directories > directly in case of EXDEV (I cannot think of any), please let me know. > > -- > Chris > --00000000000078324705a7586e03 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Very cool - thanks Chris!

In= the meantime, I've updated my build script to externalize the Guix env= ironment from the Docker container.
So far the daily builds are s= taying nice and small at ~197MB and one layer. The images and updated scrip= t are
here if anybody is curious:

<= div>
GitLab allows you to cache files between job stages and even = full pipeline runs. I first tried to cache /var/guix
and /gnu/store and = mount them inside a container in which to perform `guix pull` etc. However,= it seems
that handling hard links on externally mounted file systems fr= om within a container is problematic. I think
passing `--disable-de= duplication` to guix-daemon might resolve it, but I couldn't figure out= where/how to
change the Shepherd configuration to do that. So in= stead, I just copy the directories into and out of the
container = at start and end of the process. It seems to work. Mounting would probably = speed up the process
quite a bit if I could make it work.

Cheers,
-Steve

On Fri, Jun 5, 2020 at = 5:32 AM Christopher Marusich <cm= marusich@gmail.com> wrote:
Chris Marusich <cmmarusich@gmail.com> writes:

> Ludovic Court=C3=A8s <ludo@gnu.org> writes:
>
>>> Should Guix do anything about this?=C2=A0 We could change guix= -daemon to take
>>> correct action in the face of an XDEV error.=C2=A0 We could al= so improve the
>>> logging, since currently it silently swallows the XDEV error.<= br> >>
>> I guess we could delete recursively right away upon EXDEV.=C2=A0 I= t should be
>> just two lines of code, right?
>
> I'll try making the change and report back.=C2=A0 Yes, there are o= ther cases
> where we immediately delete without moving into the trash directory > (e.g., when the trash directory fails to be created), so it seems OK.<= br>
Here is a patch.=C2=A0 Turns out it's was just a one line change!=C2=A0= If nobody
has any further feedback on it, I'll go ahead and merge it to the maste= r
branch in the next couple days.

I tested it in one of the Docker containers provided by Stephen which
exhibited the problem.=C2=A0 I built the new Guix inside the Docker contain= er
and verified that a path which was previously unable to be GC'd due to<= br> the EXDEV error, was now able to be successfully GC'd.

My understanding is that the only reason why the guix-daemon attempts to move dead directories to the trash directory is to save time on
deleting, since large directories could take a while to fully delete.
If there is any reason why it might be unsafe to delete the directories
directly in case of EXDEV (I cannot think of any), please let me know.

--
Chris
--00000000000078324705a7586e03-- From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 06 21:31:22 2020 Received: (at 41607-close) by debbugs.gnu.org; 7 Jun 2020 01:31:22 +0000 Received: from localhost ([127.0.0.1]:52876 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhk9i-00071P-8d for submit@debbugs.gnu.org; Sat, 06 Jun 2020 21:31:22 -0400 Received: from mail-pf1-f177.google.com ([209.85.210.177]:38743) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhk9g-00071C-5u for 41607-close@debbugs.gnu.org; Sat, 06 Jun 2020 21:31:20 -0400 Received: by mail-pf1-f177.google.com with SMTP id x207so6846576pfc.5 for <41607-close@debbugs.gnu.org>; Sat, 06 Jun 2020 18:31:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=rCr/lmrW/BoAWejPj5ds6C3csQfQIR7iSfC6jTbA8fI=; b=BI+h6d14WPm8zbZ0jEtz6rsGhNoqEMtHk0ocop4ZrFL1BqbR6wwARoe2PFC6KFBd4X kT3jKtwZ4r6dNtlev8dWcUqZGX634ByXpI3Y8VDioxLrJ1ej13kWHCt3RRtM57HFQI81 h7xlijOUmW0kziTACxnJcoXmLcwy0UxvCnO1VFyMhKJuz+c+OesZNI3/T5+SOy8wfTVf CzpMuZkjJraFJwXJbyaIEU0+bNtiD6UqN7vxg33dQIxxxLYPMJYnPbsK5hyxnahFmcQr K3eMl1Ac73j/CfQ3BzquGDEf/hUCo7iyYJX3NhSbMixmMlyCR4XZQTeeiw1jXTW29j03 nLjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=rCr/lmrW/BoAWejPj5ds6C3csQfQIR7iSfC6jTbA8fI=; b=TNbulD7R3t50IK2cjXexR5ZDh5lej0bj4KTIov0W+xMxlGeWpzJMCp9wVzI0k+wvLw ikvyXB3Oh31drFJYU81uuHYttJJO+VNqLNDYZYREsfuGpoP1VkMABScYxRJfN6EhOyYX VgBDT4EO8dH2Sv8eacDfiB570rbRnltyNfZC9vDYrvpW7IyyLr1bq/28Xdutz1g9ZPJg 7FXbREElxCiSbA3MrdVwLSj5k0smEBwdjb7rDyesKyOS9RqG+KGZlfqHEcwDN6UzE02Y 3dXI2k6/7JDvCjVh0wPVKr5hIGTE1OCqsTuONWnf3MfJecmWmh5E6SsEl86Qc+SNU93H 4v0A== X-Gm-Message-State: AOAM5333eWigMYixgbneqlyJuQVbKu3CW45ctUaUSE0Nx4SPGaEChm6g uDu1h0ZI2qd8UfqTk587r2Q= X-Google-Smtp-Source: ABdhPJzbcHuM3jdB4KNfL2FBezpvWHdGbRzE3VK+WISyegqYYwPfXjZ0gHtAzpK+Q6BVTXiGj0Y/9w== X-Received: by 2002:a63:1142:: with SMTP id 2mr1947186pgr.172.1591493474082; Sat, 06 Jun 2020 18:31:14 -0700 (PDT) Received: from garuda-lan (c-73-97-103-127.hsd1.wa.comcast.net. [73.97.103.127]) by smtp.gmail.com with ESMTPSA id p16sm2625226pgj.53.2020.06.06.18.31.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 06 Jun 2020 18:31:12 -0700 (PDT) From: Chris Marusich To: Ludovic =?utf-8?Q?Court=C3=A8s?= , Stephen Scheck , zimoun Subject: Re: bug#41607: Deleted store items are not actually deleted References: <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> <87r1uu1x9e.fsf@gmail.com> <87pnad6eor.fsf@gmail.com> <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> <87r1uu1x9e.fsf@gmail.com> <87pnad6eor.fsf@gmail.com> <877dwl32l7.fsf@gnu.org> Date: Sat, 06 Jun 2020 18:31:08 -0700 In-Reply-To: <877dwl32l7.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Fri, 05 Jun 2020 18:21:56 +0200, Fri, 5 Jun 2020 12:05:24 -0400") Message-ID: <87r1urmzkz.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607-close Cc: 41607-close@debbugs.gnu.org, Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi everyone, I have committed the fix in d445c30ea6 and updated the guix package definition in ecbde6505c to ensure that the next time "guix pull" is run, the new guix-daemon version will be used. Ludovic Court=C3=A8s writes: > For consistency with (most) of the code, I=E2=80=99d suggest a /* */ comm= ent. OK - I did this in d445c30ea6. Stephen Scheck writes: > I've updated my build script to externalize the Guix environment from > the Docker container. So far the daily builds are staying nice and > small at ~197MB and one layer. The images and updated script are here > if anybody is curious: > > https://hub.docker.com/r/singularsyntax/guix/tags > https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-c= i.yml Neat! I'm glad you were figure out a way to to keep the image from growing in size. It's good to know of a way to do that. Thank you for your help and have a nice day! I'm closing this bug report. =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAl7cQ1wACgkQ3UCaFdgi Rp2uyw/+KFqobULrNAJlv5sKE2H7KpHKyMMx2v3X0UxZO1CAATzRi49wr93KqDIY 1Rmly1/fDmnSAWEM+4VeE4aN86z7ZLLwWhftVmpRQ3260PywPFkCLLxvRBKpj9Je HgWgdxglzPbSXuePxAr9hreMtRWnjThMlBqwN9m9qP5fj5QyLKAHr2HoVTEakxet eJzYTcyuHEjK95oUFAsgdZI9OQlGg1DDeCclY/VPJHrLN20DktZkbqOFli7RySam 6cY+nzuqJuc6Clid72i9ZD7oReGxWP1KRMjQQq3T0h+AYBbD5X2XbTxDvSbBqQIm 0YidhUqyEVO8d0NSRLzN7+Sd6qvp7neur6ZQr03Tvl6QU8+aDnsuVDDHXNmLeeIX eHv0sOB2Duf+FVT1jkZuRFy4oyXSqxf3bA5UHtGa/b8wC232kYiOKWLimm+hDrBT J9eLL7QAI/EFsHj+W8WoYf3dMs1FreLMQnt5XszRHXXfPf+1ScvmI0rSHB5rWJbp TC3waFcQMFJlqASlZ0dfv0GyZbeyDWABmIsI2s9c7+nFMDTuVAk1dQ71EFaVqWtL eWC/5BwosKA+We4+P5bNMGu7iJNcq3RSLp/6aKga4lhUmQpCE9oeTxwhUgKNbCKW 5N9W5gjvW7+zlLqLxcDvJWpsBzLiakI62aJQqj9uUeXH7pq1Qog= =2mYj -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 07 06:07:48 2020 Received: (at 41607-close) by debbugs.gnu.org; 7 Jun 2020 10:07:48 +0000 Received: from localhost ([127.0.0.1]:53377 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhsDU-0005GD-2p for submit@debbugs.gnu.org; Sun, 07 Jun 2020 06:07:48 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:46869) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhsDS-0005G0-Lq for 41607-close@debbugs.gnu.org; Sun, 07 Jun 2020 06:07:47 -0400 Received: by mail-qt1-f193.google.com with SMTP id g18so12354969qtu.13 for <41607-close@debbugs.gnu.org>; Sun, 07 Jun 2020 03:07:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=a6x5yg25SpKmlUmVsoGsVNos1pLhBv8PRj6KuoIRz9k=; b=T9dny/7LsHjjqqVqIYCAfvMuCdgdj3/yP/hZl/JwxvDhSY/mZ1Ds/kZ2DUuVVwkmqW ZXfjRSZdnH/xhwJbokE9ugizYoj/gZeb/7gA87reMgWKTbGwbWwBKf4U8/GEEbiRGHVH pdrHmFc2l9BA59ZS8+S7+bc9IPSt93w1qKBGzSAsUObEXD+0qzAsoGLK19Sm/bezq2DD +X41Xz+SHxP9rgFhQLHl0+cPDA41G9rwwnTvronc8M7v95hykUjM8wR9yFydal9L4gdD kYseP4XcC/Gheotasvh997k6F7du24dSB0gsYMegHAF99HvN8hPCWUlDAks5JrAWhwjA 1sIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=a6x5yg25SpKmlUmVsoGsVNos1pLhBv8PRj6KuoIRz9k=; b=aWWTe5GX0aPwp2ESEH0zNwgCKxCJu2tCu2zdAuBIiFyItidMlMHYj0wIkPwj/ih+uA FKUAYh/w7jMLHbanDplaMwZUMgxY4HLP/Gx/SepxTG6NVJt8sUfhzdIK9Si0jYJV47Ag tjYxi5+K5PDMF07oq1AGQqY7AhjtMwke8bcnWm1N6lR1g/AkebVFFPU0oaW3Kwz9Sosn Q7+0PkDi7zeBunNP3a4l5wRmx+pc6S9irXXCVZec13QGi/9yo+Sxz1Ju5JdXXllpgTdE LAxhR+uvPJO0hPY4dFrSi/JDCPBKx8RAhSFlfsiBArle3n+Ce2btvWZsrNZdvlal4E/x QYiw== X-Gm-Message-State: AOAM531hyXSQ+wpmKgaY3VtvS+6qHLD3mTtx2RNdprbCYbIzk7JblMHv P9G72pHr75A7L6TbDjBqG7/OUkZ/lQMF0IZu4BY= X-Google-Smtp-Source: ABdhPJyJ+/U2D+zvS/Dwy6Ve5gVIfTqbVWfzrBGAgWURiBh/uN/y7dvKKZihGo9VfmTvqsHBUfm30mQ6oPevqLL7gWU= X-Received: by 2002:aed:3169:: with SMTP id 96mr17519120qtg.211.1591524461199; Sun, 07 Jun 2020 03:07:41 -0700 (PDT) MIME-Version: 1.0 References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> <87r1uu1x9e.fsf@gmail.com> <87pnad6eor.fsf@gmail.com> <877dwl32l7.fsf@gnu.org> <87r1urmzkz.fsf_-_@gmail.com> In-Reply-To: <87r1urmzkz.fsf_-_@gmail.com> From: zimoun Date: Sun, 7 Jun 2020 12:07:29 +0200 Message-ID: Subject: Re: bug#41607: Deleted store items are not actually deleted To: Chris Marusich Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41607-close Cc: 41607-close@debbugs.gnu.org, =?UTF-8?Q?Ludovic_Court=C3=A8s?= , Stephen Scheck , Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Chris, Thank you. On Sun, 7 Jun 2020 at 03:31, Chris Marusich wrote: > I have committed the fix in d445c30ea6 and updated the guix package > definition in ecbde6505c to ensure that the next time "guix pull" is > run, the new guix-daemon version will be used. Sorry to be late, I have things like that: -8<---------------cut here---------------start------------->8--- [548 MiB] deleting '/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f' note: can't create move `/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1 unable to rename `/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f' to `/gnu/store/trash/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f': Invalid cross-device link [553 MiB] deleting '/gnu/store/yaajjri3cks8rhkb29z08d3q5waww0dx-packages.scm' --8<---------------cut here---------------end--------------->8--- I should do something wrong... All the best, simon From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 08 03:44:00 2020 Received: (at 41607) by debbugs.gnu.org; 8 Jun 2020 07:44:00 +0000 Received: from localhost ([127.0.0.1]:55836 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiCRs-0006H5-0a for submit@debbugs.gnu.org; Mon, 08 Jun 2020 03:44:00 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:39708) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiCRp-0006Gl-Nv; Mon, 08 Jun 2020 03:43:58 -0400 Received: by mail-pg1-f193.google.com with SMTP id w20so8363647pga.6; Mon, 08 Jun 2020 00:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=fKoCQe/VFEgWfY2bRYtB4oeZADpWnEeuL4lnMVyABZc=; b=AfOrJ4dlpi9G2rWsSXPSNJ3VQ2Gp0Z7AtT6cWupD8CN+WtL/HQEiUO/rIuWIgz6gqE LJSKXCnj52mfGn5zzCJdcQHQlrN9MYMPQ4xnlUUcNuuc4DdSz8RV1DrVfjonlFhEY2/v a2yzk7vwzeX6v1LLtFTUnrbxV+UjgXblVK3X2R5LrPG73fK/hu5rxgBqQu7pQgeMRp7D eEQmQXYFid+TaEYXgZQA+3anWu2ueF4ftn/5AkXYWxwDg61990CxIj/41eInXwqaGlo0 nBNIfm6qWiaD7SJ+bjthSaPwXKDXjwSAZAFFDIJRMuCw7hWlQS+UPwfG6wkKDEnw3Awm ebUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=fKoCQe/VFEgWfY2bRYtB4oeZADpWnEeuL4lnMVyABZc=; b=imMMUSBtElWYoEih7QutPyI8Pzp1Grbz+OQ6ZeMTRqJqygcmMnm0tXzqXClQ6RuAD4 lm6nyuZUfGVd7Fg++vFDxe/vU10CsJP7mmnZgahTFa72++cmbLsmFtFtKTRwSMOCuB1p eEeilsxen/bIcSnsIX8woFWDN7sYWCbp+tiq3lmsY/G4U2vnILggmOjJKf8g+txxyMkC Gb4scOaGuR40XXYLgnXYV5VJm8ZiAZWd1bAd60KUthidVw7wr6pn+6gcSGKW78NV3M5A xs7Nb3IpQ8T9AgPpJS3FGMzsqRFsS9yICz2wKKnDTKyw4Hev4bvkbo6u8tzE68Vc7LTt +dqg== X-Gm-Message-State: AOAM531DRspXhhINP3DUxIsSMSLWAFE+XuHIUaAp0aOOmMwynORBdP+k 556BW0WazYv02RlCaF4B3hs= X-Google-Smtp-Source: ABdhPJwA82vHnu64aEZJOEdbt/FSh5y1nyKGhFQj0UgNFRGx+rIuaq2LrzwD80rPcBAaLI+HHGNYeA== X-Received: by 2002:a63:a55c:: with SMTP id r28mr18933791pgu.108.1591602231686; Mon, 08 Jun 2020 00:43:51 -0700 (PDT) Received: from garuda-lan (c-73-97-103-127.hsd1.wa.comcast.net. [73.97.103.127]) by smtp.gmail.com with ESMTPSA id 71sm6359791pfb.20.2020.06.08.00.43.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2020 00:43:50 -0700 (PDT) From: Chris Marusich To: zimoun Subject: Re: bug#41607: Deleted store items are not actually deleted References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> <87r1uu1x9e.fsf@gmail.com> <87pnad6eor.fsf@gmail.com> <877dwl32l7.fsf@gnu.org> <87r1urmzkz.fsf_-_@gmail.com> Date: Mon, 08 Jun 2020 00:43:48 -0700 In-Reply-To: (zimoun's message of "Sun, 7 Jun 2020 12:07:29 +0200") Message-ID: <874krm9f4b.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 41607 Cc: 41607-close@debbugs.gnu.org, 41607@debbugs.gnu.org, Ludovic =?utf-8?Q?Court=C3=A8s?= , Stephen Scheck , Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi zimoun, zimoun writes: > On Sun, 7 Jun 2020 at 03:31, Chris Marusich wrote: > >> I have committed the fix in d445c30ea6 and updated the guix package >> definition in ecbde6505c to ensure that the next time "guix pull" is >> run, the new guix-daemon version will be used. > > Sorry to be late, I have things like that: > > [548 MiB] deleting '/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1= .1.1f' > note: can't create move > `/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1 unable to > rename `/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f' to > `/gnu/store/trash/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f': > Invalid cross-device link > [553 MiB] deleting '/gnu/store/yaajjri3cks8rhkb29z08d3q5waww0dx-packages.= scm' That looks normal to me. Previously, the cross-device link error would be silently ignored and the directory would not be deleted - no message would be printed about the cross-device link error. Now, however, the cross-device link error is reported, and the directory is deleted. If you find that the directory is not actually deleted, please let me know so I can look into it more. When I tested it on my end, the directory was in fact deleted. =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAl7d7DQACgkQ3UCaFdgi Rp0OfA//Ug3haMHSVugQMtPIeMvsfiO/uvEAus5ccSbFgoz8MeZI0x6LMxIpbfPs lf0bv9yNBNBn+Jf/XanwRLjscOCDxJc87eXYYtUHk8B5JrxTQ3ueUK4SspS2GgHA W9yR+/s2XCiDY3WAPD8Jr3iVTblRESuR7l78K/tCT0dPL0BX3xPtIaBm1PFg/594 LAIqhIfAlmcYtkvsUnp+2NM4t0h58HvD2iMRwdVXKg8A93fx+HBlBqWps1DlT6eH 1kI6BpXYbJGPn6nR4pCTcZHwI6ETXwUtVKOGchntqitsLigfNPVi5UjkKhQu0Y2U AHXqod284e2kvW73eaOAGoHkJr0rPdyqRjUjO5nOW/uCuN5p/NLl5qE4ar+z61nk JiDy13Y3yNkUPcpMEN3cmVJUIv7vJ6/w52mQO8z2i3TakbT7696aWuYui15XJlZ+ wv8PQ/g6InTi1093KPL+vmdTzkAlQEwyab+GJKC5WOwOP1x9+STU1qzSyBRbKMfi K6aNOjL9vf2WjxOt1C4jaAlXxtaTeKuMfkUp41VEIIFaocfw1nq4F2ujVSELvVU4 RQGrgrU3N+bKvHPEY59IFYPuvGOzVULsyy29/N0IOqFTFKSGIskit5J6iWkmvGAz JlF+P16EzjFhzW1sQEMTblfgJpuicq8ZImU3JJd/RIp+S07fVVA= =agrs -----END PGP SIGNATURE----- --=-=-=-- From unknown Mon Aug 18 02:05:27 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 06 Jul 2020 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator