From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 19 08:07:21 2021 Received: (at submit) by debbugs.gnu.org; 19 Aug 2021 12:07:21 +0000 Received: from localhost ([127.0.0.1]:58747 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGgpM-0002Km-Sb for submit@debbugs.gnu.org; Thu, 19 Aug 2021 08:07:21 -0400 Received: from lists.gnu.org ([209.51.188.17]:53628) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGgpM-0002Kf-0Z for submit@debbugs.gnu.org; Thu, 19 Aug 2021 08:07:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGgpL-0001Cq-SU for bug-guix@gnu.org; Thu, 19 Aug 2021 08:07:19 -0400 Received: from mout-p-102.mailbox.org ([2001:67c:2050::465:102]:43034) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256) (Exim 4.90_1) (envelope-from ) id 1mGgpJ-00060S-8q for bug-guix@gnu.org; Thu, 19 Aug 2021 08:07:19 -0400 Received: from smtp1.mailbox.org (smtp1.mailbox.org [80.241.60.240]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-102.mailbox.org (Postfix) with ESMTPS id 4Gr3RK1818zQjTy for ; Thu, 19 Aug 2021 14:07:09 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6xq.net; s=MBO0001; t=1629374827; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=hip4WWL0yj+LCidJ0ONV74EaBMrKYibwWR+yJaUJvEU=; b=fvJMZUHj71vWEA5d0aRAfwqf8uZyoTKqC/b2BeV26nOcGsqrQczLVggKZfTjppoZTWRNUL A/dlLM059GQXVRFDlkY3DwWcjjkzPqywiyxcUGI31FKIBzOs17wFUoXkiWpO5JG/GjZpTY wMklhAdF/Bm4+urq9/G8gN6HbssdJ+k8IU/maivgkQeES6logk463qc3m3NvdqVhD4euak XKEOEmGq94h/O/AyUT7r6sdnArb8xm0bbwE8jwaIAugOyFRmIxahOToWNZkqVDiYBPVv4y J/45y9ZxUvJIB+lKz9Epw9uNm0EPIifwXSWav3sivmH1dY1CrtMZKIXuzvLWhQ== Received: from smtp1.mailbox.org ([80.241.60.240]) by spamfilter06.heinlein-hosting.de (spamfilter06.heinlein-hosting.de [80.241.56.125]) (amavisd-new, port 10030) with ESMTP id z7SfcixetRIL for ; Thu, 19 Aug 2021 14:07:02 +0200 (CEST) Date: Thu, 19 Aug 2021 14:07:01 +0200 From: Lars-Dominik Braun To: bug-guix@gnu.org Subject: Deduplication breaks store item repair Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="vxz+bEefRHM5YCrX" Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: E69C618C2 X-Rspamd-UID: 4c6aa9 Received-SPF: pass client-ip=2001:67c:2050::465:102; envelope-from=lars@6xq.net; helo=mout-p-102.mailbox.org X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.6 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.6 (--) --vxz+bEefRHM5YCrX Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Hi, I’ve been having issues with the filesystem that holds /gnu/store recently, causing corrupted/broken files. When trying to repair these broken files with `guix gc --verify=repair,contents` it properly detects that store items’ hashes do not match the ones recorded in the database and redownloads/rebuilds them. However, the corrupted store items are never actually repaired – not by `guix gc` and not by `guix build --repair`. Attached is a testcase showing that deduplication is the problem, because repairing will just hardlink the (broken) deduplicated file instead of replacing it with the downloaded/built file. I tried the daemon’s `--disable-deduplication` too with same results. Cheers, Lars --vxz+bEefRHM5YCrX Content-Type: application/x-sh Content-Disposition: attachment; filename="testcase.sh" Content-Transfer-Encoding: quoted-printable # use =E2=80=9Chello=E2=80=9D as a test subject=0Apath=3D`guix build hello`= =0Ahello=3D$path/bin/hello=0Ahellohash=3D`guix hash -r $hello`=0Als -lh $he= llo=0Aecho "$hello $hellohash"=0A# =E2=80=9Ccorrupt=E2=80=9D the file=0Atru= ncate --size=3D0 $hello=0Als -lh $hello=0A# Now try to repair it=0Aguix gc = --verify=3Drepair,contents=0A# But nothing happens=0Als -lh $hello=0A# Try = again=0Aguix build --repair hello=0A# Again nothing happens=0Als -lh $hello= =0A# Fix the problem by removing the broken file from the deduplication=0A#= store.=0Arm /gnu/store/.links/$hellohash=0Aguix build --repair hello=0A# A= nd it=E2=80=99s fine again.=0Als -lh $hello=0A --vxz+bEefRHM5YCrX Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename=log Content-Transfer-Encoding: 8bit /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello 0jxnp4f4rac2scvq9lhcvpr4n4w0zrx5wdhcqml4w7zfybbszswp -r-xr-xr-x 2 root root 0 19. Aug 13:54 /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello reading the store... checking path existence... checking hashes... path `/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10' was modified! expected hash `9c61184c4b1af09639cee8148bc0c3d7aced4a671615a6e0a3e7ccb927848ffa', got `3330b928ba2d3cf6acfdb0ef3a359fb686eac7ee6e9d49a6515ef7b1701537cf' fetching path `/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10'... Downloading https://ci.guix.gnu.org/nar/lzip/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10... hello-2.10 51KiB 528KiB/s 00:00 [##################] 100.0% -r-xr-xr-x 2 root root 0 1. Jan 1970 /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello path `/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10' is corrupted or missing! Substituiere /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 … Lade von https://ci.guix.gnu.org/nar/lzip/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 herunter … hello-2.10 51KiB 482KiB/s 00:00 [##################] 100.0% /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 -r-xr-xr-x 2 root root 0 1. Jan 1970 /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello path `/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10' is corrupted or missing! Substituiere /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 … Lade von https://ci.guix.gnu.org/nar/lzip/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 herunter … hello-2.10 51KiB 455KiB/s 00:00 [##################] 100.0% /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 -r-xr-xr-x 2 root root 37K 1. Jan 1970 /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello --vxz+bEefRHM5YCrX-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 25 17:43:03 2022 Received: (at control) by debbugs.gnu.org; 25 Jan 2022 22:43:03 +0000 Received: from localhost ([127.0.0.1]:50452 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCUWl-0002cy-It for submit@debbugs.gnu.org; Tue, 25 Jan 2022 17:43:03 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:43596) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCUWk-0002cL-Ig for control@debbugs.gnu.org; Tue, 25 Jan 2022 17:43:02 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 26F953F0 for ; Tue, 25 Jan 2022 23:42:57 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m-OkIQW_BJeS for ; Tue, 25 Jan 2022 23:42:56 +0100 (CET) Received: from ribbon (91-160-117-201.subs.proxad.net [91.160.117.201]) by hera.aquilenet.fr (Postfix) with ESMTPSA id BBCB0A4 for ; Tue, 25 Jan 2022 23:42:56 +0100 (CET) Date: Tue, 25 Jan 2022 23:42:56 +0100 Message-Id: <871r0vl9n3.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #50121 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spamd-Bar: / Authentication-Results: hera.aquilenet.fr; none X-Rspamd-Server: hera X-Rspamd-Queue-Id: 26F953F0 X-Spamd-Result: default: False [0.61 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[control@debbugs.gnu.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; R_MIXED_CHARSET(0.71)[subject]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) severity 50121 important quit