From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 05 09:43:34 2021 Received: (at submit) by debbugs.gnu.org; 5 Jan 2021 14:43:34 +0000 Received: from localhost ([127.0.0.1]:41142 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwnYc-0007F3-HA for submit@debbugs.gnu.org; Tue, 05 Jan 2021 09:43:34 -0500 Received: from lists.gnu.org ([209.51.188.17]:44080) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwnYa-0007Ev-E7 for submit@debbugs.gnu.org; Tue, 05 Jan 2021 09:43:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33268) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kwnYY-0007Pf-VT for bug-guix@gnu.org; Tue, 05 Jan 2021 09:43:32 -0500 Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]:34191) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kwnYV-0001x5-So for bug-guix@gnu.org; Tue, 05 Jan 2021 09:43:30 -0500 Received: by mail-lf1-x12a.google.com with SMTP id o19so73163605lfo.1 for ; Tue, 05 Jan 2021 06:43:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:user-agent:mime-version; bh=z9ZFnmfBUtwirDsu2p1GtnrtxxiYdT9Fu0DcYhQ3gRo=; b=cDAaFIw9GDVTTovOKWUM3Gc5ncMfgqyqdFrV/dPRdc2Xa7lcu8iUbjX1X6oLIvZoMB 9TxjAq2RQrkjblt6tSx7GFSz+gISeTjomBiRqtCADLmeAhwJ25ySu38W3+7WWS8arRW7 eClxkbjJw4HqE4L8nSHcQdTzkT23bSSjOshjIrd3r5vpQVhiSudlN7nSSLBa1u6gGSqt plXNUV0fCWgxEh6KzqkVYRlzQrOSWs8ThfT0Txap1XgvfAreK53QBLBmquUSqzT1rLq/ FD4a7ZIcIhXo7KdeaFqetx+wVQRO/VyHB0exGS4PsCtUHBcjv+GgMW0YkKdrc+/WQRbT ccDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:user-agent :mime-version; bh=z9ZFnmfBUtwirDsu2p1GtnrtxxiYdT9Fu0DcYhQ3gRo=; b=mlbLdoUZChTCDlLpE+h4QkEFCj3mCO9FLxZ4qRz1GihQY55Nxo62ii9J3eN74cfq8i pY0+MJAzv5T/QnWivyCnThLL0teZezFZGxL3G5GeRQ29KcUjg9dnu3nDzakMRND3d5nN LehmCOmFzvqGO+CsvNR+PDGzsFU1dFHl0VwJTR1/M1xwOigTLC/Nw4bdmsU5d85JjMo/ BzY1OjnEESzxm00PgSpw8FjiVZsLneJNFKfu0dvzLH5Jgo6ENWUac1m77isqkw2w9JO+ XW2YRUx8Y1h/FIZP1AK3hRsJeeYPsAMVh4nYfKUt1IEqleSFBQomLHdxsVLGh9X0GYZN X+nQ== X-Gm-Message-State: AOAM5306wQ8upmNfcdNigb0GR+Qu9uPE2QEu1T+ciGABw9D0yawGeqLS koRzSyTcrJtiSqJNAwUh4vnXIuaoTOQ= X-Google-Smtp-Source: ABdhPJz5V3mHQz59ZUY7k70uPoX8QlNxy6SXwHaItxP1hHrCKximSdHbjA2B8ktd7tF+HaMhReVhFw== X-Received: by 2002:adf:decb:: with SMTP id i11mr85542819wrn.26.1609857370742; Tue, 05 Jan 2021 06:36:10 -0800 (PST) Received: from unfall (36.193.158.146.dynamic.jazztel.es. [146.158.193.36]) by smtp.gmail.com with ESMTPSA id r7sm4423678wmh.2.2021.01.05.06.36.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Jan 2021 06:36:10 -0800 (PST) From: =?utf-8?Q?Miguel_=C3=81ngel_Arruga_Vivas?= To: bug-guix@gnu.org Subject: Store references inside compressed data Date: Tue, 05 Jan 2021 15:36:07 +0100 Message-ID: <8735zf30yw.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2a00:1450:4864:20::12a; envelope-from=rosen644835@gmail.com; helo=mail-lf1-x12a.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.1 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.1 (--) There are several binary formats that allow compression of the executable image, or some of its data, which is decompress at runtime: - Kernel images. - Compressed libraries: e.g. Smalltalk modules. - Compressed executable or data files: e.g. library.el.gz. These aren't taken into account by the grafting process, which may lead to issues when store paths are located inside that kind of files. From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 05 15:22:22 2021 Received: (at 45676) by debbugs.gnu.org; 5 Jan 2021 20:22:22 +0000 Received: from localhost ([127.0.0.1]:42753 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwsqT-0005Tu-WB for submit@debbugs.gnu.org; Tue, 05 Jan 2021 15:22:22 -0500 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:56703) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwsqR-0005Tg-G2 for 45676@debbugs.gnu.org; Tue, 05 Jan 2021 15:22:19 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id A158DFE0; Tue, 5 Jan 2021 15:22:13 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 05 Jan 2021 15:22:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=famulari.name; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-transfer-encoding:in-reply-to; s=mesmtp; bh=OJpvC98aKVT4D/IP9ysWsqqnHi+Xn1tYXzyQJyr/vWk=; b=O89nLZ1RJ3CO jy2PMp4ag42c0oC3Y/BqqYT87r3eckOXhvK+VLFGm/Z7+OdtabSjEDRlrZVY293C 65S7UJuGGPPuMUv36piLhoMN24WTPtuktSncBE2nt7WlMpHONeipbX2V1AWhWWdU 6uAx0rLb+Hge9RGiXRgSUpXF1DuYOSk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=OJpvC98aKVT4D/IP9ysWsqqnHi+Xn1tYXzyQJyr/v Wk=; b=GzbWMVkQJiz8oRxC1l2FDvmNmUbZG/eI/FT9KKe+XZjukyCXWAp/H2nRS EiTjtkBeiKVhObZecr+JrQ14fu+uiwAKWLnlqF+122fHtC2QVNmfdzgvESlgybGk mffRefs4bTLoQSpvX0S3I5OHGW/ke9OMTn+24bOoDPWIcQSk8ME9Q0dxt0uCjwwM CksDZgluz738RdWd7iXgZqAduDJXJB2iNpLVE7A7dmd67lprn/X6OIWU0ExpCJnu XnbReM8wjihHAxtdghw1r3IgVc6xjwdxtJ8FIOj6ddwdi9JXNOIDuMZHxlK84cmk 7sAYO5nCmP/szxiZ8ej/IPdGlCIRg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdefjedguddttdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtugfgjgesth ekredttddtudenucfhrhhomhepnfgvohcuhfgrmhhulhgrrhhiuceolhgvohesfhgrmhhu lhgrrhhirdhnrghmvgeqnecuggftrfgrthhtvghrnhepgeeileetueeuudejheefhfdtud fgfeefheeutdeuveefhffhffffgfevieeludfhnecuffhomhgrihhnpehgnhhurdhorhhg necukfhppeejfedrudeguddruddvjedrudegieenucevlhhushhtvghrufhiiigvpedtne curfgrrhgrmhepmhgrihhlfhhrohhmpehlvghosehfrghmuhhlrghrihdrnhgrmhgv X-ME-Proxy: Received: from localhost (c-73-141-127-146.hsd1.pa.comcast.net [73.141.127.146]) by mail.messagingengine.com (Postfix) with ESMTPA id A1EF3108005F; Tue, 5 Jan 2021 15:22:12 -0500 (EST) Date: Tue, 5 Jan 2021 15:22:10 -0500 From: Leo Famulari To: Miguel =?iso-8859-1?Q?=C1ngel?= Arruga Vivas Subject: Re: bug#45676: Store references inside compressed data Message-ID: References: <8735zf30yw.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8735zf30yw.fsf@gmail.com> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45676 Cc: 45676@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Tue, Jan 05, 2021 at 03:36:07PM +0100, Miguel Ángel Arruga Vivas wrote: > There are several binary formats that allow compression of the > executable image, or some of its data, which is decompress at runtime: > > - Kernel images. > - Compressed libraries: e.g. Smalltalk modules. > - Compressed executable or data files: e.g. library.el.gz. > > These aren't taken into account by the grafting process, which may lead > to issues when store paths are located inside that kind of files. It's a serious problem, and not just because of grafting. These obscured references can cause things to be garbage collected inappropriately. Here is an older case of the same problem: https://bugs.gnu.org/24703 It was resolved by patching GCC. From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 05 15:22:42 2021 Received: (at 45676) by debbugs.gnu.org; 5 Jan 2021 20:22:42 +0000 Received: from localhost ([127.0.0.1]:42757 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwsqo-0005UU-93 for submit@debbugs.gnu.org; Tue, 05 Jan 2021 15:22:42 -0500 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:41413) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwsqn-0005UH-47 for 45676@debbugs.gnu.org; Tue, 05 Jan 2021 15:22:41 -0500 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 6F7F8FE0; Tue, 5 Jan 2021 15:22:35 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Tue, 05 Jan 2021 15:22:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=famulari.name; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-transfer-encoding:in-reply-to; s=mesmtp; bh=vxrS8hTntS3JJ2kBpVTFlbodQJUqyncOYMVqalfmfXE=; b=QD1kFKJMEr1m EzC0H6BV16Ai/TDXyllKLwW3VLLR+eBfnxaac8iuNFHYnrsdAE44l/xgfDyrzpei h2KD9owudryyIMUbQBfHE01LJVMhcvm9W3CzT2K0N5WHIqh3KgjKmDwxlREm//ob wyvkfYTioHNCdM6D0W+jaiprQL+GcIE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=vxrS8hTntS3JJ2kBpVTFlbodQJUqyncOYMVqalfmf XE=; b=MrA6Oq4KIshlVQXi5f70kayjXRq9ar1WFDwUDxmJNEDiUcFCp6SwL6u7q wA9c8fB23WZORVabqsazHfGOKBDdAihstKEfy05/0PCiNBs7ndKIWEpoDvu3nuwr lFu5mw122hRc9WdzA7YkyDbLEs15z38OXuMDbVJWXq4OWoimzW2BjdBdaX0ZxVoo ju5eyV/dNCvBPAAVVAFntzXwMmbpEFzIeBLBta7o5O/uwvdfCEeS3EIF2+t0XKq4 /9kRbA+U3SdEM01Twn/Sqzxml2b+OJ2hrSdZWxjD9UNrXT9OPeKtL98l2WcRONww m0VOUzS4rtIQS/+4JLhCI/VH7b7GA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdefjedguddttdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtugfgjgesth ekredttddtudenucfhrhhomhepnfgvohcuhfgrmhhulhgrrhhiuceolhgvohesfhgrmhhu lhgrrhhirdhnrghmvgeqnecuggftrfgrthhtvghrnhepueeugedugfdvgfevuedvledule efjefhgefgjeetgfegkedukefhvdefleejhefgnecukfhppeejfedrudeguddruddvjedr udegieenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hlvghosehfrghmuhhlrghrihdrnhgrmhgv X-ME-Proxy: Received: from localhost (c-73-141-127-146.hsd1.pa.comcast.net [73.141.127.146]) by mail.messagingengine.com (Postfix) with ESMTPA id CAE201080059; Tue, 5 Jan 2021 15:22:34 -0500 (EST) Date: Tue, 5 Jan 2021 15:22:33 -0500 From: Leo Famulari To: Miguel =?iso-8859-1?Q?=C1ngel?= Arruga Vivas Subject: Re: bug#45676: Store references inside compressed data Message-ID: References: <8735zf30yw.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8735zf30yw.fsf@gmail.com> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 45676 Cc: 45676@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Tue, Jan 05, 2021 at 03:36:07PM +0100, Miguel Ángel Arruga Vivas wrote: > There are several binary formats that allow compression of the > executable image, or some of its data, which is decompress at runtime: > > - Kernel images. > - Compressed libraries: e.g. Smalltalk modules. > - Compressed executable or data files: e.g. library.el.gz. > > These aren't taken into account by the grafting process, which may lead > to issues when store paths are located inside that kind of files. If you have specific instances of this type of bug, please report them. From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 05 17:34:14 2021 Received: (at submit) by debbugs.gnu.org; 5 Jan 2021 22:34:14 +0000 Received: from localhost ([127.0.0.1]:42900 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwuu5-0000lB-RZ for submit@debbugs.gnu.org; Tue, 05 Jan 2021 17:34:14 -0500 Received: from lists.gnu.org ([209.51.188.17]:56122) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwuu3-0000l3-Kb for submit@debbugs.gnu.org; Tue, 05 Jan 2021 17:34:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46612) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kwuu3-0005d4-DB for bug-guix@gnu.org; Tue, 05 Jan 2021 17:34:11 -0500 Received: from tobias.gr ([2a02:c205:2020:6054::1]:50722) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kwuu1-0002AK-Aj for bug-guix@gnu.org; Tue, 05 Jan 2021 17:34:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tobias.gr; s=2018; bh=Z4r1Nw8EDW9v6fP4WIn0eMrNVuMWnj5qn+iuUL4fzwA=; h=date:in-reply-to: references:subject:cc:to:from; b=BycB56QwGOxTc72sojjOCGlEfqjKjF1rGtbZT yRMU3AZPtKFirDt4P8wNiTXndhHR6ughcpCt1TRRNYreYf+XTtBvlrryMYGRW0Bh1ux2u3 7d9fUC+ULkA5BbiHVXugbM/trqxm25Eg1IgBGsJMDb+teXZ5H1DYcHWORcJMNNn+wwBePU N01kXqo3//6RFzL05ksqZas8fnSkdPyC1UU/+87uZntP2YdHvMRowZU2K8oP10rw8aRINk Sn9bEODFP43+KVVcVxXAjuiKtG5IaoJ7PbLXnWRA4c3TvL7XjnLKP152dhGQ2cq7d/KHwP aK8F+jXzv4wVxI/ze19XWblVQ== Received: by submission.tobias.gr (OpenSMTPD) with ESMTPSA id c08dce1b (TLSv1.2:ECDHE-ECDSA-AES256-GCM-SHA384:256:NO); Tue, 5 Jan 2021 22:34:32 +0000 (UTC) BIMI-Selector: v=BIMI1; s=default; From: Tobias Geerinckx-Rice To: Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas Subject: Re: bug#45676: Store references inside compressed data References: <8735zf30yw.fsf@gmail.com> In-reply-to: <8735zf30yw.fsf@gmail.com> Date: Tue, 05 Jan 2021 23:33:59 +0100 Message-ID: <871rezf1yg.fsf@nckx> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=2a02:c205:2020:6054::1; envelope-from=me@tobias.gr; helo=tobias.gr X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: 45676@debbugs.gnu.org, bug-guix@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) --=-=-= Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Hi! Miguel =C3=81ngel Arruga Vivas wrote: > These aren't taken into account by the grafting process, which=20 > may lead > to issues when store paths are located inside that kind of=20 > files. It's true. It's a known trade-off of an otherwise=20 almost-zero-effort yet fast reference scanner. I don't think it's=20 a bug per se, but it is something of which to be aware. I also=20 think this trade-off is worth it. Luckily, this case is easier to fix than the infamous=20 , because the right solution is=20 simple: > - Compressed libraries: e.g. Smalltalk modules. > - Compressed executable or data files: e.g. library.el.gz. Let's stop installing compressed executables & data files. We=20 already avoid compressed .jars and other renamed zip files. It=20 ain't right. It's not 1998, my hard drive isn't 1.1GB, and I didn't just=20 reinstall Slackware because I =E2=80=98accidentally=E2=80=99 gzexe'd gzip. Gzipping a tiny handful of Lisp or Smalltalk files is pointless=20 when zstd {,de}compresses my entire 500GB SSD better and faster,=20 at the file system level where it now squarely belongs. Without=20 breaking Guix. Kind regards, T G-R --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCX/TpVw0cbWVAdG9iaWFz LmdyAAoJEA2w/4hPVW15fqUA/AguLGDH8mb0d95sWnOYLYTkg0BwteblZfmqDZa4 GH6zAP4oG7O7QPczb4CXwElweEdbpsYuzZf2n3fc8mI7wWzsCw== =92PV -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 06 03:54:32 2021 Received: (at 45676) by debbugs.gnu.org; 6 Jan 2021 08:54:32 +0000 Received: from localhost ([127.0.0.1]:43368 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kx4aN-0000He-TV for submit@debbugs.gnu.org; Wed, 06 Jan 2021 03:54:32 -0500 Received: from mailrelay.tugraz.at ([129.27.2.202]:53763) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kx4aL-0000HS-Ry for 45676@debbugs.gnu.org; Wed, 06 Jan 2021 03:54:31 -0500 Received: from nijino.local (217-149-174-13.nat.highway.telekom.at [217.149.174.13]) by mailrelay.tugraz.at (Postfix) with ESMTPSA id 4D9jpn6wpwz1LBF8; Wed, 6 Jan 2021 09:54:25 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.11.0 mailrelay.tugraz.at 4D9jpn6wpwz1LBF8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tugraz.at; s=mailrelay; t=1609923266; bh=PP/QIIgF8Mm0eRFLaTBb7U75vQ36PrKOGxYiO1MoAeQ=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=RE6UgVAxWAGaDRirEoXFA/Echod91tK6+9zS6NLxe39lZKhH1xPXmQkD/ZIDxC/FW f0vFF9T3Zus/egfHK0Rr6C2xdr8OG3qLavkgbFg2qvn8MHLQeo6kaqrgRNf4DAsABe 1/U7tcM9lusVcUZflkLl7b3FN0SXbBt/GeVdhDW8= Message-ID: <06ba5c0f24bdcdb706990c9169093aba72463302.camel@student.tugraz.at> Subject: Re: bug#45676: Store references inside compressed data From: Leo Prikler To: Tobias Geerinckx-Rice , Miguel =?ISO-8859-1?Q?=C1ngel?= Arruga Vivas Date: Wed, 06 Jan 2021 09:54:25 +0100 In-Reply-To: <871rezf1yg.fsf@nckx> References: <8735zf30yw.fsf@gmail.com> <871rezf1yg.fsf@nckx> Content-Type: multipart/mixed; boundary="=-UZ7bSDYiGU4d4OZNtwAU" User-Agent: Evolution 3.34.2 MIME-Version: 1.0 X-TUG-Backscatter-control: bt4lQm5Tva3SBgCuw0EnZw X-Spam-Scanner: SpamAssassin 3.003001 X-Spam-Score-relay: -1.9 X-Scanned-By: MIMEDefang 2.74 on 129.27.10.117 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45676 Cc: 45676@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-UZ7bSDYiGU4d4OZNtwAU Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Hi! Am Dienstag, den 05.01.2021, 23:33 +0100 schrieb Tobias Geerinckx-Rice: > Let's stop installing compressed executables & data files. We > already avoid compressed .jars and other renamed zip files. It > ain't right. Would this be strictly necessary even if the same references are kept through other files, e.g. uncompressed binaries? I'll attach a patch, that fixes Emacs just in case. Regards, Leo --=-UZ7bSDYiGU4d4OZNtwAU Content-Disposition: attachment; filename="0001-gnu-emacs-Don-t-install-compressed-archives.patch" Content-Transfer-Encoding: base64 Content-Type: text/x-patch; name="0001-gnu-emacs-Don-t-install-compressed-archives.patch"; charset="UTF-8" RnJvbSA1N2MyM2JmNmVjYWM3OWMzOTdjYjQ5ZmYyNTExNzZlYzNhN2IxY2Y1IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBMZW8gUHJpa2xlciA8bGVvLnByaWtsZXJAc3R1ZGVudC50dWdy YXouYXQ+CkRhdGU6IFdlZCwgNiBKYW4gMjAyMSAwOToyNDowNyArMDEwMApTdWJqZWN0OiBbUEFU Q0hdIGdudTogZW1hY3M6IERvbid0IGluc3RhbGwgY29tcHJlc3NlZCBhcmNoaXZlcy4KClNlZSA8 aHR0cDovL2lzc3Vlcy5ndWl4LmdudS5vcmcvNDU2NzYjMz4uCgoqIGdudS9wYWNrYWdlcy9lbWFj cy5zY20gKGVtYWNzKVsjOmNvbmZpZ3VyZS1mbGFnc106CkFkZCAtLXdpdGhvdXQtY29tcHJlc3Mt aW5zdGFsbC4KKGVtYWNzLW1pbmltYWwpWyM6Y29uZmlndXJlLWZsYWdzXTogTGlrZXdpc2UuCi0t LQogZ251L3BhY2thZ2VzL2VtYWNzLnNjbSB8IDQgKysrLQogMSBmaWxlIGNoYW5nZWQsIDMgaW5z ZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQoKZGlmZiAtLWdpdCBhL2dudS9wYWNrYWdlcy9lbWFj cy5zY20gYi9nbnUvcGFja2FnZXMvZW1hY3Muc2NtCmluZGV4IGNhMTQ1ODRhZGEuLmFhNjM2Yjhj OWIgMTAwNjQ0Ci0tLSBhL2dudS9wYWNrYWdlcy9lbWFjcy5zY20KKysrIGIvZ251L3BhY2thZ2Vz L2VtYWNzLnNjbQpAQCAtMTI0LDYgKzEyNCw3IEBACiAgICAgIGAoIzp0ZXN0cz8gI2YgICAgICAg ICAgICAgICAgICAgICAgOyBubyBjaGVjayB0YXJnZXQKICAgICAgICAjOmNvbmZpZ3VyZS1mbGFn cyAobGlzdCAiLS13aXRoLW1vZHVsZXMiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg Ii0td2l0aC1jYWlybyIKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiLS13aXRob3V0 LWNvbXByZXNzLWluc3RhbGwiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIi0tZGlz YWJsZS1idWlsZC1kZXRhaWxzIikKICAgICAgICAjOnBoYXNlcwogICAgICAgIChtb2RpZnktcGhh c2VzICVzdGFuZGFyZC1waGFzZXMKQEAgLTM1NSw3ICszNTYsOCBAQCBhbHNvIGVuYWJsZWQgYW5k IHdvcmtzIHdpdGhvdXQgZ2xpdGNoZXMgZXZlbiBvbiBYIHNlcnZlci4iKSkpKQogICAgIChhcmd1 bWVudHMKICAgICAgKHN1YnN0aXR1dGUta2V5d29yZC1hcmd1bWVudHMgKHBhY2thZ2UtYXJndW1l bnRzIGVtYWNzKQogICAgICAgICgoIzpjb25maWd1cmUtZmxhZ3MgZmxhZ3MgJycoKSkKLSAgICAg ICAgYChsaXN0ICItLXdpdGgtZ251dGxzPW5vIiAiLS1kaXNhYmxlLWJ1aWxkLWRldGFpbHMiKSkK KyAgICAgICAgYChsaXN0ICItLXdpdGgtZ251dGxzPW5vIiAiLS1kaXNhYmxlLWJ1aWxkLWRldGFp bHMiCisgICAgICAgICAgICAgICAiLS13aXRob3V0LWNvbXByZXNzLWluc3RhbGwiKSkKICAgICAg ICAoKCM6cGhhc2VzIHBoYXNlcykKICAgICAgICAgYChtb2RpZnktcGhhc2VzICxwaGFzZXMKICAg ICAgICAgICAgKGRlbGV0ZSAncmVzdG9yZS1lbWFjcy1wZG1wKQotLSAKMi4zMC4wCgo= --=-UZ7bSDYiGU4d4OZNtwAU-- From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 06 06:35:43 2021 Received: (at 45676) by debbugs.gnu.org; 6 Jan 2021 11:35:43 +0000 Received: from localhost ([127.0.0.1]:43621 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kx76N-0006y7-Al for submit@debbugs.gnu.org; Wed, 06 Jan 2021 06:35:43 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47778) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kx76M-0006xk-7b for 45676@debbugs.gnu.org; Wed, 06 Jan 2021 06:35:42 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58412) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kx76G-0004xb-Un; Wed, 06 Jan 2021 06:35:36 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41504 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kx76F-00078F-FD; Wed, 06 Jan 2021 06:35:36 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Leo Famulari Subject: Re: bug#45676: Store references inside compressed data References: <8735zf30yw.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 17 =?utf-8?Q?Niv=C3=B4se?= an 229 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 06 Jan 2021 12:35:34 +0100 In-Reply-To: (Leo Famulari's message of "Tue, 5 Jan 2021 15:22:33 -0500") Message-ID: <87eeiymh6h.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45676 Cc: 45676@debbugs.gnu.org, Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Leo Famulari skribis: > On Tue, Jan 05, 2021 at 03:36:07PM +0100, Miguel =C3=81ngel Arruga Vivas = wrote: >> There are several binary formats that allow compression of the >> executable image, or some of its data, which is decompress at runtime: >>=20 >> - Kernel images. >> - Compressed libraries: e.g. Smalltalk modules. >> - Compressed executable or data files: e.g. library.el.gz. >>=20 >> These aren't taken into account by the grafting process, which may lead >> to issues when store paths are located inside that kind of files. > > If you have specific instances of this type of bug, please report them. Agreed. The general issue is =E2=80=9Cwell known=E2=80=9D as we say, but w= hat I think we need to do is look for specific instances and address them. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 06 10:03:28 2021 Received: (at control) by debbugs.gnu.org; 6 Jan 2021 15:03:28 +0000 Received: from localhost ([127.0.0.1]:45255 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxALP-00076a-Po for submit@debbugs.gnu.org; Wed, 06 Jan 2021 10:03:27 -0500 Received: from mail-wr1-f46.google.com ([209.85.221.46]:32903) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxALO-00076K-JI for control@debbugs.gnu.org; Wed, 06 Jan 2021 10:03:26 -0500 Received: by mail-wr1-f46.google.com with SMTP id t30so2740060wrb.0 for ; Wed, 06 Jan 2021 07:03:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:to:from:subject:mime-version :content-transfer-encoding; bh=s/cDyQff5S45wuaNXrXJLf+zQySpqe41ABwi39WOUpc=; b=L1PNheikj/IutqpA08JcUiNo9diTs87Fg80qrGxgFbZHdEZq2X9va2Jb3q3voMEiwS lkWiUQeLqmXgNo7fgriGcdOvhVbPp5ukI6FQItVhKhTIzpb5do8ZeZdrDZdz6vLNwMRu vZIed0Y3QJmaRVhexDkkClNqhSaR1uEZguZdkY/jRAi7M8nKdHBBFjJ0lLGlHTkNuXUI oj1VOYXcU1JB8JPfMGJQ95j2SdCrJuMaW4dZyzTRLsjSEaeFHWe2MweH2CK5iT6AlTFu 3DYmTBLvo+I1L/1K8tgu9CXqoqJpUrb+hr+MDRdGvi/hkcVwyMJ5NcgMDA5cfz2Lur+N tQBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:to:from:subject:mime-version :content-transfer-encoding; bh=s/cDyQff5S45wuaNXrXJLf+zQySpqe41ABwi39WOUpc=; b=VaaUKjEUX7VdJgUsVZlVE44/eN42ysNh+2Ky1Nj+QjezshjCc6uEFwz5PeM8/3FGG9 xeUL2w8hiL9M1q9Nqj0NUBnkOc1oASfKWK1EOyOCndn9EL7EUGwRVEcqrYE+uZZoMtyX AvV76CAhubXjcqt2a1Z1qXlWVywRyEWka6ZK+q/X6FYicld2vvjB8VE4BPErV5uwHxxr sFrxdaD5BFw4SYAwuffQIasvrHN04pnryNsGVcYo2hZ56Pa99yMLBN/f3Y9tzUBmLgCZ c0hLoVNGa45xEue1Nj3QjZRhCR4h/Gz7pIwMRsAEYLUYh2yXrpMfEcNIdteVUM5gB6SM 7GCQ== X-Gm-Message-State: AOAM530iNgQIJYyjoIehm1MkpcejbSNa+UKTGoAuJPMBWQb2aCAGc+eo VQR5uiPUnf2cQWnTLU4JSQt3rQg2EEM= X-Google-Smtp-Source: ABdhPJzdZF4XCI9mZUaUYfoqye3xQQ/r++bQDosblMVCWxbWcbNTudVxWjNAu/cf8eO6tE5owqxFtw== X-Received: by 2002:adf:84c1:: with SMTP id 59mr4840416wrg.409.1609945400720; Wed, 06 Jan 2021 07:03:20 -0800 (PST) Received: from unfall (36.193.158.146.dynamic.jazztel.es. [146.158.193.36]) by smtp.gmail.com with ESMTPSA id z63sm3558675wme.8.2021.01.06.07.03.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 07:03:20 -0800 (PST) Date: Wed, 06 Jan 2021 16:03:12 +0100 Message-Id: <87y2h6151r.fsf@gmail.com> To: control@debbugs.gnu.org From: =?utf-8?Q?Miguel_=C3=81ngel_Arruga_Vivas?= Subject: control message for bug #45676 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 2.2 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: severity 45676 wishlist quit Content analysis details: (2.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (rosen644835[at]gmail.com) 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in digit (rosen644835[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.221.46 listed in wl.mailspike.net] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.221.46 listed in list.dnswl.org] 1.9 MALFORMED_FREEMAIL Bad headers on message from free email service X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.2 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: severity 45676 wishlist quit Content analysis details: (1.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.221.46 listed in wl.mailspike.net] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.221.46 listed in list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (rosen644835[at]gmail.com) 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in digit (rosen644835[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 1.9 MALFORMED_FREEMAIL Bad headers on message from free email service severity 45676 wishlist quit From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 06 11:57:54 2021 Received: (at 45676) by debbugs.gnu.org; 6 Jan 2021 16:57:54 +0000 Received: from localhost ([127.0.0.1]:45399 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxC89-0003eC-OU for submit@debbugs.gnu.org; Wed, 06 Jan 2021 11:57:54 -0500 Received: from mail-wr1-f43.google.com ([209.85.221.43]:41388) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxC87-0003dr-N4 for 45676@debbugs.gnu.org; Wed, 06 Jan 2021 11:57:52 -0500 Received: by mail-wr1-f43.google.com with SMTP id a12so3035204wrv.8 for <45676@debbugs.gnu.org>; Wed, 06 Jan 2021 08:57:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=SlfGiJ3YrjX15eiy3EcObtDkJf61CD9886K3FKZqfS8=; b=cervPh9S4S6Ihk3pSaDb/JI/TJGJK88JXJj5/EgUuusKKsL8uU155vE2/+c0EvYjVu u82s9Jk1heGb/ZJhgVLs11oKrz5ipVMDvubW+aJ2LC3MhSNR0B4U2Cs5USen+lhC4yJn h8evtk2npE4aJfRN70Y04DVLO8KeJB07yVxHiwb+iFnGU8SLLcwPDf82bGLiW3Y60Ycg EDwvg/T7p3Lt6pt89FCX9KNwFePMrBH6DgLg/BbmTNJwMedxtL0FAWWSdFkJwkzLxOA7 Y/bk2qvsiSQ/3m8E37d7YEErBuNJ2tcUa4P2lO0wQGiISML5j+9s2hKAB9+vxXawbeFR Abkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=SlfGiJ3YrjX15eiy3EcObtDkJf61CD9886K3FKZqfS8=; b=i1ajNMOKadKrI6DY5/wtHZUxvLZYc9sqyWWKjgU2Cvue+phOo0DlQEuVBoo4cbECMc z29AYq2tAz8nDkkumRC2GY2AnUrr2GwyNGTFazko9V5hAhO9ihrUfOW77/K77OUFPkck Po2YqUM3vRWCTUR013L/EQgwyzlhw5ECYIseifg8PjVipNyEVUmXLV0LE0fOQcCGkKwM bEt2MWJvT5tChz3dbE8Ka1+8DoDrpEsetqGdXh7Q42UBqCtYrI8vxdGE8rtbVEB+nRtT nlYQycGuAR1XilIyPCh8bnYvGNt9WLNcCgcrNUOzAJQL7IbrF7qf8izK4vN8P5yP2UH+ 3qOw== X-Gm-Message-State: AOAM533w9dx2ER9lBm+Hhay2V8oHllHcz56ua0HCKHveFxMyzP739w2z wWdTn36To5PqpdiWDHuPzGJIVa3X1wA= X-Google-Smtp-Source: ABdhPJwEIxinPbMYm0mfa0GzUfsdL8k4rw/3CeIAx8vF7jPS5qT6XgaduUcerOktkh/PbvG6S527bw== X-Received: by 2002:adf:d082:: with SMTP id y2mr5175773wrh.301.1609952265664; Wed, 06 Jan 2021 08:57:45 -0800 (PST) Received: from unfall (36.193.158.146.dynamic.jazztel.es. [146.158.193.36]) by smtp.gmail.com with ESMTPSA id b83sm3821634wmd.48.2021.01.06.08.57.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 08:57:44 -0800 (PST) From: =?utf-8?Q?Miguel_=C3=81ngel_Arruga_Vivas?= To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#45676: Store references inside compressed data References: <8735zf30yw.fsf@gmail.com> <87eeiymh6h.fsf@gnu.org> Date: Wed, 06 Jan 2021 17:57:43 +0100 In-Reply-To: <87eeiymh6h.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 06 Jan 2021 12:35:34 +0100") Message-ID: <87mtxm0zqw.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 45676 Cc: 45676@debbugs.gnu.org, Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Hi Ludo and Leo, Ludovic Court=C3=A8s writes: > Hi, > > Leo Famulari skribis: > >> On Tue, Jan 05, 2021 at 03:36:07PM +0100, Miguel =C3=81ngel Arruga Vivas= wrote: >>> There are several binary formats that allow compression of the >>> executable image, or some of its data, which is decompress at runtime: >>>=20 >>> - Kernel images. >>> - Compressed libraries: e.g. Smalltalk modules. >>> - Compressed executable or data files: e.g. library.el.gz. >>>=20 >>> These aren't taken into account by the grafting process, which may lead >>> to issues when store paths are located inside that kind of files. >> >> If you have specific instances of this type of bug, please report them. > > Agreed. The general issue is =E2=80=9Cwell known=E2=80=9D as we say, but= what I think > we need to do is look for specific instances and address them. It can be tagged it notabug if you consider so. I've tagged it as wishlist (I should have been done it before) for that reason (it's "well known"), but I haven't found any specific instance yet. OTOH, I think it might be closely related to #33848, as the solution for both issues could be solved by the extension on the dumpPath code path---or the Scheme implementation equivalent, as pointed there. Happy hacking! Miguel From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 06 13:41:11 2021 Received: (at 45676) by debbugs.gnu.org; 6 Jan 2021 18:41:12 +0000 Received: from localhost ([127.0.0.1]:45528 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxDk7-0008F2-Kj for submit@debbugs.gnu.org; Wed, 06 Jan 2021 13:41:11 -0500 Received: from mail-wr1-f51.google.com ([209.85.221.51]:34310) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxDk5-0008Ep-65 for 45676@debbugs.gnu.org; Wed, 06 Jan 2021 13:41:10 -0500 Received: by mail-wr1-f51.google.com with SMTP id q18so3353920wrn.1 for <45676@debbugs.gnu.org>; Wed, 06 Jan 2021 10:41:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version; bh=l4D7RGUK4DqePxFLmZN0FFPwnVhpJwr/A0VwH/ncaQQ=; b=NhydUk9ENIQRhONs6CrZbvO3EKk3Op5Y6QQpFrzygpdi3/T2eFbDa5uEzFiEYxwyMw eEZYYuYnzHFjTBgwCI3onr4fW/G9QAHnYg6qr05ij6x5K+HpX3FkpW2tq8ZoApZqBfdk zqCrQ3dQtzu/gTY2GT9S/sNd9y/W8m1fjKn4JjWyFx676I6ENMxZg1HuCrEnIcSh7VPS FTKuJ46wEKScipSB3HjbjQR5IOxu9ncOM+foqf1LtMQA6B5k1++xqysv6+E0WOVyetoC XfhLvPJ75CrtPzLaOx5wZepis+Nr6Je82V7nLqEHQng7pMWdvJBYZG18iut44DluMGbh UoMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references :user-agent:date:message-id:mime-version; bh=l4D7RGUK4DqePxFLmZN0FFPwnVhpJwr/A0VwH/ncaQQ=; b=FaRsb1EcRLpJ8Jl6DTUc9noYPUh/t9Nv1KHi37Oir0JGGIyJ8s2qC7PvLUFK95OT6L 8NTf5S3W04z/MU0pMV5E8kQucPwne9jnyY0aiqqJWDfsof8TDgEQYsSJ/nQCHEye0ygp boi+5UmFlkiNsEdupe/tSNAWRrxNVyuiB7iRZVoJq22TDx/wcVkSJ7N/SGhAkmo113Ud r/IAQI+L/ceDJPK7vfVMhREDTDuefhGarUZUAjJTsxPte49++Xlnt7o4EpE2Opuid3lv c0pEEoi3S4FI3D/YafB7UU5maNf1cSfsM90rf9rPwNtMIrD/CpF0fpQGIKKPoLYmLdNH 33zg== X-Gm-Message-State: AOAM533KR539QOhLJoL+yJoeC8dRPxi8TyePWMGvwSuxjzmcmAdIdIud L2n4dhSKwZzP7Jc9u1jgRi2eI2M6AlE= X-Google-Smtp-Source: ABdhPJwAzwmWeIprjRAgT9pfH22bjEBnOYFhxDJmqn12UPyzEAbw76jUqSMeuP77zh48+JWyMxkzAQ== X-Received: by 2002:a5d:58d7:: with SMTP id o23mr5660643wrf.288.1609958463447; Wed, 06 Jan 2021 10:41:03 -0800 (PST) Received: from unfall (36.193.158.146.dynamic.jazztel.es. [146.158.193.36]) by smtp.gmail.com with ESMTPSA id t1sm4489160wro.27.2021.01.06.10.41.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 10:41:02 -0800 (PST) From: =?utf-8?Q?Miguel_=C3=81ngel_Arruga_Vivas?= To: Tobias Geerinckx-Rice Subject: Re: bug#45676: Store references inside compressed data In-Reply-To: <871rezf1yg.fsf@nckx> (Tobias Geerinckx-Rice's message of "Tue, 05 Jan 2021 23:33:59 +0100") References: <8735zf30yw.fsf@gmail.com> <871rezf1yg.fsf@nckx> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) Date: Wed, 06 Jan 2021 19:40:55 +0100 Message-ID: <87h7nt29jc.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 45676 Cc: 45676@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Hi! Tobias Geerinckx-Rice writes: > It's true. It's a known trade-off of an otherwise almost-zero-effort > yet fast reference scanner. I don't think it's a bug per se, but it > is something of which to be aware. > > Let's stop installing compressed executables & data files. We already > avoid compressed .jars and other renamed zip files. This is the current trade-off between build time and closure size for executable code, but it isn't the current status regarding data files. > Gzipping a tiny handful of Lisp or Smalltalk files is pointless when > zstd {,de}compresses my entire 500GB SSD better and faster, at the > file system level where it now squarely belongs. Not every system has a file system with compression, nor most of us mortals have a SSD to test that. ;-) > Without breaking Guix. Software bugs are related to the number of lines, and this probably would end up adding more, so I get that idea, hehe. :-P With your proposal closures wouldn't benefit from the "standard tricks" used by package maintainers to reduce their footprint for uncompressed file systems. Having an option to remove that compression seems best for treating it at the file system level---perhaps only some wrappers for the compression tools to use always -0 could do most of the trick---but I'd still like to have the option of paying at build/graft time the storage savings. Of course, this is still only a wish. Happy hacking! Miguel From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 07 06:05:45 2021 Received: (at 45676) by debbugs.gnu.org; 7 Jan 2021 11:05:45 +0000 Received: from localhost ([127.0.0.1]:46644 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxT6u-00074M-Sf for submit@debbugs.gnu.org; Thu, 07 Jan 2021 06:05:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58042) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxT6u-000749-3g for 45676@debbugs.gnu.org; Thu, 07 Jan 2021 06:05:44 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53646) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kxT6o-0001RR-1h; Thu, 07 Jan 2021 06:05:38 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44290 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kxT6h-0001W1-Ov; Thu, 07 Jan 2021 06:05:33 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas Subject: Re: bug#45676: Store references inside compressed data References: <8735zf30yw.fsf@gmail.com> <87eeiymh6h.fsf@gnu.org> <87mtxm0zqw.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 18 =?utf-8?Q?Niv=C3=B4se?= an 229 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 07 Jan 2021 12:05:30 +0100 In-Reply-To: <87mtxm0zqw.fsf@gmail.com> ("Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas"'s message of "Wed, 06 Jan 2021 17:57:43 +0100") Message-ID: <871rexhurp.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45676 Cc: 45676@debbugs.gnu.org, Leo Famulari X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Howdy, Miguel =C3=81ngel Arruga Vivas skribis: > Ludovic Court=C3=A8s writes: > >> Hi, >> >> Leo Famulari skribis: >> >>> On Tue, Jan 05, 2021 at 03:36:07PM +0100, Miguel =C3=81ngel Arruga Viva= s wrote: >>>> There are several binary formats that allow compression of the >>>> executable image, or some of its data, which is decompress at runtime: >>>>=20 >>>> - Kernel images. >>>> - Compressed libraries: e.g. Smalltalk modules. >>>> - Compressed executable or data files: e.g. library.el.gz. >>>>=20 >>>> These aren't taken into account by the grafting process, which may lead >>>> to issues when store paths are located inside that kind of files. >>> >>> If you have specific instances of this type of bug, please report them. >> >> Agreed. The general issue is =E2=80=9Cwell known=E2=80=9D as we say, bu= t what I think >> we need to do is look for specific instances and address them. > > It can be tagged it notabug if you consider so. I've tagged it as > wishlist (I should have been done it before) for that reason (it's "well > known"), but I haven't found any specific instance yet. OTOH, I think > it might be closely related to #33848, as the solution for both issues > could be solved by the extension on the dumpPath code path---or the > Scheme implementation equivalent, as pointed there. Yes, though I=E2=80=99d prefer simple workarounds if possible=E2=80=94after= all, we=E2=80=99ve lived with it since the beginning and there=E2=80=99s only ever been a hand= ful of instances of that problem (one of them was really tricky, see =E2=80=98gcc-strmov-store-file-names.patch=E2=80=99=E2=80=A6). Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 14 16:31:44 2021 Received: (at 45676) by debbugs.gnu.org; 14 Jan 2021 21:31:44 +0000 Received: from localhost ([127.0.0.1]:39465 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0ADY-0007OI-2A for submit@debbugs.gnu.org; Thu, 14 Jan 2021 16:31:44 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51262) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0ADU-0007O4-NN for 45676@debbugs.gnu.org; Thu, 14 Jan 2021 16:31:42 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60256) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0ADN-0004jQ-9T; Thu, 14 Jan 2021 16:31:33 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=52732 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l0ADM-0003nW-1a; Thu, 14 Jan 2021 16:31:32 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Leo Prikler Subject: Re: bug#45676: Store references inside compressed data References: <8735zf30yw.fsf@gmail.com> <871rezf1yg.fsf@nckx> <06ba5c0f24bdcdb706990c9169093aba72463302.camel@student.tugraz.at> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 25 =?utf-8?Q?Niv=C3=B4se?= an 229 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 14 Jan 2021 22:31:30 +0100 In-Reply-To: <06ba5c0f24bdcdb706990c9169093aba72463302.camel@student.tugraz.at> (Leo Prikler's message of "Wed, 06 Jan 2021 09:54:25 +0100") Message-ID: <87turj19zh.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45676 Cc: 45676@debbugs.gnu.org, Tobias Geerinckx-Rice , Miguel =?utf-8?Q?=C3=81ngel?= Arruga Vivas X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Leo, Leo Prikler skribis: > From 57c23bf6ecac79c397cb49ff251176ec3a7b1cf5 Mon Sep 17 00:00:00 2001 > From: Leo Prikler > Date: Wed, 6 Jan 2021 09:24:07 +0100 > Subject: [PATCH] gnu: emacs: Don't install compressed archives. > > See . Perhaps make it a comment next to the option. > * gnu/packages/emacs.scm (emacs)[#:configure-flags]: > Add --without-compress-install. > (emacs-minimal)[#:configure-flags]: Likewise. [...] > + "--without-compress-install" Does that disable .el file compression altogether for Emacs=E2=80=99 own fi= les? If so, isn=E2=80=99t it too much? Do these file currently contain store fi= le names? (I know EMMS .el files for instance are full of store file names, so that one should definitely not be gzipped, but Emacs itself may be fine?) Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 14 17:24:27 2021 Received: (at 45676) by debbugs.gnu.org; 14 Jan 2021 22:24:27 +0000 Received: from localhost ([127.0.0.1]:39544 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0B2Y-0000Ew-Kw for submit@debbugs.gnu.org; Thu, 14 Jan 2021 17:24:26 -0500 Received: from mailrelay.tugraz.at ([129.27.2.202]:30785) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0B2V-0000Em-PL for 45676@debbugs.gnu.org; Thu, 14 Jan 2021 17:24:25 -0500 Received: from nijino.local (217-149-173-242.nat.highway.telekom.at [217.149.173.242]) by mailrelay.tugraz.at (Postfix) with ESMTPSA id 4DGzPb3R5Nz1LLyW; Thu, 14 Jan 2021 23:24:19 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.11.0 mailrelay.tugraz.at 4DGzPb3R5Nz1LLyW DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tugraz.at; s=mailrelay; t=1610663059; bh=LbRBwWvOBFY8qDqCMcvAg9pIFh+ee/JmQdFrgjAA7HY=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=duDbv3NsmcHg/fJorCdllR4nWpHgWbszBTbh4FS0RAFt9ulBAnlvUWuX50dbmAFnB Q7ef2QkTnrqpa0l4TMsJtJ86p3AH2xdfWXqm3Dp24YSpXFZZqUxohTegSMWvK5xRjv aWn982c3awTZEE5SvJFFF0mbfxo67rfSWs5H4VUI= Message-ID: <98965ea00ce05ba435535e4347dfdfc409c10ec2.camel@student.tugraz.at> Subject: Re: bug#45676: Store references inside compressed data From: Leo Prikler To: Ludovic =?ISO-8859-1?Q?Court=E8s?= Date: Thu, 14 Jan 2021 23:24:18 +0100 In-Reply-To: <87turj19zh.fsf@gnu.org> References: <8735zf30yw.fsf@gmail.com> <871rezf1yg.fsf@nckx> <06ba5c0f24bdcdb706990c9169093aba72463302.camel@student.tugraz.at> <87turj19zh.fsf@gnu.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.34.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUG-Backscatter-control: bt4lQm5Tva3SBgCuw0EnZw X-Spam-Scanner: SpamAssassin 3.003001 X-Spam-Score-relay: -1.9 X-Scanned-By: MIMEDefang 2.74 on 129.27.10.117 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45676 Cc: 45676@debbugs.gnu.org, Tobias Geerinckx-Rice , Miguel =?ISO-8859-1?Q?=C1ngel?= Arruga Vivas X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Ludo, Am Donnerstag, den 14.01.2021, 22:31 +0100 schrieb Ludovic Courtès: > Hi Leo, > > Leo Prikler skribis: > > > From 57c23bf6ecac79c397cb49ff251176ec3a7b1cf5 Mon Sep 17 00:00:00 > > 2001 > > From: Leo Prikler > > Date: Wed, 6 Jan 2021 09:24:07 +0100 > > Subject: [PATCH] gnu: emacs: Don't install compressed archives. > > > > See ;. > > Perhaps make it a comment next to the option. I'll keep that in mind, but I wasn't going to commit this unless it is absolutely needed. > > * gnu/packages/emacs.scm (emacs)[#:configure-flags]: > > Add --without-compress-install. > > (emacs-minimal)[#:configure-flags]: Likewise. > > [...] > > > + "--without-compress-install" > > Does that disable .el file compression altogether for Emacs’ own > files? > > If so, isn’t it too much? Do these file currently contain store file > names? > > (I know EMMS .el files for instance are full of store file names, so > that one should definitely not be gzipped, but Emacs itself may be > fine?) As far as I know, this is an all or nothing deal. If I'm not mistaken, however, all those references should still exist in the compiled (and not compressed) .go files however, hence it making little difference. Perhaps time stamps could be added during compression, but I think our Emacs reproducibility issues lie elsewhere as well. All in all, I don't think there's a technical reason to do this (yet), merely the somewhat purist stance of "no compressed source files". Regards, Leo