From unknown Thu Jun 19 14:06:26 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#21942 <21942@debbugs.gnu.org> To: bug#21942 <21942@debbugs.gnu.org> Subject: Status: Files with incorrect file sizes Reply-To: bug#21942 <21942@debbugs.gnu.org> Date: Thu, 19 Jun 2025 21:06:26 +0000 retitle 21942 Files with incorrect file sizes reassign 21942 diffutils submitter 21942 Stephan M=C3=BCller severity 21942 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Tue Nov 17 12:43:14 2015 Received: (at submit) by debbugs.gnu.org; 17 Nov 2015 17:43:14 +0000 Received: from localhost ([127.0.0.1]:41978 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZykHi-0005gq-58 for submit@debbugs.gnu.org; Tue, 17 Nov 2015 12:43:14 -0500 Received: from eggs.gnu.org ([208.118.235.92]:59989) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZyegL-0001Ht-JE for submit@debbugs.gnu.org; Tue, 17 Nov 2015 06:44:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZyegK-0001eN-N7 for submit@debbugs.gnu.org; Tue, 17 Nov 2015 06:44:17 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:48288) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZyegK-0001eJ-K7 for submit@debbugs.gnu.org; Tue, 17 Nov 2015 06:44:16 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41758) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZyegJ-0005ZC-OZ for bug-diffutils@gnu.org; Tue, 17 Nov 2015 06:44:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZyegG-0001do-FL for bug-diffutils@gnu.org; Tue, 17 Nov 2015 06:44:15 -0500 Received: from mail-wm0-x22a.google.com ([2a00:1450:400c:c09::22a]:38482) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZyegG-0001dh-A4 for bug-diffutils@gnu.org; Tue, 17 Nov 2015 06:44:12 -0500 Received: by wmec201 with SMTP id c201so21793933wme.1 for ; Tue, 17 Nov 2015 03:44:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-transfer-encoding; bh=Yy9YiLytiqvZgKpLv3iuxZdPquknRrK2supzDkVAn6o=; b=YOnPgVbX4735BQbFPinlAPXSuXxSVAQifAMAaPCF18+3Q4c7lWuOYZYTcbq2fKM8Za M43asQI1k3V8z/n6cu4xOSwWzEgiK83rWlPERq1H4j5s1K9fvzuxWNDQTRrmrEt6bg7R iFQyejzFalHMpAK610rdF0Ltr7XCHXw7xRfEms618OXhx8wtuePxZ/6piu/noLL94ayl tupiQ18WI/sIFBzh7UlaupMetL+U0g5oBVhbfY4FMvn0py7p6s3NYk9dckN1LDc6T/ny IQGZSOiB+UN9gOnkg0MJ0fnXkV2Y0CBjp5vvgZN8OQiShcEN9QaiBbomP0mMOMEhUEIl Xa7Q== X-Received: by 10.194.249.97 with SMTP id yt1mr41712741wjc.89.1447760651800; Tue, 17 Nov 2015 03:44:11 -0800 (PST) Received: from fuckup.fritz.box (adsl-89-217-233-109.adslplus.ch. [89.217.233.109]) by smtp.gmail.com with ESMTPSA id cv3sm38154574wjc.20.2015.11.17.03.44.11 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Nov 2015 03:44:11 -0800 (PST) Date: Tue, 17 Nov 2015 12:44:08 +0100 From: Stephan =?UTF-8?B?TcO8bGxlcg==?= To: bug-diffutils@gnu.org Subject: Files with incorrect file sizes Message-ID: <20151117124408.27185c58@fuckup.fritz.box> X-Mailer: Claws Mail 3.12.0 (GTK+ 2.24.28; i686-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Tue, 17 Nov 2015 12:43:12 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) Hi there, recently I had to debug weird problem. Finally I figured it out. Virtual file systems like /sys or /proc usually don't care about file sizes. All files have a size of 0. This leads to difficulties as diff sometimes looks for file sizes. Say you do: > $ cp /proc/cmdline my_cmdline > $ diff /proc/cmdline my_cmdline ; echo $? > 0 // ok, files don't differ > $ diff --brief /proc/cmdline my_cmdline > Files /proc/cmdline and mycmdline differ The --brief option triggers a binary compare, as we aren't interested in the actual differences this makes sense. As a first step, file sizes are compared (0 vs ~150) and the files are reported as different. I am not sure how to fix this. I.e. are the files different or not? What is the correct behavior. In any case --brief should have no influence on the overall result! I see the following options (I prefer the 2nd): 1: Don't use file sizes in comparing. This is rather expensive but would certainly fix it. 2: add a --use-file-sizes flag which enables decisions based on file sizes. Without this flag, always look into the files. And --brief should report, if the files differ because of their data or size. That is > $ diff /proc/cmdline my_cmdline ; echo $? > 0 // ok, files don't differ > $ diff --brief /proc/cmdline my_cmdline ; echo $0 > 0 > $ diff --brief --use-file-sizes /proc/cmdline my_cmdline > Files /proc/cmdline and mycmdline differ in file size 3: Leave it at it is and add a warning in the man page. However that would be very inconvenient. best, stephan From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 20 12:29:17 2015 Received: (at 21942) by debbugs.gnu.org; 20 Nov 2015 17:29:17 +0000 Received: from localhost ([127.0.0.1]:46380 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZzpUr-000786-0x for submit@debbugs.gnu.org; Fri, 20 Nov 2015 12:29:17 -0500 Received: from mail-vk0-f54.google.com ([209.85.213.54]:33413) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZzpUX-00077B-Ip for 21942@debbugs.gnu.org; Fri, 20 Nov 2015 12:29:16 -0500 Received: by vkfr145 with SMTP id r145so2042244vkf.0 for <21942@debbugs.gnu.org>; Fri, 20 Nov 2015 09:28:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=IFpDuph2UhOGGSIBiLpRvHIhbXT43vGJ++v9AbFdJmU=; b=wFL7j6hfa31HfWwIVffAHE9pTpyCU1jaMedY4RjSglNZVGAFL5ZqSQ4ntIk4dWNkF6 aaxpqJuUEpCKIFgwe23tKM3lpbkS301MIZvxMOHtHseRWiUHOJIuHfRHbLLBt8+jrkTD lNXqlbYwVxe6JPmLskjScWJoT+jqxmA3GiAxfzNWR+N4BtxnUWxyEDP6aTX2pCJYxIwc AyhMdFWbbEXU++UGvxyX3KLZxBxQOdc5hBqikJvBtdVEZExYSfeQHo882XdBb+Wv6fmQ cmVu18NDUs7qchnUnzmf2bhqvLg3Eb6gU19Y4ts2Y3ICiUGDQrQjQY1G8GAbgetYfA93 C/ZQ== X-Received: by 10.31.41.131 with SMTP id p125mr510374vkp.133.1448040536801; Fri, 20 Nov 2015 09:28:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.31.84.198 with HTTP; Fri, 20 Nov 2015 09:28:37 -0800 (PST) In-Reply-To: <20151117124408.27185c58@fuckup.fritz.box> References: <20151117124408.27185c58@fuckup.fritz.box> From: Jim Meyering Date: Fri, 20 Nov 2015 18:28:37 +0100 X-Google-Sender-Auth: X8A8qxUY-LIK52wClPblla2I1LI Message-ID: Subject: Re: [bug-diffutils] bug#21942: Files with incorrect file sizes To: =?UTF-8?Q?Stephan_M=C3=BCller?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21942 Cc: 21942@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Tue, Nov 17, 2015 at 12:44 PM, Stephan M=C3=BCller = wrote: > recently I had to debug weird problem. Finally I figured it out. > > Virtual file systems like /sys or /proc usually don't care about file > sizes. All files have a size of 0. This leads to difficulties as diff > sometimes looks for file sizes. > > Say you do: > >> $ cp /proc/cmdline my_cmdline >> $ diff /proc/cmdline my_cmdline ; echo $? >> 0 // ok, files don't differ >> $ diff --brief /proc/cmdline my_cmdline >> Files /proc/cmdline and mycmdline differ > > The --brief option triggers a binary compare, as we aren't interested > in the actual differences this makes sense. As a first step, file sizes > are compared (0 vs ~150) and the files are reported as different. thanks for the report. What version of diffutils are you using? I think this has been fixed for some time. I was unable to reproduce with 2.8.1 nor with the latest built from git. I.e., I created an empty file and used diff-2.8.1 to compare it with the nominally- zero-length /proc/cmdline file, and diff did the right thing. Also, I ran stat to show st_size of each file is indeed 0: $ : > /tmp/k; /p/p/diffutils-2.8.1/bin/diff /proc/cmdline /tmp/k; \ stat --format %s /proc/cmdline /tmp/k 1d0 < ro root=3DLABEL=3D... 0 0 In fact, I went ahead and built all available versions and tested them like this: $ for i in /p/p/*/bin/diff; do p=3Ddiffutils-$i; echo $i; $i /proc/cmdline /tmp/k > /dev/null && echo bad; done /p/p/diffutils-2.7/bin/diff /p/p/diffutils-2.8.1/bin/diff /p/p/diffutils-2.8/bin/diff /p/p/diffutils-2.9/bin/diff /p/p/diffutils-3.0/bin/diff /p/p/diffutils-3.1/bin/diff /p/p/diffutils-3.2/bin/diff /p/p/diffutils-3.3/bin/diff From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 20 15:07:05 2015 Received: (at 21942) by debbugs.gnu.org; 20 Nov 2015 20:07:05 +0000 Received: from localhost ([127.0.0.1]:46433 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZzrxY-00034R-I5 for submit@debbugs.gnu.org; Fri, 20 Nov 2015 15:07:04 -0500 Received: from mail-wm0-f48.google.com ([74.125.82.48]:37479) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZzrxV-000343-O3 for 21942@debbugs.gnu.org; Fri, 20 Nov 2015 15:07:02 -0500 Received: by wmww144 with SMTP id w144so34887103wmw.0 for <21942@debbugs.gnu.org>; Fri, 20 Nov 2015 12:07:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=UKvHnkjjJYLc2WNAvECQ9+uus4XndK9TWTmZC436uZU=; b=HSCBKThIfcD9pHn/nejevPDn2VnH3lls6HgGPt3JrgEfV2rrsQboIur8AUENqwyYiN aJHG35NveIURw57ikN1LZuGeftbaRuCc4myRXcl4c/OEXVIW4si2M31jVor5oNgy7JXb xoc8J4JXqH9pDf4OnskUA2xfus1XfY/khm7AKMr94lfBYl0aYJ+hLgkF/KcWsMdVgK9P Lclt+rPIWaGXL6bQuQ+qQdlZ+YQx8fDlbqZgsCaxJ5WGI5i8Nx3QfK55LRTWqpmYcYEn xdLGWr0+AhFISIIjz9Ct7RR2RNFZDtIUYiwjLD7OfvAO3EEnIZ0GOxzDgQj+gix+v93R jFQA== X-Received: by 10.194.117.68 with SMTP id kc4mr4259941wjb.77.1448050020511; Fri, 20 Nov 2015 12:07:00 -0800 (PST) Received: from fuckup.fritz.box (adsl-89-217-233-109.adslplus.ch. [89.217.233.109]) by smtp.gmail.com with ESMTPSA id t133sm930053wmf.24.2015.11.20.12.07.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Nov 2015 12:07:00 -0800 (PST) Date: Fri, 20 Nov 2015 21:06:54 +0100 From: Stephan =?UTF-8?B?TcO8bGxlcg==?= To: Jim Meyering Subject: Re: [bug-diffutils] bug#21942: Files with incorrect file sizes Message-ID: <20151120210654.779970ad@fuckup.fritz.box> In-Reply-To: References: <20151117124408.27185c58@fuckup.fritz.box> X-Mailer: Claws Mail 3.12.0 (GTK+ 2.24.28; i686-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21942 Cc: 21942@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Am Fri, 20 Nov 2015 18:28:37 +0100 schrieb Jim Meyering : > On Tue, Nov 17, 2015 at 12:44 PM, Stephan M=C3=BCller > wrote: > > recently I had to debug weird problem. Finally I figured it out. > > > > Virtual file systems like /sys or /proc usually don't care about > > file sizes. All files have a size of 0. This leads to difficulties > > as diff sometimes looks for file sizes. > > > > Say you do: > > > >> $ cp /proc/cmdline my_cmdline > >> $ diff /proc/cmdline my_cmdline ; echo $? > >> 0 // ok, files don't differ > >> $ diff --brief /proc/cmdline my_cmdline > >> Files /proc/cmdline and mycmdline differ > > > > The --brief option triggers a binary compare, as we aren't > > interested in the actual differences this makes sense. As a first > > step, file sizes are compared (0 vs ~150) and the files are > > reported as different. >=20 > thanks for the report. > What version of diffutils are you using? > I think this has been fixed for some time. > I was unable to reproduce with 2.8.1 nor with the latest built from > git. I.e., I created an empty file and used diff-2.8.1 to compare it > with the nominally- > zero-length /proc/cmdline file, and diff did the right thing. > Also, I ran stat to show st_size of each file is indeed 0: >=20 > $ : > /tmp/k; /p/p/diffutils-2.8.1/bin/diff /proc/cmdline /tmp/k; \ > stat --format %s /proc/cmdline /tmp/k > 1d0 > < ro root=3DLABEL=3D... > 0 > 0 >=20 > In fact, I went ahead and built all available versions and tested them > like this: >=20 > $ for i in /p/p/*/bin/diff; do p=3Ddiffutils-$i; echo $i; $i > /proc/cmdline /tmp/k > /dev/null && echo bad; done > /p/p/diffutils-2.7/bin/diff > /p/p/diffutils-2.8.1/bin/diff > /p/p/diffutils-2.8/bin/diff > /p/p/diffutils-2.9/bin/diff > /p/p/diffutils-3.0/bin/diff > /p/p/diffutils-3.1/bin/diff > /p/p/diffutils-3.2/bin/diff > /p/p/diffutils-3.3/bin/diff Hi, I am using v.3.3 of diffutils $ diff -v=20 diff (GNU diffutils) 3.3 but I think you misunderstood the problem. Sorry for being ambiguous. I am not diffing against an empty file. That works well. The point is procfs doesn't care about size, but 'normal' file systems do. So for example on my system I have (after cp /proc/cmdline mycmdline) $ stat --format %s /proc/cmdline mycmdline 0 140 The result of diffing /proc/cmdline against mycmdline depends on the --brief flag. STEPS TO REPRODUCE: cp /proc/cmdline mycmdline diff --brief /proc/cmdline mycmdline > /dev/null ; echo ?$ 1 diff /proc/cmdline mycmdline ; echo $? 0 EXPECTED RESULT: cp /proc/cmdline mycmdline diff --brief /proc/cmdline mycmdline > /dev/null ; echo ?$ 0 diff /proc/cmdline mycmdline ; echo $? 0 ~stephan From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 20 17:13:04 2015 Received: (at 21942) by debbugs.gnu.org; 20 Nov 2015 22:13:04 +0000 Received: from localhost ([127.0.0.1]:46483 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZztvT-0006Dg-9f for submit@debbugs.gnu.org; Fri, 20 Nov 2015 17:13:03 -0500 Received: from mail-vk0-f50.google.com ([209.85.213.50]:35526) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZztvQ-0006DF-SX for 21942@debbugs.gnu.org; Fri, 20 Nov 2015 17:13:01 -0500 Received: by vkha189 with SMTP id a189so6923960vkh.2 for <21942@debbugs.gnu.org>; Fri, 20 Nov 2015 14:13:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=IiXYBX2ZVNLIOVkP5pq9QYKiG/XIoS6DBIesTyw04mI=; b=IZ30tv0Thqeq1hSvT6oXKOqX2VPrZ+cMo8lPygjvnKHW/QYFJ4gIFH4HhYxTMgyO4J ASkpJq7ANL3TheNCe6A2E1zAeBDtKWT1SEu4FIn0ZQzOWIn+VB+c3NvC6XM9oPiMzpVW irM/BWj1AnE31LwqcPKaiX2XcqZvfSoySxRs/NpPx1RYHN8oT6Dhu6hP8rAKej1Dy7W6 /BhrnuOzd1aH2HfXvM46lGwpuYYCqR8dHZij9h1ScDWbBljySMQuZ0+dOJs8Vun87Rp7 3MIOZkUBMH+9ls1PClK5YGNjolqZ6h+6OAEu++9aqGm1kP653nNHDqG75J/gMZ4dIxHD q00w== X-Received: by 10.31.41.131 with SMTP id p125mr1881566vkp.133.1448057580398; Fri, 20 Nov 2015 14:13:00 -0800 (PST) MIME-Version: 1.0 Received: by 10.31.84.198 with HTTP; Fri, 20 Nov 2015 14:12:40 -0800 (PST) In-Reply-To: <20151120210654.779970ad@fuckup.fritz.box> References: <20151117124408.27185c58@fuckup.fritz.box> <20151120210654.779970ad@fuckup.fritz.box> From: Jim Meyering Date: Fri, 20 Nov 2015 23:12:40 +0100 X-Google-Sender-Auth: 2ZMGQ3vV9iP63--qEgnB5K55kQM Message-ID: Subject: Re: [bug-diffutils] bug#21942: Files with incorrect file sizes To: =?UTF-8?Q?Stephan_M=C3=BCller?= Content-Type: multipart/mixed; boundary=001a113f0e46b1c1300525002d33 X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21942 Cc: 21942@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --001a113f0e46b1c1300525002d33 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Fri, Nov 20, 2015 at 9:06 PM, Stephan M=C3=BCller = wrote: > Am Fri, 20 Nov 2015 18:28:37 +0100 > schrieb Jim Meyering : > >> On Tue, Nov 17, 2015 at 12:44 PM, Stephan M=C3=BCller >> wrote: >> > recently I had to debug weird problem. Finally I figured it out. >> > >> > Virtual file systems like /sys or /proc usually don't care about >> > file sizes. All files have a size of 0. This leads to difficulties >> > as diff sometimes looks for file sizes. >> > >> > Say you do: >> > >> >> $ cp /proc/cmdline my_cmdline >> >> $ diff /proc/cmdline my_cmdline ; echo $? >> >> 0 // ok, files don't differ >> >> $ diff --brief /proc/cmdline my_cmdline >> >> Files /proc/cmdline and mycmdline differ >> > >> > The --brief option triggers a binary compare, as we aren't >> > interested in the actual differences this makes sense. As a first >> > step, file sizes are compared (0 vs ~150) and the files are >> > reported as different. >> >> thanks for the report. >> What version of diffutils are you using? >> I think this has been fixed for some time. >> I was unable to reproduce with 2.8.1 nor with the latest built from >> git. I.e., I created an empty file and used diff-2.8.1 to compare it >> with the nominally- >> zero-length /proc/cmdline file, and diff did the right thing. >> Also, I ran stat to show st_size of each file is indeed 0: >> >> $ : > /tmp/k; /p/p/diffutils-2.8.1/bin/diff /proc/cmdline /tmp/k; \ >> stat --format %s /proc/cmdline /tmp/k >> 1d0 >> < ro root=3DLABEL=3D... >> 0 >> 0 >> >> In fact, I went ahead and built all available versions and tested them >> like this: >> >> $ for i in /p/p/*/bin/diff; do p=3Ddiffutils-$i; echo $i; $i >> /proc/cmdline /tmp/k > /dev/null && echo bad; done >> /p/p/diffutils-2.7/bin/diff >> /p/p/diffutils-2.8.1/bin/diff >> /p/p/diffutils-2.8/bin/diff >> /p/p/diffutils-2.9/bin/diff >> /p/p/diffutils-3.0/bin/diff >> /p/p/diffutils-3.1/bin/diff >> /p/p/diffutils-3.2/bin/diff >> /p/p/diffutils-3.3/bin/diff > > Hi, > > I am using v.3.3 of diffutils > > $ diff -v > diff (GNU diffutils) 3.3 > > but I think you misunderstood the problem. Sorry for being ambiguous. I > am not diffing against an empty file. That works well. The point is > procfs doesn't care about size, but 'normal' file systems do. So for > example on my system I have (after cp /proc/cmdline mycmdline) > > $ stat --format %s /proc/cmdline mycmdline > 0 > 140 > > The result of diffing /proc/cmdline against mycmdline depends on the > --brief flag. > > STEPS TO REPRODUCE: > > cp /proc/cmdline mycmdline > diff --brief /proc/cmdline mycmdline > /dev/null ; echo ?$ > 1 > diff /proc/cmdline mycmdline ; echo $? > 0 > > EXPECTED RESULT: > > cp /proc/cmdline mycmdline > diff --brief /proc/cmdline mycmdline > /dev/null ; echo ?$ > 0 > diff /proc/cmdline mycmdline ; echo $? > 0 Oh, indeed. Thank you for clarifying. That feels like a bug. Here's a knee-jerk patch that refrains from using the st_size-comparing heuristic when either of the sizes is zero. This may well be wrong. I have only barely tested the diff.c code path. --001a113f0e46b1c1300525002d33 Content-Type: text/x-patch; charset=US-ASCII; name="st_size-is-zero.patch" Content-Disposition: attachment; filename="st_size-is-zero.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ih881vmf0 ZGlmZiAtLWdpdCBhL3NyYy9hbmFseXplLmMgYi9zcmMvYW5hbHl6ZS5jCmluZGV4IDg5M2QwN2Mu LjI2MjI4MTAgMTAwNjQ0Ci0tLSBhL3NyYy9hbmFseXplLmMKKysrIGIvc3JjL2FuYWx5emUuYwpA QCAtNDc3LDYgKzQ3Nyw4IEBAIGRpZmZfMl9maWxlcyAoc3RydWN0IGNvbXBhcmlzb24gKmNtcCkK ICAgICB7CiAgICAgICAvKiBGaWxlcyB3aXRoIGRpZmZlcmVudCBsZW5ndGhzIG11c3QgYmUgZGlm ZmVyZW50LiAgKi8KICAgICAgIGlmIChjbXAtPmZpbGVbMF0uc3RhdC5zdF9zaXplICE9IGNtcC0+ ZmlsZVsxXS5zdGF0LnN0X3NpemUKKwkgICYmIDAgPCBjbXAtPmZpbGVbMF0uc3RhdC5zdF9zaXpl CisJICAmJiAwIDwgY21wLT5maWxlWzFdLnN0YXQuc3Rfc2l6ZQogCSAgJiYgKGNtcC0+ZmlsZVsw XS5kZXNjIDwgMCB8fCBTX0lTUkVHIChjbXAtPmZpbGVbMF0uc3RhdC5zdF9tb2RlKSkKIAkgICYm IChjbXAtPmZpbGVbMV0uZGVzYyA8IDAgfHwgU19JU1JFRyAoY21wLT5maWxlWzFdLnN0YXQuc3Rf bW9kZSkpKQogCWNoYW5nZXMgPSAxOwpkaWZmIC0tZ2l0IGEvc3JjL2RpZmYuYyBiL3NyYy9kaWZm LmMKaW5kZXggZWZkN2U0Ny4uYjJhOTBhYSAxMDA2NDQKLS0tIGEvc3JjL2RpZmYuYworKysgYi9z cmMvZGlmZi5jCkBAIC0xMzQ0LDcgKzEzNDQsOSBAQCBjb21wYXJlX2ZpbGVzIChzdHJ1Y3QgY29t cGFyaXNvbiBjb25zdCAqcGFyZW50LAogICBlbHNlIGlmIChmaWxlc19jYW5fYmVfdHJlYXRlZF9h c19iaW5hcnkKIAkgICAmJiBTX0lTUkVHIChjbXAuZmlsZVswXS5zdGF0LnN0X21vZGUpCiAJICAg JiYgU19JU1JFRyAoY21wLmZpbGVbMV0uc3RhdC5zdF9tb2RlKQotCSAgICYmIGNtcC5maWxlWzBd LnN0YXQuc3Rfc2l6ZSAhPSBjbXAuZmlsZVsxXS5zdGF0LnN0X3NpemUpCisJICAgJiYgY21wLmZp bGVbMF0uc3RhdC5zdF9zaXplICE9IGNtcC5maWxlWzFdLnN0YXQuc3Rfc2l6ZQorCSAgICYmIDAg PCBjbXAuZmlsZVswXS5zdGF0LnN0X3NpemUKKwkgICAmJiAwIDwgY21wLmZpbGVbMV0uc3RhdC5z dF9zaXplKQogICAgIHsKICAgICAgIG1lc3NhZ2UgKCJGaWxlcyAlcyBhbmQgJXMgZGlmZmVyXG4i LAogCSAgICAgICBmaWxlX2xhYmVsWzBdID8gZmlsZV9sYWJlbFswXSA6IGNtcC5maWxlWzBdLm5h bWUsCg== --001a113f0e46b1c1300525002d33-- From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 20 17:33:59 2015 Received: (at 21942) by debbugs.gnu.org; 20 Nov 2015 22:33:59 +0000 Received: from localhost ([127.0.0.1]:46502 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZzuFi-0006ld-Jv for submit@debbugs.gnu.org; Fri, 20 Nov 2015 17:33:59 -0500 Received: from mail-wm0-f48.google.com ([74.125.82.48]:35382) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZzuFO-0006kv-MR for 21942@debbugs.gnu.org; Fri, 20 Nov 2015 17:33:57 -0500 Received: by wmdw130 with SMTP id w130so36183095wmd.0 for <21942@debbugs.gnu.org>; Fri, 20 Nov 2015 14:33:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=LDP4p3xE8CjoJsSo/SPWTzaEgcqSHDAxe0WCgESJznM=; b=P/sj35TLCHSNFHd2mAyKj1sqtSBbWWY0duIBr/2vL6olFCN62E2tFn+uxC2XVf0eKb Z/lCNtX223xU14wiTLD7YYFH1w9mpkoS0wsA6ezesqZXd2BQm1+vVhREGudKZwzoSJU6 WbSk6LUKSRhlYd9XGXm691CeQxzPDbbWTtJTwIip+dgfUZ2Pqj4UDDzNSiqeA7m2V9yE dg7sYxGi8ASQOKWHzQdRK/iY5f1/p487j44dEpiNranqtQpcWZpJIeZLp9aOf557y0SL sxxG5Y78AlkqmWUG/FG8q+L5FxvojwDLL/vMkyKsdSoKgV+zzrpWvrGq/5rTDFXGR9d6 IpVw== X-Received: by 10.194.116.100 with SMTP id jv4mr16195570wjb.67.1448058818116; Fri, 20 Nov 2015 14:33:38 -0800 (PST) Received: from fuckup.fritz.box (adsl-89-217-233-109.adslplus.ch. [89.217.233.109]) by smtp.gmail.com with ESMTPSA id gl10sm1423134wjb.30.2015.11.20.14.33.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Nov 2015 14:33:37 -0800 (PST) Date: Fri, 20 Nov 2015 23:33:34 +0100 From: Stephan =?UTF-8?B?TcO8bGxlcg==?= To: Jim Meyering Subject: Re: [bug-diffutils] bug#21942: Files with incorrect file sizes Message-ID: <20151120233334.7e93f488@fuckup.fritz.box> In-Reply-To: References: <20151117124408.27185c58@fuckup.fritz.box> <20151120210654.779970ad@fuckup.fritz.box> X-Mailer: Claws Mail 3.12.0 (GTK+ 2.24.28; i686-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21942 Cc: 21942@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Am Fri, 20 Nov 2015 23:12:40 +0100 schrieb Jim Meyering : > On Fri, Nov 20, 2015 at 9:06 PM, Stephan M=C3=BCller > wrote: > > Am Fri, 20 Nov 2015 18:28:37 +0100 > > schrieb Jim Meyering : > > > >> On Tue, Nov 17, 2015 at 12:44 PM, Stephan M=C3=BCller > >> wrote: > >> > recently I had to debug weird problem. Finally I figured it out. > >> > > >> > Virtual file systems like /sys or /proc usually don't care about > >> > file sizes. All files have a size of 0. This leads to > >> > difficulties as diff sometimes looks for file sizes. > >> > > >> > Say you do: > >> > > >> >> $ cp /proc/cmdline my_cmdline > >> >> $ diff /proc/cmdline my_cmdline ; echo $? > >> >> 0 // ok, files don't differ > >> >> $ diff --brief /proc/cmdline my_cmdline > >> >> Files /proc/cmdline and mycmdline differ > >> > > >> > The --brief option triggers a binary compare, as we aren't > >> > interested in the actual differences this makes sense. As a first > >> > step, file sizes are compared (0 vs ~150) and the files are > >> > reported as different. > >> > >> thanks for the report. > >> What version of diffutils are you using? > >> I think this has been fixed for some time. > >> I was unable to reproduce with 2.8.1 nor with the latest built from > >> git. I.e., I created an empty file and used diff-2.8.1 to compare > >> it with the nominally- > >> zero-length /proc/cmdline file, and diff did the right thing. > >> Also, I ran stat to show st_size of each file is indeed 0: > >> > >> $ : > >> > /tmp/k; /p/p/diffutils-2.8.1/bin/diff /proc/cmdline /tmp/k; \ > >> > stat --format %s /proc/cmdline /tmp/k > >> 1d0 > >> < ro root=3DLABEL=3D... > >> 0 > >> 0 > >> > >> In fact, I went ahead and built all available versions and tested > >> them like this: > >> > >> $ for i in /p/p/*/bin/diff; do p=3Ddiffutils-$i; echo $i; $i > >> /proc/cmdline /tmp/k > /dev/null && echo bad; done > >> /p/p/diffutils-2.7/bin/diff > >> /p/p/diffutils-2.8.1/bin/diff > >> /p/p/diffutils-2.8/bin/diff > >> /p/p/diffutils-2.9/bin/diff > >> /p/p/diffutils-3.0/bin/diff > >> /p/p/diffutils-3.1/bin/diff > >> /p/p/diffutils-3.2/bin/diff > >> /p/p/diffutils-3.3/bin/diff > > > > Hi, > > > > I am using v.3.3 of diffutils > > > > $ diff -v > > diff (GNU diffutils) 3.3 > > > > but I think you misunderstood the problem. Sorry for being > > ambiguous. I am not diffing against an empty file. That works well. > > The point is procfs doesn't care about size, but 'normal' file > > systems do. So for example on my system I have (after > > cp /proc/cmdline mycmdline) > > > > $ stat --format %s /proc/cmdline mycmdline > > 0 > > 140 > > > > The result of diffing /proc/cmdline against mycmdline depends on the > > --brief flag. > > > > STEPS TO REPRODUCE: > > > > cp /proc/cmdline mycmdline > > diff --brief /proc/cmdline mycmdline > /dev/null ; echo ?$ > > 1 > > diff /proc/cmdline mycmdline ; echo $? > > 0 > > > > EXPECTED RESULT: > > > > cp /proc/cmdline mycmdline > > diff --brief /proc/cmdline mycmdline > /dev/null ; echo ?$ > > 0 > > diff /proc/cmdline mycmdline ; echo $? > > 0 >=20 > Oh, indeed. Thank you for clarifying. That feels like a bug. > Here's a knee-jerk patch that refrains from using the > st_size-comparing heuristic when either of the sizes is zero. This > may well be wrong. I have only barely tested the diff.c code path. Thanks, that makes the problem at least (even) less unlikely. But if we cant trust file sizes we're doomed. What do you think about a flag controlling comparison by size and a notice if files differ by size. I can craft a patch for this. ~s From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 20 17:40:03 2015 Received: (at 21942) by debbugs.gnu.org; 20 Nov 2015 22:40:03 +0000 Received: from localhost ([127.0.0.1]:46507 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZzuLa-0006v0-LM for submit@debbugs.gnu.org; Fri, 20 Nov 2015 17:40:03 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:59742) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZzuLY-0006uS-Bu for 21942@debbugs.gnu.org; Fri, 20 Nov 2015 17:40:00 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 1128E160D72; Fri, 20 Nov 2015 14:39:58 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id AOY9-Ev6YB9q; Fri, 20 Nov 2015 14:39:57 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 69AF1160DFD; Fri, 20 Nov 2015 14:39:57 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id RmvhN21ora9Y; Fri, 20 Nov 2015 14:39:57 -0800 (PST) Received: from [192.168.1.9] (pool-100-32-155-148.lsanca.fios.verizon.net [100.32.155.148]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 4A666160D72; Fri, 20 Nov 2015 14:39:57 -0800 (PST) Subject: Re: [bug-diffutils] bug#21942: bug#21942: Files with incorrect file sizes To: =?UTF-8?Q?Stephan_M=c3=bcller?= , Jim Meyering References: <20151117124408.27185c58@fuckup.fritz.box> <20151120210654.779970ad@fuckup.fritz.box> <20151120233334.7e93f488@fuckup.fritz.box> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <564FA13D.9030401@cs.ucla.edu> Date: Fri, 20 Nov 2015 14:39:57 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151120233334.7e93f488@fuckup.fritz.box> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.6 (/) X-Debbugs-Envelope-To: 21942 Cc: 21942@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.6 (/) Stephan M=C3=BCller wrote: > What do you think about a flag I'd rather avoid a flag. User-visible complexity has a cost, and we're be= tter=20 off keeping things simple. Is there some easy way to tell whether we're on a file system like procfs= that=20 has the bug? Perhaps we can have a list of known good file systems. But= this=20 is lower priority than getting the bug fixed. From debbugs-submit-bounces@debbugs.gnu.org Sat Nov 21 06:33:37 2015 Received: (at 21942) by debbugs.gnu.org; 21 Nov 2015 11:33:37 +0000 Received: from localhost ([127.0.0.1]:46650 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a06QD-0001kU-A0 for submit@debbugs.gnu.org; Sat, 21 Nov 2015 06:33:37 -0500 Received: from mail-wm0-f51.google.com ([74.125.82.51]:33028) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a06Ps-0001k3-Sb for 21942@debbugs.gnu.org; Sat, 21 Nov 2015 06:33:35 -0500 Received: by wmec201 with SMTP id c201so103849140wme.0 for <21942@debbugs.gnu.org>; Sat, 21 Nov 2015 03:33:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=pKqcohp5Tg9lvGoOVn52623eK0/V3r8hvzB1yOfheco=; b=TzKToOKvpr0MaxJU9wLGHgakcs2DrR7kA41soa9LEKID2PD6u2ZYXMOq/CiVJIfGJ+ npfB/mkZz0hfilVX6Ipm+2BMFdWSBAjZEpkN/JSFPC+7+n2LgqG+vnseeSsAgkumLdVv Iml6EoVOIaWGISZ41XrTAdBn+73kkxje2bCkpSTCPfMAwEYViwge+h8ykfHVvkbD+lFB pw8v3hnDcUKFdc0EbSklVcd5JD2akRxTuNADhsT69Fjsp0e2u6jFfd9Y/iaSOoBRpE7l 8qU1rkpr4w97WHkRTt1AahroeRZ7JZKoTdwNvQtZo7+ucgoLxMn9uqY6NM9CTeFxDGXN pxkQ== X-Received: by 10.28.130.7 with SMTP id e7mr5242326wmd.68.1448105596176; Sat, 21 Nov 2015 03:33:16 -0800 (PST) Received: from fuckup.fritz.box (adsl-89-217-233-109.adslplus.ch. [89.217.233.109]) by smtp.gmail.com with ESMTPSA id m11sm3734968wma.5.2015.11.21.03.33.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Nov 2015 03:33:16 -0800 (PST) Date: Sat, 21 Nov 2015 12:33:12 +0100 From: Stephan =?UTF-8?B?TcO8bGxlcg==?= To: Paul Eggert Subject: Re: [bug-diffutils] bug#21942: bug#21942: Files with incorrect file sizes Message-ID: <20151121123312.7af8070f@fuckup.fritz.box> In-Reply-To: <564FA13D.9030401@cs.ucla.edu> References: <20151117124408.27185c58@fuckup.fritz.box> <20151120210654.779970ad@fuckup.fritz.box> <20151120233334.7e93f488@fuckup.fritz.box> <564FA13D.9030401@cs.ucla.edu> X-Mailer: Claws Mail 3.12.0 (GTK+ 2.24.28; i686-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21942 Cc: Jim Meyering , 21942@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Am Fri, 20 Nov 2015 14:39:57 -0800 schrieb Paul Eggert : > I'd rather avoid a flag. User-visible complexity has a cost, and > we're better off keeping things simple. > > Is there some easy way to tell whether we're on a file system like > procfs that has the bug? Perhaps we can have a list of known good > file systems. But this is lower priority than getting the bug fixed. Ok, I agree. A flag is not optimal. I did some reading but I think you cant tell if a file system has that issue. I slightly more elaborate approach than just checking for file size 0 (as Jim did) would be to actually try to read such a file and see if you can get anything from it. If yes, than diff by looking into the content. However, still procfs or worse debugfs could report anything as size. The only proper fix would be to ignore file sizes entirely. ~stephan From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 06 14:27:45 2015 Received: (at 21942) by debbugs.gnu.org; 6 Dec 2015 19:27:45 +0000 Received: from localhost ([127.0.0.1]:40953 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a5eyG-0007Rz-IG for submit@debbugs.gnu.org; Sun, 06 Dec 2015 14:27:45 -0500 Received: from mail-vk0-f41.google.com ([209.85.213.41]:36615) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a5eyD-0007Rq-4I for 21942@debbugs.gnu.org; Sun, 06 Dec 2015 14:27:42 -0500 Received: by vkay187 with SMTP id y187so91385522vka.3 for <21942@debbugs.gnu.org>; Sun, 06 Dec 2015 11:27:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=0ilBQn5yz5ibN70q8uS8/CftMBu1r/73mE2fmXkrH7M=; b=PLa7j06R26EXCjdoguK3VH22eR2J/gcddldqBRy7ey2cMHTXFipIUgUW7yA4zmA1CQ h2yii6zuoCV8XpzD+MuJxT+wZEKygtN3YlU4q/I/DYi2DI0FbBxr118+0nKISSI7GODL gv1EtDJcDLf50+S2FCPfM3ak3z51ZfYb6FfmvAk74KT+Vhy3VXQEtXkFCdrMHPqZcvVz RaSkgWggc4eJWomY77KMJ2Mdjr5c8s257lzv4lOehNDxRfqctVI084XUJisIsEo4nPHp 3j9pzUnqOkMqV4K7TODoZvltffwQLWFTZgn6KjyyX2/Kk9Vg3NNuNY3CP7aXxlMt7BcD w3/A== X-Received: by 10.31.170.199 with SMTP id t190mr18229023vke.66.1449430060564; Sun, 06 Dec 2015 11:27:40 -0800 (PST) MIME-Version: 1.0 Received: by 10.31.84.198 with HTTP; Sun, 6 Dec 2015 11:27:20 -0800 (PST) In-Reply-To: <20151120233334.7e93f488@fuckup.fritz.box> References: <20151117124408.27185c58@fuckup.fritz.box> <20151120210654.779970ad@fuckup.fritz.box> <20151120233334.7e93f488@fuckup.fritz.box> From: Jim Meyering Date: Sun, 6 Dec 2015 11:27:20 -0800 X-Google-Sender-Auth: 2IGPrSbTf1kCyp7QOJyVXWTGZo0 Message-ID: Subject: Re: [bug-diffutils] bug#21942: Files with incorrect file sizes To: =?UTF-8?Q?Stephan_M=C3=BCller?= Content-Type: multipart/mixed; boundary=001a11431b0ce31a3105263fbbef X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21942 Cc: 21942@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --001a11431b0ce31a3105263fbbef Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Fri, Nov 20, 2015 at 2:33 PM, Stephan M=C3=BCller = wrote: > Am Fri, 20 Nov 2015 23:12:40 +0100 > schrieb Jim Meyering : > >> On Fri, Nov 20, 2015 at 9:06 PM, Stephan M=C3=BCller >> wrote: >> > Am Fri, 20 Nov 2015 18:28:37 +0100 >> > schrieb Jim Meyering : >> > >> >> On Tue, Nov 17, 2015 at 12:44 PM, Stephan M=C3=BCller >> >> wrote: >> >> > recently I had to debug weird problem. Finally I figured it out. >> >> > >> >> > Virtual file systems like /sys or /proc usually don't care about >> >> > file sizes. All files have a size of 0. This leads to >> >> > difficulties as diff sometimes looks for file sizes. >> >> > >> >> > Say you do: >> >> > >> >> >> $ cp /proc/cmdline my_cmdline >> >> >> $ diff /proc/cmdline my_cmdline ; echo $? >> >> >> 0 // ok, files don't differ >> >> >> $ diff --brief /proc/cmdline my_cmdline >> >> >> Files /proc/cmdline and mycmdline differ >> >> > >> >> > The --brief option triggers a binary compare, as we aren't >> >> > interested in the actual differences this makes sense. As a first >> >> > step, file sizes are compared (0 vs ~150) and the files are >> >> > reported as different. >> >> >> >> thanks for the report. >> >> What version of diffutils are you using? >> >> I think this has been fixed for some time. >> >> I was unable to reproduce with 2.8.1 nor with the latest built from >> >> git. I.e., I created an empty file and used diff-2.8.1 to compare >> >> it with the nominally- >> >> zero-length /proc/cmdline file, and diff did the right thing. >> >> Also, I ran stat to show st_size of each file is indeed 0: >> >> >> >> $ : >> >> > /tmp/k; /p/p/diffutils-2.8.1/bin/diff /proc/cmdline /tmp/k; \ >> >> > stat --format %s /proc/cmdline /tmp/k >> >> 1d0 >> >> < ro root=3DLABEL=3D... >> >> 0 >> >> 0 >> >> >> >> In fact, I went ahead and built all available versions and tested >> >> them like this: >> >> >> >> $ for i in /p/p/*/bin/diff; do p=3Ddiffutils-$i; echo $i; $i >> >> /proc/cmdline /tmp/k > /dev/null && echo bad; done >> >> /p/p/diffutils-2.7/bin/diff >> >> /p/p/diffutils-2.8.1/bin/diff >> >> /p/p/diffutils-2.8/bin/diff >> >> /p/p/diffutils-2.9/bin/diff >> >> /p/p/diffutils-3.0/bin/diff >> >> /p/p/diffutils-3.1/bin/diff >> >> /p/p/diffutils-3.2/bin/diff >> >> /p/p/diffutils-3.3/bin/diff >> > >> > Hi, >> > >> > I am using v.3.3 of diffutils >> > >> > $ diff -v >> > diff (GNU diffutils) 3.3 >> > >> > but I think you misunderstood the problem. Sorry for being >> > ambiguous. I am not diffing against an empty file. That works well. >> > The point is procfs doesn't care about size, but 'normal' file >> > systems do. So for example on my system I have (after >> > cp /proc/cmdline mycmdline) >> > >> > $ stat --format %s /proc/cmdline mycmdline >> > 0 >> > 140 >> > >> > The result of diffing /proc/cmdline against mycmdline depends on the >> > --brief flag. >> > >> > STEPS TO REPRODUCE: >> > >> > cp /proc/cmdline mycmdline >> > diff --brief /proc/cmdline mycmdline > /dev/null ; echo ?$ >> > 1 >> > diff /proc/cmdline mycmdline ; echo $? >> > 0 >> > >> > EXPECTED RESULT: >> > >> > cp /proc/cmdline mycmdline >> > diff --brief /proc/cmdline mycmdline > /dev/null ; echo ?$ >> > 0 >> > diff /proc/cmdline mycmdline ; echo $? >> > 0 >> >> Oh, indeed. Thank you for clarifying. That feels like a bug. >> Here's a knee-jerk patch that refrains from using the >> st_size-comparing heuristic when either of the sizes is zero. This >> may well be wrong. I have only barely tested the diff.c code path. > > Thanks, that makes the problem at least (even) less unlikely. But if we > cant trust file sizes we're doomed. What do you think about a flag > controlling comparison by size and a notice if files differ by size. > > I can craft a patch for this. Thank you, but I don't want to have to specify some new option to avoid this misbehavior, so will push the attached patch shortly. If someone finds a system for which a falsely reported stat.st_size is nonzero, we can revisit this. --001a11431b0ce31a3105263fbbef Content-Type: text/x-patch; charset=UTF-8; name="0001-diff-brief-no-longer-mistakenly-reports-diff.-with-0.patch" Content-Disposition: attachment; filename="0001-diff-brief-no-longer-mistakenly-reports-diff.-with-0.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ihux6nut0 RnJvbSAxMjA5ZDYxYWMwYzk0MGJkZDJkZDExNDQ4Y2M5ZTFiM2U5YjU5YTI5IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBKaW0gTWV5ZXJpbmcgPG1leWVyaW5nQGZiLmNvbT4KRGF0ZTog U2F0LCAyOCBOb3YgMjAxNSAxODowMjowNSAtMDgwMApTdWJqZWN0OiBbUEFUQ0hdIGRpZmYgLS1i cmllZiBubyBsb25nZXIgbWlzdGFrZW5seSByZXBvcnRzIGRpZmYuIHdpdGggMC1zaXplZAogL3By b2MvIGZpbGVzCk1JTUUtVmVyc2lvbjogMS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hh cnNldD1VVEYtOApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0CgoqIHNyYy9hbmFseXpl LmMgKGRpZmZfMl9maWxlcyk6IERvbid0IGRlY2xhcmUgZGlmZmVyZW5jZSB3aGVuIGFueQpzdF9z aXplIGlzIDAuCiogc3JjL2RpZmYuYyAoY29tcGFyZV9maWxlcyk6IExpa2V3aXNlLgoqIHRlc3Rz L2JyaWVmLXZzLXByb2Mtc3RhdC16ZXJvOiBOZXcgdGVzdC4KKiB0ZXN0cy9NYWtlZmlsZS5hbTog QWRkIGl0LgoqIE5FV1MgKEJ1ZyBmaXhlcyk6IERlc2NyaWJlIGl0LgpSZXBvcnRlZCBieSBTdGVw aGFuIE3DvGxsZXIgaW4gaHR0cDovL2RlYmJ1Z3MuZ251Lm9yZy8yMTk0MgotLS0KIE5FV1MgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICA4ICsrKysrKysrCiBzcmMvYW5hbHl6ZS5j ICAgICAgICAgICAgICAgICAgICAgICAgfCAgMiArKwogc3JjL2RpZmYuYyAgICAgICAgICAgICAg ICAgICAgICAgICAgIHwgIDQgKysrLQogdGVzdHMvTWFrZWZpbGUuYW0gICAgICAgICAgICAgICAg ICAgIHwgIDEgKwogdGVzdHMvYnJpZWYtdnMtc3RhdC16ZXJvLWtlcm5lbC1saWVzIHwgMzAgKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrCiA1IGZpbGVzIGNoYW5nZWQsIDQ0IGluc2VydGlv bnMoKyksIDEgZGVsZXRpb24oLSkKIGNyZWF0ZSBtb2RlIDEwMDc1NSB0ZXN0cy9icmllZi12cy1z dGF0LXplcm8ta2VybmVsLWxpZXMKCmRpZmYgLS1naXQgYS9ORVdTIGIvTkVXUwppbmRleCAwODhm MTNiLi4xNzI0Yzc0IDEwMDY0NAotLS0gYS9ORVdTCisrKyBiL05FV1MKQEAgLTIzLDYgKzIzLDE0 IEBAIEdOVSBkaWZmdXRpbHMgTkVXUyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IC0qLSBvdXRsaW5lIC0qLQogICBkaWZmIC1CIG5vIGxvbmdlciBnZW5lcmF0ZXMgaW5jb3JyZWN0 IG91dHB1dCBpZiB0aGUgdHdvIGlucHV0cwogICBlYWNoIGVuZCB3aXRoIGEgb25lLWJ5dGUgaW5j b21wbGV0ZSBsaW5lLgoKKyAgZGlmZiAtLWJyaWVmIG5vIGxvbmdlciByZXBvcnRzIGEgZGlmZmVy ZW5jZSBmb3IgdW51c3VhbCBpZGVudGljYWwgZmlsZXMuCisgIEZvciBleGFtcGxlLCB3aGVuIGNv bXBhcmluZyBhIGZpbGUgbGlrZSAvcHJvYy9jbWRsaW5lIChmb3Igd2hpY2ggdGhlIGxpbnV4Cisg IGtlcm5lbCByZXBvcnRzIHN0X3NpemUgb2YgMCBldmVuIHRob3VnaCBpdCBpcyBub3QgYW4gZW1w dHkgZmlsZSkgdG8gYQorICBjb3B5IG9mIHRoYXQgZmlsZSdzIGNvbnRlbnRzIHJlc2lkaW5nIG9u IGEgIm5vcm1hbCIgZmlsZSBzeXN0ZW06CisgICAgJCBmPS9wcm9jL2NtZGxpbmU7IGNwICRmIGs7 IGRpZmYgLS1icmllZiAkZiBrCisgICAgRmlsZXMgL3Byb2MvY21kbGluZSBhbmQgayBkaWZmZXIK KworCiAqKiBQZXJmb3JtYW5jZSBjaGFuZ2VzCgogICBkaWZmJ3MgZGVmYXVsdCBhbGdvcml0aG0g aGFzIGJlZW4gYWRqdXN0ZWQgdG8gb3V0cHV0IGhpZ2hlci1xdWFsaXR5CmRpZmYgLS1naXQgYS9z cmMvYW5hbHl6ZS5jIGIvc3JjL2FuYWx5emUuYwppbmRleCA4OTNkMDdjLi4yNjIyODEwIDEwMDY0 NAotLS0gYS9zcmMvYW5hbHl6ZS5jCisrKyBiL3NyYy9hbmFseXplLmMKQEAgLTQ3Nyw2ICs0Nzcs OCBAQCBkaWZmXzJfZmlsZXMgKHN0cnVjdCBjb21wYXJpc29uICpjbXApCiAgICAgewogICAgICAg LyogRmlsZXMgd2l0aCBkaWZmZXJlbnQgbGVuZ3RocyBtdXN0IGJlIGRpZmZlcmVudC4gICovCiAg ICAgICBpZiAoY21wLT5maWxlWzBdLnN0YXQuc3Rfc2l6ZSAhPSBjbXAtPmZpbGVbMV0uc3RhdC5z dF9zaXplCisJICAmJiAwIDwgY21wLT5maWxlWzBdLnN0YXQuc3Rfc2l6ZQorCSAgJiYgMCA8IGNt cC0+ZmlsZVsxXS5zdGF0LnN0X3NpemUKIAkgICYmIChjbXAtPmZpbGVbMF0uZGVzYyA8IDAgfHwg U19JU1JFRyAoY21wLT5maWxlWzBdLnN0YXQuc3RfbW9kZSkpCiAJICAmJiAoY21wLT5maWxlWzFd LmRlc2MgPCAwIHx8IFNfSVNSRUcgKGNtcC0+ZmlsZVsxXS5zdGF0LnN0X21vZGUpKSkKIAljaGFu Z2VzID0gMTsKZGlmZiAtLWdpdCBhL3NyYy9kaWZmLmMgYi9zcmMvZGlmZi5jCmluZGV4IDQ2YWM5 OWQuLmNmZjIzZjAgMTAwNjQ0Ci0tLSBhL3NyYy9kaWZmLmMKKysrIGIvc3JjL2RpZmYuYwpAQCAt MTM3Nyw3ICsxMzc3LDkgQEAgY29tcGFyZV9maWxlcyAoc3RydWN0IGNvbXBhcmlzb24gY29uc3Qg KnBhcmVudCwKICAgZWxzZSBpZiAoZmlsZXNfY2FuX2JlX3RyZWF0ZWRfYXNfYmluYXJ5CiAJICAg JiYgU19JU1JFRyAoY21wLmZpbGVbMF0uc3RhdC5zdF9tb2RlKQogCSAgICYmIFNfSVNSRUcgKGNt cC5maWxlWzFdLnN0YXQuc3RfbW9kZSkKLQkgICAmJiBjbXAuZmlsZVswXS5zdGF0LnN0X3NpemUg IT0gY21wLmZpbGVbMV0uc3RhdC5zdF9zaXplKQorCSAgICYmIGNtcC5maWxlWzBdLnN0YXQuc3Rf c2l6ZSAhPSBjbXAuZmlsZVsxXS5zdGF0LnN0X3NpemUKKwkgICAmJiAwIDwgY21wLmZpbGVbMF0u c3RhdC5zdF9zaXplCisJICAgJiYgMCA8IGNtcC5maWxlWzFdLnN0YXQuc3Rfc2l6ZSkKICAgICB7 CiAgICAgICBtZXNzYWdlICgiRmlsZXMgJXMgYW5kICVzIGRpZmZlclxuIiwKIAkgICAgICAgZmls ZV9sYWJlbFswXSA/IGZpbGVfbGFiZWxbMF0gOiBjbXAuZmlsZVswXS5uYW1lLApkaWZmIC0tZ2l0 IGEvdGVzdHMvTWFrZWZpbGUuYW0gYi90ZXN0cy9NYWtlZmlsZS5hbQppbmRleCA4MDVjY2MyLi5i MDcwYTc2IDEwMDY0NAotLS0gYS90ZXN0cy9NYWtlZmlsZS5hbQorKysgYi90ZXN0cy9NYWtlZmls ZS5hbQpAQCAtNCw2ICs0LDcgQEAgVEVTVFMgPSBcCiAgIGJhc2ljIFwKICAgYmlnbnVtIFwKICAg YmluYXJ5IFwKKyAgYnJpZWYtdnMtc3RhdC16ZXJvLWtlcm5lbC1saWVzIFwKICAgY29sbGlkaW5n LWZpbGUtbmFtZXMgXAogICBleGNlc3Mtc2xhc2ggXAogICBoZWxwLXZlcnNpb24JXApkaWZmIC0t Z2l0IGEvdGVzdHMvYnJpZWYtdnMtc3RhdC16ZXJvLWtlcm5lbC1saWVzIGIvdGVzdHMvYnJpZWYt dnMtc3RhdC16ZXJvLWtlcm5lbC1saWVzCm5ldyBmaWxlIG1vZGUgMTAwNzU1CmluZGV4IDAwMDAw MDAuLjliMjcyYzYKLS0tIC9kZXYvbnVsbAorKysgYi90ZXN0cy9icmllZi12cy1zdGF0LXplcm8t a2VybmVsLWxpZXMKQEAgLTAsMCArMSwzMCBAQAorIyEvYmluL3NoCisjIEJlZm9yZSBkaWZmLTMu NCwgZGlmZiAtLWJyaWVmIGNvdWxkIG1pc3Rha2VubHkgZGVjbGFyZSBhIGRpZmZlcmVuY2UuCisj IEZvciBleGFtcGxlLCB3aGVuIGNvbXBhcmluZyBhIGZpbGUgbGlrZSAvcHJvYy9jbWRsaW5lIChm b3Igd2hpY2ggdGhlIGxpbnV4CisjIGtlcm5lbCByZXBvcnRzIGEgc3Rfc2l6ZSBvZiAwIGV2ZW4g dGhvdWdoIGl0IGlzIG5vdCBhbiBlbXB0eSBmaWxlKSB0byBhCisjIGNvcHkgb2YgdGhhdCBmaWxl J3MgY29udGVudHMgcmVzaWRpbmcgb24gYSAibm9ybWFsIiBmaWxlIHN5c3RlbS4KKworLiAiJHtz cmNkaXI9Ln0vaW5pdC5zaCI7IHBhdGhfcHJlcGVuZF8gLi4vc3JjCisKK2ZhaWw9MAorCisjIFNr aXAgdGhlIHRlc3QgdW5sZXNzIHdlIGhhdmUgYW4gYXBwcm9wcmlhdGUgZmlsZS4KK2Jvb3Q9L3By b2MvY21kbGluZQordGVzdCAtZiAkYm9vdCB8fCBza2lwXyBubyAkYm9vdCBmaWxlCitzej0kKHN0 YXQgLS1mb3JtYXQgJXMgJGJvb3QpIHx8IHNraXBfIHN0YXQgLS1mb3JtYXQgJXMgZG9lcyBub3Qg d29yawordGVzdCAkc3ogPSAwIHx8IHNraXBfICRib290IGhhcyBub256ZXJvIHNpemUKKworIyBU aGVyZSBhcmUgdHdvIGNvZGUgcGF0aHMgdG8gdGVzdDogb25lIGZvciBub24tYmluYXJ5IGFuZCBv bmUgZm9yIGJpbmFyeSBmaWxlcy4KKyMgJGJvb3QgaXMgbm9uLWJpbmFyeS4KK2NhdCAkYm9vdCA+ IHJlZiB8fCBmcmFtZXdvcmtfZmFpbHVyZV8KK2RpZmYgLS1icmllZiAkYm9vdCByZWYgPiBvdXQg Mj4mMSB8fCBmYWlsPTEKK2NvbXBhcmUgL2Rldi9udWxsIG91dCB8fCBmYWlsPTEKKworIyAvcHJv Yy9zZWxmL2NtZGxpbmUgaXMgYSBOVUwtdGVybWluYXRlZCBsaXN0IG9mIGFyZ3YgdmFsdWVzLAor IyBzbyBjb25zdHJ1Y3QgdGhlIGV4cGVjdGVkIG91dHB1dCBoZXJlOgorcHJpbnRmICdkaWZmXDAt LWJyaWVmXDAvcHJvYy9zZWxmL2NtZGxpbmVcMGJpblwwJyA+IGJpbiB8fCBmcmFtZXdvcmtfZmFp bHVyZV8KKyMgQW5kIHJ1biB0aGUgY29tbWFuZCB0aGF0IGlzIGVtYmVkZGVkIGluIHRoYXQgb3V0 cHV0OgorZGlmZiAtLWJyaWVmIC9wcm9jL3NlbGYvY21kbGluZSBiaW4gPiBvdXQgMj4mMSB8fCBm YWlsPTEKK2NvbXBhcmUgL2Rldi9udWxsIG91dCB8fCBmYWlsPTEKKworRXhpdCAkZmFpbAotLSAK Mi42LjIKCg== --001a11431b0ce31a3105263fbbef-- From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 06 14:40:25 2015 Received: (at 21942-done) by debbugs.gnu.org; 6 Dec 2015 19:40:25 +0000 Received: from localhost ([127.0.0.1]:40957 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a5fAW-0007lM-JH for submit@debbugs.gnu.org; Sun, 06 Dec 2015 14:40:24 -0500 Received: from mail-vk0-f53.google.com ([209.85.213.53]:35063) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a5fAC-0007kt-EB for 21942-done@debbugs.gnu.org; Sun, 06 Dec 2015 14:40:23 -0500 Received: by vkha189 with SMTP id a189so91412400vkh.2 for <21942-done@debbugs.gnu.org>; Sun, 06 Dec 2015 11:40:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=lQtJkECtUzSl9APF25s5/z2pUv9ykNhOdmYf4uZ8qv4=; b=MxfXPkI1qFh6sJzs+7qGl+F+m7z/e8SBHeVjPjqR7sodahWRG0dJIzVFF/HoDetoPW Cpc3VZPM50QrbBdirUQlLn2163stJSCID1ecx3N8f3QgI1VCwvXEEoB3nwYSYhb2aHDa x7JhIuhvm9G6YqxXx5pXi9unIBSagzhbvE6f2weWyn+v3WHrXb2ZHZbqCB9xTh8+DDuE fAgLis0H+Wg2yKefKU+65gN0+gkUFa9juiQxVdCi1lPF3N8jqQG/nnYGVDelLOEX4HwY 7WH6K9fSaL5B0lwxcFxHRftDy9lw1CQGKK+niDluwuaJsmJL66CvJutN5PKyKDNxy4Vh Hi8Q== X-Received: by 10.31.48.11 with SMTP id w11mr16359056vkw.114.1449430803909; Sun, 06 Dec 2015 11:40:03 -0800 (PST) MIME-Version: 1.0 Received: by 10.31.84.198 with HTTP; Sun, 6 Dec 2015 11:39:44 -0800 (PST) In-Reply-To: References: <20151117124408.27185c58@fuckup.fritz.box> <20151120210654.779970ad@fuckup.fritz.box> <20151120233334.7e93f488@fuckup.fritz.box> From: Jim Meyering Date: Sun, 6 Dec 2015 11:39:44 -0800 X-Google-Sender-Auth: fMTBvY5CPK3p_lU7FDiABQWbrMs Message-ID: Subject: Re: [bug-diffutils] bug#21942: Files with incorrect file sizes To: =?UTF-8?Q?Stephan_M=C3=BCller?= Content-Type: text/plain; charset=UTF-8 X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21942-done Cc: 21942-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Sun, Dec 6, 2015 at 11:27 AM, Jim Meyering wrote: ... > Thank you, but I don't want to have to specify some new option to > avoid this misbehavior, so will push the attached patch shortly. > If someone finds a system for which a falsely reported stat.st_size > is nonzero, we can revisit this. Pushed with commit log changes and one fewer blank line in NEWS. From unknown Thu Jun 19 14:06:26 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 04 Jan 2016 12:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator