GNU bug report logs -
#50097
gzip: add "--keep" option to keep original files unchanged
Previous Next
To reply to this bug, email your comments to 50097 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Tue, 17 Aug 2021 17:45:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Krzysztof Żelechowski <giecrilj <at> stegny.2a.pl>:
New bug report received and forwarded. Copy sent to
bug-gzip <at> gnu.org.
(Tue, 17 Aug 2021 17:45:01 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Dnia sobota, 9 lutego 2013 16:21:13 CEST Rodrigo Campos pisze:
> Hi,
>
> Please keep me in Cc: since I'm not subscribed.
>
> This path just adds the "--keep" option to keep original files unchanged.
> The patch is very simple and I have tried it here (manpage and binary) and
> it seems to work just fine.
>
> I tried to respect the coding style (it is a very small patch, so hopefully
> I did) and the commit format. But I wasn't sure how the commit format for
> this patch should be. So please let me know which changes should I do.
This patch has the following problem: it still refuses to zip a file with
multiple links for no reason (the original file is unaffected).
Please fix,
Chris
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Wed, 18 Aug 2021 22:44:02 GMT)
Full text and
rfc822 format available.
Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):
On 8/17/21 7:43 PM, Krzysztof Żelechowski wrote:
> Dnia sobota, 9 lutego 2013 16:21:13 CEST Rodrigo Campos pisze:
>> Hi,
>>
>> Please keep me in Cc: since I'm not subscribed.
>>
>> This path just adds the "--keep" option to keep original files unchanged.
>> The patch is very simple and I have tried it here (manpage and binary) and
>> it seems to work just fine.
>>
>> I tried to respect the coding style (it is a very small patch, so hopefully
>> I did) and the commit format. But I wasn't sure how the commit format for
>> this patch should be. So please let me know which changes should I do.
>
> This patch has the following problem: it still refuses to zip a file with
> multiple links for no reason (the original file is unaffected).
Wow, this patch was sent (and applied) ~8 years ago already. Congrats
for finding the thread :D
Can you please explain why, if the original file is not affected, this
patch is related? The patch just makes sure the original files are
unchanged, as the topic says, and that is working as far as I understand.
Also, can you please provide specific steps to reproduce?
Best,
Rodrigo
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Thu, 19 Aug 2021 00:31:01 GMT)
Full text and
rfc822 format available.
Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):
Dnia czwartek, 19 sierpnia 2021 00:43:26 CEST Rodrigo Campos pisze:
> On 8/17/21 7:43 PM, Krzysztof Żelechowski wrote:
> > Dnia sobota, 9 lutego 2013 16:21:13 CEST Rodrigo Campos pisze:
> >> Hi,
> >>
> >> Please keep me in Cc: since I'm not subscribed.
> >>
> >> This path just adds the "--keep" option to keep original files unchanged.
> >> The patch is very simple and I have tried it here (manpage and binary)
> >> and
> >> it seems to work just fine.
> >>
> >> I tried to respect the coding style (it is a very small patch, so
> >> hopefully
> >> I did) and the commit format. But I wasn't sure how the commit format for
> >> this patch should be. So please let me know which changes should I do.
> >
> > This patch has the following problem: it still refuses to zip a file with
> > multiple links for no reason (the original file is unaffected).
>
> Wow, this patch was sent (and applied) ~8 years ago already. Congrats
> for finding the thread :D
>
> Can you please explain why, if the original file is not affected, this
> patch is related?
This patch allows the original file to be left unaffected.
> The patch just makes sure the original files are
> unchanged, as the topic says, and that is working as far as I understand.
It is.
>
>
> Also, can you please provide specific steps to reproduce?
{ >>a && ln a b && gzip -k b
}
>
>
>
> Best,
> Rodrigo
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Thu, 19 Aug 2021 12:02:02 GMT)
Full text and
rfc822 format available.
Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):
On 8/19/21 2:29 AM, Krzysztof Żelechowski wrote:
>
>> The patch just makes sure the original files are
>> unchanged, as the topic says, and that is working as far as I understand.
>
> It is >
>> Also, can you please provide specific steps to reproduce?
>
> { >>a && ln a b && gzip -k b
> }
Exactly, the original file is unaffected, therefore nothing to fix in
this specific patch. Note the same happens without "-k" too.
If this is a bug, please report with the regular methods. Don't revive
this unrelated thread from the mailing list :)
Best,
Rodrigo
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Thu, 19 Aug 2021 13:52:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 50097 <at> debbugs.gnu.org (full text, mbox):
Rodrigo Campos wrote:
>> { >>a && ln a b && gzip -k b
>> }
>
> Exactly, the original file is unaffected, therefore nothing to fix in
> this specific patch. Note the same happens without "-k" too.
I think the point of Krzysztof is that the patch is incomplete because, in
addition to keep the original file, it should also allow the (de)compression
of links without forcing the user to type also '-f' for no apparent reason.
$ touch a && ln a b && ls -go *
-rw-r--r-- 2 0 Aug 19 15:29 a
-rw-r--r-- 2 0 Aug 19 15:29 b
$ gzip -k b
gzip: b has 1 other link -- unchanged
$ lzip -k b
$ ls -go *
-rw-r--r-- 2 0 Aug 19 15:29 a
-rw-r--r-- 2 0 Aug 19 15:29 b
-rw-r--r-- 1 36 Aug 19 15:29 b.lz
$ gzip -k -f b
$ ls -go *
-rw-r--r-- 2 0 Aug 19 15:29 a
-rw-r--r-- 2 0 Aug 19 15:29 b
-rw-r--r-- 1 22 Aug 19 15:29 b.gz
-rw-r--r-- 1 36 Aug 19 15:29 b.lz
Best regards,
Antonio.
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Thu, 19 Aug 2021 15:29:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 50097 <at> debbugs.gnu.org (full text, mbox):
On 8/19/21 3:50 PM, Antonio Diaz Diaz wrote:
> Rodrigo Campos wrote:
>>> { >>a && ln a b && gzip -k b
>>> }
>>
>> Exactly, the original file is unaffected, therefore nothing to fix in
>> this specific patch. Note the same happens without "-k" too.
>
> I think the point of Krzysztof is that the patch is incomplete because,
Oh, thanks for chiming in :)
> in addition to keep the original file, it should also allow the
> (de)compression of links without forcing the user to type also '-f' for
> no apparent reason.
It is not the -k flag that disallows this.
>
> $ touch a && ln a b && ls -go *
> -rw-r--r-- 2 0 Aug 19 15:29 a
> -rw-r--r-- 2 0 Aug 19 15:29 b
> $ gzip -k b
> gzip: b has 1 other link -- unchanged
Yeap, but the same happens if you run this instead: gzip b (i.e. without
-k).
It is not the -k flag that gets in the way. This is what happens in
gzip, even without using the flag.
Am I missing something?
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Thu, 19 Aug 2021 20:45:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 50097 <at> debbugs.gnu.org (full text, mbox):
Dnia czwartek, 19 sierpnia 2021 17:28:51 CEST Rodrigo Campos pisze:
> On 8/19/21 3:50 PM, Antonio Diaz Diaz wrote:
> > Rodrigo Campos wrote:
> > $ touch a && ln a b && ls -go *
> > -rw-r--r-- 2 0 Aug 19 15:29 a
> > -rw-r--r-- 2 0 Aug 19 15:29 b
> > $ gzip -k b
> > gzip: b has 1 other link -- unchanged
>
> Yeap, but the same happens if you run this instead: gzip b (i.e. without
> -k).
>
> It is not the -k flag that gets in the way. This is what happens in
> gzip, even without using the flag.
>
> Am I missing something?
The command { gzip b; } fails because it would compress [a] without renaming
it.
The command { gzip -k b; } would not compress [a], so there is no reason to
fail.
HTH,
Chris
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Thu, 19 Aug 2021 21:33:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 50097 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 8/19/21 1:43 PM, Krzysztof Żelechowski wrote:
> The command { gzip b; } fails because it would compress [a] without renaming
> it.
> The command { gzip -k b; } would not compress [a], so there is no reason to
> fail.
There is a safety reason to fail. A naive user might do this:
gzip -k b && gzip -cd b.gz >b
which would trash the hard-linked file as well.
There's a similar safety issue with symbolic links:
$ ln -s a b
$ touch a
$ gzip -k b && gzip -cd b.gz >b
gzip: b: Too many levels of symbolic links
where gzip's failure prevents the naive user from trashing the linked-to
file.
Of course one can use -f to go ahead and compress anyway.
Similar diagnostics say "file ignored" or "ignored", and that is clearer
than saying "unchanged", so I installed the attached patch.
[0001-gzip-clarify-other-links-diagnostic.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Fri, 20 Aug 2021 12:21:01 GMT)
Full text and
rfc822 format available.
Message #29 received at 50097 <at> debbugs.gnu.org (full text, mbox):
Dnia czwartek, 19 sierpnia 2021 23:32:44 CEST Paul Eggert pisze:
> On 8/19/21 1:43 PM, Krzysztof Żelechowski wrote:
> > The command { gzip b; } fails because it would compress [a] without
> > renaming it.
> > The command { gzip -k b; } would not compress [a], so there is no reason
> > to
> > fail.
>
> There is a safety reason to fail. A naive user might do this:
>
> gzip -k b && gzip -cd b.gz >b
>
> which would trash the hard-linked file as well.
That would replace the content of the file b with the content of the file b.
How bad can that be? ;-)
Chris
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Fri, 20 Aug 2021 12:32:01 GMT)
Full text and
rfc822 format available.
Message #32 received at 50097 <at> debbugs.gnu.org (full text, mbox):
Dnia czwartek, 19 sierpnia 2021 23:32:44 CEST Paul Eggert pisze:
> On 8/19/21 1:43 PM, Krzysztof Żelechowski wrote:
> > The command { gzip b; } fails because it would compress [a] without
> > renaming it.
> > The command { gzip -k b; } would not compress [a], so there is no reason
> > to
> > fail.
> Of course one can use -f to go ahead and compress anyway.
That would silently replace an existing target, which is bad.
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Fri, 20 Aug 2021 14:44:02 GMT)
Full text and
rfc822 format available.
Message #35 received at 50097 <at> debbugs.gnu.org (full text, mbox):
On 8/20/21 5:20 AM, Krzysztof Żelechowski wrote:
>> gzip -k b && gzip -cd b.gz >b
>>
>> which would trash the hard-linked file as well.
> That would replace the content of the file b with the content of the file b.
> How bad can that be? ;-)
It could be a problem as-is if some other file reads 'a' in the middle
of the process. And the problem would be more serious in other examples,
such as:
echo hello >a
ln a b
gzip -k b && gzip -cd b.gz >>b
This would trash 'a' if gzip -k succeeded. (Same with 'ln -s'.)
>> > Of course one can use -f to go ahead and compress anyway.
>
> That would silently replace an existing target, which is bad.
Sure. The only question here is what things "gzip -k" should be able to
do, without the things being considered "bad".
Perhaps if you gave us the surrounding context; that might help explain
why it would be useful to change the behavior of gzip -k on links. Bug
report 50097 seems to start up in the middle of a conversation that I'm
not privy to.
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Fri, 20 Aug 2021 14:54:01 GMT)
Full text and
rfc822 format available.
Message #38 received at 50097 <at> debbugs.gnu.org (full text, mbox):
Dnia piątek, 20 sierpnia 2021 16:42:54 CEST Paul Eggert pisze:
> On 8/20/21 5:20 AM, Krzysztof Żelechowski wrote:
> >> gzip -k b && gzip -cd b.gz >b
> >>
> >> which would trash the hard-linked file as well.
> >
> > That would replace the content of the file b with the content of the file
> > b. How bad can that be? ;-)
>
> It could be a problem as-is if some other file reads 'a' in the middle
> of the process. And the problem would be more serious in other examples,
> such as:
>
> echo hello >a
> ln a b
> gzip -k b && gzip -cd b.gz >>b
>
> This would trash 'a' if gzip -k succeeded. (Same with 'ln -s'.)
The appending redirection operator has nothing to do with gzip. It is a shell
feature. If you need to protect dumb users from themselves, you need to design
and implement { sh -o noclobberlinks; }.
>
> >> > Of course one can use -f to go ahead and compress anyway.
> >
> > That would silently replace an existing target, which is bad.
>
> Sure. The only question here is what things "gzip -k" should be able to
> do, without the things being considered "bad".
>
> Perhaps if you gave us the surrounding context; that might help explain
> why it would be useful to change the behavior of gzip -k on links. Bug
> report 50097 seems to start up in the middle of a conversation that I'm
> not privy to.
My normal usage of (hard) links is as follows:
Create a file page.txt. Link it to page.html.
Want to view? Open page.html.
Want to edit? Open page.txt.
Want to attach? gzip -k page.html … oops.
See ya,
Chris
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Fri, 20 Aug 2021 15:16:02 GMT)
Full text and
rfc822 format available.
Message #41 received at 50097 <at> debbugs.gnu.org (full text, mbox):
On 8/20/21 7:52 AM, Krzysztof Żelechowski wrote:
>> echo hello >a
>> ln a b
>> gzip -k b && gzip -cd b.gz >>b
>>
>> This would trash 'a' if gzip -k succeeded. (Same with 'ln -s'.)
>
> The appending redirection operator has nothing to do with gzip.
That's true even without -k: if plain 'gzip b' succeeded in this case,
any problem that the user ran into later would have nothing to do with
gzip. Plain 'gzip b' ignores 'b' here, not because compressing 'b' would
cause any problem with gzip itself, but because it could cause confusion
*later*.
> My normal usage of (hard) links is as follows:
> Create a file page.txt. Link it to page.html.
> Want to view? Open page.html.
> Want to edit? Open page.txt.
> Want to attach? gzip -k page.html … oops.
Oh my. That is ... unusual. Let's see if others chime in about whether
this usage would justify changing the behavior of gzip -k.
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Fri, 20 Aug 2021 15:19:01 GMT)
Full text and
rfc822 format available.
Message #44 received at 50097 <at> debbugs.gnu.org (full text, mbox):
Dnia piątek, 20 sierpnia 2021 17:15:43 CEST Paul Eggert pisze:
> On 8/20/21 7:52 AM, Krzysztof Żelechowski wrote:
> >> echo hello >a
> >> ln a b
> >> gzip -k b && gzip -cd b.gz >>b
> >>
> >> This would trash 'a' if gzip -k succeeded. (Same with 'ln -s'.)
> >
> > The appending redirection operator has nothing to do with gzip.
>
> That's true even without -k: if plain 'gzip b' succeeded in this case,
> any problem that the user ran into later would have nothing to do with
> gzip. Plain 'gzip b' ignores 'b' here, not because compressing 'b' would
> cause any problem with gzip itself, but because it could cause confusion
> *later*.
If plain { gzip b; } succeeded, we have a problem because [a] is compressed
but not renamed.
Chris
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Sat, 21 Aug 2021 13:57:02 GMT)
Full text and
rfc822 format available.
Message #47 received at 50097 <at> debbugs.gnu.org (full text, mbox):
On Fri, Aug 20, 2021 at 5:16 PM Paul Eggert <eggert <at> cs.ucla.edu> wrote:
> On 8/20/21 7:52 AM, Krzysztof Żelechowski wrote:
>
> >> echo hello >a
> >> ln a b
> >> gzip -k b && gzip -cd b.gz >>b
> >>
> >> This would trash 'a' if gzip -k succeeded. (Same with 'ln -s'.)
> >
> > The appending redirection operator has nothing to do with gzip.
>
> That's true even without -k: if plain 'gzip b' succeeded in this case,
> any problem that the user ran into later would have nothing to do with
> gzip. Plain 'gzip b' ignores 'b' here, not because compressing 'b' would
> cause any problem with gzip itself, but because it could cause confusion
> *later*.
>
>
> > My normal usage of (hard) links is as follows:
> > Create a file page.txt. Link it to page.html.
> > Want to view? Open page.html.
> > Want to edit? Open page.txt.
> > Want to attach? gzip -k page.html … oops.
>
> Oh my. That is ... unusual. Let's see if others chime in about whether
> this usage would justify changing the behavior of gzip -k.
We have decades of history across gzip, bzip2 and xz for the current
semantics of -k.
Generally, we have an obligation to try hard to avoid changing
semantics of existing options.
I.e., we have to weigh "what could go wrong?" against "what would be improved?"
In this case, the weights are small on each side. That alone argues against
making a semantics-changing modification to such a tool.
To compress a single file and keep the original, one can always read
it via standard input.
To summarize, while tempting to make the change,
it feels like too small a gain to justify changing semantics in a tool
like gzip.
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Sun, 22 Aug 2021 01:04:02 GMT)
Full text and
rfc822 format available.
Message #50 received at 50097 <at> debbugs.gnu.org (full text, mbox):
Dnia sobota, 21 sierpnia 2021 15:55:45 CEST Jim Meyering pisze:
> I.e., we have to weigh "what could go wrong?" against "what would be
> improved?" In this case, the weights are small on each side. That alone
> argues against making a semantics-changing modification to such a tool.
The bad weight on the "for" side is 0, the counterexamples are unconvincing.
BR,
Chris
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Sun, 22 Aug 2021 11:22:01 GMT)
Full text and
rfc822 format available.
Message #53 received at 50097 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
gzip theologians,
Just for information, up to now, pigz would merrily unlink the hard link of the input file, even if it has other hard links. While I’m not sure I understand why it shouldn’t do that (that is in fact what you asked it to do), in the spirit of behaving mostly like gzip, it will now* not unlink the input file if it has other hard links. It will let you know it didn’t do that, unless -k was provided. However it will proceed with the compression (or decompression) and create the file with the .gz added (or the .gz removed). If -f is specified, it will unlink the input file regardless of the number of hard links it has.
Mark
* “now” means on the develop branch of pigz on github. I have not yet released a new version with that commit.
On Aug 21, 2021, at 6:02 PM, Krzysztof Żelechowski <giecrilj <at> stegny.2a.pl<mailto:giecrilj <at> stegny.2a.pl>> wrote:
Dnia sobota, 21 sierpnia 2021 15:55:45 CEST Jim Meyering pisze:
I.e., we have to weigh "what could go wrong?" against "what would be
improved?" In this case, the weights are small on each side. That alone
argues against making a semantics-changing modification to such a tool.
The bad weight on the "for" side is 0, the counterexamples are unconvincing.
BR,
Chris
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gzip <at> gnu.org:
bug#50097; Package
gzip.
(Sun, 22 Aug 2021 11:32:02 GMT)
Full text and
rfc822 format available.
Message #56 received at 50097 <at> debbugs.gnu.org (full text, mbox):
Dnia niedziela, 22 sierpnia 2021 13:21:45 CEST Adler, Mark pisze:
> Just for information, up to now, pigz would merrily unlink the hard link of
> the input file, even if it has other hard links. While I’m not sure I
> understand why it shouldn’t do that (that is in fact what you asked it to
> do), in the spirit of behaving mostly like gzip, it will now* not
I assumed we wanted to prevent the possibility that we would overwrite and
rename the file instead of removing it. I understand that it is not the
current approach and it stinks of data loss but it possible to do it that way,
e.g. if the underlying file system is transactional; in that case, the
difference would not be observable.
Chris
Severity set to 'wishlist' from 'normal'
Request was from
Paul Eggert <eggert <at> cs.ucla.edu>
to
control <at> debbugs.gnu.org.
(Wed, 30 Mar 2022 00:24:02 GMT)
Full text and
rfc822 format available.
This bug report was last modified 3 years and 77 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.