GNU bug report logs - #11922
df doesn't handle \n in mount entries appropriately

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 11922 in the body.
You can then email your comments to 11922 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Report bugs to <bug-coreutils <at> gnu.org>
Subject: df doesn't handle \n in mount entries appropriately
Date: Thu, 12 Jul 2012 12:50:51 +0100

The edge case where '\n' is in any of the /proc/mounts fields,
will cause problems with errant new lines being output by `df`.
(I notice `findmnt` from util-linux too has the same issue).

So df uses getmntent(3) to get the list, and from its man page we have:

"Since fields in the mtab and fstab files are separated  by  whitespace,
octal  escapes  are used to represent the four characters space (\040),
tab (\011), newline (\012) and backslash (\134)  in  those  files  when
they  occur in one of the four strings in a mntent structure.  The rou‐
tines addmntent() and getmntent() will convert from string  representa‐
tion to escaped representation and back."

So how to handle this. I don't think it's appropriate to
reapply all the escaping in -P mode (which explicitly wants to avoid
multiline entries), since scripts might already be
handling the more common cases of <space> and <tab>.
So how about we just change \n -> \012.
This will at least improve the output for humans.
Scripts will not be able to directly use such entries,
but they couldn't anyway, and also processing of other
entries is no longer impacted.

Also `df` since coreutils 8.11 no longer wraps long lines,
preferring to expand alignment. This is easier to read
and avoids a common gotcha in scripts.
So even for non -P mode, we should do the \n -> \012 change.

It's tempting to do this transformation in read_file_system_list()
in gnulib, but that would cause that lib to output an ambiguous
format, and so it's probably best do the processing in each app
as required?

cheers,
Pádraig.

Message #8 received at 11922 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 11922 <at> debbugs.gnu.org
Subject: Re: bug#11922: df doesn't handle \n in mount entries appropriately
Date: Thu, 12 Jul 2012 08:14:35 -0700

On 07/12/2012 04:50 AM, Pádraig Brady wrote:
> So how about we just change \n -> \012.

That would mean we also need to change \ -> \\
(otherwise the output would be ambiguous),
and at that point it might make sense to escape
space and tab consistently with the the lower
level interface.

Message #11 received at 11922 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 11922 <at> debbugs.gnu.org
Subject: Re: bug#11922: df doesn't handle \n in mount entries appropriately
Date: Thu, 12 Jul 2012 16:28:59 +0100

On 07/12/2012 04:14 PM, Paul Eggert wrote:
> On 07/12/2012 04:50 AM, Pádraig Brady wrote:
>> So how about we just change \n -> \012.
> 
> That would mean we also need to change \ -> \\
> (otherwise the output would be ambiguous),
> and at that point it might make sense to escape
> space and tab consistently with the the lower
> level interface.

Right that would be the most consistent and general.
I detailed in the previous mail why I was wary.

It would change the output for df etc. in possibly more
common cases of space and tab used in names.
Maybe that case is uncommon enough to not worry about, but...

more problematically it would change the output from
the lib, so that whereas before any program could unambiguously
access the returned entries directly, now they'd have
to unescape.

seems a bit dangerous to me.

cheers,
Pádraig.

Message #14 received at 11922 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 11922 <at> debbugs.gnu.org
Subject: Re: bug#11922: df doesn't handle \n in mount entries appropriately
Date: Thu, 12 Jul 2012 08:35:29 -0700

On 07/12/2012 08:28 AM, Pádraig Brady wrote:
> more problematically it would change the output from
> the lib

Ah, sorry, I was talking only about df's output,
independently of how it gets the info from the underlying
library.  Surely the C library API should return a string
with raw newlines etc. in it, leaving it up to 'df' how to
display it?

Message #17 received at 11922 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 11922 <at> debbugs.gnu.org
Subject: Re: bug#11922: df doesn't handle \n in mount entries appropriately
Date: Thu, 12 Jul 2012 17:11:39 +0100

On 07/12/2012 04:35 PM, Paul Eggert wrote:
> On 07/12/2012 08:28 AM, Pádraig Brady wrote:
>> more problematically it would change the output from
>> the lib
> 
> Ah, sorry, I was talking only about df's output,
> independently of how it gets the info from the underlying
> library.  Surely the C library API should return a string
> with raw newlines etc. in it, leaving it up to 'df' how to
> display it?
> 

Oops, right.
I misread "with the lower level" as "in the lower level" :)

So just considering df, the tradeoff is ambiguous
output in the presence of \n when just escaping \n,
or unambiguous output which is backwards incompatible
in the presence of <space>,<tab>,<newline>.

It's interesting that the low level interface only
considers the above 3 chars. I guess that's so
that _programs_ can use isblank() and getline() to
parse the output robustly.
However it might be better for df to escape all
control characters (\r,\f,...) so that _humans_
can parse too.

cheers,
Pádraig.

Message #20 received at 11922 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 11922 <at> debbugs.gnu.org
Subject: Re: bug#11922: df doesn't handle \n in mount entries appropriately
Date: Fri, 13 Jul 2012 15:57:54 +0100

[Message part 1 (text/plain, inline)]

On 07/12/2012 05:11 PM, Pádraig Brady wrote:
> On 07/12/2012 04:35 PM, Paul Eggert wrote:
>> On 07/12/2012 08:28 AM, Pádraig Brady wrote:
>>> more problematically it would change the output from
>>> the lib
>>
>> Ah, sorry, I was talking only about df's output,
>> independently of how it gets the info from the underlying
>> library.  Surely the C library API should return a string
>> with raw newlines etc. in it, leaving it up to 'df' how to
>> display it?
>>
> 
> Oops, right.
> I misread "with the lower level" as "in the lower level" :)
> 
> So just considering df, the tradeoff is ambiguous
> output in the presence of \n when just escaping \n,
> or unambiguous output which is backwards incompatible
> in the presence of <space>,<tab>,<newline>.
> 
> It's interesting that the low level interface only
> considers the above 3 chars. I guess that's so
> that _programs_ can use isblank() and getline() to
> parse the output robustly.
> However it might be better for df to escape all
> control characters (\r,\f,...) so that _humans_
> can parse too.

That attached patch takes the simplest approach
and just replaces '\n' with '?'.
Reasons are detailed in the commit message.

I was considering doing something a bit more general,
along the lines of unicode_displayable() in fslint:
http://code.google.com/p/fslint/source/browse/tags/2.42/fslint-gui#92
While that could be generally useful within coreutils,
I've gone with the simpler approach for now.

cheers,
Pádraig.

[df-newlines.diff (text/plain, attachment)]

Message #23 received at 11922 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 11922 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#11922: df doesn't handle \n in mount entries appropriately
Date: Fri, 13 Jul 2012 17:03:18 +0200

Pádraig Brady wrote:
> On 07/12/2012 05:11 PM, Pádraig Brady wrote:
>> On 07/12/2012 04:35 PM, Paul Eggert wrote:
>>> On 07/12/2012 08:28 AM, Pádraig Brady wrote:
>>>> more problematically it would change the output from
>>>> the lib
>>>
>>> Ah, sorry, I was talking only about df's output,
>>> independently of how it gets the info from the underlying
>>> library.  Surely the C library API should return a string
>>> with raw newlines etc. in it, leaving it up to 'df' how to
>>> display it?
>>>
>>
>> Oops, right.
>> I misread "with the lower level" as "in the lower level" :)
>>
>> So just considering df, the tradeoff is ambiguous
>> output in the presence of \n when just escaping \n,
>> or unambiguous output which is backwards incompatible
>> in the presence of <space>,<tab>,<newline>.
>>
>> It's interesting that the low level interface only
>> considers the above 3 chars. I guess that's so
>> that _programs_ can use isblank() and getline() to
>> parse the output robustly.
>> However it might be better for df to escape all
>> control characters (\r,\f,...) so that _humans_
>> can parse too.
>
> That attached patch takes the simplest approach
> and just replaces '\n' with '?'.
...
>
> * src/df.c (hide_problematic_chars): Replace '\n' with '\?'.

Thanks.  That looks fine, but shouldn't the '\?' above be just '?'

> * NEWS: Mention the fix.
> ---
>  NEWS     |    5 +++++
>  src/df.c |   24 +++++++++++++++++++++++-
>  2 files changed, 28 insertions(+), 1 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index b8d3cbb..b4bd7e4 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -13,6 +13,11 @@ GNU coreutils NEWS                                    -*- outline -*-
>    date: invalid date '\260'
>    [This bug was present in "the beginning".]
>
> +  df no longer outputs '\n' characters present in the mount point name.
> +  Such characters are replaced with '?' so for example, scripts consuming
> +  lines output by df, can work reliably.
> +  [This bug was present in "the beginning".]

Message #26 received at 11922 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 11922 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#11922: df doesn't handle \n in mount entries appropriately
Date: Fri, 13 Jul 2012 16:05:34 +0100

On 07/13/2012 04:03 PM, Jim Meyering wrote:
> Pádraig Brady wrote:
>> On 07/12/2012 05:11 PM, Pádraig Brady wrote:

>> * src/df.c (hide_problematic_chars): Replace '\n' with '\?'.
> 
> Thanks.  That looks fine, but shouldn't the '\?' above be just '?'

oops thanks.

Message #31 received at 11922-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 11922-done <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#11922: df doesn't handle \n in mount entries appropriately
Date: Mon, 16 Jul 2012 02:46:19 +0100

[Message part 1 (text/plain, inline)]

On 07/13/2012 04:05 PM, Pádraig Brady wrote:
> On 07/13/2012 04:03 PM, Jim Meyering wrote:
>> Pádraig Brady wrote:
>>> On 07/12/2012 05:11 PM, Pádraig Brady wrote:
> 
>>> * src/df.c (hide_problematic_chars): Replace '\n' with '\?'.
>>
>> Thanks.  That looks fine, but shouldn't the '\?' above be just '?'
> 
> oops thanks.

I adjusted a bit and now replace all control chars (< 0x20) with '?'.
They're easy to handle in all encodings and can only be problematic.
I added a test too.

I've pushed the attached, so I'm marking this done.

cheers,
Pádraig.

[df-newlines.diff (text/plain, attachment)]

Message #32 received at 11922-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 11922-done <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#11922: df doesn't handle \n in mount entries appropriately
Date: Mon, 16 Jul 2012 09:32:10 +0200

Pádraig Brady wrote:
...
> I adjusted a bit and now replace all control chars (< 0x20) with '?'.
> They're easy to handle in all encodings and can only be problematic.
> I added a test too.
>
> I've pushed the attached, so I'm marking this done.

Tests passed here.
Thanks!

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.