GNU bug report logs - #10376
[Bug 908354] Re: tail -0f /var/log/kern.log never prints anything (livecd cow overlayfs)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10376 in the body.
You can then email your comments to 10376 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bug 908354 <908354 <at> bugs.launchpad.net>
Cc: bug-coreutils <at> gnu.org
Subject: Re: [Bug 908354] Re: tail -0f /var/log/kern.log never prints anything
	(livecd cow overlayfs)
Date: Tue, 27 Dec 2011 10:48:10 +0100

[ following up to this report,
  http://bugs.launchpad.net/+source/coreutils/+bug/908354
  ]

Tuomas Heino wrote:
> ubuntu <at> ubuntu:~$ stat -f --format %t:%T /var/log
> 1021994:tmpfs

Thanks.

Unlike most file system types, overlayFS appears to have no magic number.
Now we're seeing how using files on such a file system can cause trouble.
I see no direct way to distinguish this file (for which inotify does not
work) from any other on a tmpfs file system, for which inotify works
just fine.

Ugly work-around: tail -f could try using both inotify and polling,
and, eventually, if polling spots a change for which there was no
inotify event, it would give up on using inotify for that file.
As soon as tail sees an inotify event for a file, however, it could
not stop polling: for most remote FS types, inotify works with changes
generated locally, but not with those generated remotely.  So polling
can still be useful when using inotify.

We'll see...

Message #8 received at 10376 <at> debbugs.gnu.org (full text, mbox):

From: Sven Breuner <sven.breuner <at> itwm.fraunhofer.de>
To: Jim Meyering <jim <at> meyering.net>
Cc: Bug 908354 <908354 <at> bugs.launchpad.net>, 10376 <at> debbugs.gnu.org
Subject: Re: bug#10376: [Bug 908354] Re: tail -0f /var/log/kern.log never
	prints anything (livecd cow overlayfs)
Date: Sun, 01 Jan 2012 23:12:58 +0100

Jim Meyering wrote on 12/27/2011 10:48 AM:
> Unlike most file system types, overlayFS appears to have no magic number.
> Now we're seeing how using files on such a file system can cause trouble.
> I see no direct way to distinguish this file (for which inotify does not
> work) from any other on a tmpfs file system, for which inotify works
> just fine.
>
> Ugly work-around: tail -f could try using both inotify and polling,
> and, eventually, if polling spots a change for which there was no
> inotify event, it would give up on using inotify for that file.
> As soon as tail sees an inotify event for a file, however, it could
> not stop polling: for most remote FS types, inotify works with changes
> generated locally, but not with those generated remotely.  So polling
> can still be useful when using inotify.


I wouldn't see this as an ugly solution, but rather as a normal hybrid 
approach where tail would just loop and wait with poll(2) on the inotify fd:
- If poll() reports an event, read the file modifications.
- If poll() times out after one second or so, use stat() for the old 
stat-based change detection.
- Either way, poll() again afterwards with the same timeout (i.e. no 
need to disable inotify only because stat() detects a change, which 
inotify didn't detect).

As this would work with local and remote file systems, it could 
completely eliminate the need for different local/remote FS code paths 
in tail (at least from the Linux point of view). It would also bring one 
of the inotify advantages (immediate updates instead of one second 
delay) to the remote file systems, if the writing process and the 
watching process are running on the same machine.

However, the obvious disadvantage here is that tail would need to do 
stat() even on local file systems, where it typically shouldn't be 
necessary at all. (Don't know how important that is.)

Anyways, the actual problem here is of course that overlayFS pretends to 
be another file system, but doesn't behave like applications would 
expect from that other file system. Thus, it's rather a problem with 
overlayFS than a problem with tail - and this might also affect other 
applications, not only tail. So overlayFS should get its own magic number.

Best regards,
Sven

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.