From unknown Sat Jun 14 19:24:21 2025 X-Loop: help-debbugs@gnu.org Subject: bug#10376: [Bug 908354] Re: tail -0f /var/log/kern.log never prints anything (livecd cow overlayfs) Resent-From: Jim Meyering Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 27 Dec 2011 09:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 10376 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Bug 908354 <908354@bugs.launchpad.net> Cc: 10376@debbugs.gnu.org X-Debbugs-Original-Cc: bug-coreutils@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.132497946220703 (code B ref -1); Tue, 27 Dec 2011 09:52:02 +0000 Received: (at submit) by debbugs.gnu.org; 27 Dec 2011 09:51:02 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RfTgU-0005Ns-8G for submit@debbugs.gnu.org; Tue, 27 Dec 2011 04:51:02 -0500 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RfTgR-0005NU-Gu for submit@debbugs.gnu.org; Tue, 27 Dec 2011 04:51:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RfTds-00027b-Ob for submit@debbugs.gnu.org; Tue, 27 Dec 2011 04:48:21 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=unavailable version=3.3.2 Received: from lists.gnu.org ([140.186.70.17]:43129) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RfTds-00027X-NC for submit@debbugs.gnu.org; Tue, 27 Dec 2011 04:48:20 -0500 Received: from eggs.gnu.org ([140.186.70.92]:44033) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RfTdo-0005KU-Q0 for bug-coreutils@gnu.org; Tue, 27 Dec 2011 04:48:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RfTdk-000266-KK for bug-coreutils@gnu.org; Tue, 27 Dec 2011 04:48:16 -0500 Received: from mx.meyering.net ([88.168.87.75]:35261) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RfTdk-00025e-62 for bug-coreutils@gnu.org; Tue, 27 Dec 2011 04:48:12 -0500 Received: from rho.meyering.net (localhost.localdomain [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id 694FF6005B; Tue, 27 Dec 2011 10:48:10 +0100 (CET) From: Jim Meyering In-Reply-To: <20111227013651.31528.69562.malone@gac.canonical.com> (Tuomas Heino's message of "Tue, 27 Dec 2011 01:36:51 -0000") References: <20111224093845.17188.46468.malonedeb@wampee.canonical.com> <20111227013651.31528.69562.malone@gac.canonical.com> Date: Tue, 27 Dec 2011 10:48:10 +0100 Message-ID: <87sjk69wh1.fsf@rho.meyering.net> Lines: 25 MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -4.7 (----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.7 (----) [ following up to this report, http://bugs.launchpad.net/+source/coreutils/+bug/908354 ] Tuomas Heino wrote: > ubuntu@ubuntu:~$ stat -f --format %t:%T /var/log > 1021994:tmpfs Thanks. Unlike most file system types, overlayFS appears to have no magic number. Now we're seeing how using files on such a file system can cause trouble. I see no direct way to distinguish this file (for which inotify does not work) from any other on a tmpfs file system, for which inotify works just fine. Ugly work-around: tail -f could try using both inotify and polling, and, eventually, if polling spots a change for which there was no inotify event, it would give up on using inotify for that file. As soon as tail sees an inotify event for a file, however, it could not stop polling: for most remote FS types, inotify works with changes generated locally, but not with those generated remotely. So polling can still be useful when using inotify. We'll see... From unknown Sat Jun 14 19:24:21 2025 X-Loop: help-debbugs@gnu.org Subject: bug#10376: [Bug 908354] Re: tail -0f /var/log/kern.log never prints anything (livecd cow overlayfs) Resent-From: Sven Breuner Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Sun, 01 Jan 2012 22:17:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 10376 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Jim Meyering Cc: Bug 908354 <908354@bugs.launchpad.net>, 10376@debbugs.gnu.org Received: via spool by 10376-submit@debbugs.gnu.org id=B10376.132545617425355 (code B ref 10376); Sun, 01 Jan 2012 22:17:01 +0000 Received: (at 10376) by debbugs.gnu.org; 1 Jan 2012 22:16:14 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RhThO-0006au-Mu for submit@debbugs.gnu.org; Sun, 01 Jan 2012 17:16:14 -0500 Received: from mailgw1.uni-kl.de ([131.246.120.220]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RhThL-0006al-7N for 10376@debbugs.gnu.org; Sun, 01 Jan 2012 17:16:12 -0500 Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3]) by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id q01MD0d2011905 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT) for <10376@debbugs.gnu.org>; Sun, 1 Jan 2012 23:13:00 +0100 Received: from mail2.itwm.fhg.de ([131.246.191.79]:43535) by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168) (/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1) id 1RhTeF-0006G9-Fg; Sun, 01 Jan 2012 23:12:59 +0100 Message-ID: <4F00DA6A.3010709@itwm.fraunhofer.de> Date: Sun, 01 Jan 2012 23:12:58 +0100 From: Sven Breuner User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 References: <20111224093845.17188.46468.malonedeb@wampee.canonical.com> <20111227013651.31528.69562.malone@gac.canonical.com> <87sjk69wh1.fsf@rho.meyering.net> In-Reply-To: <87sjk69wh1.fsf@rho.meyering.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ITWM-CharSet: ISO-8859-1 X-ITWM-Scanned-By: mail2.itwm.fhg.de X-Spam-Score: -6.6 (------) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.6 (------) Jim Meyering wrote on 12/27/2011 10:48 AM: > Unlike most file system types, overlayFS appears to have no magic number. > Now we're seeing how using files on such a file system can cause trouble. > I see no direct way to distinguish this file (for which inotify does not > work) from any other on a tmpfs file system, for which inotify works > just fine. > > Ugly work-around: tail -f could try using both inotify and polling, > and, eventually, if polling spots a change for which there was no > inotify event, it would give up on using inotify for that file. > As soon as tail sees an inotify event for a file, however, it could > not stop polling: for most remote FS types, inotify works with changes > generated locally, but not with those generated remotely. So polling > can still be useful when using inotify. I wouldn't see this as an ugly solution, but rather as a normal hybrid approach where tail would just loop and wait with poll(2) on the inotify fd: - If poll() reports an event, read the file modifications. - If poll() times out after one second or so, use stat() for the old stat-based change detection. - Either way, poll() again afterwards with the same timeout (i.e. no need to disable inotify only because stat() detects a change, which inotify didn't detect). As this would work with local and remote file systems, it could completely eliminate the need for different local/remote FS code paths in tail (at least from the Linux point of view). It would also bring one of the inotify advantages (immediate updates instead of one second delay) to the remote file systems, if the writing process and the watching process are running on the same machine. However, the obvious disadvantage here is that tail would need to do stat() even on local file systems, where it typically shouldn't be necessary at all. (Don't know how important that is.) Anyways, the actual problem here is of course that overlayFS pretends to be another file system, but doesn't behave like applications would expect from that other file system. Thus, it's rather a problem with overlayFS than a problem with tail - and this might also affect other applications, not only tail. So overlayFS should get its own magic number. Best regards, Sven From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 11 18:41:23 2018 Received: (at control) by debbugs.gnu.org; 11 Oct 2018 22:41:23 +0000 Received: from localhost ([127.0.0.1]:45718 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gAjdz-0005AC-3c for submit@debbugs.gnu.org; Thu, 11 Oct 2018 18:41:23 -0400 Received: from mail-pl1-f174.google.com ([209.85.214.174]:37739) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gAjdx-00059z-2T for control@debbugs.gnu.org; Thu, 11 Oct 2018 18:41:21 -0400 Received: by mail-pl1-f174.google.com with SMTP id u6-v6so2169559plz.4 for ; Thu, 11 Oct 2018 15:41:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:message-id:date:user-agent:mime-version:content-language :content-transfer-encoding; bh=uJKUdpAUHaUoHWmSrSdjOngBf5819XF15F7yXY2d6yk=; b=Ted2UJ0CXKcyLJ9bRmBhAY4LP/nopOPhaN8F3uo0Mc0fpqWCTMn3Mi6Qddn4kHDuv0 0EuMijAhVzTFETDOPm4uSRrh/wa3wvrQDCo79ZC6JtQ6AJfOHwxh3U8MnDfpjHplDrlz odoviTjdXT4MwzbkJ1hjTb3cQv/i7ENEUA/8Dut2AR/FQit/c5tZd3iyTRoZ8fwJGEB9 juw271JIkKxhmxCh0Nihp2zUoRJlEQg7UdTH1zsJ6yGa0om1OBD5sClO2deWvL78DiFI mQ2Ja0Zn++QMkLP5p2AmgulerKIDMpYexray475QFTbzHNe1Lu0D+dWyGDASymOtg57g uy/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:message-id:date:user-agent:mime-version :content-language:content-transfer-encoding; bh=uJKUdpAUHaUoHWmSrSdjOngBf5819XF15F7yXY2d6yk=; b=Cu/4J3Ih6JT9tYmOYE1wbmFrHTJdy/lW/hvP0+tiPVs/9vj8pp2o5Ft0bocu2CsIAp v3VvSGCnUdvxX3+dhToQk0+LrfaPZcyb0KeZTjzOkkhu7vL12BIS85OthjEKcXsC5Xdo 2FGdazuROmUaLHgdbIG0VzJbyzEz5sZgTt/UHI0v7VlPfhvpgnI/+yXgGR0jft5S27Ss HUopiQOgp3IzyuDnNdHFSt2DMrWvb+h4DEUOyBflUJfDRwmsJqsFlbD/RpGh5jQMNUq7 /ShX8CKPnoR90Shj0GNn1EwIaRepIxe63p8MnU29GrEQeQMxOW9Rv6kGfHE0bze38/kM 74JQ== X-Gm-Message-State: ABuFfojfKoYWdznwia8ldeKWKGu6vqC/fSZKp3xvrQHzX48JoeDl3/v2 bs+3Zsp2wJAj7nS8nm+0K3fxRhar X-Google-Smtp-Source: ACcGV615390budogDdJ4QyFlFPD6w3s+H4at71Ss2fiEnDUzrvW9sskA0n/LxN0WITy1mM9Ig48daw== X-Received: by 2002:a17:902:6e17:: with SMTP id u23-v6mr3441305plk.28.1539297674702; Thu, 11 Oct 2018 15:41:14 -0700 (PDT) Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38]) by smtp.googlemail.com with ESMTPSA id x23-v6sm16505975pfh.56.2018.10.11.15.41.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Oct 2018 15:41:13 -0700 (PDT) To: control@debbugs.gnu.org From: Assaf Gordon Message-ID: <82dcd5cf-e31a-1a83-8aa9-fedba61edde7@gmail.com> Date: Thu, 11 Oct 2018 16:41:11 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: close 10436 tags 10376 notabug close 10376 close 10430 [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [209.85.214.174 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (assafgordon[at]gmail.com) 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.85.214.174 listed in wl.mailspike.net] 1.8 MISSING_SUBJECT Missing Subject: header 0.2 NO_SUBJECT Extra score for no subject 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) close 10436 tags 10376 notabug close 10376 close 10430 tags 7597 fixed close 7597