From unknown Sun Jun 22 22:48:01 2025 X-Loop: help-debbugs@gnu.org Subject: bug#19760: [bug] "tail -f" with inotify fails to follow a file after a rename() Resent-From: Stephane Chazelas Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 03 Feb 2015 22:05:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 19760 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: 19760@debbugs.gnu.org X-Debbugs-Original-To: bug-coreutils@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.142300106725019 (code B ref -1); Tue, 03 Feb 2015 22:05:02 +0000 Received: (at submit) by debbugs.gnu.org; 3 Feb 2015 22:04:27 +0000 Received: from localhost ([127.0.0.1]:33199 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YIla6-0006VS-TY for submit@debbugs.gnu.org; Tue, 03 Feb 2015 17:04:27 -0500 Received: from eggs.gnu.org ([208.118.235.92]:49102) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YIla5-0006VF-C8 for submit@debbugs.gnu.org; Tue, 03 Feb 2015 17:04:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YIlZz-0000Hw-8B for submit@debbugs.gnu.org; Tue, 03 Feb 2015 17:04:20 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:45766) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YIlZz-0000Hs-60 for submit@debbugs.gnu.org; Tue, 03 Feb 2015 17:04:19 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50795) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YIlZy-0006A1-35 for bug-coreutils@gnu.org; Tue, 03 Feb 2015 17:04:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YIlZv-0000Gu-4Y for bug-coreutils@gnu.org; Tue, 03 Feb 2015 17:04:18 -0500 Received: from mail-wi0-x234.google.com ([2a00:1450:400c:c05::234]:41961) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YIlZu-0000Ge-Tc for bug-coreutils@gnu.org; Tue, 03 Feb 2015 17:04:15 -0500 Received: by mail-wi0-f180.google.com with SMTP id h11so24877434wiw.1 for ; Tue, 03 Feb 2015 14:04:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=HUrjGbrEUVc4i+4Qk8fEu1aA89tTMzDS7W4ivdv0bao=; b=ssrNgXrtAWGBXrlwUatj+fI1mgwybhq5t5o/pscQWMrkWtyn9Z3Wew6id533xqbhNI GpQrk0uT0vluavYivoh0O+lW6SIrGmNsMwTqXmQuAfLacgx1R4elVvN9gToK/CCC8fkd PLhx+L6dmxG7tbI8a7lPItRQDwsQGAiR78zaQ3JK0NQFwbP+ENv9Q1NTmnaJkVpWu84k BZDHc7OFsS1+0j/h5wQB0b+It63Y85V2il2EzHEQ5Nb4Yh4yxTFKr1SyQIKzDDifMKhq rSJZNjLv5gBBhjI2hjzslQ2uuyT4DFokdPmwb7dmlMxJbVRBotvMdLgK/a8OlXuZLuVa pIPw== X-Received: by 10.181.13.176 with SMTP id ez16mr38302539wid.78.1423001053173; Tue, 03 Feb 2015 14:04:13 -0800 (PST) Received: from chaz.gmail.com (5ec33bc9.skybroadband.com. [94.195.59.201]) by mx.google.com with ESMTPSA id q4sm338399wiz.4.2015.02.03.14.04.12 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 03 Feb 2015 14:04:12 -0800 (PST) Date: Tue, 3 Feb 2015 22:04:11 +0000 From: Stephane Chazelas Message-ID: <20150203220411.GA6648@chaz.gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="PEIAKu/WMn1b1Hv9" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) --PEIAKu/WMn1b1Hv9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, On Linux, when inotify is used, tail -f file follows a file only until it's renamed. After it is renamed, the inotify watch is removed, which means tail sits there doing nothing and any further modifications to the file are ignored. To reproduce: echo 1 > file tail -f file & exec 3>> file echo 2 >&3 sleep 1 mv file file2 sleep 1 echo 3 >&3 sleep 1 : > file2 "3" is not displayed. No message about the file being truncated either. Work arounds: tail ---disable-inotify -f file tail -f < file # effectively disables inotify or rename the file with a link() followed by an unlink() ln file newfile && rm -f file Note that the IN_DELETED_SELF event is not reached in follow-descriptor mode because tail has the file open preventing it from being deleted even after it's unlinked from the last directory. Path attached (on the current git head). I don't think IN_DELETED_SELF is useful in follow-name mode either, but I've not removed it. -- Stephane --PEIAKu/WMn1b1Hv9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-tail-fix-rename-in-follow-descriptor-mode.patch" >From f97355fa31f5e13f95667a50908506388d62df64 Mon Sep 17 00:00:00 2001 From: Stephane Chazelas Date: Tue, 3 Feb 2015 21:22:06 +0000 Subject: [PATCH] tail: fix rename in follow-descriptor mode In follow-descriptor mode, even when using inotify, we don't care if the file has been renamed or deleted. We still want to watch it. --- src/tail.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/src/tail.c b/src/tail.c index b512c2a..864c71f 100644 --- a/src/tail.c +++ b/src/tail.c @@ -159,13 +159,6 @@ struct File_spec uintmax_t n_unchanged_stats; }; -#if HAVE_INOTIFY -/* The events mask used with inotify on files. This mask is not used on - directories. */ -static const uint32_t inotify_wd_mask = (IN_MODIFY | IN_ATTRIB - | IN_DELETE_SELF | IN_MOVE_SELF); -#endif - /* Keep trying to open a file even if it is inaccessible when tail starts or if it becomes inaccessible later -- useful only with -f. */ static bool reopen_inaccessible_files; @@ -1385,11 +1378,22 @@ tail_forever_inotify (int wd, struct File_spec *f, size_t n_files, char *evbuf; size_t evbuf_off = 0; size_t len = 0; + uint32_t inotify_wd_mask; wd_to_name = hash_initialize (n_files, NULL, wd_hasher, wd_comparator, NULL); if (! wd_to_name) xalloc_die (); + /* The events mask used with inotify on files. This mask is not used on + directories. */ + if (follow_mode == Follow_name) + inotify_wd_mask = (IN_MODIFY | IN_ATTRIB | IN_DELETE_SELF | IN_MOVE_SELF); + else + /* if following by descriptor, we're watching the same file even if + unlinked or renamed. */ + inotify_wd_mask = IN_MODIFY; + + /* Add an inotify watch for each watched file. If -F is specified then watch its parent directory too, in this way when they re-appear we can add them again to the watch list. */ -- 1.9.1 --PEIAKu/WMn1b1Hv9-- From unknown Sun Jun 22 22:48:01 2025 X-Loop: help-debbugs@gnu.org Subject: bug#19760: [bug] "tail -f" with inotify fails to follow a file after a rename() Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 03 Feb 2015 23:32:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 19760 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Stephane Chazelas , 19760@debbugs.gnu.org Received: via spool by 19760-submit@debbugs.gnu.org id=B19760.142300627910829 (code B ref 19760); Tue, 03 Feb 2015 23:32:01 +0000 Received: (at 19760) by debbugs.gnu.org; 3 Feb 2015 23:31:19 +0000 Received: from localhost ([127.0.0.1]:33259 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YImwB-0002oG-8b for submit@debbugs.gnu.org; Tue, 03 Feb 2015 18:31:19 -0500 Received: from mail4.vodafone.ie ([213.233.128.170]:48149) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YImw9-0002fO-2j for 19760@debbugs.gnu.org; Tue, 03 Feb 2015 18:31:17 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjUFAI9Z0VRtTJGS/2dsb2JhbABagwaDXcV7gk8CgRZDAQEBAQF9hA0BBTIBVgsNCwkWDwkDAgECAUUGAQwIAQGILQHFdpBsAQsgj3+EKQEEnwaMCyKCMoE8PoJzAQEB Received: from unknown (HELO localhost.localdomain) ([109.76.145.146]) by mail3.vodafone.ie with ESMTP; 03 Feb 2015 23:30:35 +0000 Message-ID: <54D15A1A.6020704@draigBrady.com> Date: Tue, 03 Feb 2015 23:30:34 +0000 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 References: <20150203220411.GA6648@chaz.gmail.com> In-Reply-To: <20150203220411.GA6648@chaz.gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) On 03/02/15 22:04, Stephane Chazelas wrote: > Hello, > > On Linux, when inotify is used, > > tail -f file > > follows a file only until it's renamed. After it is renamed, the > inotify watch is removed, which means tail sits there doing > nothing and any further modifications to the file are ignored. > > To reproduce: > > echo 1 > file > tail -f file & > exec 3>> file > echo 2 >&3 > sleep 1 > mv file file2 > sleep 1 > echo 3 >&3 > sleep 1 > : > file2 > > "3" is not displayed. No message about the file being truncated > either. > > Work arounds: > > tail ---disable-inotify -f file > tail -f < file # effectively disables inotify > > or rename the file with a link() followed by an unlink() > ln file newfile && rm -f file > > Note that the IN_DELETED_SELF event is not reached in > follow-descriptor mode because tail has the file open preventing > it from being deleted even after it's unlinked from the last > directory. > > Patch attached (on the current git head). Ouch. The patch makes sense on first glance, and all existing tests pass with it. I'll check some more and add a test. thanks! Pádraig. From unknown Sun Jun 22 22:48:01 2025 X-Loop: help-debbugs@gnu.org Subject: bug#19760: [bug] "tail -f" with inotify fails to follow a file after a rename() Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 31 Mar 2015 03:16:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 19760 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Stephane Chazelas , 19760@debbugs.gnu.org Received: via spool by 19760-submit@debbugs.gnu.org id=B19760.142777174017211 (code B ref 19760); Tue, 31 Mar 2015 03:16:01 +0000 Received: (at 19760) by debbugs.gnu.org; 31 Mar 2015 03:15:40 +0000 Received: from localhost ([127.0.0.1]:40791 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YcmeQ-0004TV-2f for submit@debbugs.gnu.org; Mon, 30 Mar 2015 23:15:38 -0400 Received: from mail1.vodafone.ie ([213.233.128.43]:62469) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YcmeN-0004TH-Q0 for 19760@debbugs.gnu.org; Mon, 30 Mar 2015 23:15:37 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhAFABYQGlVtT9Bt/2dsb2JhbABcgwZSVwW7fYlhhXQBAgKBQ0wBAQEBAQF9hBQBAQEEJ2ILDQQDAQIBCRQCDwkDAgECAQ8uCAYBDAYCAQEFEYgBAxUBAwW4aY1qDYU6AQEBAQEBBAEBAQEBAQEBGospgkeCIBgGhCcFhhqKH4IOgTJUhDUBgmk6hUeGbQeCXINIIoIBARwUgT09MYJDAQEB Received: from unknown (HELO localhost.localdomain) ([109.79.208.109]) by mail1.vodafone.ie with ESMTP; 31 Mar 2015 04:15:28 +0100 Message-ID: <551A114F.9040009@draigBrady.com> Date: Tue, 31 Mar 2015 04:15:27 +0100 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 References: <20150203220411.GA6648@chaz.gmail.com> <54D15A1A.6020704@draigBrady.com> In-Reply-To: <54D15A1A.6020704@draigBrady.com> Content-Type: multipart/mixed; boundary="------------020607090704030909050406" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) This is a multi-part message in MIME format. --------------020607090704030909050406 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit On 03/02/15 23:30, Pádraig Brady wrote: > On 03/02/15 22:04, Stephane Chazelas wrote: >> Hello, >> >> On Linux, when inotify is used, >> >> tail -f file >> >> follows a file only until it's renamed. After it is renamed, the >> inotify watch is removed, which means tail sits there doing >> nothing and any further modifications to the file are ignored. >> >> To reproduce: >> >> echo 1 > file >> tail -f file & >> exec 3>> file >> echo 2 >&3 >> sleep 1 >> mv file file2 >> sleep 1 >> echo 3 >&3 >> sleep 1 >> : > file2 >> >> "3" is not displayed. No message about the file being truncated >> either. >> >> Work arounds: >> >> tail ---disable-inotify -f file >> tail -f < file # effectively disables inotify >> >> or rename the file with a link() followed by an unlink() >> ln file newfile && rm -f file >> >> Note that the IN_DELETED_SELF event is not reached in >> follow-descriptor mode because tail has the file open preventing >> it from being deleted even after it's unlinked from the last >> directory. >> >> Patch attached (on the current git head). > > Ouch. The patch makes sense on first glance, > and all existing tests pass with it. > I'll check some more and add a test. Sorry for the delay. I'll apply the attached in your name soon. thanks, Pádraig. --------------020607090704030909050406 Content-Type: text/x-patch; name="tail-rename-desc.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tail-rename-desc.patch" >From 9c23049e17a76f4ec8f38c04b088f149a49b4851 Mon Sep 17 00:00:00 2001 From: Stephane Chazelas Date: Tue, 3 Feb 2015 21:22:06 +0000 Subject: [PATCH] tail: fix -f to follow changes after a rename * src/tail.c (tail_forever_inotify): Only monitor write()s and truncate()s to files in --follow=descriptor mode, thus avoiding the bug where we removed the watch on renamed files. Also adjust the inotify event processing code which that is now significant only in --follow=name mode. * tests/tail-2/f-vs-rename.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the bug. Fixes http://bugs.gnu.org/19760 --- NEWS | 3 +++ src/tail.c | 31 +++++++++++++-------------- tests/local.mk | 1 + tests/tail-2/f-vs-rename.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 69 insertions(+), 17 deletions(-) create mode 100755 tests/tail-2/f-vs-rename.sh diff --git a/NEWS b/NEWS index 81031c6..214db08 100644 --- a/NEWS +++ b/NEWS @@ -39,6 +39,9 @@ GNU coreutils NEWS -*- outline -*- resources with many files, or with -F if files were replaced many times. [bug introduced in coreutils-7.5] + tail -f continues to follow changes to a file even after it's renamed. + [bug introduced in coreutils-7.5] + ** New features chroot accepts the new --skip-chdir option to not change the working directory diff --git a/src/tail.c b/src/tail.c index c5380cb..f75d7a9 100644 --- a/src/tail.c +++ b/src/tail.c @@ -159,13 +159,6 @@ struct File_spec uintmax_t n_unchanged_stats; }; -#if HAVE_INOTIFY -/* The events mask used with inotify on files. This mask is not used on - directories. */ -static const uint32_t inotify_wd_mask = (IN_MODIFY | IN_ATTRIB - | IN_DELETE_SELF | IN_MOVE_SELF); -#endif - /* Keep trying to open a file even if it is inaccessible when tail starts or if it becomes inaccessible later -- useful only with -f. */ static bool reopen_inaccessible_files; @@ -1390,6 +1383,13 @@ tail_forever_inotify (int wd, struct File_spec *f, size_t n_files, if (! wd_to_name) xalloc_die (); + /* The events mask used with inotify on files (not directories). */ + uint32_t inotify_wd_mask = IN_MODIFY; + /* TODO: Perhaps monitor these events in Follow_descriptor mode also, + to tag reported file names with "deleted", "moved" etc. */ + if (follow_mode == Follow_name) + inotify_wd_mask |= (IN_ATTRIB | IN_DELETE_SELF | IN_MOVE_SELF); + /* Add an inotify watch for each watched file. If -F is specified then watch its parent directory too, in this way when they re-appear we can add them again to the watch list. */ @@ -1641,20 +1641,17 @@ tail_forever_inotify (int wd, struct File_spec *f, size_t n_files, if (ev->mask & (IN_ATTRIB | IN_DELETE_SELF | IN_MOVE_SELF)) { - /* For IN_DELETE_SELF, we always want to remove the watch. - However, for IN_MOVE_SELF (the file we're watching has - been clobbered via a rename), when tailing by NAME, we - must continue to watch the file. It's only when following - by file descriptor that we must remove the watch. */ - if ((ev->mask & IN_DELETE_SELF) - || ((ev->mask & IN_MOVE_SELF) - && follow_mode == Follow_descriptor)) + /* Note for IN_MOVE_SELF (the file we're watching has + been clobbered via a rename) we leave the watch + in place since it may still be part of the set + of watched names. */ + if (ev->mask & IN_DELETE_SELF) { inotify_rm_watch (wd, fspec->wd); hash_delete (wd_to_name, fspec); } - if (follow_mode == Follow_name) - recheck (fspec, false); + + recheck (fspec, false); continue; } diff --git a/tests/local.mk b/tests/local.mk index 56cba69..1be31ad 100644 --- a/tests/local.mk +++ b/tests/local.mk @@ -171,6 +171,7 @@ all_tests = \ tests/tail-2/inotify-hash-abuse2.sh \ tests/tail-2/F-vs-missing.sh \ tests/tail-2/F-vs-rename.sh \ + tests/tail-2/f-vs-rename.sh \ tests/tail-2/inotify-rotate.sh \ tests/tail-2/inotify-rotate-resources.sh \ tests/chmod/no-x.sh \ diff --git a/tests/tail-2/f-vs-rename.sh b/tests/tail-2/f-vs-rename.sh new file mode 100755 index 0000000..efb0a40 --- /dev/null +++ b/tests/tail-2/f-vs-rename.sh @@ -0,0 +1,51 @@ +#!/bin/sh +# demonstrate that tail -f works when renaming the tailed files +# Before coreutils-8.24, tail -f a would stop tracking additions to b +# after "mv a b". + +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src +print_ver_ tail + +touch a || framework_failure_ + +debug='---disable-inotify' +debug= +tail $debug -f -s.1 a > out 2>&1 & pid=$! + +check_tail_output() +{ + local delay="$1" + grep "$tail_re" out > /dev/null || + { sleep $delay; return 1; } +} + +# Wait up to 12.7s for tail to start +echo x > a +tail_re='^x$' retry_delay_ check_tail_output .1 7 || fail=1 + +mv a b || fail=1 + +echo y >> b +# Wait up to 12.7s for "y" to appear in the output: +tail_re='^y$' retry_delay_ check_tail_output .1 7 || fail=1 + +kill $pid + +wait + +Exit $fail -- 2.1.0 --------------020607090704030909050406-- From unknown Sun Jun 22 22:48:01 2025 X-Loop: help-debbugs@gnu.org Subject: bug#19760: [bug] "tail -f" with inotify fails to follow a file after a rename() Resent-From: Bernhard Voelker Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 31 Mar 2015 06:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 19760 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: =?UTF-8?Q?P=C3=A1draig?= Brady , Stephane Chazelas , 19760@debbugs.gnu.org Received: via spool by 19760-submit@debbugs.gnu.org id=B19760.14277834332564 (code B ref 19760); Tue, 31 Mar 2015 06:31:02 +0000 Received: (at 19760) by debbugs.gnu.org; 31 Mar 2015 06:30:33 +0000 Received: from localhost ([127.0.0.1]:40828 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Ycph1-0000fG-Bd for submit@debbugs.gnu.org; Tue, 31 Mar 2015 02:30:32 -0400 Received: from mout.kundenserver.de ([212.227.126.131]:59257) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Ycpgu-0000ey-7i for 19760@debbugs.gnu.org; Tue, 31 Mar 2015 02:30:25 -0400 Received: from [192.168.1.10] ([217.86.68.70]) by mrelayeu.kundenserver.de (mreue003) with ESMTPSA (Nemesis) id 0Le960-1ZFP1B3nXI-00pu0m; Tue, 31 Mar 2015 08:30:18 +0200 Message-ID: <551A3EF9.7000304@bernhard-voelker.de> Date: Tue, 31 Mar 2015 08:30:17 +0200 From: Bernhard Voelker User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 References: <20150203220411.GA6648@chaz.gmail.com> <54D15A1A.6020704@draigBrady.com> <551A114F.9040009@draigBrady.com> In-Reply-To: <551A114F.9040009@draigBrady.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:IkfAAafkMtEOkZE6w7jeA+gpU6T0lPC+Jct+eHQDOtLo0ES1lm8 B8/yk2uEvOAxJ7M0wQQXQtfLqHKAD6soIN2YhPqQCfSy+ScNQb2/bSppwTqzekZCD5XB2Ry m9HylxX4+ouLfHouONnJ1ooM/4ZnyGkJJWRW+cxK52XWlXpr6EMUS3O2Vzm/aSTKNWOOrF+ GW/xO8G2ZMP0fCAKZyF9w== X-UI-Out-Filterresults: notjunk:1; X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) On 03/31/2015 05:15 AM, Pádraig Brady wrote: > + tail -f continues to follow changes to a file even after it's renamed. > + [bug introduced in coreutils-7.5] > + It is not 100% clear to me by this sentence what was the actual change; maybe a little "again" or "now" would help? > --- /dev/null > +++ b/tests/tail-2/f-vs-rename.sh > @@ -0,0 +1,51 @@ > +#!/bin/sh > +# demonstrate that tail -f works when renaming the tailed files s/^d/D/; s/$/./ > +# Before coreutils-8.24, tail -f a would stop tracking additions to b > +# after "mv a b". > + > +# Copyright (C) 2015 Free Software Foundation, Inc. > + > +# This program is free software: you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation, either version 3 of the License, or > +# (at your option) any later version. > + > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > + > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see . > + > +. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src > +print_ver_ tail > + > +touch a || framework_failure_ > + > +debug='---disable-inotify' > +debug= > +tail $debug -f -s.1 a > out 2>&1 & pid=$! Shouldn't $debug be removed? Otherwise maybe a loop over both the inotify and the non-inotify mode would make sense? > + > +check_tail_output() > +{ > + local delay="$1" > + grep "$tail_re" out > /dev/null || > + { sleep $delay; return 1; } > +} Please don't discard grep's output: reading the test's log file is easier with this included. > + > +# Wait up to 12.7s for tail to start s/$/./ > +echo x > a > +tail_re='^x$' retry_delay_ check_tail_output .1 7 || fail=1 > + > +mv a b || fail=1 > + > +echo y >> b > +# Wait up to 12.7s for "y" to appear in the output: > +tail_re='^y$' retry_delay_ check_tail_output .1 7 || fail=1 > + > +kill $pid > + > +wait > + > +Exit $fail Otherwise +1 (including the changes in tail.c). Thanks & have a nice day, Berny From unknown Sun Jun 22 22:48:01 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.503 (Entity 5.503) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Stephane Chazelas Subject: bug#19760: closed (Re: bug#19760: [bug] "tail -f" with inotify fails to follow a file after a rename()) Message-ID: References: <551A7D68.4020901@draigBrady.com> <20150203220411.GA6648@chaz.gmail.com> X-Gnu-PR-Message: they-closed 19760 X-Gnu-PR-Package: coreutils Reply-To: 19760@debbugs.gnu.org Date: Tue, 31 Mar 2015 10:57:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1427799422-26914-1" This is a multi-part message in MIME format... ------------=_1427799422-26914-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #19760: [bug] "tail -f" with inotify fails to follow a file after a rename() which was filed against the coreutils package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 19760@debbugs.gnu.org. --=20 19760: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D19760 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1427799422-26914-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 19760-done) by debbugs.gnu.org; 31 Mar 2015 10:56:52 +0000 Received: from localhost ([127.0.0.1]:40931 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Yctql-0006zi-9o for submit@debbugs.gnu.org; Tue, 31 Mar 2015 06:56:52 -0400 Received: from mail5.vodafone.ie ([213.233.128.176]:4591) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Yctqh-0006zS-Mf for 19760-done@debbugs.gnu.org; Tue, 31 Mar 2015 06:56:49 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArkFABx8GlVtTW5N/2dsb2JhbABcgwZSVwWCRrlDiWKFdAECAoE/TAEBAQEBAX2EFAEBAQQnYgsNAQMDAQIBCRYPCQMCAQIBDy4IBgEMBgIBAQURiAEDFQEDBblHjW0NhToBAQEBAQUBAQEBAQEBARqLKYJHgiAYhC0Fhh2KIoIPgTJUhDUBgmo6gniCUIZxB4Jcg0giggEBHBSBPT0xgkMBAQE Received: from unknown (HELO localhost.localdomain) ([109.77.110.77]) by mail3.vodafone.ie with ESMTP; 31 Mar 2015 11:56:40 +0100 Message-ID: <551A7D68.4020901@draigBrady.com> Date: Tue, 31 Mar 2015 11:56:40 +0100 From: =?windows-1252?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Bernhard Voelker , Stephane Chazelas , 19760-done@debbugs.gnu.org Subject: Re: bug#19760: [bug] "tail -f" with inotify fails to follow a file after a rename() References: <20150203220411.GA6648@chaz.gmail.com> <54D15A1A.6020704@draigBrady.com> <551A114F.9040009@draigBrady.com> <551A3EF9.7000304@bernhard-voelker.de> In-Reply-To: <551A3EF9.7000304@bernhard-voelker.de> Content-Type: multipart/mixed; boundary="------------060705000603000003000108" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 19760-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) This is a multi-part message in MIME format. --------------060705000603000003000108 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit On 31/03/15 07:30, Bernhard Voelker wrote: > On 03/31/2015 05:15 AM, Pádraig Brady wrote: >> + tail -f continues to follow changes to a file even after it's renamed. >> + [bug introduced in coreutils-7.5] >> + > > It is not 100% clear to me by this sentence what was the actual change; > maybe a little "again" or "now" would help? > >> --- /dev/null >> +++ b/tests/tail-2/f-vs-rename.sh >> @@ -0,0 +1,51 @@ >> +#!/bin/sh >> +# demonstrate that tail -f works when renaming the tailed files > > s/^d/D/; s/$/./ > >> +# Before coreutils-8.24, tail -f a would stop tracking additions to b >> +# after "mv a b". >> + >> +# Copyright (C) 2015 Free Software Foundation, Inc. >> + >> +# This program is free software: you can redistribute it and/or modify >> +# it under the terms of the GNU General Public License as published by >> +# the Free Software Foundation, either version 3 of the License, or >> +# (at your option) any later version. >> + >> +# This program is distributed in the hope that it will be useful, >> +# but WITHOUT ANY WARRANTY; without even the implied warranty of >> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >> +# GNU General Public License for more details. >> + >> +# You should have received a copy of the GNU General Public License >> +# along with this program. If not, see . >> + >> +. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src >> +print_ver_ tail >> + >> +touch a || framework_failure_ >> + >> +debug='---disable-inotify' >> +debug= >> +tail $debug -f -s.1 a > out 2>&1 & pid=$! > > Shouldn't $debug be removed? Otherwise maybe a loop over both the > inotify and the non-inotify mode would make sense? > >> + >> +check_tail_output() >> +{ >> + local delay="$1" >> + grep "$tail_re" out > /dev/null || >> + { sleep $delay; return 1; } >> +} > > Please don't discard grep's output: reading the test's log file is > easier with this included. > >> + >> +# Wait up to 12.7s for tail to start > > s/$/./ > >> +echo x > a >> +tail_re='^x$' retry_delay_ check_tail_output .1 7 || fail=1 >> + >> +mv a b || fail=1 >> + >> +echo y >> b >> +# Wait up to 12.7s for "y" to appear in the output: >> +tail_re='^y$' retry_delay_ check_tail_output .1 7 || fail=1 >> + >> +kill $pid >> + >> +wait >> + >> +Exit $fail > > Otherwise +1 (including the changes in tail.c). All good suggestions. Latest attached. thanks for the review! Pádraig. --------------060705000603000003000108 Content-Type: text/x-patch; name="tail-rename-desc.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tail-rename-desc.patch" >From d313a0b24234d3366ec263111469f219f5b4634f Mon Sep 17 00:00:00 2001 From: Stephane Chazelas Date: Tue, 3 Feb 2015 21:22:06 +0000 Subject: [PATCH] tail: fix -f to follow changes after a rename * src/tail.c (tail_forever_inotify): Only monitor write()s and truncate()s to files in --follow=descriptor mode, thus avoiding the bug where we removed the watch on renamed files. Also adjust the inotify event processing code that is now significant only in --follow=name mode. * tests/tail-2/F-vs-rename.sh: Improve this existing test by running in both polling and inotify modes. * tests/tail-2/f-vs-rename.sh: A new test based on the existing one. * tests/local.mk: Reference the new test. * NEWS: Mention the bug. Fixes http://bugs.gnu.org/19760 --- NEWS | 3 ++ src/tail.c | 31 +++++++-------- tests/local.mk | 1 + tests/tail-2/F-vs-rename.sh | 94 +++++++++++++++++++++++---------------------- tests/tail-2/f-vs-rename.sh | 51 ++++++++++++++++++++++++ 5 files changed, 117 insertions(+), 63 deletions(-) create mode 100755 tests/tail-2/f-vs-rename.sh diff --git a/NEWS b/NEWS index 81031c6..4b12e46 100644 --- a/NEWS +++ b/NEWS @@ -39,6 +39,9 @@ GNU coreutils NEWS -*- outline -*- resources with many files, or with -F if files were replaced many times. [bug introduced in coreutils-7.5] + tail -f again follows changes to a file after it's renamed. + [bug introduced in coreutils-7.5] + ** New features chroot accepts the new --skip-chdir option to not change the working directory diff --git a/src/tail.c b/src/tail.c index c5380cb..f75d7a9 100644 --- a/src/tail.c +++ b/src/tail.c @@ -159,13 +159,6 @@ struct File_spec uintmax_t n_unchanged_stats; }; -#if HAVE_INOTIFY -/* The events mask used with inotify on files. This mask is not used on - directories. */ -static const uint32_t inotify_wd_mask = (IN_MODIFY | IN_ATTRIB - | IN_DELETE_SELF | IN_MOVE_SELF); -#endif - /* Keep trying to open a file even if it is inaccessible when tail starts or if it becomes inaccessible later -- useful only with -f. */ static bool reopen_inaccessible_files; @@ -1390,6 +1383,13 @@ tail_forever_inotify (int wd, struct File_spec *f, size_t n_files, if (! wd_to_name) xalloc_die (); + /* The events mask used with inotify on files (not directories). */ + uint32_t inotify_wd_mask = IN_MODIFY; + /* TODO: Perhaps monitor these events in Follow_descriptor mode also, + to tag reported file names with "deleted", "moved" etc. */ + if (follow_mode == Follow_name) + inotify_wd_mask |= (IN_ATTRIB | IN_DELETE_SELF | IN_MOVE_SELF); + /* Add an inotify watch for each watched file. If -F is specified then watch its parent directory too, in this way when they re-appear we can add them again to the watch list. */ @@ -1641,20 +1641,17 @@ tail_forever_inotify (int wd, struct File_spec *f, size_t n_files, if (ev->mask & (IN_ATTRIB | IN_DELETE_SELF | IN_MOVE_SELF)) { - /* For IN_DELETE_SELF, we always want to remove the watch. - However, for IN_MOVE_SELF (the file we're watching has - been clobbered via a rename), when tailing by NAME, we - must continue to watch the file. It's only when following - by file descriptor that we must remove the watch. */ - if ((ev->mask & IN_DELETE_SELF) - || ((ev->mask & IN_MOVE_SELF) - && follow_mode == Follow_descriptor)) + /* Note for IN_MOVE_SELF (the file we're watching has + been clobbered via a rename) we leave the watch + in place since it may still be part of the set + of watched names. */ + if (ev->mask & IN_DELETE_SELF) { inotify_rm_watch (wd, fspec->wd); hash_delete (wd_to_name, fspec); } - if (follow_mode == Follow_name) - recheck (fspec, false); + + recheck (fspec, false); continue; } diff --git a/tests/local.mk b/tests/local.mk index 56cba69..1be31ad 100644 --- a/tests/local.mk +++ b/tests/local.mk @@ -171,6 +171,7 @@ all_tests = \ tests/tail-2/inotify-hash-abuse2.sh \ tests/tail-2/F-vs-missing.sh \ tests/tail-2/F-vs-rename.sh \ + tests/tail-2/f-vs-rename.sh \ tests/tail-2/inotify-rotate.sh \ tests/tail-2/inotify-rotate-resources.sh \ tests/chmod/no-x.sh \ diff --git a/tests/tail-2/F-vs-rename.sh b/tests/tail-2/F-vs-rename.sh index cb6a3ed..f95b71a 100755 --- a/tests/tail-2/F-vs-rename.sh +++ b/tests/tail-2/F-vs-rename.sh @@ -1,7 +1,7 @@ #!/bin/sh -# demonstrate that tail -F works when renaming the tailed files -# Before coreutils-8.3, tail -F a b would stop tracking additions to b -# after "mv a b". +# Demonstrate that tail -F works when renaming the tailed files. +# Between coreutils 7.5 and 8.2 inclusive, 'tail -F a b' would +# stop tracking additions to b after 'mv a b'. # Copyright (C) 2009-2015 Free Software Foundation, Inc. @@ -21,55 +21,57 @@ . "${srcdir=.}/tests/init.sh"; path_prepend_ ./src print_ver_ tail -touch a b || framework_failure_ - -debug='---disable-inotify' -debug= -tail $debug -F -s.1 a b > out 2>&1 & pid=$! - check_tail_output() { local delay="$1" - grep "$tail_re" out > /dev/null || + grep "$tail_re" out || { sleep $delay; return 1; } } -# Wait up to 12.7s for tail to start -echo x > a -tail_re='^x$' retry_delay_ check_tail_output .1 7 || fail=1 - -mv a b || fail=1 - -# Wait 12.7s for this diagnostic: -# tail: 'a' has become inaccessible: No such file or directory -tail_re='inaccessible' retry_delay_ check_tail_output .1 7 || fail=1 - -echo x > a -# Wait up to 12.7s for this to appear in the output: -# "tail: '...' has appeared; following end of new file" -tail_re='has appeared' retry_delay_ check_tail_output .1 7 || - { echo "$0: a: unexpected delay?"; cat out; fail=1; } - -echo y >> b -# Wait up to 12.7s for "y" to appear in the output: -tail_f_vs_rename_2() { - local delay="$1" - tr '\n' @ < out | grep '@@==> b <==@y@$' > /dev/null || - { sleep $delay; return 1; } -} -retry_delay_ tail_f_vs_rename_2 .1 7 || - { echo "$0: b: unexpected delay?"; cat out; fail=1; } - -echo z >> a -# Wait up to 12.7s for "z" to appear in the output: -tail_f_vs_rename_3() { - local delay="$1" - tr '\n' @ < out | grep '@@==> a <==@z@$' > /dev/null || - { sleep $delay; return 1; } -} -retry_delay_ tail_f_vs_rename_3 .1 7 || - { echo "$0: a: unexpected delay?"; cat out; fail=1; } +touch a b || framework_failure_ -kill -HUP $pid +for mode in '' '---disable-inotify'; do + tail $mode -F -s.1 a b > out 2>&1 & pid=$! + + # Wait up to 12.7s for tail to start. + echo x > a + tail_re='^x$' retry_delay_ check_tail_output .1 7 || fail=1 + + mv a b || fail=1 + + # Wait 12.7s for this diagnostic: + # tail: 'a' has become inaccessible: No such file or directory + tail_re='inaccessible' retry_delay_ check_tail_output .1 7 || fail=1 + + echo x > a + # Wait up to 12.7s for this to appear in the output: + # "tail: '...' has appeared; following end of new file" + tail_re='has appeared' retry_delay_ check_tail_output .1 7 || + { echo "$0: a: unexpected delay?"; cat out; fail=1; } + + echo y >> b + # Wait up to 12.7s for "y" to appear in the output: + tail_f_vs_rename_2() { + local delay="$1" + tr '\n' @ < out | grep '@@==> b <==@y@$' > /dev/null || + { sleep $delay; return 1; } + } + retry_delay_ tail_f_vs_rename_2 .1 7 || + { echo "$0: b: unexpected delay?"; cat out; fail=1; } + + echo z >> a + # Wait up to 12.7s for "z" to appear in the output: + tail_f_vs_rename_3() { + local delay="$1" + tr '\n' @ < out | grep '@@==> a <==@z@$' > /dev/null || + { sleep $delay; return 1; } + } + retry_delay_ tail_f_vs_rename_3 .1 7 || + { echo "$0: a: unexpected delay?"; cat out; fail=1; } + + kill $pid + + wait $pid +done Exit $fail diff --git a/tests/tail-2/f-vs-rename.sh b/tests/tail-2/f-vs-rename.sh new file mode 100755 index 0000000..4bff41c --- /dev/null +++ b/tests/tail-2/f-vs-rename.sh @@ -0,0 +1,51 @@ +#!/bin/sh +# Demonstrate that tail -f works when renaming the tailed files. +# Between coreutils 7.5 and 8.23 inclusive, 'tail -f a' would +# stop tracking additions to b after 'mv a b'. + +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src +print_ver_ tail + +check_tail_output() +{ + local delay="$1" + grep "$tail_re" out || + { sleep $delay; return 1; } +} + +touch a || framework_failure_ + +for mode in '' '---disable-inotify'; do + tail $mode -f -s.1 a > out 2>&1 & pid=$! + + # Wait up to 12.7s for tail to start. + echo x > a + tail_re='^x$' retry_delay_ check_tail_output .1 7 || fail=1 + + mv a b || fail=1 + + echo y >> b + # Wait up to 12.7s for "y" to appear in the output: + tail_re='^y$' retry_delay_ check_tail_output .1 7 || fail=1 + + kill $pid + + wait $pid +done + +Exit $fail -- 2.1.0 --------------060705000603000003000108-- ------------=_1427799422-26914-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 3 Feb 2015 22:04:27 +0000 Received: from localhost ([127.0.0.1]:33199 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YIla6-0006VS-TY for submit@debbugs.gnu.org; Tue, 03 Feb 2015 17:04:27 -0500 Received: from eggs.gnu.org ([208.118.235.92]:49102) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YIla5-0006VF-C8 for submit@debbugs.gnu.org; Tue, 03 Feb 2015 17:04:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YIlZz-0000Hw-8B for submit@debbugs.gnu.org; Tue, 03 Feb 2015 17:04:20 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:45766) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YIlZz-0000Hs-60 for submit@debbugs.gnu.org; Tue, 03 Feb 2015 17:04:19 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50795) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YIlZy-0006A1-35 for bug-coreutils@gnu.org; Tue, 03 Feb 2015 17:04:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YIlZv-0000Gu-4Y for bug-coreutils@gnu.org; Tue, 03 Feb 2015 17:04:18 -0500 Received: from mail-wi0-x234.google.com ([2a00:1450:400c:c05::234]:41961) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YIlZu-0000Ge-Tc for bug-coreutils@gnu.org; Tue, 03 Feb 2015 17:04:15 -0500 Received: by mail-wi0-f180.google.com with SMTP id h11so24877434wiw.1 for ; Tue, 03 Feb 2015 14:04:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=HUrjGbrEUVc4i+4Qk8fEu1aA89tTMzDS7W4ivdv0bao=; b=ssrNgXrtAWGBXrlwUatj+fI1mgwybhq5t5o/pscQWMrkWtyn9Z3Wew6id533xqbhNI GpQrk0uT0vluavYivoh0O+lW6SIrGmNsMwTqXmQuAfLacgx1R4elVvN9gToK/CCC8fkd PLhx+L6dmxG7tbI8a7lPItRQDwsQGAiR78zaQ3JK0NQFwbP+ENv9Q1NTmnaJkVpWu84k BZDHc7OFsS1+0j/h5wQB0b+It63Y85V2il2EzHEQ5Nb4Yh4yxTFKr1SyQIKzDDifMKhq rSJZNjLv5gBBhjI2hjzslQ2uuyT4DFokdPmwb7dmlMxJbVRBotvMdLgK/a8OlXuZLuVa pIPw== X-Received: by 10.181.13.176 with SMTP id ez16mr38302539wid.78.1423001053173; Tue, 03 Feb 2015 14:04:13 -0800 (PST) Received: from chaz.gmail.com (5ec33bc9.skybroadband.com. [94.195.59.201]) by mx.google.com with ESMTPSA id q4sm338399wiz.4.2015.02.03.14.04.12 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 03 Feb 2015 14:04:12 -0800 (PST) Date: Tue, 3 Feb 2015 22:04:11 +0000 From: Stephane Chazelas To: bug-coreutils@gnu.org Subject: [bug] "tail -f" with inotify fails to follow a file after a rename() Message-ID: <20150203220411.GA6648@chaz.gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="PEIAKu/WMn1b1Hv9" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) --PEIAKu/WMn1b1Hv9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, On Linux, when inotify is used, tail -f file follows a file only until it's renamed. After it is renamed, the inotify watch is removed, which means tail sits there doing nothing and any further modifications to the file are ignored. To reproduce: echo 1 > file tail -f file & exec 3>> file echo 2 >&3 sleep 1 mv file file2 sleep 1 echo 3 >&3 sleep 1 : > file2 "3" is not displayed. No message about the file being truncated either. Work arounds: tail ---disable-inotify -f file tail -f < file # effectively disables inotify or rename the file with a link() followed by an unlink() ln file newfile && rm -f file Note that the IN_DELETED_SELF event is not reached in follow-descriptor mode because tail has the file open preventing it from being deleted even after it's unlinked from the last directory. Path attached (on the current git head). I don't think IN_DELETED_SELF is useful in follow-name mode either, but I've not removed it. -- Stephane --PEIAKu/WMn1b1Hv9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-tail-fix-rename-in-follow-descriptor-mode.patch" >From f97355fa31f5e13f95667a50908506388d62df64 Mon Sep 17 00:00:00 2001 From: Stephane Chazelas Date: Tue, 3 Feb 2015 21:22:06 +0000 Subject: [PATCH] tail: fix rename in follow-descriptor mode In follow-descriptor mode, even when using inotify, we don't care if the file has been renamed or deleted. We still want to watch it. --- src/tail.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/src/tail.c b/src/tail.c index b512c2a..864c71f 100644 --- a/src/tail.c +++ b/src/tail.c @@ -159,13 +159,6 @@ struct File_spec uintmax_t n_unchanged_stats; }; -#if HAVE_INOTIFY -/* The events mask used with inotify on files. This mask is not used on - directories. */ -static const uint32_t inotify_wd_mask = (IN_MODIFY | IN_ATTRIB - | IN_DELETE_SELF | IN_MOVE_SELF); -#endif - /* Keep trying to open a file even if it is inaccessible when tail starts or if it becomes inaccessible later -- useful only with -f. */ static bool reopen_inaccessible_files; @@ -1385,11 +1378,22 @@ tail_forever_inotify (int wd, struct File_spec *f, size_t n_files, char *evbuf; size_t evbuf_off = 0; size_t len = 0; + uint32_t inotify_wd_mask; wd_to_name = hash_initialize (n_files, NULL, wd_hasher, wd_comparator, NULL); if (! wd_to_name) xalloc_die (); + /* The events mask used with inotify on files. This mask is not used on + directories. */ + if (follow_mode == Follow_name) + inotify_wd_mask = (IN_MODIFY | IN_ATTRIB | IN_DELETE_SELF | IN_MOVE_SELF); + else + /* if following by descriptor, we're watching the same file even if + unlinked or renamed. */ + inotify_wd_mask = IN_MODIFY; + + /* Add an inotify watch for each watched file. If -F is specified then watch its parent directory too, in this way when they re-appear we can add them again to the watch list. */ -- 1.9.1 --PEIAKu/WMn1b1Hv9-- ------------=_1427799422-26914-1-- From unknown Sun Jun 22 22:48:01 2025 X-Loop: help-debbugs@gnu.org Subject: bug#19760: [bug] "tail -f" with inotify fails to follow a file after a rename() Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 31 Mar 2015 11:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 19760 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: Bernhard Voelker , Stephane Chazelas , 19760@debbugs.gnu.org Received: via spool by 19760-submit@debbugs.gnu.org id=B19760.14278017613576 (code B ref 19760); Tue, 31 Mar 2015 11:36:02 +0000 Received: (at 19760) by debbugs.gnu.org; 31 Mar 2015 11:36:01 +0000 Received: from localhost ([127.0.0.1]:40962 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YcuSe-0000va-Js for submit@debbugs.gnu.org; Tue, 31 Mar 2015 07:36:00 -0400 Received: from mail6.vodafone.ie ([213.233.128.184]:27168) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YcuSb-0000vG-HU for 19760@debbugs.gnu.org; Tue, 31 Mar 2015 07:35:58 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhcMAIKFGlVtTW5N/2dsb2JhbABcgwaCYIEUySUCgT1MAQEBAQEBfUEBg1MBBTIBVgsNARMWDwkDAgECAUUGAQwIAQGILwG5XZM1DCCLKYR/hC0BBKF2jRwigjKBPT2CdAEBAQ Received: from unknown (HELO localhost.localdomain) ([109.77.110.77]) by mail3.vodafone.ie with ESMTP; 31 Mar 2015 12:35:50 +0100 Message-ID: <551A8695.7060802@draigBrady.com> Date: Tue, 31 Mar 2015 12:35:49 +0100 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 References: <20150203220411.GA6648@chaz.gmail.com> <54D15A1A.6020704@draigBrady.com> <551A114F.9040009@draigBrady.com> <551A3EF9.7000304@bernhard-voelker.de> <551A7D68.4020901@draigBrady.com> In-Reply-To: <551A7D68.4020901@draigBrady.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) BTW given that -f was broken for so long (6 years) it lends more weight to making -f behave like -F by default. Note POSIX allows (and even implies this), and openBSD -f behaves like -F for example. Not something appropriate for coreutils 8.x, and I'd be 60:40 against changing in a later major release, but it's worth mentioning. cheers, Pádraig. From unknown Sun Jun 22 22:48:01 2025 X-Loop: help-debbugs@gnu.org Subject: bug#19760: [bug] "tail -f" with inotify fails to follow a file after a rename() Resent-From: Bob Proulx Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Wed, 01 Apr 2015 20:24:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 19760 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: 19760@debbugs.gnu.org Cc: Stephane Chazelas Received: via spool by 19760-submit@debbugs.gnu.org id=B19760.14279197976800 (code B ref 19760); Wed, 01 Apr 2015 20:24:02 +0000 Received: (at 19760) by debbugs.gnu.org; 1 Apr 2015 20:23:17 +0000 Received: from localhost ([127.0.0.1]:42704 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YdPAT-0001lc-CZ for submit@debbugs.gnu.org; Wed, 01 Apr 2015 16:23:17 -0400 Received: from joseki.proulx.com ([216.17.153.58]:57741) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YdPAQ-0001lS-85 for 19760@debbugs.gnu.org; Wed, 01 Apr 2015 16:23:15 -0400 Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 2616B21230; Wed, 1 Apr 2015 14:23:09 -0600 (MDT) Received: by hysteria.proulx.com (Postfix, from userid 1000) id F08172DC35; Wed, 1 Apr 2015 14:23:08 -0600 (MDT) Date: Wed, 1 Apr 2015 14:23:08 -0600 From: Bob Proulx Message-ID: <20150401140626645437479@bob.proulx.com> References: <20150203220411.GA6648@chaz.gmail.com> <54D15A1A.6020704@draigBrady.com> <551A114F.9040009@draigBrady.com> <551A3EF9.7000304@bernhard-voelker.de> <551A7D68.4020901@draigBrady.com> <551A8695.7060802@draigBrady.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <551A8695.7060802@draigBrady.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Pádraig Brady wrote: > BTW given that -f was broken for so long (6 years) In retrospect the inotify support, while appearing nifty, has been the source of many bugs and problems for such a simple utility. It certainly destabilized it. > it lends more weight to making -f behave like -F by default. > > Note POSIX allows (and even implies this), > and openBSD -f behaves like -F for example. I think the reason this problem isn't often noticed is because once a file has been renamed it usually is no longer written anymore. This makes the behavior be mostly invisible in the typical case. Except for the cases where it isn't and the people count on it working as expected and it isn't. Then it is a shame that such a simple utility is misbehaving. > Not something appropriate for coreutils 8.x, > and I'd be 60:40 against changing in a later major release, > but it's worth mentioning. I use the -F behavior often but I think changing to -F behavior by default actually makes it more complicated. For example the -f behavior is that tail follows a file. And if I rename the file? Then it follows the original file. If I unlink the file? Then it follows the original file. No complexity. Simple. Does what it says. (Or at least should.) The -F is more complicated and needs explanation for how each of those cases is handled. Bob