From unknown Sun Jun 22 08:08:26 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67490: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Resent-From: dann frazier Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Mon, 27 Nov 2023 17:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 67490 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: 67490@debbugs.gnu.org X-Debbugs-Original-To: bug-coreutils@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.170110622024166 (code B ref -1); Mon, 27 Nov 2023 17:31:02 +0000 Received: (at submit) by debbugs.gnu.org; 27 Nov 2023 17:30:20 +0000 Received: from localhost ([127.0.0.1]:44715 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7fR4-0006Hh-Us for submit@debbugs.gnu.org; Mon, 27 Nov 2023 12:30:20 -0500 Received: from lists.gnu.org ([2001:470:142::17]:53992) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7ePy-0004UP-6x for submit@debbugs.gnu.org; Mon, 27 Nov 2023 11:25:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7ePm-0004wl-Db for bug-coreutils@gnu.org; Mon, 27 Nov 2023 11:24:54 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7ePk-0005Bj-I9 for bug-coreutils@gnu.org; Mon, 27 Nov 2023 11:24:54 -0500 Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 434003F11F for ; Mon, 27 Nov 2023 16:24:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701102289; bh=eRxRZp9frYjOz8Y7px/sfeG47wN3RQH42qLQTHKYPjs=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=GCdo+X3H+exnv3jU81J1FrF0vgiafqSrtdgofU/r/Z3u6Wr9f+jKBqsjGZZrp//7t JsU2BI5YDdtD3r//noATHRyaO32ncZW+GW68Cv3eqbxYYkBoJO0pbd+vKL556iScyo qHB9DXxRKH/4N0lk6Zeu/JkDa2asJ09wSSopF6QK3GsH0Yfosj1DDe0tH+DwndMi6+ wMs/wxc7UTVgfrYafvclBiwUD+wx398SgGmViJdpTgRByXpjWMzI9frkNrRml5vMjT Zy+5rk8eNweNf8jL5Eng5nWP9ZxDScwn6hq0p/37LP84OGuLE2AUhOO2AXooItXRw1 lBJAKfVA4VBJQ== Received: by mail-io1-f70.google.com with SMTP id ca18e2360f4ac-7b39a5cc776so157294039f.2 for ; Mon, 27 Nov 2023 08:24:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701102287; x=1701707087; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eRxRZp9frYjOz8Y7px/sfeG47wN3RQH42qLQTHKYPjs=; b=fAJGtks7MtuAa2yEH8ybtdFJZn3k94JzAE7NdnJJln3CXccYaTyjGDDq5P65AeG9Ld 3hWHijXC13GN6zelZnACo4o4xt07q3DGFT8hZwZqCQrPvM9GSv/WB8fVD+0qo/rxeoQ0 cLPt/gPhateyNBKd0xmORmQ5a9mXPMvsj7CBV7BUFKeOEXngq3dKZT2FqRzEcUe/MwNj r4mLtNTYdSe5AA4JY+c/cUXZp2RxzLDpZI5ZwKm8nLddRicdAbPW0GfkWpD0zUblG4j0 0IyBz6BZjnqthaXt3SdQ24pU3txyLUlmibyLYwvxLqFXUFMq+mhwzjYwc2Js9ga6niLq 5/yQ== X-Gm-Message-State: AOJu0YwBKD6BaLQjZ2s9DmVQmHVvN4yPEO78Bj18Xso6XyEhv5UANtlE +MeHq4AziXF9wbzQyjfFOJ3beRWpvkjVMnL3Wd3bRk1gwnHVoZMjPLCgWRFKw66J8eHkQfhduz8 qnS/QFbDAAu0vCle5GUMGokbW1LKMacTiZczbPSQw X-Received: by 2002:a5d:850f:0:b0:7ac:cb6b:616a with SMTP id q15-20020a5d850f000000b007accb6b616amr7495310ion.8.1701102287671; Mon, 27 Nov 2023 08:24:47 -0800 (PST) X-Google-Smtp-Source: AGHT+IGgrK7+UeWHO4J9D3FhIZyrgR96OJ4om7eFAiMil5VyqS/tWG8ootifE2nZqofVxygA2mUT5A== X-Received: by 2002:a5d:850f:0:b0:7ac:cb6b:616a with SMTP id q15-20020a5d850f000000b007accb6b616amr7495295ion.8.1701102287386; Mon, 27 Nov 2023 08:24:47 -0800 (PST) Received: from localhost ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id b13-20020a02a58d000000b00466b9ec90ffsm831595jam.115.2023.11.27.08.24.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Nov 2023 08:24:46 -0800 (PST) From: dann frazier Date: Mon, 27 Nov 2023 09:24:33 -0700 Message-ID: <20231127162433.924238-1-dann.frazier@canonical.com> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=185.125.188.122; envelope-from=dann.frazier@canonical.com; helo=smtp-relay-internal-0.canonical.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Mailman-Approved-At: Mon, 27 Nov 2023 12:30:17 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) * src/tail.c (file_lines): Use fstat() to determine a file's block size and dynamically allocate a buffer of that size for traversing backwards. --- src/tail.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/src/tail.c b/src/tail.c index c45f3b65a..437a38204 100644 --- a/src/tail.c +++ b/src/tail.c @@ -518,18 +518,30 @@ static bool file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, off_t start_pos, off_t end_pos, uintmax_t *read_pos) { - char buffer[BUFSIZ]; + char *buffer; size_t bytes_read; + blksize_t bufsize; off_t pos = end_pos; + bool ok = true; + struct stat stats; if (n_lines == 0) return true; + if (fstat (fd, &stats) != 0) + { + error (0, errno, _("cannot fstat %s"), quoteaf (pretty_filename)); + return false; + } + + bufsize = ST_BLKSIZE (stats); + buffer = xmalloc (bufsize); + /* Set 'bytes_read' to the size of the last, probably partial, buffer; 0 < 'bytes_read' <= 'BUFSIZ'. */ - bytes_read = (pos - start_pos) % BUFSIZ; + bytes_read = (pos - start_pos) % bufsize; if (bytes_read == 0) - bytes_read = BUFSIZ; + bytes_read = bufsize; /* Make 'pos' a multiple of 'BUFSIZ' (0 if the file is short), so that all reads will be on block boundaries, which might increase efficiency. */ pos -= bytes_read; @@ -538,7 +550,8 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; @@ -565,7 +578,7 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xwrite_stdout (nl + 1, bytes_read - (n + 1)); *read_pos += dump_remainder (false, pretty_filename, fd, end_pos - (pos + bytes_read)); - return true; + goto free_buffer; } } @@ -577,23 +590,26 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xlseek (fd, start_pos, SEEK_SET, pretty_filename); *read_pos = start_pos + dump_remainder (false, pretty_filename, fd, end_pos); - return true; + goto free_buffer; } - pos -= BUFSIZ; + pos -= bufsize; xlseek (fd, pos, SEEK_SET, pretty_filename); - bytes_read = safe_read (fd, buffer, BUFSIZ); + bytes_read = safe_read (fd, buffer, bufsize); if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; } while (bytes_read > 0); - return true; +free_buffer: + free (buffer); + return ok; } /* Print the last N_LINES lines from the end of the standard input, -- 2.42.0 From unknown Sun Jun 22 08:08:26 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67490: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Mon, 27 Nov 2023 19:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67490 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: dann frazier , 67490@debbugs.gnu.org Received: via spool by 67490-submit@debbugs.gnu.org id=B67490.170111224413054 (code B ref 67490); Mon, 27 Nov 2023 19:11:02 +0000 Received: (at 67490) by debbugs.gnu.org; 27 Nov 2023 19:10:44 +0000 Received: from localhost ([127.0.0.1]:44800 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7h0G-0003OU-FY for submit@debbugs.gnu.org; Mon, 27 Nov 2023 14:10:44 -0500 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]:50533) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7h0E-0003OC-PS for 67490@debbugs.gnu.org; Mon, 27 Nov 2023 14:10:43 -0500 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-4079ed65582so32403415e9.1 for <67490@debbugs.gnu.org>; Mon, 27 Nov 2023 11:10:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701112229; x=1701717029; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=0YAj7wn8KHnxKyBnPpSSUB68xLzyu/SbuHY3ivUhmCA=; b=hzgm350OUiJC1h90WVDORMAxAP5r8tAwZTV2veThe3FVyPNIplYjojTUPQ5JoKT7mk h9pCOLOY2NE/ZkObHmd7IzumI9UZz6hR++GvUHoCg2IWZfhnfc6SSPvo2A3iOzgPVkT/ CZk247fcRtF5cLLIcY2Qs/sEmI+DfM/ChGoz3aLU3Ucy0fO4ipxfruspwCKuJTtYVX3I gMKyFQ13ZlufzL8tDgWkEJGSsRePc1IQBVGXVijmHzGMocXyTIN/vrK65Zytkpxqgcp/ m48f21nBAkt8dovRMu28CvZ0/PgqjsOtb/Ew1WE47zQRjGPEWPu8i8D8ZWzKShkvRj1r D/KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701112229; x=1701717029; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0YAj7wn8KHnxKyBnPpSSUB68xLzyu/SbuHY3ivUhmCA=; b=gBK93O1jrCILjzyjtyBxpSwEMb+797w5z6lOmfO+js72dgXKtKWvN6Ykr0dEFJFlO9 4WjqB5ccUAbkDPGL8sPtJiYg9Wdp4SrZkco0pr4nFXAvWuc9C1KOlZy1SfS3EmQL7BND pHTiFESMjTlMVff1AcE+aME1BrjqsbCS/EVv6og5ACriOCgUgaQpxtapEwmRTgbypFDw Hs2QKc5HIOQth7hck/l/3ZVFgs6xoPkS38Lnu7prMJpoUV3cryPhL9Aw6/EcNj6H5Cqd eQVH0kt06QjbSMKSF8LphC0bP1NxoSL8LF8FafkeE2g5mBLw/xcjDihcMYko/Tqf13Q1 yrMg== X-Gm-Message-State: AOJu0YzReRsvxae5EqwaVidj3IxDyMn4e4b3ZPWpuxhr/SDYgRr7lYlT ptSWvfwUpZbQvKpho4JdC8A= X-Google-Smtp-Source: AGHT+IE9eJKSbZ5K4nhwrmmkyFKZL79TJt+7WkLrIEuzi+g49GDk/ju7tzIZiR8MLKFs/XsdOpd5CA== X-Received: by 2002:a05:600c:1f8c:b0:409:50b4:3da8 with SMTP id je12-20020a05600c1f8c00b0040950b43da8mr10034602wmb.38.1701112228971; Mon, 27 Nov 2023 11:10:28 -0800 (PST) Received: from [192.168.1.20] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id g21-20020a05600c311500b0040b3645a7c2sm14894482wmo.40.2023.11.27.11.10.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 27 Nov 2023 11:10:28 -0800 (PST) Message-ID: Date: Mon, 27 Nov 2023 19:10:26 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US References: <20231127162433.924238-1-dann.frazier@canonical.com> From: =?UTF-8?Q?P=C3=A1draig?= Brady In-Reply-To: <20231127162433.924238-1-dann.frazier@canonical.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 27/11/2023 16:24, dann frazier wrote: > * src/tail.c (file_lines): Use fstat() to determine a file's block > size and dynamically allocate a buffer of that size for traversing > backwards. Thanks for the patch. Could you describe it a bit more. What happens if we use smaller reads? Also what about all the other safe_read() calls in tail.c ? thanks, Pádraig From unknown Sun Jun 22 08:08:26 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67490: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Mon, 27 Nov 2023 19:19:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67490 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: dann frazier , 67490@debbugs.gnu.org Received: via spool by 67490-submit@debbugs.gnu.org id=B67490.170111269213871 (code B ref 67490); Mon, 27 Nov 2023 19:19:01 +0000 Received: (at 67490) by debbugs.gnu.org; 27 Nov 2023 19:18:12 +0000 Received: from localhost ([127.0.0.1]:44805 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7h7U-0003bf-CK for submit@debbugs.gnu.org; Mon, 27 Nov 2023 14:18:12 -0500 Received: from mail.cs.ucla.edu ([131.179.128.66]:41520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7h7S-0003bP-FH for 67490@debbugs.gnu.org; Mon, 27 Nov 2023 14:18:11 -0500 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id D4D003C011BDC; Mon, 27 Nov 2023 11:17:57 -0800 (PST) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 4rGk9X-Ze9iW; Mon, 27 Nov 2023 11:17:57 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 8F17C3C011BDD; Mon, 27 Nov 2023 11:17:57 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu 8F17C3C011BDD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1701112677; bh=kNtnop5Ip0hgnW64ie6+6p4rVyoex/CSWE+n9Yzg5hE=; h=Message-ID:Date:MIME-Version:To:From; b=WFM3S+j03tbRrLVznmomkS0JBmVj/PzyeHAx7YKbcb4GhAve/mRFvP3NMbsxkKSCs dS3jzqPZ84qYI7QXTK9Qg8SXkcS/YsBCdJq34W6htBINnmzabLNn/JtvFGL+nE4yxd NA7gdA2XyGcYeDpMlIWmzERO/ZuG5LnVY6QcaJxu9v12SZ4StBV+kV+P566SrYRlca 6azgEI0U+PavL2VydVPIkUk73PsarreTFBWxf2TLzgZVbZi+RO02FAxoMPxSFan//W S/0pliHdZbs3lN129I4qiro4ch4Xus1qiWpSWfZy3v3WuxsFpBtMnRZUFAYp0jTuJ0 MaXk1Pw9121Pg== X-Virus-Scanned: amavisd-new at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ZN7lw7DgaikP; Mon, 27 Nov 2023 11:17:57 -0800 (PST) Received: from [192.168.254.12] (unknown [47.148.192.211]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id 6EB633C011BDC; Mon, 27 Nov 2023 11:17:57 -0800 (PST) Message-ID: Date: Mon, 27 Nov 2023 11:17:57 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US References: <20231127162433.924238-1-dann.frazier@canonical.com> From: Paul Eggert Organization: UCLA Computer Science Department In-Reply-To: <20231127162433.924238-1-dann.frazier@canonical.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 2023-11-27 08:24, dann frazier wrote: > + if (fstat (fd, &stats) != 0) > + { > + error (0, errno, _("cannot fstat %s"), quoteaf (pretty_filename)); > + return false; > + } > + > + bufsize = ST_BLKSIZE (stats); If fstat fails, that's no reason to exit. Just use bufsize = BUFSIZ and keep going. fstat can fail for reasons unrelated to what 'tail' needs (e.g., time_t overflow). From unknown Sun Jun 22 08:08:26 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67490: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Resent-From: dann frazier Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Wed, 29 Nov 2023 01:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67490 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: Paul Eggert Cc: 67490@debbugs.gnu.org Received: via spool by 67490-submit@debbugs.gnu.org id=B67490.170122309319721 (code B ref 67490); Wed, 29 Nov 2023 01:59:02 +0000 Received: (at 67490) by debbugs.gnu.org; 29 Nov 2023 01:58:13 +0000 Received: from localhost ([127.0.0.1]:48503 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r89q5-00057y-S0 for submit@debbugs.gnu.org; Tue, 28 Nov 2023 20:58:13 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]:56772) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r88bd-00035E-H2 for 67490@debbugs.gnu.org; Tue, 28 Nov 2023 19:39:13 -0500 Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 0CC933F078 for <67490@debbugs.gnu.org>; Wed, 29 Nov 2023 00:38:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701218336; bh=f5gli4jc9Yl9WHmKn49y+95WIkbAcsWqYqJDFjgWO+w=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=NGkrTqX3h/MIlkbDqMWdCQcXkLrVfKzCLDHGy94Dm8umrJyaxeL9kaITteFurdVQ3 /G1nj7q9zBRhzJuB3IRuoytUvCqVx50YNiHuBlPEn9/kLHrNNiFjNlh+Kv01jGlWbK lshwxlmqqJAAm4wiyfntoUvNCO2gPycl8HVSHVnoADscOjG8OoZ4YgE5gAMfmgPS9e SncGHwK65nY6GQGgKjts78SEPY4l4joIDXzaD2t37CL7LlQYgoQ/VKiAiPMxP5MmJa tBRpTTkKaVfMc78qzOfFS3huTXQz7vqujYUwHlndtB74yNgm4jbqXYSd4p7+beGr3m pDYMrTKygahrA== Received: by mail-io1-f72.google.com with SMTP id ca18e2360f4ac-7b36a881314so565185439f.2 for <67490@debbugs.gnu.org>; Tue, 28 Nov 2023 16:38:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701218335; x=1701823135; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=f5gli4jc9Yl9WHmKn49y+95WIkbAcsWqYqJDFjgWO+w=; b=EbSmF0kDizQ6K+g1UK7NnCkS4aqPEHuBv85RPq54DBed6TJrdPlKY222+KTqz3t0l1 3QnI1/fuNnFFsOKFxU3SIP9S02FpFFIDQtVaDtXSX2bTq6Wlem+ij/k/jRt4iVvsOarJ CLR75oeLb+pGGV6AaOxynxHoyJndkGrTO+/cIx54Qx57swxsg4DrZs2txWH8plgfIEPz hn6edHnoI93V5/sx31ORhP9WKAe3CUYBNh5k99fTMi1tjQJDoHeFHA5OjIEkBODDzMym gp2ZO4iTHF9rrEc2KJ9CNJflnE3dfgr52PJwPNEDr/Z5p6L5W89hOybimmvTIbWyeBPK Q+cA== X-Gm-Message-State: AOJu0Yy+bOOQj+uj1naPg6IOejTxoll8yTuj47vwtKuf3ez/zI8Wfumm hTh0FhqoIcE9bNsYf6/zD8ov9pWpE3UzC9KyT/XjBmZjgVKsYwyHznH+HUpc3sm/+OTBD/0ElXp pDLyL3BEuPOX1QTx8/EL8yfj+0sxc7Vlr X-Received: by 2002:a6b:ce16:0:b0:7b0:a8d7:3bb3 with SMTP id p22-20020a6bce16000000b007b0a8d73bb3mr18748450iob.16.1701218334810; Tue, 28 Nov 2023 16:38:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IHWN1zfHjPARh9wPOkVftmQ1rnmoNUAASnEkcdsBkNXQV6RBD1wtMGdfX3D0BggtWYparzsJA== X-Received: by 2002:a6b:ce16:0:b0:7b0:a8d7:3bb3 with SMTP id p22-20020a6bce16000000b007b0a8d73bb3mr18748435iob.16.1701218334478; Tue, 28 Nov 2023 16:38:54 -0800 (PST) Received: from xps13.dannf ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id m21-20020a0566380ad500b0042b3e04510asm3164896jab.98.2023.11.28.16.38.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Nov 2023 16:38:54 -0800 (PST) Date: Tue, 28 Nov 2023 17:38:52 -0700 From: dann frazier Message-ID: References: <20231127162433.924238-1-dann.frazier@canonical.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Score: -2.3 (--) X-Mailman-Approved-At: Tue, 28 Nov 2023 20:58:08 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.8 (-) On Mon, Nov 27, 2023 at 11:17:57AM -0800, Paul Eggert wrote: > On 2023-11-27 08:24, dann frazier wrote: > > + if (fstat (fd, &stats) != 0) > > + { > > + error (0, errno, _("cannot fstat %s"), quoteaf (pretty_filename)); > > + return false; > > + } > > + > > + bufsize = ST_BLKSIZE (stats); > > > If fstat fails, that's no reason to exit. Just use bufsize > = BUFSIZ and keep going. fstat can fail for reasons > unrelated to what 'tail' needs (e.g., time_t overflow). Thanks for the suggestion - I'll make that change. -dann From unknown Sun Jun 22 08:08:26 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67490: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Resent-From: dann frazier Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Wed, 29 Nov 2023 14:35:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67490 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: =?UTF-8?Q?P=C3=A1draig?= Brady Cc: 67490@debbugs.gnu.org Received: via spool by 67490-submit@debbugs.gnu.org id=B67490.170126849310637 (code B ref 67490); Wed, 29 Nov 2023 14:35:02 +0000 Received: (at 67490) by debbugs.gnu.org; 29 Nov 2023 14:34:53 +0000 Received: from localhost ([127.0.0.1]:49248 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8LeP-0002lV-Cc for submit@debbugs.gnu.org; Wed, 29 Nov 2023 09:34:53 -0500 Received: from smtp-relay-internal-1.canonical.com ([185.125.188.123]:41406) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8LeM-0002lI-TR for 67490@debbugs.gnu.org; Wed, 29 Nov 2023 09:34:52 -0500 Received: from mail-il1-f199.google.com (mail-il1-f199.google.com [209.85.166.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id B710440185 for <67490@debbugs.gnu.org>; Wed, 29 Nov 2023 14:34:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701268475; bh=i529c8yrbVZwM2nRyE2LeuuJOwkeAbFAIbNSDRP/NNA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=SQz/LBWRj9h2lcvLREOnyVd0ROePg8PdftktDeZB3PMXEt7t5AtbeFJKscxM10YeM WNh7VaRGX+PN0iYFqvLEMEw57nn6VbpYvucM4JKQ6YADaAo0CEVTUkR/oaIbWOXKYW CEoCff1hWCmryXF59rywS6X7s9dEW1VnVtqoDd89oJGQ+TrAzj1jqcwPeBtBSP4Uvo uCuuKDj3BqY9nbshEKjCm2qwJXzx4vm/vbqgn55dq3G3vuzluJLUUPeZxC4UQKFKVD zZLhq2MSwG+aadyt0wzLTXXqCPMxemWwwPiFE2J77QRdE14Apz+9SWxbsLMU7kRlSO rkqQWv4Tpe6hw== Received: by mail-il1-f199.google.com with SMTP id e9e14a558f8ab-35cc2083d25so36693825ab.2 for <67490@debbugs.gnu.org>; Wed, 29 Nov 2023 06:34:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701268469; x=1701873269; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=i529c8yrbVZwM2nRyE2LeuuJOwkeAbFAIbNSDRP/NNA=; b=fW5hZIyVCw19MofWL8lk06EggUNy+QwdMe/W63S2DVQYL1FsYputEZxCrdkRb1VVSF fUvzrQcH5BqG3VdpQt1ByuhKUa/6I6i8jPa6zN5pEaEK1SV8ddmqMDDuHr9WwR8XYutw dvNll/e7o8KKunigw05+c82Qa/F89VdyAM/8yjpId1HzaGY0B+reIWhK0krZ30p71lEE sfoGOgbI0CLtLxvz7kESTf7lmRGXTZzbHcN3346ifqD116meRJtFjOFDGMZ7wgSbJIyp Yfz2D2sW/yyxIA7RavrlzZpzMXdbRt/YOk0xbQM7eWUix1D6Kelg9SvjEhYdMWiM0aRj YlmQ== X-Gm-Message-State: AOJu0Yyr/eeRz8cZz1SzJOvoIyndr/KiAiqXA6WtUGSi68vEBTG1RPTW KxWEKRLTOujWCf2/Ubwv0743Hkb12Ub1e28KFxy9bsBvsaBc2XHgCGTDg1Zc+r78jyu93Y7htuj roT+rKZMzhiIyXQBbnHjdRJFnw7ZIHTJcKCGh751k X-Received: by 2002:a05:6e02:970:b0:35c:ca7c:ebb2 with SMTP id q16-20020a056e02097000b0035cca7cebb2mr12136251ilt.20.1701268469650; Wed, 29 Nov 2023 06:34:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IHQhpSu0q+/ul6T0zCGCb9tWzDGyDMUhoDYzLL/FZyEomZHfynSQQHQtdrV7jUKvI2UJ2wFMw== X-Received: by 2002:a05:6e02:970:b0:35c:ca7c:ebb2 with SMTP id q16-20020a056e02097000b0035cca7cebb2mr12136231ilt.20.1701268469270; Wed, 29 Nov 2023 06:34:29 -0800 (PST) Received: from xps13.dannf ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id s9-20020a056638218900b004667f079d40sm3563325jaj.60.2023.11.29.06.34.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Nov 2023 06:34:28 -0800 (PST) Date: Wed, 29 Nov 2023 07:34:26 -0700 From: dann frazier Message-ID: References: <20231127162433.924238-1-dann.frazier@canonical.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On Mon, Nov 27, 2023 at 07:10:26PM +0000, Pádraig Brady wrote: > On 27/11/2023 16:24, dann frazier wrote: > > * src/tail.c (file_lines): Use fstat() to determine a file's block > > size and dynamically allocate a buffer of that size for traversing > > backwards. > > Thanks for the patch. > Could you describe it a bit more. > What happens if we use smaller reads? > Also what about all the other safe_read() calls in tail.c ? Yes, this warrants more detail - I'll try to capture that in a comment. Thanks Pádraig! -dann From unknown Sun Jun 22 08:08:26 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels References: <20231127162433.924238-1-dann.frazier@canonical.com> In-Reply-To: <20231127162433.924238-1-dann.frazier@canonical.com> Resent-From: dann frazier Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Thu, 30 Nov 2023 01:34:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67490 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: 67490@debbugs.gnu.org Received: via spool by 67490-submit@debbugs.gnu.org id=B67490.17013079896012 (code B ref 67490); Thu, 30 Nov 2023 01:34:01 +0000 Received: (at 67490) by debbugs.gnu.org; 30 Nov 2023 01:33:09 +0000 Received: from localhost ([127.0.0.1]:51807 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8VvQ-0001Ys-M8 for submit@debbugs.gnu.org; Wed, 29 Nov 2023 20:33:09 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]:35218) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8VvN-0001YJ-PY for 67490@debbugs.gnu.org; Wed, 29 Nov 2023 20:33:07 -0500 Received: from mail-il1-f199.google.com (mail-il1-f199.google.com [209.85.166.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id AFC453F078 for <67490@debbugs.gnu.org>; Thu, 30 Nov 2023 01:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701307971; bh=nj6RJP8ZprDdFKxs2E3B6Ts6+T/c7UHkneqHovXDMv4=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=Uzc4NjnV0L/8tTiMB+TGC38q74npVpx3AS8nVdp+zUv+waZMMLEraHCtA2J8AoaBi IaXEe+DTqZ0B4+7deXhV5fhyViyWd/MY5LHybCeOhw+fCmsijekSkzyKtqsUSU6RZ5 MNpkDc9j+BFRsgVde7qITRa5C7H0uzbcIjRgdajMhvdjPrFk0IXsGQ88a6H71KzZsW RIOSs2L4EXCdzovfHFMeyhAG0C2KEVeFAmw7C+Yc2lDMuHyY/ZO5mvn2+M8yZGTslU w1TflqfO2YmEXh6Y6BiA/4392s5XbgDdUvW5jHytrWNxsYkokwgLQXOlSdtHY+oa56 P8rBlqp6hmPzw== Received: by mail-il1-f199.google.com with SMTP id e9e14a558f8ab-35ca1eadf5fso2582775ab.1 for <67490@debbugs.gnu.org>; Wed, 29 Nov 2023 17:32:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701307969; x=1701912769; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nj6RJP8ZprDdFKxs2E3B6Ts6+T/c7UHkneqHovXDMv4=; b=iw8jToGWsmJqvSD7Tjr22qWpQlYka7X/yhAak747uShOu5c4zVsfMn3cizfoDOdjX4 sugcfmGxBbSrdCDggJANGtJvzj578Xg8e+q7CpqLspd0PNgkmD8a7AuuDSWjBA3+gsx7 IQ6ux7xD2NCraQ/hN5W0dUG0gwxgzjBpE3oErNVybAlwHxiqCZLjU/K9XkNGvhjTHnIa lwBParBbKYyRQl1rZXODmFOJwvrr8BVHpjrYAJ7XOqdmzzBZjYpyB+A+/4PiEWM1yq1K +7B/qaPj5Pvq9lFglKKvibm0IRpx0RW6Ln9GKOOzHw2C4fZw72p1G+leoAUXE6lpp+nX D/HQ== X-Gm-Message-State: AOJu0YxmpoihRzoqLsK/+Fu6089Ug7ZG4kyPl5oUn+mCdDjl9YAJzZiX DA2UogMdcdldrO5K0FcLBCcmnWNpzW7bx9W/8HCIYcpchN6Ips71owTtdrwXa5UyYFTuaJchOLZ IfccmbWd6XVdFi5OgtLdBEfWEEqucuRs3jM/MNRpL X-Received: by 2002:a92:d305:0:b0:35c:76ff:f211 with SMTP id x5-20020a92d305000000b0035c76fff211mr17388331ila.0.1701307969015; Wed, 29 Nov 2023 17:32:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IGM3UzEgGReXxPYXASammAr2PN3FLinknmCWdzs0xIWlPOzKGjcOPnWi2yxqyi1aE1ugJjbyw== X-Received: by 2002:a92:d305:0:b0:35c:76ff:f211 with SMTP id x5-20020a92d305000000b0035c76fff211mr17388314ila.0.1701307968571; Wed, 29 Nov 2023 17:32:48 -0800 (PST) Received: from localhost ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id k16-20020a02a710000000b004666dff32d7sm22258jam.148.2023.11.29.17.32.48 for <67490@debbugs.gnu.org> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Nov 2023 17:32:48 -0800 (PST) From: dann frazier Date: Wed, 29 Nov 2023 18:32:34 -0700 Message-ID: <20231130013234.135285-1-dann.frazier@canonical.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Changes from v1: - Fallback to BUFSIZ if fstat fails instead of exiting. - Add a comment block to explain why the blksize is used in file_lines() (which I hope also clarifies why it is not needed elsewhere). - Only use blksize if it is > BUFSIZ (I had regressed to 4K reads, oops). - Update comments to reference bufsize instead of BUFSIZ. - Update subject because /proc is not affected, and it isn't limited to follow mode. - Add a test. * src/tail.c (file_lines): Consider a full blocksize when searching backwards to find sysfs file content on systems with large page sizes. * tests/tail/tail-1-sysfs.sh: Add a new test. * tests/local.mk: Reference the new test. Fixes https://bugs.gnu.org/67490 --- src/tail.c | 41 +++++++++++++++++++++++++++----------- tests/local.mk | 1 + tests/tail/tail-1-sysfs.sh | 31 ++++++++++++++++++++++++++++ 3 files changed, 61 insertions(+), 12 deletions(-) create mode 100755 tests/tail/tail-1-sysfs.sh diff --git a/src/tail.c b/src/tail.c index c45f3b65a..f3219757d 100644 --- a/src/tail.c +++ b/src/tail.c @@ -518,19 +518,32 @@ static bool file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, off_t start_pos, off_t end_pos, uintmax_t *read_pos) { - char buffer[BUFSIZ]; + char *buffer; size_t bytes_read; + blksize_t bufsize = BUFSIZ; off_t pos = end_pos; + bool ok = true; + struct stat stats; if (n_lines == 0) return true; + if (fstat (fd, &stats) == 0) + /* sysfs files have a blksize == kernel page size, which can be > BUFSIZ. + lseek SEEK_END moves to the next page, which is likely more than + BUFSIZ away from the end of the actual data. Use a buffer large enough + to hold an entire page, so that the window we search for data is large + enough to find it. */ + bufsize = MAX(BUFSIZ, ST_BLKSIZE (stats)); + + buffer = xmalloc (bufsize); + /* Set 'bytes_read' to the size of the last, probably partial, buffer; - 0 < 'bytes_read' <= 'BUFSIZ'. */ - bytes_read = (pos - start_pos) % BUFSIZ; + 0 < 'bytes_read' <= 'bufsize'. */ + bytes_read = (pos - start_pos) % bufsize; if (bytes_read == 0) - bytes_read = BUFSIZ; - /* Make 'pos' a multiple of 'BUFSIZ' (0 if the file is short), so that all + bytes_read = bufsize; + /* Make 'pos' a multiple of 'bufsize' (0 if the file is short), so that all reads will be on block boundaries, which might increase efficiency. */ pos -= bytes_read; xlseek (fd, pos, SEEK_SET, pretty_filename); @@ -538,7 +551,8 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; @@ -565,7 +579,7 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xwrite_stdout (nl + 1, bytes_read - (n + 1)); *read_pos += dump_remainder (false, pretty_filename, fd, end_pos - (pos + bytes_read)); - return true; + goto free_buffer; } } @@ -577,23 +591,26 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xlseek (fd, start_pos, SEEK_SET, pretty_filename); *read_pos = start_pos + dump_remainder (false, pretty_filename, fd, end_pos); - return true; + goto free_buffer; } - pos -= BUFSIZ; + pos -= bufsize; xlseek (fd, pos, SEEK_SET, pretty_filename); - bytes_read = safe_read (fd, buffer, BUFSIZ); + bytes_read = safe_read (fd, buffer, bufsize); if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; } while (bytes_read > 0); - return true; +free_buffer: + free (buffer); + return ok; } /* Print the last N_LINES lines from the end of the standard input, diff --git a/tests/local.mk b/tests/local.mk index a5fb62d96..af41337b1 100644 --- a/tests/local.mk +++ b/tests/local.mk @@ -257,6 +257,7 @@ all_tests = \ tests/seq/seq-precision.sh \ tests/head/head.pl \ tests/head/head-elide-tail.pl \ + tests/tail/tail-1-sysfs.sh \ tests/tail/tail-n0f.sh \ tests/ls/ls-misc.pl \ tests/date/date.pl \ diff --git a/tests/tail/tail-1-sysfs.sh b/tests/tail/tail-1-sysfs.sh new file mode 100755 index 000000000..078366588 --- /dev/null +++ b/tests/tail/tail-1-sysfs.sh @@ -0,0 +1,31 @@ +#!/bin/sh +# sysfs files have weird properties that can be influenced by page size + +# Copyright 2023 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src +print_ver_ tail + +file=/sys/kernel/profiling + +if test -r $file; then + cp -f $file exp || framework_failure_ + tail -1 $file > out || fail=1 + + compare exp out || fail=1 +fi + +Exit $fail -- 2.43.0 From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 29 20:34:41 2023 Received: (at control) by debbugs.gnu.org; 30 Nov 2023 01:34:41 +0000 Received: from localhost ([127.0.0.1]:51812 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8Vwv-0001bT-Jl for submit@debbugs.gnu.org; Wed, 29 Nov 2023 20:34:41 -0500 Received: from smtp-relay-internal-1.canonical.com ([185.125.188.123]:44512) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8Vws-0001bB-Iw for control@debbugs.gnu.org; Wed, 29 Nov 2023 20:34:39 -0500 Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 85BFF40185 for ; Thu, 30 Nov 2023 01:34:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701308064; bh=srTqsuws6Mktg8R/C+woPM9Y8VQwdWqsdZbDT280OtE=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=OUluaQxnNfpYdlDMVahye3l0NqSuCR6nqBxjNoWMQwQePmua0eODZMWo0ZQNOasJ2 gJPWXlQ7+aerNitmO8mU82CMHb+0FbRLJfu3UQDX+4E1S0smdDkjT7ItAWS/W2TOwR NefGAcQVo4TnhEDbOH5EIBkln2QY6bWFIve1fpHZLFkif6p+di01LG+bFA3XfZHvUr GSv1x4XUPHqyNkUJd+GK/z4UZZ76xLuHjXGte7HjsbD1ulaZa6hMGNqz0dCFPGw7N+ aLzLAkUfY3bpg7iSfO2JhWFuzoYtzH3urdcHEIefrlzf4iRsAKnCqQGtFg6/ztHXhM xckb+pkvDoKxg== Received: by mail-io1-f72.google.com with SMTP id ca18e2360f4ac-7b39e0d8185so30658439f.0 for ; Wed, 29 Nov 2023 17:34:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701308062; x=1701912862; h=content-disposition:mime-version:message-id:subject:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=srTqsuws6Mktg8R/C+woPM9Y8VQwdWqsdZbDT280OtE=; b=EKVOtwdj0BoJxps3qE4hFbnSE6T9AEMzL92cNtwCwN03vJnYZz/Pnm+8mBCbS2P7nu pz0FBwCiEaCaWukbCF0cPhtm41lK9sn4tVKNf0Qx6AyFzNfPrm/cIiIbqjs/rf9s7WuD J0j9Q5XamNQu+LQYCTZIcMTLQ7OFmyZiqY1gVS8LyCec5XwgAQaWa78FqTOclAClQ70T u0By7Owk3qdgezFnSw6PKoCBCQiPfEzfpDFGdePmb4n9VILm7d1s086rxY5qMiZmw8bI MejRJvADYD3BO6iaIRO/rclTqsYsK92oM4ovricRcIaJktW6a4Rm/ez4Pl18NxPsQe3y RQKA== X-Gm-Message-State: AOJu0YzglowopUF8qaNuLJ90gxX60w055HzcDsuDPv/ukGWkmKfhsPSV 7awlVXuPBuIuI709eHT2i9b5J9S7T4qUHzedRef2aDfps7L24BN3F9Fyte+H8kWt3M2snsmMt/J i0VW96GS9Iyx7Aksbh7htBJFt0KcHkE7hbOYDVdpiwrI= X-Received: by 2002:a05:6e02:214f:b0:35c:ca58:645 with SMTP id d15-20020a056e02214f00b0035cca580645mr12621493ilv.13.1701308062451; Wed, 29 Nov 2023 17:34:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IGgMJktmYITwJW9JZVtneROcNT6jd1F9WZD7f25fn6hZPOnDinRsuikJ8DdptUnhf25CGGqkg== X-Received: by 2002:a05:6e02:214f:b0:35c:ca58:645 with SMTP id d15-20020a056e02214f00b0035cca580645mr12621487ilv.13.1701308062167; Wed, 29 Nov 2023 17:34:22 -0800 (PST) Received: from xps13.dannf ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id o17-20020a92d4d1000000b0035d249ed77csm43437ilm.35.2023.11.29.17.34.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Nov 2023 17:34:21 -0800 (PST) Date: Wed, 29 Nov 2023 18:34:19 -0700 From: dann frazier To: control@debbugs.gnu.org Subject: retitle Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.8 (-) retitle 67490 [PATCH v2] tail: fix tailing sysfs files on large page kernels thanks From unknown Sun Jun 22 08:08:26 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Thu, 30 Nov 2023 20:13:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67490 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: dann frazier , 67490@debbugs.gnu.org Received: via spool by 67490-submit@debbugs.gnu.org id=B67490.17013751354114 (code B ref 67490); Thu, 30 Nov 2023 20:13:01 +0000 Received: (at 67490) by debbugs.gnu.org; 30 Nov 2023 20:12:15 +0000 Received: from localhost ([127.0.0.1]:54539 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8nOP-00014H-JB for submit@debbugs.gnu.org; Thu, 30 Nov 2023 15:12:15 -0500 Received: from mail-wr1-x42e.google.com ([2a00:1450:4864:20::42e]:50547) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8nOJ-00013z-9z for 67490@debbugs.gnu.org; Thu, 30 Nov 2023 15:12:11 -0500 Received: by mail-wr1-x42e.google.com with SMTP id ffacd0b85a97d-332e40315bdso998362f8f.1 for <67490@debbugs.gnu.org>; Thu, 30 Nov 2023 12:11:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701375113; x=1701979913; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=eD+kjAHp+4EIebzmLwHQK4ZFlL0pGUnKFIiwQMyP/ZE=; b=aew6DdH3WEwoQiAi6EHSmLUPmaHloATiwbcK2LZLAqXk6eGSj232/NPN5ruOcmHyKz 9+vNOVw9lb02HcR8odbmCFRUF8a6rY9qsGvR6cDzQ6ueNClyqE3pY6OTU/ZodJH0ftRJ huWqe1UyyCGuiKyzazWXltHKdPPfxeRERgfgbZM0MiHKPCRLdW3OFr+KB2vox97aDsUa tTXGylQRyxSAHR6ZkBdGpHE5FLx9AKyFpNPKFB0lICA/xirtwHspBCXmg3N3S2V2sN7M wxcAWfK7f7woL8XzJNWlVyBQzYv9N34D1T4tBZfn/eoTvAAeHFWcc9SmAnxVG4fEYgaB tc/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701375113; x=1701979913; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=eD+kjAHp+4EIebzmLwHQK4ZFlL0pGUnKFIiwQMyP/ZE=; b=NjoxHbVn0FOhsQZEwVwKrymHC02IPpogAToFw9kDJfcP+h6GET8WKJWYW2QDx8zeSV nSgx/X9WIdYr5gGxIIB4jv39ikg+NqLVfZGjsHh0BUbOwJTuFQfjJ23MICLcIxtPUWwi wooyCq9FA9vneR1wJmVNqT6qWC0ys1hhIU6HFoLma1yQW77ac5grN6FSQyLwkUrf4Twj gJr2KPPKd3VS564WskMzfTCe8P9lyM+y4khBH67QKh5cou9bWGh1X4PNcvbglI38JX7W emtF12Nkt6cbseMaNw34B8peW9kkZ5ze2elpzQ63B41cz6BohYEi3gCcK0wN25fXn7FQ lpQQ== X-Gm-Message-State: AOJu0YyZZ+Fup1QpXJMC8QnteZgd+H1woY5hItw48CzGKtS7EsuUii+E yYDnYYLVmJF81kEZcMWcPzQ= X-Google-Smtp-Source: AGHT+IHG9X7jqh4xT8Tvrvi2nRhhRT8yasnY5BQE83ZeGpDl6PSc/qOZH2SIapM29H1RBKiOUsLizQ== X-Received: by 2002:a5d:618f:0:b0:32f:dd59:fd0 with SMTP id j15-20020a5d618f000000b0032fdd590fd0mr88039wru.52.1701375112921; Thu, 30 Nov 2023 12:11:52 -0800 (PST) Received: from [192.168.1.20] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id n10-20020a05600c4f8a00b004053e9276easm6675789wmq.32.2023.11.30.12.11.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Nov 2023 12:11:52 -0800 (PST) Message-ID: <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> Date: Thu, 30 Nov 2023 20:11:52 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US References: <20231127162433.924238-1-dann.frazier@canonical.com> <20231130013234.135285-1-dann.frazier@canonical.com> From: =?UTF-8?Q?P=C3=A1draig?= Brady In-Reply-To: <20231130013234.135285-1-dann.frazier@canonical.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) Much clearer thanks. On my system: $ stat /sys/kernel/profiling File: /sys/kernel/profiling Size: 4096 Blocks: 0 IO Block: 4096 regular file I can easily repro by setting the buffer size < PAGE_SIZE. So this patch handles the case where sysfs reports a file is a certain size, but it isn't really. In that case seeking to anywhere other than the start doesn't give an error, but reading returns nothing. So we use a buffer size large enough (>= PAGE_SIZE as inferred from st_blksize) so that we'll be reading from the start of the file in this case. Note st_blksize can have unusual values, so it might be better to use the io_blksize() wrapper to sanitize the values. Though that will generally give 128K, which is good when processing all of a file, but perhaps overkill when processing just the last part of a file. So perhaps it's better to use buffer_size = MAX (BUFSIZ, getpagesize ()) and avoid all the st_blksize edge cases. I'll think about it a little, and make the adjustments. thanks! Pádraig. From unknown Sun Jun 22 08:08:26 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels Resent-From: dann frazier Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Thu, 30 Nov 2023 20:38:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67490 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: =?UTF-8?Q?P=C3=A1draig?= Brady Cc: 67490@debbugs.gnu.org Received: via spool by 67490-submit@debbugs.gnu.org id=B67490.17013766676630 (code B ref 67490); Thu, 30 Nov 2023 20:38:01 +0000 Received: (at 67490) by debbugs.gnu.org; 30 Nov 2023 20:37:47 +0000 Received: from localhost ([127.0.0.1]:54546 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8nn8-0001ir-Lp for submit@debbugs.gnu.org; Thu, 30 Nov 2023 15:37:46 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]:57842) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8nn5-0001ie-W4 for 67490@debbugs.gnu.org; Thu, 30 Nov 2023 15:37:45 -0500 Received: from mail-io1-f69.google.com (mail-io1-f69.google.com [209.85.166.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 4995B3FA52 for <67490@debbugs.gnu.org>; Thu, 30 Nov 2023 20:37:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701376649; bh=pOd7/rOw35h8fnBEcstb9QOFdzjndMjV8hDXusnNWMA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=MyVRp666q7l03dAalS0ig8KGya7CorO9zxFhB45M048OlwERgthrTwzDyvR/bdqBf 4hfZKLn4qCrRCMtemC4IS0B9jeCqBf6imj53MkUJLasMuftaORtW1JR45Jl4zm2Ht+ aGQClpyeJTmB4+X+xOW0YWbG3/2PlmOJG7X07n8fcBWXtDSWl4jIKkJdCxNc03CFTC QNUOP3wYb58T93sjciTkNDsgL2lIUXE1ILeFMC23QhzZpEpIO4ZgzVXA2ENo80XHxW juS4EooYQAVeGnVJ1vyOgyDed4lJVwPusypkyJTF3HBbDckU90RjH5jY6fYQj1tjvM dwA5dU+agyPlA== Received: by mail-io1-f69.google.com with SMTP id ca18e2360f4ac-7b3f80e659aso126714839f.2 for <67490@debbugs.gnu.org>; Thu, 30 Nov 2023 12:37:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701376648; x=1701981448; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pOd7/rOw35h8fnBEcstb9QOFdzjndMjV8hDXusnNWMA=; b=n8tsDSabqfRDS8XDcFC/cZAysPICBAQlyDV1dHwH3E3WsDCIgogKgdwlppWD801RrO 3viGl1dc0tYC2ArOwSCf1fh4Zf/eIATcNhi5BFhonxSsWyZARxjzrdK3nEJmih9LMxDE IjwfmCwv88SiekWmgAVkh7ZhrMwzIeuvrWxOAd8ZqsiHV4FV+F9MFUUoxzDNdzZn7iTr x6gsxOneLJjl7NQ9HsAUJMqPvSYz8NlSZ2Hlvbt/CsiAAV08mgTCt7IUMutqsTAnDYNa 9X4tyzvx7edDkBi3PCvlXGtkBjzmt15ZAIeH/JOXOGcXJ/yNN2KSt/pCDkeaJcv8Rez+ FlQA== X-Gm-Message-State: AOJu0YxYktlwaBYk5kp5SrgsB8t7YyC9F8K+OPWB6DJ543twG4MRK8Ga vD6q3U2ZVGYu7lvon0q2IUkR5CXm1EgNdHIXg6zXunYjkDiy/7UXnAsrMjF7Kex4HPk+3WK9tWB 5NyanNcHjCi5eVmeUp0MkFfgXGrH3BL1N X-Received: by 2002:a5d:9c10:0:b0:7b0:aee5:a413 with SMTP id 16-20020a5d9c10000000b007b0aee5a413mr25257871ioe.15.1701376648183; Thu, 30 Nov 2023 12:37:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IEQKiGzWCsK0hWXMs70Cq/3F6+vV2o4nG+j/4EOKgm/JeXRfabKp7lr9C/O4fGUK78pdtVffQ== X-Received: by 2002:a5d:9c10:0:b0:7b0:aee5:a413 with SMTP id 16-20020a5d9c10000000b007b0aee5a413mr25257853ioe.15.1701376647790; Thu, 30 Nov 2023 12:37:27 -0800 (PST) Received: from xps13.dannf ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id h12-20020a6b784c000000b007b05b1e5029sm559472iop.28.2023.11.30.12.37.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 12:37:27 -0800 (PST) Date: Thu, 30 Nov 2023 13:37:25 -0700 From: dann frazier Message-ID: References: <20231127162433.924238-1-dann.frazier@canonical.com> <20231130013234.135285-1-dann.frazier@canonical.com> <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On Thu, Nov 30, 2023 at 08:11:52PM +0000, Pádraig Brady wrote: > Much clearer thanks. > > On my system: > > $ stat /sys/kernel/profiling > File: /sys/kernel/profiling > Size: 4096 Blocks: 0 IO Block: 4096 regular file > > I can easily repro by setting the buffer size < PAGE_SIZE. Oh, clever. > So this patch handles the case where sysfs reports a file is a certain size, > but it isn't really. In that case seeking to anywhere other than the start > doesn't give an error, but reading returns nothing. So we use a buffer size > large enough (>= PAGE_SIZE as inferred from st_blksize) so that we'll be > reading from the start of the file in this case. Exactly. > Note st_blksize can have unusual values, so it might be better > to use the io_blksize() wrapper to sanitize the values. > Though that will generally give 128K, which is good when processing all of a file, > but perhaps overkill when processing just the last part of a file. > > So perhaps it's better to use buffer_size = MAX (BUFSIZ, getpagesize ()) > and avoid all the st_blksize edge cases. Is there any performance concern about switching to 64K reads for "normal" files (on these systems)? > I'll think about it a little, and make the adjustments. Appreciated! -dann > thanks! > Pádraig. From unknown Sun Jun 22 08:08:26 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-coreutils@gnu.org Resent-Date: Fri, 01 Dec 2023 01:55:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67490 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: =?UTF-8?Q?P=C3=A1draig?= Brady , dann frazier , 67490@debbugs.gnu.org Received: via spool by 67490-submit@debbugs.gnu.org id=B67490.17013956647409 (code B ref 67490); Fri, 01 Dec 2023 01:55:02 +0000 Received: (at 67490) by debbugs.gnu.org; 1 Dec 2023 01:54:24 +0000 Received: from localhost ([127.0.0.1]:54807 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8sjX-0001vR-Oe for submit@debbugs.gnu.org; Thu, 30 Nov 2023 20:54:24 -0500 Received: from mail.cs.ucla.edu ([131.179.128.66]:51520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8sjV-0001vA-JY for 67490@debbugs.gnu.org; Thu, 30 Nov 2023 20:54:22 -0500 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id EBD403C011BDA; Thu, 30 Nov 2023 17:54:06 -0800 (PST) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id q9YG8v0ONPMc; Thu, 30 Nov 2023 17:54:06 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id A2FB03C011BDB; Thu, 30 Nov 2023 17:54:06 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu A2FB03C011BDB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1701395646; bh=ikb2wVRiiycFxDzFbRLjQofqCG63WVP/C6isd1hCG5E=; h=Message-ID:Date:MIME-Version:To:From; b=hU9IvdqnyW7W2yLYLXZecoL+nZWCc+TtFIeK1C4zyCNMSeM+yqfFdFUq0IvmJRxjx Aa3pC7OO254VTky8h8hbxamsJvlPWs3I0yYzYAvGsnF+0Mf5jj/yam8vw7Bvgibd+W jVDp/UsCMktVY8HW9MBuHcIxrbnei+70ijpJCe82p3cwGzRblxXeRzWRwSnU8jWE9u Q0UbV9uI4HfdHocbB/EzgyarP1sHM9lFwgJeikw5wiVfC26ANnTTHVsPVgbowUuusu knjeYzkmiJV6VV25B00MK0Cf0NbsIFSLEOmxpw5bcln7pxvA97Ipw/G6E9Nb/tJiks VyzW8trdXY/Bw== X-Virus-Scanned: amavisd-new at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id UwBN3UGL9cxr; Thu, 30 Nov 2023 17:54:06 -0800 (PST) Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id 899DB3C011BDA; Thu, 30 Nov 2023 17:54:06 -0800 (PST) Message-ID: Date: Thu, 30 Nov 2023 17:54:03 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird References: <20231127162433.924238-1-dann.frazier@canonical.com> <20231130013234.135285-1-dann.frazier@canonical.com> <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> Content-Language: en-US From: Paul Eggert Organization: UCLA Computer Science Department In-Reply-To: <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 11/30/23 12:11, P=C3=A1draig Brady wrote: > Though that will generally give 128K, which is good when processing all= =20 > of a file, > but perhaps overkill when processing just the last part of a file. The 128 KiB number was computed as being better for apps like 'sed' that=20 typically read all or most of the file. 'tail' sometimes behaves that=20 way (e.g., 'tail -c +10') and so 'tail' should use 128 KiB in those=20 cases. The simplest way to do that is for 'tail' to use 128 KiB all the=20 time - that would cost little for uses like plain 'tail' and it could be=20 a significant win for uses like 'tail -c +10'. (As an aside, the 128 KiB number was computed in 2014. These days 256=20 KiB might be better if someone could take the time to measure....) From unknown Sun Jun 22 08:08:26 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: dann frazier Subject: bug#67490: closed (Re: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels) Message-ID: References: <52ff0042-1eea-acac-e355-7e7dd62f5956@draigBrady.com> <20231127162433.924238-1-dann.frazier@canonical.com> X-Gnu-PR-Message: they-closed 67490 X-Gnu-PR-Package: coreutils X-Gnu-PR-Keywords: patch Reply-To: 67490@debbugs.gnu.org Date: Fri, 01 Dec 2023 23:24:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1701473042-21162-1" This is a multi-part message in MIME format... ------------=_1701473042-21162-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels which was filed against the coreutils package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 67490@debbugs.gnu.org. --=20 67490: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D67490 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1701473042-21162-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 67490-done) by debbugs.gnu.org; 1 Dec 2023 23:23:39 +0000 Received: from localhost ([127.0.0.1]:56347 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r9CrD-0005Ue-GS for submit@debbugs.gnu.org; Fri, 01 Dec 2023 18:23:39 -0500 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]:56548) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r9CrA-0005UM-Et for 67490-done@debbugs.gnu.org; Fri, 01 Dec 2023 18:23:38 -0500 Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-40b30308c67so24425755e9.0 for <67490-done@debbugs.gnu.org>; Fri, 01 Dec 2023 15:23:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701473001; x=1702077801; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=Nu2ibAAcBzX4A51A2otvTuiYVYsYVIGK4vUIYk2GNZc=; b=boJ1Y2Mpxh/yM1cmyBMyRj/nF/Q/FelhCTyqENAc6lYJ0+3BBdyomJr+xo91pMF4ns N1OBQoAfHubwpI2zuywyGnLorre1uS8lyqseQz+efPvc4U7G8805WE6qD7CCrz+FzQ1g Wy2XRdvCaVuW/Q2/gVpThVLWQcBLFGuzUyUyujrBhPM997ZKJYZ1UGKxFIU0OjBBRTTE QbWUeVHAJ5sMh30DcOa2jreb1EhH/a2R9u1IhsmX/UqlwbxQMXHd07R8sJQINM2KbYQ4 40/wANGh2L7b4jD9BKxHXBIing4mFBSs7/ZUk3lrROZs2qi7zfh1phbi7zNERZNlLqX+ GCAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701473001; x=1702077801; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Nu2ibAAcBzX4A51A2otvTuiYVYsYVIGK4vUIYk2GNZc=; b=toW/l5LEpmArQjIy1JIWDDQ39EmULJsQEQ+9gnYeHonNfgv4CdtJwXaMmBV0kZacPi s20n7ig138BB7zrEHLvVwJvXHcskokol+4/LiRO4UDhurht51WLb2rBLFUq0iMbxump/ ZdBdMT01Wgy+cjLNuvHUwexoPUrQ+l/pAwvN4kti+RlzAPq6zHK170QSeyIx7OJCxgep T3HgRYI70tvadeh+QF9FNiZSr7j7E8N1tE7AfTbtgZ52hCKw+NC75Y0Nc5N+4m8+4OS4 3mlQwC6YTwc95cWjrdOWgjwfmsMhyImvlLwHNOArasLJ7IKuyKhJlV2QDn/PPYwWu6KI uiUg== X-Gm-Message-State: AOJu0YzRXTBQPm3psTt/a4jCumG5412A61LfpJSGfjYc3AHlk2XWmEz+ hBfpO5zqVuc0PeD7EeibnvY= X-Google-Smtp-Source: AGHT+IGgSumgtQ0gfcHJBBjQvEZS0VsbSUHCv2pud7LewTd+dtlCxue2zz5cL150PGasqdeyv5cQ8g== X-Received: by 2002:a5d:6ad2:0:b0:333:2fd2:6f7b with SMTP id u18-20020a5d6ad2000000b003332fd26f7bmr1214618wrw.133.1701473000844; Fri, 01 Dec 2023 15:23:20 -0800 (PST) Received: from [192.168.1.20] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id dm11-20020a0560000bcb00b0033334625bdbsm1716028wrb.13.2023.12.01.15.23.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 01 Dec 2023 15:23:20 -0800 (PST) Message-ID: <52ff0042-1eea-acac-e355-7e7dd62f5956@draigBrady.com> Date: Fri, 1 Dec 2023 23:23:19 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels Content-Language: en-US To: Paul Eggert , dann frazier , 67490-done@debbugs.gnu.org References: <20231127162433.924238-1-dann.frazier@canonical.com> <20231130013234.135285-1-dann.frazier@canonical.com> <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> From: =?UTF-8?Q?P=C3=A1draig_Brady?= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 67490-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 01/12/2023 01:54, Paul Eggert wrote: > On 11/30/23 12:11, Pádraig Brady wrote: >> Though that will generally give 128K, which is good when processing all >> of a file, >> but perhaps overkill when processing just the last part of a file. > > The 128 KiB number was computed as being better for apps like 'sed' that > typically read all or most of the file. 'tail' sometimes behaves that > way (e.g., 'tail -c +10') and so 'tail' should use 128 KiB in those > cases. The simplest way to do that is for 'tail' to use 128 KiB all the > time - that would cost little for uses like plain 'tail' and it could be > a significant win for uses like 'tail -c +10'. Yes I agree we should use io_blksize() in other routines in tail where we may dump lots of a file. However in this (most common) case the routine is dealing with the end of a regular file, so it's probably best to somewhat minimize the amount of data read, and more directly check the page_size which is issue at hand. I've pushed the fix at https://github.com/coreutils/coreutils/commit/73d119f4f where the adjustment (which also corresponds to what we do in wc) is: if (sb->st_size % page_size == 0) bufsize = MAX (BUFSIZ, page_size); I'll follow up with another patch to address the performance aspect, which uses io_blksize() where appropriate. > (As an aside, the 128 KiB number was computed in 2014. These days 256 > KiB might be better if someone could take the time to measure....) I periodically check with the documented script, but I should update the comment when I do that. I'll update the date in the comment now at least. I quickly tested a few systems here, which suggests 256KiB _may_ be a more appropriate default now. More testing required. Marking this as done. thanks, Pádraig ------------=_1701473042-21162-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 27 Nov 2023 17:30:20 +0000 Received: from localhost ([127.0.0.1]:44715 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7fR4-0006Hh-Us for submit@debbugs.gnu.org; Mon, 27 Nov 2023 12:30:20 -0500 Received: from lists.gnu.org ([2001:470:142::17]:53992) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7ePy-0004UP-6x for submit@debbugs.gnu.org; Mon, 27 Nov 2023 11:25:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7ePm-0004wl-Db for bug-coreutils@gnu.org; Mon, 27 Nov 2023 11:24:54 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7ePk-0005Bj-I9 for bug-coreutils@gnu.org; Mon, 27 Nov 2023 11:24:54 -0500 Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 434003F11F for ; Mon, 27 Nov 2023 16:24:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701102289; bh=eRxRZp9frYjOz8Y7px/sfeG47wN3RQH42qLQTHKYPjs=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=GCdo+X3H+exnv3jU81J1FrF0vgiafqSrtdgofU/r/Z3u6Wr9f+jKBqsjGZZrp//7t JsU2BI5YDdtD3r//noATHRyaO32ncZW+GW68Cv3eqbxYYkBoJO0pbd+vKL556iScyo qHB9DXxRKH/4N0lk6Zeu/JkDa2asJ09wSSopF6QK3GsH0Yfosj1DDe0tH+DwndMi6+ wMs/wxc7UTVgfrYafvclBiwUD+wx398SgGmViJdpTgRByXpjWMzI9frkNrRml5vMjT Zy+5rk8eNweNf8jL5Eng5nWP9ZxDScwn6hq0p/37LP84OGuLE2AUhOO2AXooItXRw1 lBJAKfVA4VBJQ== Received: by mail-io1-f70.google.com with SMTP id ca18e2360f4ac-7b39a5cc776so157294039f.2 for ; Mon, 27 Nov 2023 08:24:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701102287; x=1701707087; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eRxRZp9frYjOz8Y7px/sfeG47wN3RQH42qLQTHKYPjs=; b=fAJGtks7MtuAa2yEH8ybtdFJZn3k94JzAE7NdnJJln3CXccYaTyjGDDq5P65AeG9Ld 3hWHijXC13GN6zelZnACo4o4xt07q3DGFT8hZwZqCQrPvM9GSv/WB8fVD+0qo/rxeoQ0 cLPt/gPhateyNBKd0xmORmQ5a9mXPMvsj7CBV7BUFKeOEXngq3dKZT2FqRzEcUe/MwNj r4mLtNTYdSe5AA4JY+c/cUXZp2RxzLDpZI5ZwKm8nLddRicdAbPW0GfkWpD0zUblG4j0 0IyBz6BZjnqthaXt3SdQ24pU3txyLUlmibyLYwvxLqFXUFMq+mhwzjYwc2Js9ga6niLq 5/yQ== X-Gm-Message-State: AOJu0YwBKD6BaLQjZ2s9DmVQmHVvN4yPEO78Bj18Xso6XyEhv5UANtlE +MeHq4AziXF9wbzQyjfFOJ3beRWpvkjVMnL3Wd3bRk1gwnHVoZMjPLCgWRFKw66J8eHkQfhduz8 qnS/QFbDAAu0vCle5GUMGokbW1LKMacTiZczbPSQw X-Received: by 2002:a5d:850f:0:b0:7ac:cb6b:616a with SMTP id q15-20020a5d850f000000b007accb6b616amr7495310ion.8.1701102287671; Mon, 27 Nov 2023 08:24:47 -0800 (PST) X-Google-Smtp-Source: AGHT+IGgrK7+UeWHO4J9D3FhIZyrgR96OJ4om7eFAiMil5VyqS/tWG8ootifE2nZqofVxygA2mUT5A== X-Received: by 2002:a5d:850f:0:b0:7ac:cb6b:616a with SMTP id q15-20020a5d850f000000b007accb6b616amr7495295ion.8.1701102287386; Mon, 27 Nov 2023 08:24:47 -0800 (PST) Received: from localhost ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id b13-20020a02a58d000000b00466b9ec90ffsm831595jam.115.2023.11.27.08.24.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Nov 2023 08:24:46 -0800 (PST) From: dann frazier To: bug-coreutils@gnu.org Subject: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Date: Mon, 27 Nov 2023 09:24:33 -0700 Message-ID: <20231127162433.924238-1-dann.frazier@canonical.com> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=185.125.188.122; envelope-from=dann.frazier@canonical.com; helo=smtp-relay-internal-0.canonical.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Mon, 27 Nov 2023 12:30:17 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) * src/tail.c (file_lines): Use fstat() to determine a file's block size and dynamically allocate a buffer of that size for traversing backwards. --- src/tail.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/src/tail.c b/src/tail.c index c45f3b65a..437a38204 100644 --- a/src/tail.c +++ b/src/tail.c @@ -518,18 +518,30 @@ static bool file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, off_t start_pos, off_t end_pos, uintmax_t *read_pos) { - char buffer[BUFSIZ]; + char *buffer; size_t bytes_read; + blksize_t bufsize; off_t pos = end_pos; + bool ok = true; + struct stat stats; if (n_lines == 0) return true; + if (fstat (fd, &stats) != 0) + { + error (0, errno, _("cannot fstat %s"), quoteaf (pretty_filename)); + return false; + } + + bufsize = ST_BLKSIZE (stats); + buffer = xmalloc (bufsize); + /* Set 'bytes_read' to the size of the last, probably partial, buffer; 0 < 'bytes_read' <= 'BUFSIZ'. */ - bytes_read = (pos - start_pos) % BUFSIZ; + bytes_read = (pos - start_pos) % bufsize; if (bytes_read == 0) - bytes_read = BUFSIZ; + bytes_read = bufsize; /* Make 'pos' a multiple of 'BUFSIZ' (0 if the file is short), so that all reads will be on block boundaries, which might increase efficiency. */ pos -= bytes_read; @@ -538,7 +550,8 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; @@ -565,7 +578,7 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xwrite_stdout (nl + 1, bytes_read - (n + 1)); *read_pos += dump_remainder (false, pretty_filename, fd, end_pos - (pos + bytes_read)); - return true; + goto free_buffer; } } @@ -577,23 +590,26 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xlseek (fd, start_pos, SEEK_SET, pretty_filename); *read_pos = start_pos + dump_remainder (false, pretty_filename, fd, end_pos); - return true; + goto free_buffer; } - pos -= BUFSIZ; + pos -= bufsize; xlseek (fd, pos, SEEK_SET, pretty_filename); - bytes_read = safe_read (fd, buffer, BUFSIZ); + bytes_read = safe_read (fd, buffer, bufsize); if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; } while (bytes_read > 0); - return true; +free_buffer: + free (buffer); + return ok; } /* Print the last N_LINES lines from the end of the standard input, -- 2.42.0 ------------=_1701473042-21162-1--