From unknown Sun Jun 22 03:56:59 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#67490 <67490@debbugs.gnu.org> To: bug#67490 <67490@debbugs.gnu.org> Subject: Status: [PATCH v2] tail: fix tailing sysfs files on large page kernels Reply-To: bug#67490 <67490@debbugs.gnu.org> Date: Sun, 22 Jun 2025 10:56:59 +0000 retitle 67490 [PATCH v2] tail: fix tailing sysfs files on large page kernels reassign 67490 coreutils submitter 67490 dann frazier severity 67490 normal tag 67490 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 27 12:30:20 2023 Received: (at submit) by debbugs.gnu.org; 27 Nov 2023 17:30:20 +0000 Received: from localhost ([127.0.0.1]:44715 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7fR4-0006Hh-Us for submit@debbugs.gnu.org; Mon, 27 Nov 2023 12:30:20 -0500 Received: from lists.gnu.org ([2001:470:142::17]:53992) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7ePy-0004UP-6x for submit@debbugs.gnu.org; Mon, 27 Nov 2023 11:25:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7ePm-0004wl-Db for bug-coreutils@gnu.org; Mon, 27 Nov 2023 11:24:54 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7ePk-0005Bj-I9 for bug-coreutils@gnu.org; Mon, 27 Nov 2023 11:24:54 -0500 Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 434003F11F for ; Mon, 27 Nov 2023 16:24:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701102289; bh=eRxRZp9frYjOz8Y7px/sfeG47wN3RQH42qLQTHKYPjs=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=GCdo+X3H+exnv3jU81J1FrF0vgiafqSrtdgofU/r/Z3u6Wr9f+jKBqsjGZZrp//7t JsU2BI5YDdtD3r//noATHRyaO32ncZW+GW68Cv3eqbxYYkBoJO0pbd+vKL556iScyo qHB9DXxRKH/4N0lk6Zeu/JkDa2asJ09wSSopF6QK3GsH0Yfosj1DDe0tH+DwndMi6+ wMs/wxc7UTVgfrYafvclBiwUD+wx398SgGmViJdpTgRByXpjWMzI9frkNrRml5vMjT Zy+5rk8eNweNf8jL5Eng5nWP9ZxDScwn6hq0p/37LP84OGuLE2AUhOO2AXooItXRw1 lBJAKfVA4VBJQ== Received: by mail-io1-f70.google.com with SMTP id ca18e2360f4ac-7b39a5cc776so157294039f.2 for ; Mon, 27 Nov 2023 08:24:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701102287; x=1701707087; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eRxRZp9frYjOz8Y7px/sfeG47wN3RQH42qLQTHKYPjs=; b=fAJGtks7MtuAa2yEH8ybtdFJZn3k94JzAE7NdnJJln3CXccYaTyjGDDq5P65AeG9Ld 3hWHijXC13GN6zelZnACo4o4xt07q3DGFT8hZwZqCQrPvM9GSv/WB8fVD+0qo/rxeoQ0 cLPt/gPhateyNBKd0xmORmQ5a9mXPMvsj7CBV7BUFKeOEXngq3dKZT2FqRzEcUe/MwNj r4mLtNTYdSe5AA4JY+c/cUXZp2RxzLDpZI5ZwKm8nLddRicdAbPW0GfkWpD0zUblG4j0 0IyBz6BZjnqthaXt3SdQ24pU3txyLUlmibyLYwvxLqFXUFMq+mhwzjYwc2Js9ga6niLq 5/yQ== X-Gm-Message-State: AOJu0YwBKD6BaLQjZ2s9DmVQmHVvN4yPEO78Bj18Xso6XyEhv5UANtlE +MeHq4AziXF9wbzQyjfFOJ3beRWpvkjVMnL3Wd3bRk1gwnHVoZMjPLCgWRFKw66J8eHkQfhduz8 qnS/QFbDAAu0vCle5GUMGokbW1LKMacTiZczbPSQw X-Received: by 2002:a5d:850f:0:b0:7ac:cb6b:616a with SMTP id q15-20020a5d850f000000b007accb6b616amr7495310ion.8.1701102287671; Mon, 27 Nov 2023 08:24:47 -0800 (PST) X-Google-Smtp-Source: AGHT+IGgrK7+UeWHO4J9D3FhIZyrgR96OJ4om7eFAiMil5VyqS/tWG8ootifE2nZqofVxygA2mUT5A== X-Received: by 2002:a5d:850f:0:b0:7ac:cb6b:616a with SMTP id q15-20020a5d850f000000b007accb6b616amr7495295ion.8.1701102287386; Mon, 27 Nov 2023 08:24:47 -0800 (PST) Received: from localhost ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id b13-20020a02a58d000000b00466b9ec90ffsm831595jam.115.2023.11.27.08.24.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Nov 2023 08:24:46 -0800 (PST) From: dann frazier To: bug-coreutils@gnu.org Subject: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Date: Mon, 27 Nov 2023 09:24:33 -0700 Message-ID: <20231127162433.924238-1-dann.frazier@canonical.com> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=185.125.188.122; envelope-from=dann.frazier@canonical.com; helo=smtp-relay-internal-0.canonical.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Mon, 27 Nov 2023 12:30:17 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) * src/tail.c (file_lines): Use fstat() to determine a file's block size and dynamically allocate a buffer of that size for traversing backwards. --- src/tail.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/src/tail.c b/src/tail.c index c45f3b65a..437a38204 100644 --- a/src/tail.c +++ b/src/tail.c @@ -518,18 +518,30 @@ static bool file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, off_t start_pos, off_t end_pos, uintmax_t *read_pos) { - char buffer[BUFSIZ]; + char *buffer; size_t bytes_read; + blksize_t bufsize; off_t pos = end_pos; + bool ok = true; + struct stat stats; if (n_lines == 0) return true; + if (fstat (fd, &stats) != 0) + { + error (0, errno, _("cannot fstat %s"), quoteaf (pretty_filename)); + return false; + } + + bufsize = ST_BLKSIZE (stats); + buffer = xmalloc (bufsize); + /* Set 'bytes_read' to the size of the last, probably partial, buffer; 0 < 'bytes_read' <= 'BUFSIZ'. */ - bytes_read = (pos - start_pos) % BUFSIZ; + bytes_read = (pos - start_pos) % bufsize; if (bytes_read == 0) - bytes_read = BUFSIZ; + bytes_read = bufsize; /* Make 'pos' a multiple of 'BUFSIZ' (0 if the file is short), so that all reads will be on block boundaries, which might increase efficiency. */ pos -= bytes_read; @@ -538,7 +550,8 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; @@ -565,7 +578,7 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xwrite_stdout (nl + 1, bytes_read - (n + 1)); *read_pos += dump_remainder (false, pretty_filename, fd, end_pos - (pos + bytes_read)); - return true; + goto free_buffer; } } @@ -577,23 +590,26 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xlseek (fd, start_pos, SEEK_SET, pretty_filename); *read_pos = start_pos + dump_remainder (false, pretty_filename, fd, end_pos); - return true; + goto free_buffer; } - pos -= BUFSIZ; + pos -= bufsize; xlseek (fd, pos, SEEK_SET, pretty_filename); - bytes_read = safe_read (fd, buffer, BUFSIZ); + bytes_read = safe_read (fd, buffer, bufsize); if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; } while (bytes_read > 0); - return true; +free_buffer: + free (buffer); + return ok; } /* Print the last N_LINES lines from the end of the standard input, -- 2.42.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 27 14:10:44 2023 Received: (at 67490) by debbugs.gnu.org; 27 Nov 2023 19:10:44 +0000 Received: from localhost ([127.0.0.1]:44800 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7h0G-0003OU-FY for submit@debbugs.gnu.org; Mon, 27 Nov 2023 14:10:44 -0500 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]:50533) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7h0E-0003OC-PS for 67490@debbugs.gnu.org; Mon, 27 Nov 2023 14:10:43 -0500 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-4079ed65582so32403415e9.1 for <67490@debbugs.gnu.org>; Mon, 27 Nov 2023 11:10:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701112229; x=1701717029; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=0YAj7wn8KHnxKyBnPpSSUB68xLzyu/SbuHY3ivUhmCA=; b=hzgm350OUiJC1h90WVDORMAxAP5r8tAwZTV2veThe3FVyPNIplYjojTUPQ5JoKT7mk h9pCOLOY2NE/ZkObHmd7IzumI9UZz6hR++GvUHoCg2IWZfhnfc6SSPvo2A3iOzgPVkT/ CZk247fcRtF5cLLIcY2Qs/sEmI+DfM/ChGoz3aLU3Ucy0fO4ipxfruspwCKuJTtYVX3I gMKyFQ13ZlufzL8tDgWkEJGSsRePc1IQBVGXVijmHzGMocXyTIN/vrK65Zytkpxqgcp/ m48f21nBAkt8dovRMu28CvZ0/PgqjsOtb/Ew1WE47zQRjGPEWPu8i8D8ZWzKShkvRj1r D/KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701112229; x=1701717029; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0YAj7wn8KHnxKyBnPpSSUB68xLzyu/SbuHY3ivUhmCA=; b=gBK93O1jrCILjzyjtyBxpSwEMb+797w5z6lOmfO+js72dgXKtKWvN6Ykr0dEFJFlO9 4WjqB5ccUAbkDPGL8sPtJiYg9Wdp4SrZkco0pr4nFXAvWuc9C1KOlZy1SfS3EmQL7BND pHTiFESMjTlMVff1AcE+aME1BrjqsbCS/EVv6og5ACriOCgUgaQpxtapEwmRTgbypFDw Hs2QKc5HIOQth7hck/l/3ZVFgs6xoPkS38Lnu7prMJpoUV3cryPhL9Aw6/EcNj6H5Cqd eQVH0kt06QjbSMKSF8LphC0bP1NxoSL8LF8FafkeE2g5mBLw/xcjDihcMYko/Tqf13Q1 yrMg== X-Gm-Message-State: AOJu0YzReRsvxae5EqwaVidj3IxDyMn4e4b3ZPWpuxhr/SDYgRr7lYlT ptSWvfwUpZbQvKpho4JdC8A= X-Google-Smtp-Source: AGHT+IE9eJKSbZ5K4nhwrmmkyFKZL79TJt+7WkLrIEuzi+g49GDk/ju7tzIZiR8MLKFs/XsdOpd5CA== X-Received: by 2002:a05:600c:1f8c:b0:409:50b4:3da8 with SMTP id je12-20020a05600c1f8c00b0040950b43da8mr10034602wmb.38.1701112228971; Mon, 27 Nov 2023 11:10:28 -0800 (PST) Received: from [192.168.1.20] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id g21-20020a05600c311500b0040b3645a7c2sm14894482wmo.40.2023.11.27.11.10.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 27 Nov 2023 11:10:28 -0800 (PST) Message-ID: Date: Mon, 27 Nov 2023 19:10:26 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#67490: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Content-Language: en-US To: dann frazier , 67490@debbugs.gnu.org References: <20231127162433.924238-1-dann.frazier@canonical.com> From: =?UTF-8?Q?P=C3=A1draig_Brady?= In-Reply-To: <20231127162433.924238-1-dann.frazier@canonical.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 67490 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 27/11/2023 16:24, dann frazier wrote: > * src/tail.c (file_lines): Use fstat() to determine a file's block > size and dynamically allocate a buffer of that size for traversing > backwards. Thanks for the patch. Could you describe it a bit more. What happens if we use smaller reads? Also what about all the other safe_read() calls in tail.c ? thanks, Pádraig From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 27 14:18:12 2023 Received: (at 67490) by debbugs.gnu.org; 27 Nov 2023 19:18:12 +0000 Received: from localhost ([127.0.0.1]:44805 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7h7U-0003bf-CK for submit@debbugs.gnu.org; Mon, 27 Nov 2023 14:18:12 -0500 Received: from mail.cs.ucla.edu ([131.179.128.66]:41520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r7h7S-0003bP-FH for 67490@debbugs.gnu.org; Mon, 27 Nov 2023 14:18:11 -0500 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id D4D003C011BDC; Mon, 27 Nov 2023 11:17:57 -0800 (PST) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 4rGk9X-Ze9iW; Mon, 27 Nov 2023 11:17:57 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 8F17C3C011BDD; Mon, 27 Nov 2023 11:17:57 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu 8F17C3C011BDD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1701112677; bh=kNtnop5Ip0hgnW64ie6+6p4rVyoex/CSWE+n9Yzg5hE=; h=Message-ID:Date:MIME-Version:To:From; b=WFM3S+j03tbRrLVznmomkS0JBmVj/PzyeHAx7YKbcb4GhAve/mRFvP3NMbsxkKSCs dS3jzqPZ84qYI7QXTK9Qg8SXkcS/YsBCdJq34W6htBINnmzabLNn/JtvFGL+nE4yxd NA7gdA2XyGcYeDpMlIWmzERO/ZuG5LnVY6QcaJxu9v12SZ4StBV+kV+P566SrYRlca 6azgEI0U+PavL2VydVPIkUk73PsarreTFBWxf2TLzgZVbZi+RO02FAxoMPxSFan//W S/0pliHdZbs3lN129I4qiro4ch4Xus1qiWpSWfZy3v3WuxsFpBtMnRZUFAYp0jTuJ0 MaXk1Pw9121Pg== X-Virus-Scanned: amavisd-new at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ZN7lw7DgaikP; Mon, 27 Nov 2023 11:17:57 -0800 (PST) Received: from [192.168.254.12] (unknown [47.148.192.211]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id 6EB633C011BDC; Mon, 27 Nov 2023 11:17:57 -0800 (PST) Message-ID: Date: Mon, 27 Nov 2023 11:17:57 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#67490: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Content-Language: en-US To: dann frazier , 67490@debbugs.gnu.org References: <20231127162433.924238-1-dann.frazier@canonical.com> From: Paul Eggert Organization: UCLA Computer Science Department In-Reply-To: <20231127162433.924238-1-dann.frazier@canonical.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67490 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 2023-11-27 08:24, dann frazier wrote: > + if (fstat (fd, &stats) != 0) > + { > + error (0, errno, _("cannot fstat %s"), quoteaf (pretty_filename)); > + return false; > + } > + > + bufsize = ST_BLKSIZE (stats); If fstat fails, that's no reason to exit. Just use bufsize = BUFSIZ and keep going. fstat can fail for reasons unrelated to what 'tail' needs (e.g., time_t overflow). From debbugs-submit-bounces@debbugs.gnu.org Tue Nov 28 20:58:13 2023 Received: (at 67490) by debbugs.gnu.org; 29 Nov 2023 01:58:13 +0000 Received: from localhost ([127.0.0.1]:48503 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r89q5-00057y-S0 for submit@debbugs.gnu.org; Tue, 28 Nov 2023 20:58:13 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]:56772) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r88bd-00035E-H2 for 67490@debbugs.gnu.org; Tue, 28 Nov 2023 19:39:13 -0500 Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 0CC933F078 for <67490@debbugs.gnu.org>; Wed, 29 Nov 2023 00:38:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701218336; bh=f5gli4jc9Yl9WHmKn49y+95WIkbAcsWqYqJDFjgWO+w=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=NGkrTqX3h/MIlkbDqMWdCQcXkLrVfKzCLDHGy94Dm8umrJyaxeL9kaITteFurdVQ3 /G1nj7q9zBRhzJuB3IRuoytUvCqVx50YNiHuBlPEn9/kLHrNNiFjNlh+Kv01jGlWbK lshwxlmqqJAAm4wiyfntoUvNCO2gPycl8HVSHVnoADscOjG8OoZ4YgE5gAMfmgPS9e SncGHwK65nY6GQGgKjts78SEPY4l4joIDXzaD2t37CL7LlQYgoQ/VKiAiPMxP5MmJa tBRpTTkKaVfMc78qzOfFS3huTXQz7vqujYUwHlndtB74yNgm4jbqXYSd4p7+beGr3m pDYMrTKygahrA== Received: by mail-io1-f72.google.com with SMTP id ca18e2360f4ac-7b36a881314so565185439f.2 for <67490@debbugs.gnu.org>; Tue, 28 Nov 2023 16:38:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701218335; x=1701823135; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=f5gli4jc9Yl9WHmKn49y+95WIkbAcsWqYqJDFjgWO+w=; b=EbSmF0kDizQ6K+g1UK7NnCkS4aqPEHuBv85RPq54DBed6TJrdPlKY222+KTqz3t0l1 3QnI1/fuNnFFsOKFxU3SIP9S02FpFFIDQtVaDtXSX2bTq6Wlem+ij/k/jRt4iVvsOarJ CLR75oeLb+pGGV6AaOxynxHoyJndkGrTO+/cIx54Qx57swxsg4DrZs2txWH8plgfIEPz hn6edHnoI93V5/sx31ORhP9WKAe3CUYBNh5k99fTMi1tjQJDoHeFHA5OjIEkBODDzMym gp2ZO4iTHF9rrEc2KJ9CNJflnE3dfgr52PJwPNEDr/Z5p6L5W89hOybimmvTIbWyeBPK Q+cA== X-Gm-Message-State: AOJu0Yy+bOOQj+uj1naPg6IOejTxoll8yTuj47vwtKuf3ez/zI8Wfumm hTh0FhqoIcE9bNsYf6/zD8ov9pWpE3UzC9KyT/XjBmZjgVKsYwyHznH+HUpc3sm/+OTBD/0ElXp pDLyL3BEuPOX1QTx8/EL8yfj+0sxc7Vlr X-Received: by 2002:a6b:ce16:0:b0:7b0:a8d7:3bb3 with SMTP id p22-20020a6bce16000000b007b0a8d73bb3mr18748450iob.16.1701218334810; Tue, 28 Nov 2023 16:38:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IHWN1zfHjPARh9wPOkVftmQ1rnmoNUAASnEkcdsBkNXQV6RBD1wtMGdfX3D0BggtWYparzsJA== X-Received: by 2002:a6b:ce16:0:b0:7b0:a8d7:3bb3 with SMTP id p22-20020a6bce16000000b007b0a8d73bb3mr18748435iob.16.1701218334478; Tue, 28 Nov 2023 16:38:54 -0800 (PST) Received: from xps13.dannf ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id m21-20020a0566380ad500b0042b3e04510asm3164896jab.98.2023.11.28.16.38.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Nov 2023 16:38:54 -0800 (PST) Date: Tue, 28 Nov 2023 17:38:52 -0700 From: dann frazier To: Paul Eggert Subject: Re: bug#67490: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Message-ID: References: <20231127162433.924238-1-dann.frazier@canonical.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 67490 X-Mailman-Approved-At: Tue, 28 Nov 2023 20:58:08 -0500 Cc: 67490@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.8 (-) On Mon, Nov 27, 2023 at 11:17:57AM -0800, Paul Eggert wrote: > On 2023-11-27 08:24, dann frazier wrote: > > + if (fstat (fd, &stats) != 0) > > + { > > + error (0, errno, _("cannot fstat %s"), quoteaf (pretty_filename)); > > + return false; > > + } > > + > > + bufsize = ST_BLKSIZE (stats); > > > If fstat fails, that's no reason to exit. Just use bufsize > = BUFSIZ and keep going. fstat can fail for reasons > unrelated to what 'tail' needs (e.g., time_t overflow). Thanks for the suggestion - I'll make that change. -dann From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 29 09:34:53 2023 Received: (at 67490) by debbugs.gnu.org; 29 Nov 2023 14:34:53 +0000 Received: from localhost ([127.0.0.1]:49248 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8LeP-0002lV-Cc for submit@debbugs.gnu.org; Wed, 29 Nov 2023 09:34:53 -0500 Received: from smtp-relay-internal-1.canonical.com ([185.125.188.123]:41406) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8LeM-0002lI-TR for 67490@debbugs.gnu.org; Wed, 29 Nov 2023 09:34:52 -0500 Received: from mail-il1-f199.google.com (mail-il1-f199.google.com [209.85.166.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id B710440185 for <67490@debbugs.gnu.org>; Wed, 29 Nov 2023 14:34:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701268475; bh=i529c8yrbVZwM2nRyE2LeuuJOwkeAbFAIbNSDRP/NNA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=SQz/LBWRj9h2lcvLREOnyVd0ROePg8PdftktDeZB3PMXEt7t5AtbeFJKscxM10YeM WNh7VaRGX+PN0iYFqvLEMEw57nn6VbpYvucM4JKQ6YADaAo0CEVTUkR/oaIbWOXKYW CEoCff1hWCmryXF59rywS6X7s9dEW1VnVtqoDd89oJGQ+TrAzj1jqcwPeBtBSP4Uvo uCuuKDj3BqY9nbshEKjCm2qwJXzx4vm/vbqgn55dq3G3vuzluJLUUPeZxC4UQKFKVD zZLhq2MSwG+aadyt0wzLTXXqCPMxemWwwPiFE2J77QRdE14Apz+9SWxbsLMU7kRlSO rkqQWv4Tpe6hw== Received: by mail-il1-f199.google.com with SMTP id e9e14a558f8ab-35cc2083d25so36693825ab.2 for <67490@debbugs.gnu.org>; Wed, 29 Nov 2023 06:34:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701268469; x=1701873269; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=i529c8yrbVZwM2nRyE2LeuuJOwkeAbFAIbNSDRP/NNA=; b=fW5hZIyVCw19MofWL8lk06EggUNy+QwdMe/W63S2DVQYL1FsYputEZxCrdkRb1VVSF fUvzrQcH5BqG3VdpQt1ByuhKUa/6I6i8jPa6zN5pEaEK1SV8ddmqMDDuHr9WwR8XYutw dvNll/e7o8KKunigw05+c82Qa/F89VdyAM/8yjpId1HzaGY0B+reIWhK0krZ30p71lEE sfoGOgbI0CLtLxvz7kESTf7lmRGXTZzbHcN3346ifqD116meRJtFjOFDGMZ7wgSbJIyp Yfz2D2sW/yyxIA7RavrlzZpzMXdbRt/YOk0xbQM7eWUix1D6Kelg9SvjEhYdMWiM0aRj YlmQ== X-Gm-Message-State: AOJu0Yyr/eeRz8cZz1SzJOvoIyndr/KiAiqXA6WtUGSi68vEBTG1RPTW KxWEKRLTOujWCf2/Ubwv0743Hkb12Ub1e28KFxy9bsBvsaBc2XHgCGTDg1Zc+r78jyu93Y7htuj roT+rKZMzhiIyXQBbnHjdRJFnw7ZIHTJcKCGh751k X-Received: by 2002:a05:6e02:970:b0:35c:ca7c:ebb2 with SMTP id q16-20020a056e02097000b0035cca7cebb2mr12136251ilt.20.1701268469650; Wed, 29 Nov 2023 06:34:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IHQhpSu0q+/ul6T0zCGCb9tWzDGyDMUhoDYzLL/FZyEomZHfynSQQHQtdrV7jUKvI2UJ2wFMw== X-Received: by 2002:a05:6e02:970:b0:35c:ca7c:ebb2 with SMTP id q16-20020a056e02097000b0035cca7cebb2mr12136231ilt.20.1701268469270; Wed, 29 Nov 2023 06:34:29 -0800 (PST) Received: from xps13.dannf ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id s9-20020a056638218900b004667f079d40sm3563325jaj.60.2023.11.29.06.34.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Nov 2023 06:34:28 -0800 (PST) Date: Wed, 29 Nov 2023 07:34:26 -0700 From: dann frazier To: =?iso-8859-1?Q?P=E1draig?= Brady Subject: Re: bug#67490: [PATCH] tail: fix following /proc and /sys files when using a 64K page size Message-ID: References: <20231127162433.924238-1-dann.frazier@canonical.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 67490 Cc: 67490@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On Mon, Nov 27, 2023 at 07:10:26PM +0000, Pádraig Brady wrote: > On 27/11/2023 16:24, dann frazier wrote: > > * src/tail.c (file_lines): Use fstat() to determine a file's block > > size and dynamically allocate a buffer of that size for traversing > > backwards. > > Thanks for the patch. > Could you describe it a bit more. > What happens if we use smaller reads? > Also what about all the other safe_read() calls in tail.c ? Yes, this warrants more detail - I'll try to capture that in a comment. Thanks Pádraig! -dann From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 29 20:33:09 2023 Received: (at 67490) by debbugs.gnu.org; 30 Nov 2023 01:33:09 +0000 Received: from localhost ([127.0.0.1]:51807 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8VvQ-0001Ys-M8 for submit@debbugs.gnu.org; Wed, 29 Nov 2023 20:33:09 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]:35218) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8VvN-0001YJ-PY for 67490@debbugs.gnu.org; Wed, 29 Nov 2023 20:33:07 -0500 Received: from mail-il1-f199.google.com (mail-il1-f199.google.com [209.85.166.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id AFC453F078 for <67490@debbugs.gnu.org>; Thu, 30 Nov 2023 01:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701307971; bh=nj6RJP8ZprDdFKxs2E3B6Ts6+T/c7UHkneqHovXDMv4=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=Uzc4NjnV0L/8tTiMB+TGC38q74npVpx3AS8nVdp+zUv+waZMMLEraHCtA2J8AoaBi IaXEe+DTqZ0B4+7deXhV5fhyViyWd/MY5LHybCeOhw+fCmsijekSkzyKtqsUSU6RZ5 MNpkDc9j+BFRsgVde7qITRa5C7H0uzbcIjRgdajMhvdjPrFk0IXsGQ88a6H71KzZsW RIOSs2L4EXCdzovfHFMeyhAG0C2KEVeFAmw7C+Yc2lDMuHyY/ZO5mvn2+M8yZGTslU w1TflqfO2YmEXh6Y6BiA/4392s5XbgDdUvW5jHytrWNxsYkokwgLQXOlSdtHY+oa56 P8rBlqp6hmPzw== Received: by mail-il1-f199.google.com with SMTP id e9e14a558f8ab-35ca1eadf5fso2582775ab.1 for <67490@debbugs.gnu.org>; Wed, 29 Nov 2023 17:32:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701307969; x=1701912769; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nj6RJP8ZprDdFKxs2E3B6Ts6+T/c7UHkneqHovXDMv4=; b=iw8jToGWsmJqvSD7Tjr22qWpQlYka7X/yhAak747uShOu5c4zVsfMn3cizfoDOdjX4 sugcfmGxBbSrdCDggJANGtJvzj578Xg8e+q7CpqLspd0PNgkmD8a7AuuDSWjBA3+gsx7 IQ6ux7xD2NCraQ/hN5W0dUG0gwxgzjBpE3oErNVybAlwHxiqCZLjU/K9XkNGvhjTHnIa lwBParBbKYyRQl1rZXODmFOJwvrr8BVHpjrYAJ7XOqdmzzBZjYpyB+A+/4PiEWM1yq1K +7B/qaPj5Pvq9lFglKKvibm0IRpx0RW6Ln9GKOOzHw2C4fZw72p1G+leoAUXE6lpp+nX D/HQ== X-Gm-Message-State: AOJu0YxmpoihRzoqLsK/+Fu6089Ug7ZG4kyPl5oUn+mCdDjl9YAJzZiX DA2UogMdcdldrO5K0FcLBCcmnWNpzW7bx9W/8HCIYcpchN6Ips71owTtdrwXa5UyYFTuaJchOLZ IfccmbWd6XVdFi5OgtLdBEfWEEqucuRs3jM/MNRpL X-Received: by 2002:a92:d305:0:b0:35c:76ff:f211 with SMTP id x5-20020a92d305000000b0035c76fff211mr17388331ila.0.1701307969015; Wed, 29 Nov 2023 17:32:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IGM3UzEgGReXxPYXASammAr2PN3FLinknmCWdzs0xIWlPOzKGjcOPnWi2yxqyi1aE1ugJjbyw== X-Received: by 2002:a92:d305:0:b0:35c:76ff:f211 with SMTP id x5-20020a92d305000000b0035c76fff211mr17388314ila.0.1701307968571; Wed, 29 Nov 2023 17:32:48 -0800 (PST) Received: from localhost ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id k16-20020a02a710000000b004666dff32d7sm22258jam.148.2023.11.29.17.32.48 for <67490@debbugs.gnu.org> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Nov 2023 17:32:48 -0800 (PST) From: dann frazier To: 67490@debbugs.gnu.org Subject: [PATCH v2] tail: fix tailing sysfs files on large page kernels Date: Wed, 29 Nov 2023 18:32:34 -0700 Message-ID: <20231130013234.135285-1-dann.frazier@canonical.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 67490 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Changes from v1: - Fallback to BUFSIZ if fstat fails instead of exiting. - Add a comment block to explain why the blksize is used in file_lines() (which I hope also clarifies why it is not needed elsewhere). - Only use blksize if it is > BUFSIZ (I had regressed to 4K reads, oops). - Update comments to reference bufsize instead of BUFSIZ. - Update subject because /proc is not affected, and it isn't limited to follow mode. - Add a test. * src/tail.c (file_lines): Consider a full blocksize when searching backwards to find sysfs file content on systems with large page sizes. * tests/tail/tail-1-sysfs.sh: Add a new test. * tests/local.mk: Reference the new test. Fixes https://bugs.gnu.org/67490 --- src/tail.c | 41 +++++++++++++++++++++++++++----------- tests/local.mk | 1 + tests/tail/tail-1-sysfs.sh | 31 ++++++++++++++++++++++++++++ 3 files changed, 61 insertions(+), 12 deletions(-) create mode 100755 tests/tail/tail-1-sysfs.sh diff --git a/src/tail.c b/src/tail.c index c45f3b65a..f3219757d 100644 --- a/src/tail.c +++ b/src/tail.c @@ -518,19 +518,32 @@ static bool file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, off_t start_pos, off_t end_pos, uintmax_t *read_pos) { - char buffer[BUFSIZ]; + char *buffer; size_t bytes_read; + blksize_t bufsize = BUFSIZ; off_t pos = end_pos; + bool ok = true; + struct stat stats; if (n_lines == 0) return true; + if (fstat (fd, &stats) == 0) + /* sysfs files have a blksize == kernel page size, which can be > BUFSIZ. + lseek SEEK_END moves to the next page, which is likely more than + BUFSIZ away from the end of the actual data. Use a buffer large enough + to hold an entire page, so that the window we search for data is large + enough to find it. */ + bufsize = MAX(BUFSIZ, ST_BLKSIZE (stats)); + + buffer = xmalloc (bufsize); + /* Set 'bytes_read' to the size of the last, probably partial, buffer; - 0 < 'bytes_read' <= 'BUFSIZ'. */ - bytes_read = (pos - start_pos) % BUFSIZ; + 0 < 'bytes_read' <= 'bufsize'. */ + bytes_read = (pos - start_pos) % bufsize; if (bytes_read == 0) - bytes_read = BUFSIZ; - /* Make 'pos' a multiple of 'BUFSIZ' (0 if the file is short), so that all + bytes_read = bufsize; + /* Make 'pos' a multiple of 'bufsize' (0 if the file is short), so that all reads will be on block boundaries, which might increase efficiency. */ pos -= bytes_read; xlseek (fd, pos, SEEK_SET, pretty_filename); @@ -538,7 +551,8 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; @@ -565,7 +579,7 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xwrite_stdout (nl + 1, bytes_read - (n + 1)); *read_pos += dump_remainder (false, pretty_filename, fd, end_pos - (pos + bytes_read)); - return true; + goto free_buffer; } } @@ -577,23 +591,26 @@ file_lines (char const *pretty_filename, int fd, uintmax_t n_lines, xlseek (fd, start_pos, SEEK_SET, pretty_filename); *read_pos = start_pos + dump_remainder (false, pretty_filename, fd, end_pos); - return true; + goto free_buffer; } - pos -= BUFSIZ; + pos -= bufsize; xlseek (fd, pos, SEEK_SET, pretty_filename); - bytes_read = safe_read (fd, buffer, BUFSIZ); + bytes_read = safe_read (fd, buffer, bufsize); if (bytes_read == SAFE_READ_ERROR) { error (0, errno, _("error reading %s"), quoteaf (pretty_filename)); - return false; + ok = false; + goto free_buffer; } *read_pos = pos + bytes_read; } while (bytes_read > 0); - return true; +free_buffer: + free (buffer); + return ok; } /* Print the last N_LINES lines from the end of the standard input, diff --git a/tests/local.mk b/tests/local.mk index a5fb62d96..af41337b1 100644 --- a/tests/local.mk +++ b/tests/local.mk @@ -257,6 +257,7 @@ all_tests = \ tests/seq/seq-precision.sh \ tests/head/head.pl \ tests/head/head-elide-tail.pl \ + tests/tail/tail-1-sysfs.sh \ tests/tail/tail-n0f.sh \ tests/ls/ls-misc.pl \ tests/date/date.pl \ diff --git a/tests/tail/tail-1-sysfs.sh b/tests/tail/tail-1-sysfs.sh new file mode 100755 index 000000000..078366588 --- /dev/null +++ b/tests/tail/tail-1-sysfs.sh @@ -0,0 +1,31 @@ +#!/bin/sh +# sysfs files have weird properties that can be influenced by page size + +# Copyright 2023 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src +print_ver_ tail + +file=/sys/kernel/profiling + +if test -r $file; then + cp -f $file exp || framework_failure_ + tail -1 $file > out || fail=1 + + compare exp out || fail=1 +fi + +Exit $fail -- 2.43.0 From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 29 20:34:41 2023 Received: (at control) by debbugs.gnu.org; 30 Nov 2023 01:34:41 +0000 Received: from localhost ([127.0.0.1]:51812 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8Vwv-0001bT-Jl for submit@debbugs.gnu.org; Wed, 29 Nov 2023 20:34:41 -0500 Received: from smtp-relay-internal-1.canonical.com ([185.125.188.123]:44512) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8Vws-0001bB-Iw for control@debbugs.gnu.org; Wed, 29 Nov 2023 20:34:39 -0500 Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 85BFF40185 for ; Thu, 30 Nov 2023 01:34:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701308064; bh=srTqsuws6Mktg8R/C+woPM9Y8VQwdWqsdZbDT280OtE=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=OUluaQxnNfpYdlDMVahye3l0NqSuCR6nqBxjNoWMQwQePmua0eODZMWo0ZQNOasJ2 gJPWXlQ7+aerNitmO8mU82CMHb+0FbRLJfu3UQDX+4E1S0smdDkjT7ItAWS/W2TOwR NefGAcQVo4TnhEDbOH5EIBkln2QY6bWFIve1fpHZLFkif6p+di01LG+bFA3XfZHvUr GSv1x4XUPHqyNkUJd+GK/z4UZZ76xLuHjXGte7HjsbD1ulaZa6hMGNqz0dCFPGw7N+ aLzLAkUfY3bpg7iSfO2JhWFuzoYtzH3urdcHEIefrlzf4iRsAKnCqQGtFg6/ztHXhM xckb+pkvDoKxg== Received: by mail-io1-f72.google.com with SMTP id ca18e2360f4ac-7b39e0d8185so30658439f.0 for ; Wed, 29 Nov 2023 17:34:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701308062; x=1701912862; h=content-disposition:mime-version:message-id:subject:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=srTqsuws6Mktg8R/C+woPM9Y8VQwdWqsdZbDT280OtE=; b=EKVOtwdj0BoJxps3qE4hFbnSE6T9AEMzL92cNtwCwN03vJnYZz/Pnm+8mBCbS2P7nu pz0FBwCiEaCaWukbCF0cPhtm41lK9sn4tVKNf0Qx6AyFzNfPrm/cIiIbqjs/rf9s7WuD J0j9Q5XamNQu+LQYCTZIcMTLQ7OFmyZiqY1gVS8LyCec5XwgAQaWa78FqTOclAClQ70T u0By7Owk3qdgezFnSw6PKoCBCQiPfEzfpDFGdePmb4n9VILm7d1s086rxY5qMiZmw8bI MejRJvADYD3BO6iaIRO/rclTqsYsK92oM4ovricRcIaJktW6a4Rm/ez4Pl18NxPsQe3y RQKA== X-Gm-Message-State: AOJu0YzglowopUF8qaNuLJ90gxX60w055HzcDsuDPv/ukGWkmKfhsPSV 7awlVXuPBuIuI709eHT2i9b5J9S7T4qUHzedRef2aDfps7L24BN3F9Fyte+H8kWt3M2snsmMt/J i0VW96GS9Iyx7Aksbh7htBJFt0KcHkE7hbOYDVdpiwrI= X-Received: by 2002:a05:6e02:214f:b0:35c:ca58:645 with SMTP id d15-20020a056e02214f00b0035cca580645mr12621493ilv.13.1701308062451; Wed, 29 Nov 2023 17:34:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IGgMJktmYITwJW9JZVtneROcNT6jd1F9WZD7f25fn6hZPOnDinRsuikJ8DdptUnhf25CGGqkg== X-Received: by 2002:a05:6e02:214f:b0:35c:ca58:645 with SMTP id d15-20020a056e02214f00b0035cca580645mr12621487ilv.13.1701308062167; Wed, 29 Nov 2023 17:34:22 -0800 (PST) Received: from xps13.dannf ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id o17-20020a92d4d1000000b0035d249ed77csm43437ilm.35.2023.11.29.17.34.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Nov 2023 17:34:21 -0800 (PST) Date: Wed, 29 Nov 2023 18:34:19 -0700 From: dann frazier To: control@debbugs.gnu.org Subject: retitle Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.8 (-) retitle 67490 [PATCH v2] tail: fix tailing sysfs files on large page kernels thanks From debbugs-submit-bounces@debbugs.gnu.org Thu Nov 30 15:12:15 2023 Received: (at 67490) by debbugs.gnu.org; 30 Nov 2023 20:12:15 +0000 Received: from localhost ([127.0.0.1]:54539 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8nOP-00014H-JB for submit@debbugs.gnu.org; Thu, 30 Nov 2023 15:12:15 -0500 Received: from mail-wr1-x42e.google.com ([2a00:1450:4864:20::42e]:50547) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8nOJ-00013z-9z for 67490@debbugs.gnu.org; Thu, 30 Nov 2023 15:12:11 -0500 Received: by mail-wr1-x42e.google.com with SMTP id ffacd0b85a97d-332e40315bdso998362f8f.1 for <67490@debbugs.gnu.org>; Thu, 30 Nov 2023 12:11:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701375113; x=1701979913; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=eD+kjAHp+4EIebzmLwHQK4ZFlL0pGUnKFIiwQMyP/ZE=; b=aew6DdH3WEwoQiAi6EHSmLUPmaHloATiwbcK2LZLAqXk6eGSj232/NPN5ruOcmHyKz 9+vNOVw9lb02HcR8odbmCFRUF8a6rY9qsGvR6cDzQ6ueNClyqE3pY6OTU/ZodJH0ftRJ huWqe1UyyCGuiKyzazWXltHKdPPfxeRERgfgbZM0MiHKPCRLdW3OFr+KB2vox97aDsUa tTXGylQRyxSAHR6ZkBdGpHE5FLx9AKyFpNPKFB0lICA/xirtwHspBCXmg3N3S2V2sN7M wxcAWfK7f7woL8XzJNWlVyBQzYv9N34D1T4tBZfn/eoTvAAeHFWcc9SmAnxVG4fEYgaB tc/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701375113; x=1701979913; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=eD+kjAHp+4EIebzmLwHQK4ZFlL0pGUnKFIiwQMyP/ZE=; b=NjoxHbVn0FOhsQZEwVwKrymHC02IPpogAToFw9kDJfcP+h6GET8WKJWYW2QDx8zeSV nSgx/X9WIdYr5gGxIIB4jv39ikg+NqLVfZGjsHh0BUbOwJTuFQfjJ23MICLcIxtPUWwi wooyCq9FA9vneR1wJmVNqT6qWC0ys1hhIU6HFoLma1yQW77ac5grN6FSQyLwkUrf4Twj gJr2KPPKd3VS564WskMzfTCe8P9lyM+y4khBH67QKh5cou9bWGh1X4PNcvbglI38JX7W emtF12Nkt6cbseMaNw34B8peW9kkZ5ze2elpzQ63B41cz6BohYEi3gCcK0wN25fXn7FQ lpQQ== X-Gm-Message-State: AOJu0YyZZ+Fup1QpXJMC8QnteZgd+H1woY5hItw48CzGKtS7EsuUii+E yYDnYYLVmJF81kEZcMWcPzQ= X-Google-Smtp-Source: AGHT+IHG9X7jqh4xT8Tvrvi2nRhhRT8yasnY5BQE83ZeGpDl6PSc/qOZH2SIapM29H1RBKiOUsLizQ== X-Received: by 2002:a5d:618f:0:b0:32f:dd59:fd0 with SMTP id j15-20020a5d618f000000b0032fdd590fd0mr88039wru.52.1701375112921; Thu, 30 Nov 2023 12:11:52 -0800 (PST) Received: from [192.168.1.20] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id n10-20020a05600c4f8a00b004053e9276easm6675789wmq.32.2023.11.30.12.11.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Nov 2023 12:11:52 -0800 (PST) Message-ID: <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> Date: Thu, 30 Nov 2023 20:11:52 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels Content-Language: en-US To: dann frazier , 67490@debbugs.gnu.org References: <20231127162433.924238-1-dann.frazier@canonical.com> <20231130013234.135285-1-dann.frazier@canonical.com> From: =?UTF-8?Q?P=C3=A1draig_Brady?= In-Reply-To: <20231130013234.135285-1-dann.frazier@canonical.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 67490 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) Much clearer thanks. On my system: $ stat /sys/kernel/profiling File: /sys/kernel/profiling Size: 4096 Blocks: 0 IO Block: 4096 regular file I can easily repro by setting the buffer size < PAGE_SIZE. So this patch handles the case where sysfs reports a file is a certain size, but it isn't really. In that case seeking to anywhere other than the start doesn't give an error, but reading returns nothing. So we use a buffer size large enough (>= PAGE_SIZE as inferred from st_blksize) so that we'll be reading from the start of the file in this case. Note st_blksize can have unusual values, so it might be better to use the io_blksize() wrapper to sanitize the values. Though that will generally give 128K, which is good when processing all of a file, but perhaps overkill when processing just the last part of a file. So perhaps it's better to use buffer_size = MAX (BUFSIZ, getpagesize ()) and avoid all the st_blksize edge cases. I'll think about it a little, and make the adjustments. thanks! Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Thu Nov 30 15:37:47 2023 Received: (at 67490) by debbugs.gnu.org; 30 Nov 2023 20:37:47 +0000 Received: from localhost ([127.0.0.1]:54546 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8nn8-0001ir-Lp for submit@debbugs.gnu.org; Thu, 30 Nov 2023 15:37:46 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]:57842) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8nn5-0001ie-W4 for 67490@debbugs.gnu.org; Thu, 30 Nov 2023 15:37:45 -0500 Received: from mail-io1-f69.google.com (mail-io1-f69.google.com [209.85.166.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 4995B3FA52 for <67490@debbugs.gnu.org>; Thu, 30 Nov 2023 20:37:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701376649; bh=pOd7/rOw35h8fnBEcstb9QOFdzjndMjV8hDXusnNWMA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=MyVRp666q7l03dAalS0ig8KGya7CorO9zxFhB45M048OlwERgthrTwzDyvR/bdqBf 4hfZKLn4qCrRCMtemC4IS0B9jeCqBf6imj53MkUJLasMuftaORtW1JR45Jl4zm2Ht+ aGQClpyeJTmB4+X+xOW0YWbG3/2PlmOJG7X07n8fcBWXtDSWl4jIKkJdCxNc03CFTC QNUOP3wYb58T93sjciTkNDsgL2lIUXE1ILeFMC23QhzZpEpIO4ZgzVXA2ENo80XHxW juS4EooYQAVeGnVJ1vyOgyDed4lJVwPusypkyJTF3HBbDckU90RjH5jY6fYQj1tjvM dwA5dU+agyPlA== Received: by mail-io1-f69.google.com with SMTP id ca18e2360f4ac-7b3f80e659aso126714839f.2 for <67490@debbugs.gnu.org>; Thu, 30 Nov 2023 12:37:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701376648; x=1701981448; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pOd7/rOw35h8fnBEcstb9QOFdzjndMjV8hDXusnNWMA=; b=n8tsDSabqfRDS8XDcFC/cZAysPICBAQlyDV1dHwH3E3WsDCIgogKgdwlppWD801RrO 3viGl1dc0tYC2ArOwSCf1fh4Zf/eIATcNhi5BFhonxSsWyZARxjzrdK3nEJmih9LMxDE IjwfmCwv88SiekWmgAVkh7ZhrMwzIeuvrWxOAd8ZqsiHV4FV+F9MFUUoxzDNdzZn7iTr x6gsxOneLJjl7NQ9HsAUJMqPvSYz8NlSZ2Hlvbt/CsiAAV08mgTCt7IUMutqsTAnDYNa 9X4tyzvx7edDkBi3PCvlXGtkBjzmt15ZAIeH/JOXOGcXJ/yNN2KSt/pCDkeaJcv8Rez+ FlQA== X-Gm-Message-State: AOJu0YxYktlwaBYk5kp5SrgsB8t7YyC9F8K+OPWB6DJ543twG4MRK8Ga vD6q3U2ZVGYu7lvon0q2IUkR5CXm1EgNdHIXg6zXunYjkDiy/7UXnAsrMjF7Kex4HPk+3WK9tWB 5NyanNcHjCi5eVmeUp0MkFfgXGrH3BL1N X-Received: by 2002:a5d:9c10:0:b0:7b0:aee5:a413 with SMTP id 16-20020a5d9c10000000b007b0aee5a413mr25257871ioe.15.1701376648183; Thu, 30 Nov 2023 12:37:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IEQKiGzWCsK0hWXMs70Cq/3F6+vV2o4nG+j/4EOKgm/JeXRfabKp7lr9C/O4fGUK78pdtVffQ== X-Received: by 2002:a5d:9c10:0:b0:7b0:aee5:a413 with SMTP id 16-20020a5d9c10000000b007b0aee5a413mr25257853ioe.15.1701376647790; Thu, 30 Nov 2023 12:37:27 -0800 (PST) Received: from xps13.dannf ([38.15.40.168]) by smtp.gmail.com with ESMTPSA id h12-20020a6b784c000000b007b05b1e5029sm559472iop.28.2023.11.30.12.37.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 12:37:27 -0800 (PST) Date: Thu, 30 Nov 2023 13:37:25 -0700 From: dann frazier To: =?iso-8859-1?Q?P=E1draig?= Brady Subject: Re: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels Message-ID: References: <20231127162433.924238-1-dann.frazier@canonical.com> <20231130013234.135285-1-dann.frazier@canonical.com> <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 67490 Cc: 67490@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On Thu, Nov 30, 2023 at 08:11:52PM +0000, Pádraig Brady wrote: > Much clearer thanks. > > On my system: > > $ stat /sys/kernel/profiling > File: /sys/kernel/profiling > Size: 4096 Blocks: 0 IO Block: 4096 regular file > > I can easily repro by setting the buffer size < PAGE_SIZE. Oh, clever. > So this patch handles the case where sysfs reports a file is a certain size, > but it isn't really. In that case seeking to anywhere other than the start > doesn't give an error, but reading returns nothing. So we use a buffer size > large enough (>= PAGE_SIZE as inferred from st_blksize) so that we'll be > reading from the start of the file in this case. Exactly. > Note st_blksize can have unusual values, so it might be better > to use the io_blksize() wrapper to sanitize the values. > Though that will generally give 128K, which is good when processing all of a file, > but perhaps overkill when processing just the last part of a file. > > So perhaps it's better to use buffer_size = MAX (BUFSIZ, getpagesize ()) > and avoid all the st_blksize edge cases. Is there any performance concern about switching to 64K reads for "normal" files (on these systems)? > I'll think about it a little, and make the adjustments. Appreciated! -dann > thanks! > Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Thu Nov 30 20:54:24 2023 Received: (at 67490) by debbugs.gnu.org; 1 Dec 2023 01:54:24 +0000 Received: from localhost ([127.0.0.1]:54807 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8sjX-0001vR-Oe for submit@debbugs.gnu.org; Thu, 30 Nov 2023 20:54:24 -0500 Received: from mail.cs.ucla.edu ([131.179.128.66]:51520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8sjV-0001vA-JY for 67490@debbugs.gnu.org; Thu, 30 Nov 2023 20:54:22 -0500 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id EBD403C011BDA; Thu, 30 Nov 2023 17:54:06 -0800 (PST) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id q9YG8v0ONPMc; Thu, 30 Nov 2023 17:54:06 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id A2FB03C011BDB; Thu, 30 Nov 2023 17:54:06 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu A2FB03C011BDB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1701395646; bh=ikb2wVRiiycFxDzFbRLjQofqCG63WVP/C6isd1hCG5E=; h=Message-ID:Date:MIME-Version:To:From; b=hU9IvdqnyW7W2yLYLXZecoL+nZWCc+TtFIeK1C4zyCNMSeM+yqfFdFUq0IvmJRxjx Aa3pC7OO254VTky8h8hbxamsJvlPWs3I0yYzYAvGsnF+0Mf5jj/yam8vw7Bvgibd+W jVDp/UsCMktVY8HW9MBuHcIxrbnei+70ijpJCe82p3cwGzRblxXeRzWRwSnU8jWE9u Q0UbV9uI4HfdHocbB/EzgyarP1sHM9lFwgJeikw5wiVfC26ANnTTHVsPVgbowUuusu knjeYzkmiJV6VV25B00MK0Cf0NbsIFSLEOmxpw5bcln7pxvA97Ipw/G6E9Nb/tJiks VyzW8trdXY/Bw== X-Virus-Scanned: amavisd-new at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id UwBN3UGL9cxr; Thu, 30 Nov 2023 17:54:06 -0800 (PST) Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id 899DB3C011BDA; Thu, 30 Nov 2023 17:54:06 -0800 (PST) Message-ID: Date: Thu, 30 Nov 2023 17:54:03 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels To: =?UTF-8?Q?P=C3=A1draig_Brady?= , dann frazier , 67490@debbugs.gnu.org References: <20231127162433.924238-1-dann.frazier@canonical.com> <20231130013234.135285-1-dann.frazier@canonical.com> <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> Content-Language: en-US From: Paul Eggert Organization: UCLA Computer Science Department In-Reply-To: <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67490 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 11/30/23 12:11, P=C3=A1draig Brady wrote: > Though that will generally give 128K, which is good when processing all= =20 > of a file, > but perhaps overkill when processing just the last part of a file. The 128 KiB number was computed as being better for apps like 'sed' that=20 typically read all or most of the file. 'tail' sometimes behaves that=20 way (e.g., 'tail -c +10') and so 'tail' should use 128 KiB in those=20 cases. The simplest way to do that is for 'tail' to use 128 KiB all the=20 time - that would cost little for uses like plain 'tail' and it could be=20 a significant win for uses like 'tail -c +10'. (As an aside, the 128 KiB number was computed in 2014. These days 256=20 KiB might be better if someone could take the time to measure....) From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 01 18:23:39 2023 Received: (at 67490-done) by debbugs.gnu.org; 1 Dec 2023 23:23:39 +0000 Received: from localhost ([127.0.0.1]:56347 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r9CrD-0005Ue-GS for submit@debbugs.gnu.org; Fri, 01 Dec 2023 18:23:39 -0500 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]:56548) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r9CrA-0005UM-Et for 67490-done@debbugs.gnu.org; Fri, 01 Dec 2023 18:23:38 -0500 Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-40b30308c67so24425755e9.0 for <67490-done@debbugs.gnu.org>; Fri, 01 Dec 2023 15:23:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701473001; x=1702077801; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=Nu2ibAAcBzX4A51A2otvTuiYVYsYVIGK4vUIYk2GNZc=; b=boJ1Y2Mpxh/yM1cmyBMyRj/nF/Q/FelhCTyqENAc6lYJ0+3BBdyomJr+xo91pMF4ns N1OBQoAfHubwpI2zuywyGnLorre1uS8lyqseQz+efPvc4U7G8805WE6qD7CCrz+FzQ1g Wy2XRdvCaVuW/Q2/gVpThVLWQcBLFGuzUyUyujrBhPM997ZKJYZ1UGKxFIU0OjBBRTTE QbWUeVHAJ5sMh30DcOa2jreb1EhH/a2R9u1IhsmX/UqlwbxQMXHd07R8sJQINM2KbYQ4 40/wANGh2L7b4jD9BKxHXBIing4mFBSs7/ZUk3lrROZs2qi7zfh1phbi7zNERZNlLqX+ GCAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701473001; x=1702077801; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Nu2ibAAcBzX4A51A2otvTuiYVYsYVIGK4vUIYk2GNZc=; b=toW/l5LEpmArQjIy1JIWDDQ39EmULJsQEQ+9gnYeHonNfgv4CdtJwXaMmBV0kZacPi s20n7ig138BB7zrEHLvVwJvXHcskokol+4/LiRO4UDhurht51WLb2rBLFUq0iMbxump/ ZdBdMT01Wgy+cjLNuvHUwexoPUrQ+l/pAwvN4kti+RlzAPq6zHK170QSeyIx7OJCxgep T3HgRYI70tvadeh+QF9FNiZSr7j7E8N1tE7AfTbtgZ52hCKw+NC75Y0Nc5N+4m8+4OS4 3mlQwC6YTwc95cWjrdOWgjwfmsMhyImvlLwHNOArasLJ7IKuyKhJlV2QDn/PPYwWu6KI uiUg== X-Gm-Message-State: AOJu0YzRXTBQPm3psTt/a4jCumG5412A61LfpJSGfjYc3AHlk2XWmEz+ hBfpO5zqVuc0PeD7EeibnvY= X-Google-Smtp-Source: AGHT+IGgSumgtQ0gfcHJBBjQvEZS0VsbSUHCv2pud7LewTd+dtlCxue2zz5cL150PGasqdeyv5cQ8g== X-Received: by 2002:a5d:6ad2:0:b0:333:2fd2:6f7b with SMTP id u18-20020a5d6ad2000000b003332fd26f7bmr1214618wrw.133.1701473000844; Fri, 01 Dec 2023 15:23:20 -0800 (PST) Received: from [192.168.1.20] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id dm11-20020a0560000bcb00b0033334625bdbsm1716028wrb.13.2023.12.01.15.23.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 01 Dec 2023 15:23:20 -0800 (PST) Message-ID: <52ff0042-1eea-acac-e355-7e7dd62f5956@draigBrady.com> Date: Fri, 1 Dec 2023 23:23:19 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#67490: [PATCH v2] tail: fix tailing sysfs files on large page kernels Content-Language: en-US To: Paul Eggert , dann frazier , 67490-done@debbugs.gnu.org References: <20231127162433.924238-1-dann.frazier@canonical.com> <20231130013234.135285-1-dann.frazier@canonical.com> <1823a6db-210a-c42c-0cc3-82cc9b880005@draigBrady.com> From: =?UTF-8?Q?P=C3=A1draig_Brady?= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 67490-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 01/12/2023 01:54, Paul Eggert wrote: > On 11/30/23 12:11, Pádraig Brady wrote: >> Though that will generally give 128K, which is good when processing all >> of a file, >> but perhaps overkill when processing just the last part of a file. > > The 128 KiB number was computed as being better for apps like 'sed' that > typically read all or most of the file. 'tail' sometimes behaves that > way (e.g., 'tail -c +10') and so 'tail' should use 128 KiB in those > cases. The simplest way to do that is for 'tail' to use 128 KiB all the > time - that would cost little for uses like plain 'tail' and it could be > a significant win for uses like 'tail -c +10'. Yes I agree we should use io_blksize() in other routines in tail where we may dump lots of a file. However in this (most common) case the routine is dealing with the end of a regular file, so it's probably best to somewhat minimize the amount of data read, and more directly check the page_size which is issue at hand. I've pushed the fix at https://github.com/coreutils/coreutils/commit/73d119f4f where the adjustment (which also corresponds to what we do in wc) is: if (sb->st_size % page_size == 0) bufsize = MAX (BUFSIZ, page_size); I'll follow up with another patch to address the performance aspect, which uses io_blksize() where appropriate. > (As an aside, the 128 KiB number was computed in 2014. These days 256 > KiB might be better if someone could take the time to measure....) I periodically check with the documented script, but I should update the comment when I do that. I'll update the date in the comment now at least. I quickly tested a few systems here, which suggests 256KiB _may_ be a more appropriate default now. More testing required. Marking this as done. thanks, Pádraig From unknown Sun Jun 22 03:56:59 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 30 Dec 2023 12:24:11 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator