From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 12 05:06:19 2022 Received: (at submit) by debbugs.gnu.org; 12 Feb 2022 10:06:19 +0000 Received: from localhost ([127.0.0.1]:33530 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nIpIJ-0007ev-4A for submit@debbugs.gnu.org; Sat, 12 Feb 2022 05:06:19 -0500 Received: from lists.gnu.org ([209.51.188.17]:42946) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nIpIH-0007en-FD for submit@debbugs.gnu.org; Sat, 12 Feb 2022 05:06:18 -0500 Received: from eggs.gnu.org ([209.51.188.92]:46768) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nIpIG-0004uG-RX for automake-patches@gnu.org; Sat, 12 Feb 2022 05:06:17 -0500 Received: from [2001:470:ea4a:1:5054:ff:fec7:86e4] (port=54855 helo=smtp.gentoo.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256) (Exim 4.90_1) (envelope-from ) id 1nIpIE-0008Re-03 for automake-patches@gnu.org; Sat, 12 Feb 2022 05:06:16 -0500 Received: by smtp.gentoo.org (Postfix, from userid 559) id A8BF7342CE0; Sat, 12 Feb 2022 10:06:05 +0000 (UTC) From: Mike Frysinger To: automake-patches@gnu.org Subject: [PATCH] m4: speed up filesystem modification checks Date: Sat, 12 Feb 2022 05:06:07 -0500 Message-Id: <20220212100607.19761-1-vapier@gentoo.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Host-Lookup-Failed: Reverse DNS lookup failed for 2001:470:ea4a:1:5054:ff:fec7:86e4 (failed) Received-SPF: pass client-ip=2001:470:ea4a:1:5054:ff:fec7:86e4; envelope-from=vapier@gentoo.org; helo=smtp.gentoo.org X-Spam_score_int: -33 X-Spam_score: -3.4 X-Spam_bar: --- X-Spam_report: (-3.4 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.7 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.7 (--) The current code sleeps at least 1 second to make sure the generated files are strictly newer than the source files. It does this for a few reasons: POSIX only guarantees that `sleep` accept integers, and filesystems have a history (c.f. Windows) of bad timestamp resolution. For the first part, we can easily probe sleep to see if it accepts a decimal number with a fractional part -- just run `sleep 0.001`. For the second part, we can create two files and then run sleep in a loop to see when one is considered newer than the other. For many projects, this 1 second delay is largely amortized by the rest of the configure script. Autoconf lends itself to being both large & slow. But in projects with many smallish configure scripts with many cached vars, the time to rerun is dominated by this single sleep call. For example, building libgloss against a compiler with many (60+) multilib configurations, we see: [Using sleep 1] $ time ./config.status real 2m28.164s user 0m33.651s sys 0m9.083s [Using sleep 0.1] $ time ./config.status real 0m39.569s user 0m33.517s sys 0m8.969s And in case anyone wonders, going below 0.1s doesn't seem to make a statistically significant difference, at least in this configuration. It appears to be within "noise" limits. [Using sleep 0.001] $ time ./config.status real 0m39.760s user 0m33.342s sys 0m9.080s * m4/sanity.m4: Determine whether `sleep` accepts fractional seconds. Determine (roughly) the filesystem timestamp resolution. Use this to sleep less when waiting for generated file timestamps to update. --- m4/sanity.m4 | 48 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 45 insertions(+), 3 deletions(-) diff --git a/m4/sanity.m4 b/m4/sanity.m4 index 4e44dd9c4096..5aa1e3ad8baf 100644 --- a/m4/sanity.m4 +++ b/m4/sanity.m4 @@ -6,10 +6,52 @@ # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. +# _AM_SLEEP_FRACTIONAL_SECONDS +# ---------------------------- +AC_DEFUN([_AM_SLEEP_FRACTIONAL_SECONDS], [dnl +AC_CACHE_CHECK([whether sleep supports fractional seconds], am_cv_sleep_fractional_seconds, [dnl +AS_IF([sleep 0.001 2>/dev/null], [am_cv_sleep_fractional_seconds=true], [am_cv_sleep_fractional_seconds=false]) +])]) + +# _AM_FILESYSTEM_TIMESTAMP_RESOLUTION +# ----------------------------------- +# Determine the filesystem timestamp resolution. Modern systems are nanosecond +# capable, but historical systems could be millisecond, second, or even 2-second +# resolution. +AC_DEFUN([_AM_FILESYSTEM_TIMESTAMP_RESOLUTION], [dnl +AC_REQUIRE([_AM_SLEEP_FRACTIONAL_SECONDS]) +AC_CACHE_CHECK([the filesystem timestamp resolution], am_cv_filesystem_timestamp_resolution, [dnl +# Use names that lexically sort older-first when the timestamps are equal. +rm -f conftest.file.a conftest.file.b +: > conftest.file.a +AS_IF([$am_cv_sleep_fractional_seconds], [dnl + am_try_sleep=0.1 am_try_loops=20 +], [dnl + am_try_sleep=1 am_try_loops=2 +]) +am_try=0 +while :; do + AS_VAR_ARITH([am_try], [$am_try + 1]) + echo "timestamp $am_try" > conftest.file.b + set X `ls -t conftest.file.a conftest.file.b` + if test "$[2]" = conftest.file.b || test $am_try -eq $am_try_loops; then + break + fi + sleep $am_try_sleep +done +rm -f conftest.file.a conftest.file.b +am_cv_filesystem_timestamp_resolution=$am_try +AS_IF([$am_cv_sleep_fractional_seconds], [dnl + AS_VAR_ARITH([am_cv_filesystem_timestamp_resolution], [$am_try / 10]) + AS_VAR_ARITH([am_fraction], [$am_try % 10]) + AS_VAR_APPEND([am_cv_filesystem_timestamp_resolution], [.$am_fraction]) +]) +])]) + # AM_SANITY_CHECK # --------------- AC_DEFUN([AM_SANITY_CHECK], -[dnl +[AC_REQUIRE([_AM_FILESYSTEM_TIMESTAMP_RESOLUTION]) rm -f conftest.file AC_CACHE_CHECK([whether build environment is sane], am_cv_build_env_is_sane, [dnl # Reject unsafe characters in $srcdir or the absolute working directory @@ -53,7 +95,7 @@ if ( break fi # Just in case. - sleep 1 + sleep $am_cv_filesystem_timestamp_resolution am_has_slept=yes done test "$[2]" = conftest.file @@ -69,7 +111,7 @@ fi # generated files are strictly newer. am_sleep_pid= if ! test -e conftest.file || grep 'slept: no' conftest.file >/dev/null 2>&1; then - ( sleep 1 ) & + ( sleep $am_cv_filesystem_timestamp_resolution ) & am_sleep_pid=$! fi AC_CONFIG_COMMANDS_PRE( -- 2.34.1 From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 12 16:51:20 2022 Received: (at 53951) by debbugs.gnu.org; 12 Feb 2022 21:51:20 +0000 Received: from localhost ([127.0.0.1]:35841 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nJ0Ia-0001WT-Hc for submit@debbugs.gnu.org; Sat, 12 Feb 2022 16:51:20 -0500 Received: from mail-yb1-f179.google.com ([209.85.219.179]:39644) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nJ0IZ-0001WG-4r for 53951@debbugs.gnu.org; Sat, 12 Feb 2022 16:51:19 -0500 Received: by mail-yb1-f179.google.com with SMTP id p19so35246084ybc.6 for <53951@debbugs.gnu.org>; Sat, 12 Feb 2022 13:51:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=U5EMJ1viFhAeiDvztYeZAsgTvTMM8VMZRIsk8+c/szM=; b=NUtyYV1fJWTuywydU4h6oIkaVa+cOwzA7Jopi2GVwWScW+Lq2gfhmF4NaobQDwueal 9REFe+7mQTMMykXsPGTwds9n5ou4oxgVprYEAAvGM1WANovpcBwSvDP5aSmvsNzHyw6a VPQKRvtZxFQeWuNdfM0K6mH8fWCO0Pf+qHYEJWHgCqmYgtif99+v6rrKYd/hGa4nNTn7 pimh74c5FWPV0jG9zSJvXvEZKXB+zbnXt2dEpFXBR1AqpfDP8tzFgVHlS7AzL0aks4Qt 8gJ+aSWa14FYHe6tAs/P4Ff/40XUX/ZL7awgpy/bMFLyVjIiOwG4/dhvequNJJKmbKcH Dy1w== X-Gm-Message-State: AOAM53064Xm5nNdy5lDyPpZ5C3g1KEPyeOXEuTgiXGkDNr+QVbfA5hIv PMdDgz3v/CMqBXh5moK8qL3c3mtBiQWOn34qbNfhIBjj X-Google-Smtp-Source: ABdhPJxeix4U2JJA9/lFezqEt2mkNg6EKUSfFr7zKW2rXhUc0YgEsC2I3lScjKkFrTCWOaC24PQmsg00rmzUxAJ5vTc= X-Received: by 2002:a81:2fc6:: with SMTP id v189mr7518482ywv.411.1644702673418; Sat, 12 Feb 2022 13:51:13 -0800 (PST) MIME-Version: 1.0 References: <20220212100607.19761-1-vapier@gentoo.org> In-Reply-To: <20220212100607.19761-1-vapier@gentoo.org> From: Jim Meyering Date: Sat, 12 Feb 2022 13:51:01 -0800 Message-ID: Subject: Re: [bug#53951] [PATCH] m4: speed up filesystem modification checks To: Mike Frysinger Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 53951 Cc: 53951@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On Sat, Feb 12, 2022 at 2:07 AM Mike Frysinger wrote: > The current code sleeps at least 1 second to make sure the generated > files are strictly newer than the source files. It does this for a > few reasons: POSIX only guarantees that `sleep` accept integers, and > filesystems have a history (c.f. Windows) of bad timestamp resolution. > > For the first part, we can easily probe sleep to see if it accepts a > decimal number with a fractional part -- just run `sleep 0.001`. > > For the second part, we can create two files and then run sleep in a > loop to see when one is considered newer than the other. > > For many projects, this 1 second delay is largely amortized by the > rest of the configure script. Autoconf lends itself to being both > large & slow. But in projects with many smallish configure scripts > with many cached vars, the time to rerun is dominated by this single > sleep call. For example, building libgloss against a compiler with > many (60+) multilib configurations, we see: > [Using sleep 1] > $ time ./config.status > real 2m28.164s > user 0m33.651s > sys 0m9.083s > [Using sleep 0.1] > $ time ./config.status > real 0m39.569s > user 0m33.517s > sys 0m8.969s > > And in case anyone wonders, going below 0.1s doesn't seem to make a > statistically significant difference, at least in this configuration. > It appears to be within "noise" limits. > [Using sleep 0.001] > $ time ./config.status > real 0m39.760s > user 0m33.342s > sys 0m9.080s > > * m4/sanity.m4: Determine whether `sleep` accepts fractional seconds. > Determine (roughly) the filesystem timestamp resolution. Use this to > sleep less when waiting for generated file timestamps to update. Nice work. I looked through the patch and didn't see any issue. From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 13 03:50:34 2023 Received: (at control) by debbugs.gnu.org; 13 Jan 2023 08:50:34 +0000 Received: from localhost ([127.0.0.1]:49530 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pGFli-00020A-IY for submit@debbugs.gnu.org; Fri, 13 Jan 2023 03:50:34 -0500 Received: from woodpecker.gentoo.org ([140.211.166.183]:57040 helo=smtp.gentoo.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pGFlg-0001zw-SE for control@debbugs.gnu.org; Fri, 13 Jan 2023 03:50:33 -0500 Received: by smtp.gentoo.org (Postfix, from userid 559) id 581CC340955; Fri, 13 Jan 2023 08:50:27 +0000 (UTC) From: Mike Frysinger To: control@debbugs.gnu.org Subject: Control message User-Agent: GNU debbugs/0 Message-Id: <20230113085027.581CC340955@smtp.gentoo.org> Date: Fri, 13 Jan 2023 08:50:27 +0000 (UTC) X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) close 53951 thankyou From unknown Tue Jun 17 21:54:23 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 10 Feb 2023 12:24:07 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator