From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 25 06:26:07 2021 Received: (at submit) by debbugs.gnu.org; 25 Mar 2021 10:26:07 +0000 Received: from localhost ([127.0.0.1]:37216 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lPNBf-0000PL-3V for submit@debbugs.gnu.org; Thu, 25 Mar 2021 06:26:07 -0400 Received: from lists.gnu.org ([209.51.188.17]:58764) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lPNBc-0000PD-U5 for submit@debbugs.gnu.org; Thu, 25 Mar 2021 06:25:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46442) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lPNBc-0008KR-Kl for bug-automake@gnu.org; Thu, 25 Mar 2021 06:25:56 -0400 Received: from mail.aegee.org ([144.76.142.78]:49598) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lPNBS-0001sV-0r for bug-automake@gnu.org; Thu, 25 Mar 2021 06:25:56 -0400 Authentication-Results: mail.aegee.org/12PAOHYd2910145; auth=pass (LOGIN) smtp.auth=didopalauzov@aegee.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=aegee.org; s=k4096; t=1616667927; i=dkim+MSA-tls@aegee.org; bh=iHjhjigSEWtxrYYYU+Y3d1rWctvbuzuG+JHF0iC8fd4=; h=Subject:From:To:Date; b=j776sh//mLRxmYW0hEP2tS7oBJRJi+twYDKhVZB9NL6gC9w2Epvhg/Efspo0TD7lM GkfAOzwTfVGpvZrDyxg+HL/n0+AtrvUNtJKauY1mnDj7DEMqoGOmFsTr6ofG13xO5h 5ml2LieWihMNR+zOxbeClQVha42JyIGpiG7J1SKi4rnXR9yh6fzvJ764Oc76/YMNoq FpjVU4xq8XmUALjkPqumvfveBylcyoiWoHWgnAeOF3Ug/YrkUBdkwZeRYYgv/nEYuE S34ktFdfDKol+50g3n/8XRUiBaQ95Wdd1rn+JiGHXHnC99QeLdUlOh2LU3ZcXo+uiN cEj11HY1wOi8wY4zmjhiqP7xOqE0iCJOyVKUNaKVgEG+OGbdNdcKxvngz7YCapWfAV /JKKTNCOfg+Il0bg+7qrbRyoOu3GINWNhWvJygNLyzwHE9FrPgUGZCDh13tltktjGz CStTf3lbEjiS2V4/F/OgjXMmhGpQambzD3wJnZRhqwlvKUVkWnIFSPL8+OCdwuzM7u GSqry4/i7Sis0Oz6FsQRJrzhGjXaP41ANjhiIJpxBp5ih+CRZHM5Zdl1aUBCWn/Pdb thvcvW6CSvXtIv8mv7p6UlSySdf8lXf7gWqHxxSPOkxrFWXgaB7tInfQYlKiGkC4hf 7byERUekuNFIkNVEJzU+vcng= Authentication-Results: mail.aegee.org/12PAOHYd2910145; dkim=none Received: from [192.168.0.199] (87.118.146.153.topnet.bg [87.118.146.153] (may be forged)) (authenticated bits=0) by mail.aegee.org (8.15.2/8.15.2) with ESMTPSA id 12PAOHYd2910145 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Thu, 25 Mar 2021 10:25:24 GMT Message-ID: <9339922d778c9576a9cbb2d7f6595dc5ab403857.camel@aegee.org> Subject: Atomic (fast) install From: =?UTF-8?Q?=D0=94=D0=B8=D0=BB=D1=8F=D0=BD_?= =?UTF-8?Q?=D0=9F=D0=B0=D0=BB=D0=B0=D1=83=D0=B7=D0=BE=D0=B2?= To: bug-automake@gnu.org Date: Thu, 25 Mar 2021 12:24:13 +0200 Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.41.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: none client-ip=144.76.142.78; envelope-from=dilyan.palauzov@aegee.org; helo=mail.aegee.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, I use automake with libtool . The latter has “fast install” option. My config.status contains: enable_fast_install='yes' fast_install=$enable_fast_install During “make install” it prints: /bin/sh ./libtool --mode=install /usr/bin/install -c lib/libcyrus_min.la lib/libcyrus.la imap/libcyrus_imap.la sieve/libcyrus_sieve.la '/usr/local/lib' libtool: install: /usr/bin/install -c lib/.libs/libcyrus_min.so.0.0.0 /usr/local/lib/libcyrus_min.so.0.0.0 libtool: install: (cd /usr/local/lib && { ln -s -f libcyrus_min.so.0.0.0 libcyrus_min.so.0 || { rm -f libcyrus_min.so.0 && ln -s libcyrus_min.so.0.0.0 libcyrus_min.so.0; }; }) libtool: install: (cd /usr/local/lib && { ln -s -f libcyrus_min.so.0.0.0 libcyrus_min.so || { rm -f libcyrus_min.so && ln -s libcyrus_min.so.0.0.0 libcyrus_min.so; }; }) libtool: install: /usr/bin/install -c lib/.libs/libcyrus_min.lai /usr/local/lib/libcyrus_min.la libtool: install: /usr/bin/install -c lib/.libs/libcyrus.so.0.0.0 /usr/local/lib/libcyrus.so.0.0.0 libtool: install: (cd /usr/local/lib && { ln -s -f libcyrus.so.0.0.0 libcyrus.so.0 || { rm -f libcyrus.so.0 && ln -s libcyrus.so.0.0.0 libcyrus.so.0; }; }) libtool: install: (cd /usr/local/lib && { ln -s -f libcyrus.so.0.0.0 libcyrus.so || { rm -f libcyrus.so && ln -s libcyrus.so.0.0.0 libcyrus.so; }; }) libtool: install: /usr/bin/install -c lib/.libs/libcyrus.lai /usr/local/lib/libcyrus.la libtool: warning: relinking 'imap/libcyrus_imap.la' for some shared libraries, but it does not print it for all shared libraries. It does not print it for libcyrus_min.so or libcyrus.so Moreover (Auto)make install calls: /bin/sh ./libtool --mode=install /usr/bin/install -c imap/httpd '/usr/local/libexec' which triggers these syscalls: [pid 346601] stat("/usr/local/libexec/httpd", {st_mode=S_IFREG|0755, st_size=3362912, ...}) = 0 [pid 346601] newfstatat(AT_FDCWD, "imap/.libs/httpd", {st_mode=S_IFREG|0755, st_size=3362912, ...}, 0) = 0 [pid 346601] newfstatat(AT_FDCWD, "/usr/local/libexec/httpd", {st_mode=S_IFREG|0755, st_size=3362912, ...}, AT_SYMLINK_NOFOLLOW) = 0 [pid 346601] unlink("/usr/local/libexec/httpd") = 0 [pid 346601] openat(AT_FDCWD, "imap/.libs/httpd", O_RDONLY) = 3 [pid 346601] fstat(3, {st_mode=S_IFREG|0755, st_size=3362912, ...}) = 0 [pid 346601] openat(AT_FDCWD, "/usr/local/libexec/httpd", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4 [pid 346601] fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0 [pid 346601] ioctl(4, BTRFS_IOC_CLONE or FICLONE, 3) = -1 EOPNOTSUPP (Operation not supported) [pid 346601] fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0 [pid 346601] mmap(NULL, 139264, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb61f408000 [pid 346601] read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\2\0>\0\1\0\0\0\260\343A\0\0\0\0\0"..., 131072) = 131072 [pid 346601] write(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\2\0>\0\1\0\0\0\260\343A\0\0\0\0\0"..., 131072) = 131072 [pid 346601] read(3, "\377H\211\302H\213\205h\372\377\377\211\331H\211\306H\307\307\4\252\35 3\212\350t\0\6\0H\213E\340"..., 131072) = 131072 [pid 346601] write(4, "\377H\211\302H\213\205h\372\377\377\211\331H\211\306H\307\307\4\252\35 3\212\350t\0\6\0H\213E\340"..., 131072) = 131072 ... [pid 346601] chmod("/usr/local/libexec/httpd", 0755) = 0 [pid 346601] lstat("/usr/local/libexec/httpd", {st_mode=S_IFREG|0755, st_size=3362912, ...}) = 0 That said, the installation is not atomic. The bytes destination file is replaced byte by byte, rather than writing the result to a temporary file and then calling rename(2), thus ensuring that the file is always executable. Cyrus IMAP is a server, consisting of several executable files (daemons). When an executable file notices, that its timestamp has changed, the daemon terminates and is then started again (from the new executable). The problem is, that during this process, there is a timegap, where the process-file has newer timestamp, but is not executable yet, as `install` currently moves data to it. As far the install executable is concerned, I raised the atomicity to coreutils at debbugs.gnu.org/cgi/bugreport.cgi?bug=47380 . • Please adjust Automake on “make install” to do (per default or with an option) atomic install of files, even when libtool is involved: the file is moved to a temporary destination first, and then rename(2)d, or the source is moved to the destination, disappearing from the source, or something like this. • Why does “fast install” relink some libraries, but others not? • Since ./libtool starts with #! /bin/sh do not call /bin/sh ./libtool --mode=install /usr/bin/install -c imap/httpd '/usr/local/libexec' but ./libtool --mode=install /usr/bin/install -c imap/httpd '/usr/local/libexec' Greetings Дилян From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 26 18:40:10 2021 Received: (at 47381) by debbugs.gnu.org; 26 Mar 2021 22:40:11 +0000 Received: from localhost ([127.0.0.1]:42800 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lPv7i-0000zL-O6 for submit@debbugs.gnu.org; Fri, 26 Mar 2021 18:40:10 -0400 Received: from freefriends.org ([96.88.95.60]:51844) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lPv7h-0000z2-0Q; Fri, 26 Mar 2021 18:40:09 -0400 X-Envelope-From: karl@freefriends.org Received: from freefriends.org (freefriends.org [96.88.95.60]) by freefriends.org (8.14.7/8.14.7) with ESMTP id 12QMd7l2022234 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 26 Mar 2021 16:39:07 -0600 Received: (from apache@localhost) by freefriends.org (8.14.7/8.14.7/Submit) id 12QMd6rj022233; Fri, 26 Mar 2021 16:39:06 -0600 Date: Fri, 26 Mar 2021 16:39:06 -0600 Message-Id: <202103262239.12QMd6rj022233@freefriends.org> From: Karl Berry To: dilyan.palauzov@aegee.org Subject: Re: bug#47381: Atomic (fast) install In-Reply-To: <9339922d778c9576a9cbb2d7f6595dc5ab403857.camel@aegee.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 47381 Cc: 47381@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Thanks for the reports, but: 1) Automake can only run whatever install program has been specified by the user. It does not seem possible (or sensible) to me to guarantee atomic installs at the automake level. (This behavior by cyrus imap also seems undesirable to me, but whatever.) 2) I have never heard of "fast install" before, and have barely ever touched libtool in general. You'll need to ask the libtool maintainers. 3) I have seen explicit invocation of "/bin/sh