GNU bug report logs - #68083
30.0.50; Intermittent build failure with native compilation

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 68083 in the body.
You can then email your comments to 68083 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.50; Intermittent build failure with native compilation
Date: Thu, 28 Dec 2023 09:05:25 -0500

On macOS, I am often getting this when building Emacs from scratch. Could this be a race condition with a parallel build?

I bisected and the problematic commit is: e670412a3e101e70dc26e021f467faece8cb7f6b

In toplevel form:
org/org-element.el:64:2: Error: Eager macro-expansion failure: (native-compiler-error (lambda (arg322 &optional arg323) (let ((f #'macroexpand)) (funcall f arg322 arg323))) "Compiling /private/var/tmp/emacs-plusA30-20231227-
10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln...
File already exists: /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln

Error: file-already-exists (\"File already exists\" \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
  mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0x1fcb3a734512f81>))
  debug-early-backtrace()
  debug-early(error (file-already-exists \"File already exists\" \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
\"))
  rename-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\" \"/private/var/tmp/emacs-plusA30-20231
227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
  comp-delete-or-replace-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\" \"/private/var/tmp/emacs-plusA30
-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\")
  comp--compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
  comp-compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
  comp-final1()
  load-with-code-conversion(\"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.e
l\" nil t)
  command-line-1((\"-l\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"))
  command-line()
  normal-top-level()
")
gmake[3]: *** [Makefile:330: org/org-element.elc] Error 1








In GNU Emacs 30.0.50 (build 1, aarch64-apple-darwin23.2.0, NS
 appkit-2487.30 Version 14.2.1 (Build 23C71)) of 2023-12-27 built on
 Aarons-MacBook-Pro.local
Windowing system distributor 'Apple', version 10.3.2487
System Description:  macOS 14.2.1

Configured using:
 'configure --disable-dependency-tracking --disable-silent-rules
 --enable-locallisppath=/opt/homebrew/share/emacs/site-lisp
 --infodir=/opt/homebrew/Cellar/emacs-plus <at> 30/30.0.50/share/info/emacs
 --prefix=/opt/homebrew/Cellar/emacs-plus <at> 30/30.0.50 --with-xml2
 --with-gnutls --with-native-compilation --without-compress-install
 --without-dbus --without-imagemagick --with-modules --with-rsvg
 --with-webp --with-ns --disable-ns-self-contained 'CFLAGS=-Os -w -pipe
 -mmacosx-version-min=14
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk
 -DFD_SETSIZE=10000 -DDARWIN_UNLIMITED_SELECT'
 'CPPFLAGS=-I/opt/homebrew/opt/zlib/include
 -I/opt/homebrew/opt/jpeg/include -I/opt/homebrew/opt/icu4c/include
 -isystem/opt/homebrew/include -F/opt/homebrew/Frameworks
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk'
 'LDFLAGS=-L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/jpeg/lib
 -L/opt/homebrew/opt/icu4c/lib -L/opt/homebrew/lib
 -F/opt/homebrew/Frameworks -Wl,-headerpad_max_install_names
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk
 -L/opt/homebrew/opt/libgccjit/lib''

Configured features:
ACL GIF GLIB GMP GNUTLS JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY KQUEUE NS PDUMPER PNG RSVG SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  windmove-mode: t
  global-evil-mc-mode: t
  evil-mc-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  global-git-commit-mode: t
  transient-posframe-mode: t
  corfu-prescient-mode: t
  corfu-history-mode: t
  eval-sexp-fu-flash-mode: t
  tabspaces-mode: t
  org-roam-db-autosync-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  vertico-prescient-mode: t
  prescient-persist-mode: t
  vertico-mouse-mode: t
  vertico-mode: t
  mini-frame-mode: t
  better-jumper-mode: t
  better-jumper-local-mode: t
  ns-auto-titlebar-mode: t
  global-anzu-mode: t
  anzu-mode: t
  which-key-posframe-mode: t
  which-key-mode: t
  gcmh-mode: t
  xterm-mouse-mode: t
  global-auto-revert-mode: t
  save-place-mode: t
  winner-mode: t
  savehist-mode: t
  delete-selection-mode: t
  recentf-mode: t
  repeat-mode: t
  +popup-mode: t
  evil-mode: t
  evil-local-mode: t
  server-mode: t
  leader-key-leader-override-mode: t
  global-leader-key-leader-override-mode: t
  elpaca-use-package-mode: t
  override-global-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tab-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  window-divider-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  auto-fill-function: #[128 \304\300\301%3#\207 [yas--auto-fill do-auto-fill :around nil apply] 5 advice]
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/Users/aaronjensen/.emacs.d/elpaca/builds/lispy/elpa hides /Users/aaronjensen/.emacs.d/elpaca/builds/ivy/elpa

Features:
(org-clock image-file image-converter org-drill persist tramp-cmds
dabbrev tab-line evil-collection-vterm vterm tramp tramp-message
trampver tramp-integration files-x tramp-compat tramp-loaddefs term
ehelp vterm-module term/xterm xterm json-mode json-snatcher lua-mode
lsp-jq lsp-zig lsp-tilt lsp-steep lsp-svelte lsp-sqls lsp-solidity
lsp-ruby-syntax-tree lsp-ruby-lsp lsp-yaml lsp-xml lsp-vimscript
lsp-vhdl lsp-volar lsp-vetur lsp-html lsp-verilog lsp-vala lsp-v
lsp-typeprof lsp-ttcn3 lsp-toml lsp-terraform lsp-tex lsp-sorbet
lsp-solargraph lsp-semgrep lsp-rust lsp-rubocop lsp-rf lsp-ruff-lsp
lsp-remark lsp-racket lsp-r lsp-purescript lsp-pylsp lsp-pyls lsp-pwsh
lsp-php lsp-pls lsp-perlnavigator lsp-perl lsp-openscad lsp-ocaml
lsp-mojo lsp-magik lsp-nix lsp-nim lsp-nginx lsp-move lsp-mint lsp-mdx
lsp-marksman lsp-markdown lsp-lua lsp-kotlin lsp-json lsp-javascript
lsp-idris lsp-haxe lsp-groovy lsp-hack lsp-graphql lsp-glsl lsp-gleam
lsp-go lsp-completion lsp-gdscript lsp-fsharp lsp-fortran lsp-eslint
lsp-erlang lsp-emmet lsp-elixir lsp-elm lsp-dockerfile lsp-dhall lsp-d
lsp-cypher lsp-css lsp-csharp lsp-crystal lsp-credo lsp-cmake
lsp-clojure lsp-semantic-tokens lsp-clangd lsp-beancount lsp-bash
lsp-astro lsp-awk lsp-ansible lsp-angular lsp-ada lsp-actionscript
evil-collection-simple lsp-mode lsp-protocol spinner markdown-mode
inline ht js c-ts-common cc-mode cc-fonts cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine sql view cal-iso diary-lib diary-loaddefs
evil-ruby-text-objects ruby-refactor bundler inf-ruby ruby-mode
sh-script smie treesit epa-file network-stream mailalias smtpmail
textsec uni-scripts idna-mapping uni-confusable textsec-check consult
magit-bookmark bookmark dired-aux shadow sort mail-extr emacsbug
evil-matchit-simple evil-matchit-prog evil-matchit evil-matchit-sdk
semantic/lex semantic/fw hippie-exp evil-collection-helpful helpful
cc-langs cc-vars cc-defs trace info-look elisp-refs hide-mode-line info
magit-patch magit-subtree magit-gitignore magit-ediff
evil-collection-ediff ediff ediff-merg ediff-mult ediff-wind ediff-diff
ediff-help ediff-init ediff-util magit-extras vc-hg vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs log-view bug-reference elpaca-log elpaca-ui
popup-mode-core windmove evil-terminal-cursor-changer color executable
evil-mc evil-mc-command-execute evil-mc-command-record
evil-mc-cursor-make evil-mc-region evil-mc-cursor-state evil-mc-undo
evil-mc-vars evil-mc-known-commands evil-mc-common magit-delta
xterm-color flycheck evil-collection-magit magit-submodule magit-blame
magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch
magit-clone magit-remote magit-commit magit-sequence magit-notes
magit-worktree magit-tag magit-merge magit-branch magit-reset
magit-files magit-refs magit-status magit package url-handlers
magit-repos magit-apply magit-wip magit-log which-func magit-diff
smerge-mode diff git-commit log-edit pcvs-util add-log magit-core
magit-autorevert magit-margin magit-transient magit-process with-editor
magit-mode transient-posframe transient magit-git magit-base crm
vertico-directory cape corfu-prescient corfu-history corfu dtrt-indent
eval-sexp-fu eros lispyville lispy lispy-inline avy etags fileloop
evil-collection-edebug edebug lispy-tags mode-local zoutline elisp-def
ert ewoc evil-collection-xref xref f f-shortdoc sotlisp skeleton
elec-pair envrc inheritenv evil-surround evil-matchit-evil-setup
tabspaces dired-x vc vc-git diff-mode vc-dispatcher flyspell ispell
org-indent org-appear orgonomic org-superstar form-feed oc-basic ol-eww
eww url-queue mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect
gnus-art mm-uu mml2015 mm-view mml-smime smime gnutls dig gnus-sum shr
pixel-fill kinsoku url-file svg dom browse-url gnus-group gnus-undo
gnus-start gnus-dbus dbus xml gnus-cloud nnimap nnmail mail-source utf7
nnoo parse-time gnus-spec gnus-int gnus-range message sendmail
yank-media rfc822 mml mml-sec epa epg rfc6068 epg-config mm-decode
mm-bodies mm-encode mailabbrev gmm-utils mailheader gnus-win gnus
nnheader gnus-util mail-utils range ol-docview doc-view jka-compr
image-mode exif dired dired-loaddefs ol-bibtex bibtex iso8601 ol-bbdb
ol-w3m ol-doi org-link-doi org-download url-http url url-proxy
url-privacy url-expand url-methods url-history mailcap url-auth
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
url-cookie generate-lisp-file url-domsuf url-util url-gw nsm puny async
vulpea vulpea-meta vulpea-select vulpea-buffer vulpea-db s vulpea-utils
vulpea-note org-roam-migrate org-roam-log org-roam-mode org-roam-capture
org-roam-id org-roam-node org-roam-db emacsql-sqlite-builtin sqlite
org-roam-utils org-roam-compat org-roam org-mac-link org-goto
org-capture org-attach emacsql-sqlite emacsql-sqlite-common emacsql
emacsql-compiler magit-section cursor-sensor dash evil-org-agenda
evil-org imenu ox-odt rng-loc rng-uri rng-parse rng-match rng-dt
rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex
ox-icalendar org-agenda ox-ascii ox-gfm ox-md ox-html table ox-publish
ox org-element org-persist xdg org-id org-refile avl-tree generator
org-tempo tempo ob-shell shell org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-src ob-comint org-pcomplete pcomplete org-list
org-footnote org-faces org-entities noutline outline ob-emacs-lisp
ob-core ob-eval org-cycle org-table ol org-fold org-fold-core org-keys
oc org-loaddefs cal-menu calendar cal-loaddefs org-version org-compat
org-macs format-spec undo-fu-session ws-butler yasnippet
vertico-prescient prescient char-fold vertico-mouse vertico mini-frame
better-jumper ns-auto-titlebar evil-anzu anzu which-key-posframe
posframe evil-collection-which-key which-key gcmh help-fns radix-tree
cl-print xt-mouse autorevert filenotify saveplace winner hl-line
evil-collection-ibuffer ibuffer ibuffer-loaddefs savehist delsel
popup-mode-hacks evil-collection-debug debug backtrace find-func recentf
tree-widget repeat orderless popup-mode popup-mode-settings
evil-collection annalist evil-little-word cus-edit cus-start cus-load
wid-edit pp evil evil-integration evil-maps evil-commands reveal
evil-jumps evil-command-window evil-types evil-search evil-ex
evil-macros evil-repeat evil-states evil-core advice evil-common
thingatpt rect evil-vars memoize nano-modeline nano-light-theme
face-remap nano-theme disp-table project gcmh-autoloads
copy-as-format-autoloads pdf-tools-autoloads tablist-autoloads
restclient-autoloads vterm-autoloads dumb-jump-autoloads popup-autoloads
haml-mode-autoloads emmet-mode-autoloads terraform-mode-autoloads
hcl-mode-autoloads dockerfile-mode-autoloads yaml-mode-autoloads
json-mode-autoloads json-snatcher-autoloads grip-mode-autoloads
lua-mode-autoloads bundler-autoloads inf-ruby-autoloads
ruby-refactor-autoloads evil-ruby-text-objects-autoloads
sotlisp-autoloads elisp-def-autoloads lispyville-autoloads
lispy-autoloads iedit-autoloads swiper-autoloads ivy-autoloads
zoutline-autoloads eros-autoloads eval-sexp-fu-autoloads
web-mode-autoloads ripgrep-capf-autoloads git-link-autoloads
consult-git-commit-autoloads git-timemachine-autoloads
magit-delta-autoloads xterm-color-autoloads prettier-autoloads
iter2-autoloads nvm-autoloads editorconfig-autoloads flycheck-autoloads
pkg-info-autoloads epl-autoloads lsp-ui-autoloads lsp-mode-autoloads
spinner-autoloads markdown-mode-autoloads denote-autoloads
imenu-list-autoloads org-superstar-autoloads ox-gfm-autoloads
org-pandoc-import-autoloads gnuplot-autoloads org-download-autoloads
async-autoloads org-journal-autoloads vulpea-autoloads
org-roam-autoloads emacsql-autoloads orgonomic-autoloads
org-drill-autoloads persist-autoloads org-appear-autoloads
org-mac-link-autoloads evil-org-autoloads
evil-terminal-cursor-changer-autoloads transient-posframe-autoloads
better-jumper-autoloads buffer-move-autoloads rotate-autoloads
mini-frame-autoloads embark-consult-autoloads embark-autoloads
consult-autoloads orderless-autoloads cape-autoloads
corfu-prescient-autoloads corfu-autoloads vertico-prescient-autoloads
vertico-autoloads prescient-autoloads tabspaces-autoloads
which-key-posframe-autoloads which-key-autoloads popup-mode-autoloads
hide-mode-line-autoloads evil-anzu-autoloads anzu-autoloads
titlecase-autoloads wgrep-autoloads yasnippet-autoloads
form-feed-autoloads drag-stuff-autoloads dtrt-indent-autoloads
ws-butler-autoloads evil-collection-autoloads annalist-autoloads
evil-mc-autoloads evil-numbers-autoloads speeddating-autoloads
evil-little-word-autoloads evil-matchit-autoloads
evil-nerd-commenter-autoloads evil-visualstar-autoloads
evil-surround-autoloads vundo-autoloads undo-fu-session-autoloads
ztree-autoloads dwim-shell-command-autoloads treemacs-tab-bar-autoloads
treemacs-magit-autoloads magit-autoloads git-commit-autoloads
magit-section-autoloads with-editor-autoloads treemacs-evil-autoloads
evil-autoloads goto-chg-autoloads treemacs-autoloads
ace-window-autoloads avy-autoloads pfuture-autoloads ht-autoloads
cfrs-autoloads all-the-icons-autoloads rainbow-mode-autoloads
posframe-autoloads ns-auto-titlebar-autoloads nano-modeline-autoloads
nano-theme-autoloads memoize-autoloads envrc-autoloads
inheritenv-autoloads helpful-autoloads f-autoloads elisp-refs-autoloads
s-autoloads dired-subtree-autoloads dired-hacks-utils-autoloads
dash-autoloads server pcase hydra lv url-parse auth-source eieio
eieio-core password-cache json map url-vars edmacro kmacro byte-opt
compdef derived leader-key bind-map no-littering compat
compdef-autoloads hydra-autoloads lv-autoloads leader-key-autoloads
bind-map-autoloads no-littering-autoloads compat-autoloads
elpaca-use-package use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key bind-key easy-mmode
use-package-core elpaca-use-package-autoloads compile
text-property-search comint ansi-osc ansi-color ring time-date comp-run
cl-macs elpaca elpaca-process elpaca-autoloads comp cl-seq comp-cstr
comp-common warnings subr-x rx gv bytecomp byte-compile cl-extra
help-mode icons cl-loaddefs cl-lib display-line-numbers rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/ns-win ns-win ucs-normalize
mule-util term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads kqueue cocoa ns lcms2 multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 1603207 1724654) (symbols 48 73841 1671) (strings 32 543300 273395)
 (string-bytes 1 15331101) (vectors 16 236430) (vector-slots 8 3553218 678973)
 (floats 8 1870 4220) (intervals 56 4454 1235) (buffers 992 19))

Message #8 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>, Andrea Corallo <acorallo <at> gnu.org>
Cc: 68083 <at> debbugs.gnu.org
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Thu, 28 Dec 2023 17:12:55 +0200

> From: Aaron Jensen <aaronjensen <at> gmail.com>
> Date: Thu, 28 Dec 2023 09:05:25 -0500
> 
> 
> On macOS, I am often getting this when building Emacs from scratch. Could this be a race condition with a parallel build?
> 
> I bisected and the problematic commit is: e670412a3e101e70dc26e021f467faece8cb7f6b
> 
> In toplevel form:
> org/org-element.el:64:2: Error: Eager macro-expansion failure: (native-compiler-error (lambda (arg322 &optional arg323) (let ((f #'macroexpand)) (funcall f arg322 arg323))) "Compiling /private/var/tmp/emacs-plusA30-20231227-
> 10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln...
> File already exists: /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
> 
> Error: file-already-exists (\"File already exists\" \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>   mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0x1fcb3a734512f81>))
>   debug-early-backtrace()
>   debug-early(error (file-already-exists \"File already exists\" \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
> \"))
>   rename-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\" \"/private/var/tmp/emacs-plusA30-20231
> 227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>   comp-delete-or-replace-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\" \"/private/var/tmp/emacs-plusA30
> -20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\")
>   comp--compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>   comp-compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>   comp-final1()
>   load-with-code-conversion(\"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.e
> l\" nil t)
>   command-line-1((\"-l\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"))
>   command-line()
>   normal-top-level()
> ")
> gmake[3]: *** [Makefile:330: org/org-element.elc] Error 1

Thanks.  Adding Andrea to the discussion.

Message #11 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <acorallo <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 68083 <at> debbugs.gnu.org, Aaron Jensen <aaronjensen <at> gmail.com>
Subject: Re: bug#68083: 30.0.50; Intermittent build failure with native
 compilation
Date: Fri, 29 Dec 2023 13:43:51 -0500

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Aaron Jensen <aaronjensen <at> gmail.com>
>> Date: Thu, 28 Dec 2023 09:05:25 -0500
>> 
>> 
>> On macOS, I am often getting this when building Emacs from scratch. Could this be a race condition with a parallel build?
>> 
>> I bisected and the problematic commit is: e670412a3e101e70dc26e021f467faece8cb7f6b
>> 
>> In toplevel form:
>> org/org-element.el:64:2: Error: Eager macro-expansion failure:
>> (native-compiler-error (lambda (arg322 &optional arg323) (let ((f
>> #'macroexpand)) (funcall f arg322 arg323))) "Compiling
>> /private/var/tmp/emacs-plusA30-20231227-
>> 10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln...
>> File already exists:
>> /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
>> 
>> Error: file-already-exists (\"File already exists\"
>> \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>>   mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0x1fcb3a734512f81>))
>>   debug-early-backtrace()
>>   debug-early(error (file-already-exists \"File already exists\"
>> \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
>> \"))
>>   rename-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\"
>> \"/private/var/tmp/emacs-plusA30-20231
>> 227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>>   comp-delete-or-replace-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\"
>> \"/private/var/tmp/emacs-plusA30
>> -20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\")
>>   comp--compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>>   comp-compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>>   comp-final1()
>>   load-with-code-conversion(\"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"
>> \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.e
>> l\" nil t)
>>   command-line-1((\"-l\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"))
>>   command-line()
>>   normal-top-level()
>> ")
>> gmake[3]: *** [Makefile:330: org/org-element.elc] Error 1
>
> Thanks.  Adding Andrea to the discussion.

Hi all,

reading from emacs-devel... is it still confirmed that the commit that
introduced this is e670412a3e101e70dc26e021f467faece8cb7f6b?

Thanks

  Andrea

Message #14 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Andrea Corallo <acorallo <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 68083 <at> debbugs.gnu.org
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Fri, 29 Dec 2023 14:09:50 -0500

[Message part 1 (text/plain, inline)]

Yes, that's what I've found. I can also confirm that compiling with 1
thread works around it. It's only a problem w/ gmake -jN where N is > 1 (I
run w/ 8 or 12 or so typically).


Aaron


On Fri, Dec 29, 2023 at 1:43 PM, Andrea Corallo <acorallo <at> gnu.org> wrote:

> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> From: Aaron Jensen <aaronjensen <at> gmail.com>
> Date: Thu, 28 Dec 2023 09:05:25 -0500
>
> On macOS, I am often getting this when building Emacs from scratch. Could
> this be a race condition with a parallel build?
>
> I bisected and the problematic commit is:
> e670412a3e101e70dc26e021f467faece8cb7f6b
>
> In toplevel form:
> org/org-element.el:64:2: Error: Eager macro-expansion failure:
> (native-compiler-error (lambda (arg322 &optional arg323) (let ((f
> #'macroexpand)) (funcall f arg322 arg323))) "Compiling
> /private/var/tmp/emacs-plusA30-20231227-
> 10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln...
> File already exists:
> /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
>
>
> Error: file-already-exists (\"File already exists\"
> \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
> mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode
> -0x1fcb3a734512f81>)) debug-early-backtrace()
> debug-early(error (file-already-exists \"File already exists\"
> \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
>
> \"))
> rename-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\"
>
> \"/private/var/tmp/emacs-plusA30-20231
> 227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
> comp-delete-or-replace-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\"
>
> \"/private/var/tmp/emacs-plusA30
> -20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\")
> comp--compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
> comp-compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
> comp-final1()
> load-with-code-conversion(\"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"
>
> \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.e
> l\" nil t)
> command-line-1((\"-l\"
> \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"))
> command-line()
> normal-top-level()
> ")
> gmake[3]: *** [Makefile:330: org/org-element.elc] Error 1
>
> Thanks. Adding Andrea to the discussion.
>
> Hi all,
>
> reading from emacs-devel... is it still confirmed that the commit that
> introduced this is e670412a3e101e70dc26e021f467faece8cb7f6b?
>
> Thanks
>
> Andrea
>

[Message part 2 (text/html, inline)]

Message #17 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <acorallo <at> gnu.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>,
 68083 <at> debbugs.gnu.org
Subject: Re: bug#68083: 30.0.50; Intermittent build failure with native
 compilation
Date: Fri, 29 Dec 2023 15:17:05 -0500

Aaron Jensen <aaronjensen <at> gmail.com> writes:

> Yes, that's what I've found. I can also confirm that compiling with 1 thread works around it. It's only a problem w/
> gmake -jN where N is > 1 (I run w/ 8 or 12 or so typically).
>
> Aaron

Intresting, adding Jens, hopefully he has some good idea.

Maybe you could re-add 'macroexpand' and 'rename-buffer' to
'native-comp-never-optimize-functions' and discover which one of the two
is triggering the bug?

Thanks

  Andrea

Message #20 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Andrea Corallo <acorallo <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>,
 68083 <at> debbugs.gnu.org
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Fri, 29 Dec 2023 15:26:58 -0500

[Message part 1 (text/plain, inline)]

On Fri, Dec 29, 2023 at 3:17 PM, Andrea Corallo <acorallo <at> gnu.org> wrote:

> Aaron Jensen <aaronjensen <at> gmail.com> writes:
>
> Yes, that's what I've found. I can also confirm that compiling with 1
> thread works around it. It's only a problem w/ gmake -jN where N is > 1 (I
> run w/ 8 or 12 or so typically).
>
> Aaron
>
> Intresting, adding Jens, hopefully he has some good idea.
>
> Maybe you could re-add 'macroexpand' and 'rename-buffer' to
> 'native-comp-never-optimize-functions' and discover which one of the two
> is triggering the bug?
>


I can't try it just now, but my trace includes: `File already exists:
/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln`
so I'm guessing it's macroexpand.

Aaron

[Message part 2 (text/html, inline)]

Message #23 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Andrea Corallo <acorallo <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>,
 68083 <at> debbugs.gnu.org
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Fri, 29 Dec 2023 16:07:38 -0500

[Message part 1 (text/plain, inline)]

Just to confirm, adding macroexpand to native-comp-never-optimize-functions
allows me to build successfully.

It also looks like comp-delete-or-replace-file can be updated to protect
rename-file against file-already-exists like it does for Windows. That
would also likely solve the problem if you want to be able to optimize
macroexpand.




Aaron


On Fri, Dec 29, 2023 at 3:26 PM, Aaron Jensen <aaronjensen <at> gmail.com> wrote:

> On Fri, Dec 29, 2023 at 3:17 PM, Andrea Corallo <acorallo <at> gnu.org> wrote:
>
> Aaron Jensen <aaronjensen <at> gmail.com> writes:
>
> Yes, that's what I've found. I can also confirm that compiling with 1
> thread works around it. It's only a problem w/ gmake -jN where N is > 1 (I
> run w/ 8 or 12 or so typically).
>
> Aaron
>
> Intresting, adding Jens, hopefully he has some good idea.
>
> Maybe you could re-add 'macroexpand' and 'rename-buffer' to
> 'native-comp-never-optimize-functions' and discover which one of the two
> is triggering the bug?
>
>
>
> I can't try it just now, but my trace includes: `File already exists:
> /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln`
> so I'm guessing it's macroexpand.
>
> Aaron
>

[Message part 2 (text/html, inline)]

Message #26 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 68083 <at> debbugs.gnu.org, acorallo <at> gnu.org, jschmidt4gnu <at> vodafonemail.de
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Sat, 30 Dec 2023 08:47:43 +0200

> From: Aaron Jensen <aaronjensen <at> gmail.com>
> Date: Fri, 29 Dec 2023 16:07:38 -0500
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 68083 <at> debbugs.gnu.org, 
> 	Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
> 
> Just to confirm, adding macroexpand to native-comp-never-optimize-functions allows me to build
> successfully.
> 
> It also looks like comp-delete-or-replace-file can be updated to protect rename-file against
> file-already-exists like it does for Windows. That would also likely solve the problem if you want to be
> able to optimize macroexpand.

Are you sure?  We do that on Windows because Windows doesn't allow us
to delete a file that is open by another program.  That shouldn't
happen on Posix systems, so I think what you see here is due to a race
between checking whether a file exists and renaming it, which is a
different problem.

However, feel free to try the same trick we use on Windows and see
whether it helps.

Message #29 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 68083 <at> debbugs.gnu.org, acorallo <at> gnu.org, jschmidt4gnu <at> vodafonemail.de
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Sat, 30 Dec 2023 10:29:16 -0500

On Sat, Dec 30, 2023 at 1:47 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> > From: Aaron Jensen <aaronjensen <at> gmail.com>
> > Date: Fri, 29 Dec 2023 16:07:38 -0500
> > Cc: Eli Zaretskii <eliz <at> gnu.org>, 68083 <at> debbugs.gnu.org,
> >       Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
> >
> > Just to confirm, adding macroexpand to native-comp-never-optimize-functions allows me to build
> > successfully.
> >
> > It also looks like comp-delete-or-replace-file can be updated to protect rename-file against
> > file-already-exists like it does for Windows. That would also likely solve the problem if you want to be
> > able to optimize macroexpand.
>
> Are you sure?  We do that on Windows because Windows doesn't allow us
> to delete a file that is open by another program.  That shouldn't
> happen on Posix systems, so I think what you see here is due to a race
> between checking whether a file exists and renaming it, which is a
> different problem.
>
> However, feel free to try the same trick we use on Windows and see
> whether it helps.

This fixes it for me:

diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
index 3b2fd25e61c..80088f935a4 100644
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -3341,7 +3341,11 @@ comp-delete-or-replace-file
         ;; is currently loaded.
         (t (delete-file oldfile)
            (when newfile
-             (rename-file newfile oldfile)))))
+             (condition-case _
+                 (rename-file newfile oldfile)
+               (file-already-exists
+                (delete-file newfile)
+                t))))))

 (defun comp--native-compile (function-or-file &optional with-late-load output)
   "Compile FUNCTION-OR-FILE into native code.

I imagine that this is worth doing just to make this operation
parallel-safe, but I wonder why macroexpand is the only instance of
this happening. I don't know if macroexpand should still be in
native-comp-never-optimize-functions or not (i.e., is there another
reason it was there other than to avoid this crash?)

Aaron

Message #32 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 68083 <at> debbugs.gnu.org, acorallo <at> gnu.org, jschmidt4gnu <at> vodafonemail.de
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Sat, 30 Dec 2023 19:42:54 +0200

> From: Aaron Jensen <aaronjensen <at> gmail.com>
> Date: Sat, 30 Dec 2023 10:29:16 -0500
> Cc: acorallo <at> gnu.org, 68083 <at> debbugs.gnu.org, jschmidt4gnu <at> vodafonemail.de
> 
> On Sat, Dec 30, 2023 at 1:47 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> > Are you sure?  We do that on Windows because Windows doesn't allow us
> > to delete a file that is open by another program.  That shouldn't
> > happen on Posix systems, so I think what you see here is due to a race
> > between checking whether a file exists and renaming it, which is a
> > different problem.
> >
> > However, feel free to try the same trick we use on Windows and see
> > whether it helps.
> 
> This fixes it for me:
> 
> diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
> index 3b2fd25e61c..80088f935a4 100644
> --- a/lisp/emacs-lisp/comp.el
> +++ b/lisp/emacs-lisp/comp.el
> @@ -3341,7 +3341,11 @@ comp-delete-or-replace-file
>          ;; is currently loaded.
>          (t (delete-file oldfile)
>             (when newfile
> -             (rename-file newfile oldfile)))))
> +             (condition-case _
> +                 (rename-file newfile oldfile)
> +               (file-already-exists
> +                (delete-file newfile)
> +                t))))))

What happens if, instead of wrapping rename-file in condition-case,
you change that to say

  (t (if newfile
	 (rename-file newfile oldfile)
       (delete-file oldfile))

> I imagine that this is worth doing just to make this operation
> parallel-safe, but I wonder why macroexpand is the only instance of
> this happening. I don't know if macroexpand should still be in
> native-comp-never-optimize-functions or not (i.e., is there another
> reason it was there other than to avoid this crash?)

rename-file is supposed to be an atomic operation on Posix
filesystems, so I don't quite understand why you see what you see, and
I'm hesitant to sweep under the carpet a problem we don't understand.

Message #35 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 68083 <at> debbugs.gnu.org, acorallo <at> gnu.org, jschmidt4gnu <at> vodafonemail.de
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Sat, 30 Dec 2023 13:06:55 -0500

On Sat, Dec 30, 2023 at 12:43 PM Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> > From: Aaron Jensen <aaronjensen <at> gmail.com>
> > Date: Sat, 30 Dec 2023 10:29:16 -0500
> > Cc: acorallo <at> gnu.org, 68083 <at> debbugs.gnu.org, jschmidt4gnu <at> vodafonemail.de
> >
> > On Sat, Dec 30, 2023 at 1:47 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> > >
> > > Are you sure?  We do that on Windows because Windows doesn't allow us
> > > to delete a file that is open by another program.  That shouldn't
> > > happen on Posix systems, so I think what you see here is due to a race
> > > between checking whether a file exists and renaming it, which is a
> > > different problem.
> > >
> > > However, feel free to try the same trick we use on Windows and see
> > > whether it helps.
> >
> > This fixes it for me:
> >
> > diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
> > index 3b2fd25e61c..80088f935a4 100644
> > --- a/lisp/emacs-lisp/comp.el
> > +++ b/lisp/emacs-lisp/comp.el
> > @@ -3341,7 +3341,11 @@ comp-delete-or-replace-file
> >          ;; is currently loaded.
> >          (t (delete-file oldfile)
> >             (when newfile
> > -             (rename-file newfile oldfile)))))
> > +             (condition-case _
> > +                 (rename-file newfile oldfile)
> > +               (file-already-exists
> > +                (delete-file newfile)
> > +                t))))))
>
> What happens if, instead of wrapping rename-file in condition-case,
> you change that to say
>
>   (t (if newfile
>          (rename-file newfile oldfile)
>        (delete-file oldfile))

I don't really understand this change. The previous version of the
code wraps the rename in a (when newfile) and deletes the oldfile
right before that. The deletion would always be necessary if the
oldfile exists unless OK-IF-ALREADY-EXISTS is specified in
rename-file.

I tried it anyway for diligence, and I get the exact same behavior
(the rename fails because the file exists).

> > I imagine that this is worth doing just to make this operation
> > parallel-safe, but I wonder why macroexpand is the only instance of
> > this happening. I don't know if macroexpand should still be in
> > native-comp-never-optimize-functions or not (i.e., is there another
> > reason it was there other than to avoid this crash?)
>
> rename-file is supposed to be an atomic operation on Posix
> filesystems, so I don't quite understand why you see what you see, and
> I'm hesitant to sweep under the carpet a problem we don't understand.

Indeed, but we aren't specifying OK-IF-ALREADY-EXISTS, so if another
process does the same compile at the same time (and therefore the same
rename), they will conflict. We could specify OK-IF-ALREADY-EXISTS
instead and that works as well:

diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
index 3b2fd25e61c..d56c69d9470 100644
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -3341,7 +3341,7 @@ comp-delete-or-replace-file
         ;; is currently loaded.
         (t (delete-file oldfile)
            (when newfile
-             (rename-file newfile oldfile)))))
+             (rename-file newfile oldfile t)))))

 (defun comp--native-compile (function-or-file &optional with-late-load output)
   "Compile FUNCTION-OR-FILE into native code.

Aaron

Message #38 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 68083 <at> debbugs.gnu.org, acorallo <at> gnu.org, jschmidt4gnu <at> vodafonemail.de
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Sat, 30 Dec 2023 20:46:08 +0200

> From: Aaron Jensen <aaronjensen <at> gmail.com>
> Date: Sat, 30 Dec 2023 13:06:55 -0500
> Cc: acorallo <at> gnu.org, 68083 <at> debbugs.gnu.org, jschmidt4gnu <at> vodafonemail.de
> 
> On Sat, Dec 30, 2023 at 12:43 PM Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> > What happens if, instead of wrapping rename-file in condition-case,
> > you change that to say
> >
> >   (t (if newfile
> >          (rename-file newfile oldfile)
> >        (delete-file oldfile))
> 
> I don't really understand this change. The previous version of the
> code wraps the rename in a (when newfile) and deletes the oldfile
> right before that. The deletion would always be necessary if the
> oldfile exists unless OK-IF-ALREADY-EXISTS is specified in
> rename-file.

Sorry, I meant to add the OK-IF-ALREADY-EXISTS argument non-nil, of
course.

The point is that you can rename a file if the OLDFILE exists with no
problem, and that is supposed to be an atomic operation, so no race
conditions.

> Indeed, but we aren't specifying OK-IF-ALREADY-EXISTS, so if another
> process does the same compile at the same time (and therefore the same
> rename), they will conflict. We could specify OK-IF-ALREADY-EXISTS
> instead and that works as well:

Then I prefer this version.

Message #41 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 68083 <at> debbugs.gnu.org, Andrea Corallo <acorallo <at> gnu.org>,
 Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Sat, 30 Dec 2023 14:54:53 -0600

[Message part 1 (text/plain, inline)]

 Works for me, though is it still an outstanding question as to whether or
not we should add macro expand back?

Also yeah if we add the t we can do what you suggested where it’s either a
delete or rename force.


Aaron

On Sat, Dec 30 2023 at 1:46 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> From: Aaron Jensen <aaronjensen <at> gmail.com>
> Date: Sat, 30 Dec 2023 13:06:55 -0500
> Cc: acorallo <at> gnu.org, 68083 <at> debbugs.gnu.org, jschmidt4gnu <at> vodafonemail.de
>
> On Sat, Dec 30, 2023 at 12:43 PM Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> > What happens if, instead of wrapping rename-file in condition-case,
> > you change that to say
> >
> > (t (if newfile
> > (rename-file newfile oldfile)
> > (delete-file oldfile))
>
> I don't really understand this change. The previous version of the code
> wraps the rename in a (when newfile) and deletes the oldfile right before
> that. The deletion would always be necessary if the oldfile exists unless
> OK-IF-ALREADY-EXISTS is specified in rename-file.
>
> Sorry, I meant to add the OK-IF-ALREADY-EXISTS argument non-nil, of
> course.
>
> The point is that you can rename a file if the OLDFILE exists with no
> problem, and that is supposed to be an atomic operation, so no race
> conditions.
>
> Indeed, but we aren't specifying OK-IF-ALREADY-EXISTS, so if another
> process does the same compile at the same time (and therefore the same
> rename), they will conflict. We could specify OK-IF-ALREADY-EXISTS instead
> and that works as well:
>
> Then I prefer this version.
>

[Message part 2 (text/html, inline)]

Message #44 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Jensen <aaronjensen <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 68083 <at> debbugs.gnu.org, Andrea Corallo <acorallo <at> gnu.org>,
 Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Sat, 30 Dec 2023 17:08:11 -0600

[Message part 1 (text/plain, inline)]

Here's a patch, if helpful. I've tested it and can build with it.


Aaron


On Sat, Dec 30, 2023 at 3:54 PM, Aaron Jensen <aaronjensen <at> gmail.com> wrote:

> Works for me, though is it still an outstanding question as to whether or
> not we should add macro expand back?
>
> Also yeah if we add the t we can do what you suggested where it’s either a
> delete or rename force.
>
>
> Aaron
>
> On Sat, Dec 30 2023 at 1:46 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> From: Aaron Jensen <aaronjensen <at> gmail.com>
> Date: Sat, 30 Dec 2023 13:06:55 -0500
> Cc: acorallo <at> gnu.org, 68083 <at> debbugs.gnu.org, jschmidt4gnu <at> vodafonemail.de
>
> On Sat, Dec 30, 2023 at 12:43 PM Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> > What happens if, instead of wrapping rename-file in condition-case,
> > you change that to say
> >
> > (t (if newfile
> > (rename-file newfile oldfile)
> > (delete-file oldfile))
>
> I don't really understand this change. The previous version of the code
> wraps the rename in a (when newfile) and deletes the oldfile right before
> that. The deletion would always be necessary if the oldfile exists unless
> OK-IF-ALREADY-EXISTS is specified in rename-file.
>
> Sorry, I meant to add the OK-IF-ALREADY-EXISTS argument non-nil, of
> course.
>
> The point is that you can rename a file if the OLDFILE exists with no
> problem, and that is supposed to be an atomic operation, so no race
> conditions.
>
> Indeed, but we aren't specifying OK-IF-ALREADY-EXISTS, so if another
> process does the same compile at the same time (and therefore the same
> rename), they will conflict. We could specify OK-IF-ALREADY-EXISTS instead
> and that works as well:
>
> Then I prefer this version.
>
>

[Message part 2 (text/html, inline)]

[0001-comp.el-comp-delete-or-replace-file-Fix-parallel-com.patch (application/octet-stream, attachment)]

Message #47 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Aaron Jensen <aaronjensen <at> gmail.com>
Cc: 68083 <at> debbugs.gnu.org, acorallo <at> gnu.org, jschmidt4gnu <at> vodafonemail.de
Subject: Re: bug#68083: 30.0.50;
 Intermittent build failure with native compilation
Date: Sun, 31 Dec 2023 08:26:43 +0200

> From: Aaron Jensen <aaronjensen <at> gmail.com>
> Date: Sat, 30 Dec 2023 14:54:53 -0600
> Cc: Andrea Corallo <acorallo <at> gnu.org>, 68083 <at> debbugs.gnu.org, 
> 	Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
> 
> Works for me, though is it still an outstanding question as to whether or not we should add macro
> expand back?

Andrea, WDYT?

Message #50 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
To: Andrea Corallo <acorallo <at> gnu.org>, Aaron Jensen <aaronjensen <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 68083 <at> debbugs.gnu.org
Subject: Re: bug#68083: 30.0.50; Intermittent build failure with native
 compilation
Date: Sun, 31 Dec 2023 21:30:52 +0100

On 2023-12-29  21:17, Andrea Corallo wrote:

> Intresting, adding Jens, hopefully he has some good idea.

Thanks for inviting me to the party and sorry for attending late,
but I'm grounded by a flu ... will check later if there is still
anything to check.

Message #53 received at 68083 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <acorallo <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 68083 <at> debbugs.gnu.org, jschmidt4gnu <at> vodafonemail.de,
 Aaron Jensen <aaronjensen <at> gmail.com>
Subject: Re: bug#68083: 30.0.50; Intermittent build failure with native
 compilation
Date: Mon, 01 Jan 2024 14:39:38 -0500

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Aaron Jensen <aaronjensen <at> gmail.com>
>> Date: Sat, 30 Dec 2023 14:54:53 -0600
>> Cc: Andrea Corallo <acorallo <at> gnu.org>, 68083 <at> debbugs.gnu.org, 
>> 	Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
>> 
>> Works for me, though is it still an outstanding question as to whether or not we should add macro
>> expand back?
>
> Andrea, WDYT?

I think we should not re-add it unless there's a specific reason.

If the proposed patch solves the issue I guess we should be fine with
that.

Thanks

  Andrea

Message #58 received at 68083-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andrea Corallo <acorallo <at> gnu.org>
Cc: 68083-done <at> debbugs.gnu.org, jschmidt4gnu <at> vodafonemail.de,
 aaronjensen <at> gmail.com
Subject: Re: bug#68083: 30.0.50; Intermittent build failure with native
 compilation
Date: Mon, 01 Jan 2024 22:12:10 +0200

> From: Andrea Corallo <acorallo <at> gnu.org>
> Cc: Aaron Jensen <aaronjensen <at> gmail.com>,  68083 <at> debbugs.gnu.org,
>   jschmidt4gnu <at> vodafonemail.de
> Date: Mon, 01 Jan 2024 14:39:38 -0500
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: Aaron Jensen <aaronjensen <at> gmail.com>
> >> Date: Sat, 30 Dec 2023 14:54:53 -0600
> >> Cc: Andrea Corallo <acorallo <at> gnu.org>, 68083 <at> debbugs.gnu.org, 
> >> 	Jens Schmidt <jschmidt4gnu <at> vodafonemail.de>
> >> 
> >> Works for me, though is it still an outstanding question as to whether or not we should add macro
> >> expand back?
> >
> > Andrea, WDYT?
> 
> I think we should not re-add it unless there's a specific reason.
> 
> If the proposed patch solves the issue I guess we should be fine with
> that.

Thanks, so I installed the last patch, and I'm closing this bug.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.