From unknown Fri Aug 15 04:04:25 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#65017 <65017@debbugs.gnu.org> To: bug#65017 <65017@debbugs.gnu.org> Subject: Status: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Reply-To: bug#65017 <65017@debbugs.gnu.org> Date: Fri, 15 Aug 2025 11:04:25 +0000 retitle 65017 29.1; Byte compiler interaction with cl-lib function objects,= removes symbol-function reassign 65017 emacs submitter 65017 Eric Marsden severity 65017 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 02 09:33:13 2023 Received: (at submit) by debbugs.gnu.org; 2 Aug 2023 13:33:13 +0000 Received: from localhost ([127.0.0.1]:49244 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRByS-0005pS-7s for submit@debbugs.gnu.org; Wed, 02 Aug 2023 09:33:13 -0400 Received: from lists.gnu.org ([2001:470:142::17]:44304) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qR9KP-0006sw-4r for submit@debbugs.gnu.org; Wed, 02 Aug 2023 06:43:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qR9KJ-00079u-Rx for bug-gnu-emacs@gnu.org; Wed, 02 Aug 2023 06:43:35 -0400 Received: from mail.risk-engineering.org ([2a01:4f8:c0c:a3f8::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qR9KG-0007XQ-TT for bug-gnu-emacs@gnu.org; Wed, 02 Aug 2023 06:43:35 -0400 DKIM-Signature: a=rsa-sha256; bh=suFlt0GSIJ2HA+ZjZsWhPNmp2l/v1dgFRI6dky9fZQw=; c=relaxed/relaxed; d=risk-engineering.org; h=Subject:Subject:Sender:To:To:Cc:From:From:Date:Date:MIME-Version:MIME-Version:Content-Type:Content-Type:Content-Transfer-Encoding:Content-Transfer-Encoding:Reply-To:In-Reply-To:Message-Id:Message-Id:References:Autocrypt:Openpgp; i=@risk-engineering.org; s=default; t=1690972105; v=1; x=1691404105; b=Fq3lMIabN8M77QRHQDvT2ATSSmYRb3Awuh8SvlG1iQ2hlH7cPbbhbUVUvDuil17ukQyUELrH gdPezQhUJ8DCdJdJeiXCWYg4DuXyG+tMcfuGDqJFIDPErxVsz3ubRvzEjP+XqLg+/DcBm5ZyZqT ni1gTd4nYwZpnJ7ug6NcCdljUOPwtTCv3zVX4bfN502DPS/qllaSPn4bL9ff6deXSoqRV3GOsde Ec10qzzn+mWgGnyKQQiTFr6UX1QauD6Hxez4G0DjWxfvnrFaJyZyi7b7GZ9FUkDaG8VUZtuI88R O6yBAqK5duk55hc1Eyb8EBtDpX0B37D59kG82Zgg6+x4Q== Received: by mail.risk-engineering.org (envelope-sender ) with ESMTPS id eb28a99d; Wed, 02 Aug 2023 12:28:25 +0200 Message-ID: Date: Wed, 2 Aug 2023 12:28:24 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:104.0) Gecko/20100101 Thunderbird/104.0 Content-Language: en-US To: bug-gnu-emacs@gnu.org From: Eric Marsden Subject: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=2a01:4f8:c0c:a3f8::1; envelope-from=eric.marsden@risk-engineering.org; helo=mail.risk-engineering.org X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Wed, 02 Aug 2023 09:33:11 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) The byte-compiler seems to erroneously remove the symbol-function for equal in the code show below. --- file "perturb.el" --- (require 'cl-lib) (defun foo ()   (cl-flet ((bar (v) (list v)))     (make-hash-table :test #'equal))) --- --- file "use.el" --- (require 'cl-lib) (require 'ert) (defun test ()   (cl-flet ((foo (x) (list x)))     (should (equal nil 42)))) --- % emacs -Q --batch --eval '(byte-compile-file "perturb.el")' -l use.el -f test Error: invalid-function (#)   mapbacktrace(#f(compiled-function (evald func args flags) #))   debug-early-backtrace()   debug-early(error (invalid-function #))   #(nil 42)   apply(# (nil 42))   (setq value-2 (apply fn-0 args-1))   (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq form-description-4 (nconc (list '(should (equal nil 42))) (list :form (cons fn-0 args-1)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (list :value value-2)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (let* ((-explainer- (and t (ert--get-explainer 'equal)))) (if -explainer- (list :explanation (apply -explainer- args-1)) nil))))) (ert--signal-should-execution form-description-4))   (if (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq form-description-4 (nconc (list '(should (equal nil 42))) (list :form (cons fn-0 args-1)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (list :value value-2)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (let* ((-explainer- (and t (ert--get-explainer 'equal)))) (if -explainer- (list :explanation (apply -explainer- args-1)) nil))))) (ert--signal-should-execution form-description-4)) nil (ert-fail form-description-4))   (let (form-description-4) (if (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq form-description-4 (nconc (list '(should (equal nil 42))) (list :form (cons fn-0 args-1)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (list :value value-2)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (let* ((-explainer- (and t (ert--get-explainer 'equal)))) (if -explainer- (list :explanation (apply -explainer- args-1)) nil))))) (ert--signal-should-execution form-description-4)) nil (ert-fail form-description-4)))   (let ((value-2 'ert-form-evaluation-aborted-3)) (let (form-description-4) (if (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq form-description-4 (nconc (list '(should (equal nil 42))) (list :form (cons fn-0 args-1)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (list :value value-2)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (let* ((-explainer- (and t (ert--get-explainer 'equal)))) (if -explainer- (list :explanation (apply -explainer- args-1)) nil))))) (ert--signal-should-execution form-description-4)) nil (ert-fail form-description-4))) value-2)   (let* ((fn-0 #'#) (args-1 (condition-case err (let ((signal-hook-function #'ert--should-signal-hook)) (list nil 42)) (error (progn (setq fn-0 #'signal) (list (car err) (cdr err))))))) (let ((value-2 'ert-form-evaluation-aborted-3)) (let (form-description-4) (if (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq form-description-4 (nconc (list '(should (equal nil 42))) (list :form (cons fn-0 args-1)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (list :value value-2)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (let* ((-explainer- (and t (ert--get-explainer 'equal)))) (if -explainer- (list :explanation (apply -explainer- args-1)) nil))))) (ert--signal-should-execution form-description-4)) nil (ert-fail form-description-4))) value-2))   (progn (let* ((fn-0 #'#) (args-1 (condition-case err (let ((signal-hook-function #'ert--should-signal-hook)) (list nil 42)) (error (progn (setq fn-0 #'signal) (list (car err) (cdr err))))))) (let ((value-2 'ert-form-evaluation-aborted-3)) (let (form-description-4) (if (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq form-description-4 (nconc (list '(should (equal nil 42))) (list :form (cons fn-0 args-1)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (list :value value-2)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (let* ((-explainer- (and t (ert--get-explainer 'equal)))) (if -explainer- (list :explanation (apply -explainer- args-1)) nil))))) (ert--signal-should-execution form-description-4)) nil (ert-fail form-description-4))) value-2)))   (let* ((--cl-foo-- #'(lambda (x) (list x)))) (progn (let* ((fn-0 #'#) (args-1 (condition-case err (let ((signal-hook-function #'ert--should-signal-hook)) (list nil 42)) (error (progn (setq fn-0 #'signal) (list (car err) (cdr err))))))) (let ((value-2 'ert-form-evaluation-aborted-3)) (let (form-description-4) (if (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq form-description-4 (nconc (list '(should (equal nil 42))) (list :form (cons fn-0 args-1)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (list :value value-2)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil (let* ((-explainer- (and t (ert--get-explainer 'equal)))) (if -explainer- (list :explanation (apply -explainer- args-1)) nil))))) (ert--signal-should-execution form-description-4)) nil (ert-fail form-description-4))) value-2))))   test()   command-line-1(("--eval" "(byte-compile-file \"perturb.el\")" "-l" "use.el" "-f" "test"))   command-line()   normal-top-level() Invalid function: # The byte-compiler seems to have erroneously removed the symbol-function for equal. In GNU Emacs 29.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.38,  cairo version 1.16.0) of 2023-08-01, modified by Debian built on  x86-ubc-02 Windowing system distributor 'The X.Org Foundation', version 11.0.12201009 System Description: Debian GNU/Linux trixie/sid Configured using:  'configure --build x86_64-linux-gnu --prefix=/usr  --sharedstatedir=/var/lib --libexecdir=/usr/libexec  --localstatedir=/var/lib --infodir=/usr/share/info  --mandir=/usr/share/man --with-libsystemd --with-pop=yes  --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/29.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/29.1/site-lisp:/usr/share/emacs/site-lisp  --with-sound=alsa --without-gconf --with-mailutils  --with-native-compilation --build x86_64-linux-gnu --prefix=/usr  --sharedstatedir=/var/lib --libexecdir=/usr/libexec  --localstatedir=/var/lib --infodir=/usr/share/info  --mandir=/usr/share/man --with-libsystemd --with-pop=yes  --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/29.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/29.1/site-lisp:/usr/share/emacs/site-lisp  --with-sound=alsa --without-gconf --with-mailutils  --with-native-compilation --with-cairo --with-x=yes  --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2  -ffile-prefix-map=/build/reproducible-path/emacs-29.1+1=.  -fstack-protector-strong -Wformat -Werror=format-security -Wall'  'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro' Configured features: ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 05:39:43 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 09:39:43 +0000 Received: from localhost ([127.0.0.1]:50904 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRUo2-0004SN-TT for submit@debbugs.gnu.org; Thu, 03 Aug 2023 05:39:43 -0400 Received: from mail-lj1-x22f.google.com ([2a00:1450:4864:20::22f]:44242) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRUo1-0004S8-Cs for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 05:39:42 -0400 Received: by mail-lj1-x22f.google.com with SMTP id 38308e7fff4ca-2b974031aeaso11006731fa.0 for <65017@debbugs.gnu.org>; Thu, 03 Aug 2023 02:39:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691055575; x=1691660375; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=e8fA0F5azK7RaBv7lPO7TQPVhXyf1+7UaJ4PZAARHp8=; b=hef7LdM6mt2rWW5gDusvPXyU6rA4njgb0g36P4vlOfS4Dy5Duhnbitcs+bI/y49Y5+ e7KhZNLV6m50XWgYVJKT1KRIL3MpHbooZD3OvfTvukGzNN33uQZWH63TrXbLswP6BrNg U+ItLj4YaTX7ZuqesoDHVGfwjgBrrnpBrFFURXa4P6gCap5D/QslR7DmICf/D8LcwaNi tLZshM3eoJRpK69Dczq4m/BI/Frw/ezlMF7qq3cF6B95zsgcgqX7LgO6eQrykXGJFrZa +AmuZtgzRQbLpzZn75cRF1t0e7rq+L6KgdaDrCYVvT9YaGr7WhD6v0QDak0ISCQE1NbI XxkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691055575; x=1691660375; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e8fA0F5azK7RaBv7lPO7TQPVhXyf1+7UaJ4PZAARHp8=; b=HN9h9y5fAxzYqp9alWBN96YP24sXO/F4O5cTjLrrojbNgc2h6itAGaX2IH1pnfrzRS g7lzPrVGx/Czle9/b7RvVlJIVTBk1jPosgMkGK/Jhqg5ifF1+LvuRjngxeaUBXAk7/R/ en7NNs2AVhscoQBJc5wjCC3BxFpgIBL8a5QXpRDad9bZKhCQHTfsmdgGya/MTuSZBmuo +m+zVpc8nnfmUhqoJxd8DQGHOwBmKF7sBg4Sh2ujZYxHU4dnnz5+JtM7kMP/ZeK4Iihc ffwxwjA4rE0pPiXS+pK06bsxRhpd2eH7z+SkBxwoA12ZN/rfPgLEhayMEZP3OG3a5uVB F4kw== X-Gm-Message-State: ABy/qLY+DazKGUqizZTtIp548yRj1J7iLKDmbTKhi/u+HTIvlfKOgapx tLOFWWpbYVNmVRCMkcUgECw= X-Google-Smtp-Source: APBJJlEpFftACXfhrxDl40z7N8kanTOYCGGrpR7SzHH438rcRX0f3nW1DR6u6VU08olV1jaRxkyCTA== X-Received: by 2002:a2e:8643:0:b0:2b6:effd:9a3b with SMTP id i3-20020a2e8643000000b002b6effd9a3bmr7200413ljj.24.1691055574757; Thu, 03 Aug 2023 02:39:34 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id a3-20020a2e8303000000b002b9fdfdab75sm821535ljh.12.2023.08.03.02.39.34 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Aug 2023 02:39:34 -0700 (PDT) From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-Id: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Thu, 3 Aug 2023 11:39:33 +0200 To: Eric Marsden X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Error: invalid-function (#) That is a symbol-with-position somehow leaking out. We can simplify your nice little test case to ------- first file ----------- (require 'cl-macs) (defun zeta () (cl-flet () #'equal)) ------- second file --------- (defun eta () (cl-flet () (funcall #'equal 12 34))) ------------------------------ and indeed, the leak is in cl--labels-convert-cache which will contain = `equal` as a symbol-with-pos after byte-compiling the first file, and = this causes trouble in the second file. cl--labels-convert-cache contains (# function #) and the function `eta` is consequently defined as (closure (t) nil (progn (# 12 34))) where 49 is the position of `equal` in the first file. Stefan and Alan should have a word here but I doubt we should hack this = in cl-macs.el somehow, should we? Making Ffuncall (etc) tolerant of symbol-with-pos isn't appealing = either. From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 10:43:33 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 14:43:34 +0000 Received: from localhost ([127.0.0.1]:52676 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRZY5-0007in-Hg for submit@debbugs.gnu.org; Thu, 03 Aug 2023 10:43:33 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:28188) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRZY3-0007iV-Ph for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 10:43:32 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id F3DD3441895; Thu, 3 Aug 2023 10:43:25 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691073799; bh=vOKgCw8ShwcaxVWV/4RR/eaJhlEFyse1MlGuoxPLMO8=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=J/iXy5z6Y+qPzqAq7wJlSbI2L5nluwLq7FDQ0cfNqk0WNIysbJXxT2Gpj4j+JHzW1 1z8xYePk5kuU4NJl8TeeZZN+nAyh4REp1VI54cLsBnMtWBLQDz1kzBLD0ehcVxsJTs RXFnc54x2loSdfXD6zgWd5Gzi0PME+ZFl99cMu/AGTKYl/BqB5rxq+VZLLsXRD/Wws +7lEl2zzyPvXdjC3b1XxCp8mW3kuNN8x00AUQC0kISGL7AVI/K2cEA4rTl1rZ1B4oW eGfwag0cmmHIo/i4QSN22z4smtX+2f2PSZIPyASYvbQcRHyHvjjiLX2PIbFcpuSgHT NgcDZ7coa0oug== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id D75A04418BF; Thu, 3 Aug 2023 10:43:19 -0400 (EDT) Received: from alfajor (unknown [181.44.118.150]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id A0D831202BC; Thu, 3 Aug 2023 10:43:18 -0400 (EDT) From: Stefan Monnier To: Mattias =?windows-1252?Q?Engdeg=E5rd?= Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> ("Mattias =?windows-1252?Q?Engdeg=E5rd=22's?= message of "Thu, 3 Aug 2023 11:39:33 +0200") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Thu, 03 Aug 2023 10:43:16 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.019 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > We can simplify your nice little test case to > > ------- first file ----------- > (require 'cl-macs) > (defun zeta () (cl-flet () #'equal)) > ------- second file --------- > (defun eta () (cl-flet () (funcall #'equal 12 34))) > ------------------------------ > > and indeed, the leak is in cl--labels-convert-cache which will contain > `equal` as a symbol-with-pos after byte-compiling the first file, and this > causes trouble in the second file. > > cl--labels-convert-cache contains > > (# function #) > > and the function `eta` is consequently defined as > > (closure (t) nil (progn (# 12 34))) > > where 49 is the position of `equal` in the first file. Thanks Mattias. AFAICT the problem is that OT1H `symbols-with-pos-enabled` is non-nil while loading the second file, but OTOH positions aren't stripped from the resulting code. So in `cl--labels-convert` when we check (eq f (car cl--labels-convert-cache)) we get when `f` is just `equal` whereas the cache contains the sympos, but that sympos is not stripped later on. I don't know why `symbols-with-pos-enabled` is non-nil at that point (I thought we only enabled it wile byte-compiling), but assuming there's a good reason for it, I tried to work around the problem with the patch below which is conceptually correct (indeed we should only return (cdr cl--labels-convert-cache) only in the case where `f` is exactly the very same object as (car cl--labels-convert-cache)). Sadly, it doesn't seem to help for a reason that escapes me. The *Messages* buffer says: For information about GNU Emacs and the GNU system, type C-h C-a. Compiling .../tmp/foo1.el... Self-rewrite # to #'# (t) Compiling .../tmp/foo1.el...done Wrote .../tmp/foo1.elc Self-rewrite equal to #'# (t) Self-rewrite # to #'# (t) where the middle "self-rewrite" is the culprit. Apparently (let ((symbols-with-pos-enabled nil)) (eq f (car cl--labels-convert-cache))) returned non-nil when `f` was the bare `equal` and the `car` returned the sympos. How can that be? I thought `eq` should really be the good old "pointer equality" when `symbols-with-pos-enabled` is nil. What am I missing? Stefan diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el index 0a3181561bd..bc71f565f3b 100644 --- a/lisp/emacs-lisp/cl-macs.el +++ b/lisp/emacs-lisp/cl-macs.el @@ -2037,7 +2039,12 @@ ;; *after* handling `function', but we want to stop macroexpansion from ;; being applied infinitely, so we use a cache to return the exact `form' ;; being expanded even though we don't receive it. - ((eq f (car cl--labels-convert-cache)) (cdr cl--labels-convert-cache)) + ((let ((symbols-with-pos-enabled nil)) + (eq f (car cl--labels-convert-cache))) + (let ((print-symbols-bare nil)) + (message "Self-rewrite %S to %S (%S)" f (cdr cl--labels-convert-cache) + symbols-with-pos-enabled)) + (cdr cl--labels-convert-cache)) (t (let* ((found (assq f macroexpand-all-environment)) (replacement (and found From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 11:37:56 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 15:37:56 +0000 Received: from localhost ([127.0.0.1]:52735 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRaOh-0000m0-Uu for submit@debbugs.gnu.org; Thu, 03 Aug 2023 11:37:56 -0400 Received: from mail-lj1-x236.google.com ([2a00:1450:4864:20::236]:59593) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRaOg-0000lb-Lk for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 11:37:55 -0400 Received: by mail-lj1-x236.google.com with SMTP id 38308e7fff4ca-2ba0f27a4c2so13783171fa.2 for <65017@debbugs.gnu.org>; Thu, 03 Aug 2023 08:37:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691077068; x=1691681868; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=qPcWHhA8F9paGfQcw7MjFcBQAv53ll5dlKnAJWA7gjg=; b=PBBOlDxtLafHQTz+bTyWuMhTiiL/7GXbnXi2rkpS+INnNyVuPHay60afIqrVRZQu/Q AyDnBJr3ae7/wh0ZwNORB77OkcdCcRF9szqe4DIbvwBQgw9b8CSpq1QE5qlgn6/9s72l NbmfL5x/20j1ap5jSCL87L/R7k+lUbnalxWcV9EEOYFE13t7RyxZF2fdhMf+ZS66w885 9P1ZeC3JcQG8oZv5w+E4KbmAg1KTeweZojxT0PDlKZ7bNXyubgM57Qkm/pUWzNLsjwpv rEo1QF6irQH+jYU8LkewNdk/NujhjPAEb1rIcXqiq5ZUmxAjHa0G1ZI3Pvsfya6px0p9 984A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691077068; x=1691681868; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=qPcWHhA8F9paGfQcw7MjFcBQAv53ll5dlKnAJWA7gjg=; b=Oyscsdthz5ZzUCwU/G59soNl0tRLJSN/hYSXgb1WZbBafLsNN1AaaC/JBjnJ/gIhPB JzLfzZU4X3oIIVPnSQDUOm7NcRxO8HKtSMHffznZh2FdQm0q29urTLnQ/V4SiOD1LVE1 EcDcpjXYKudpZk/4Yy3qZg3bXRXsTue1q/f4zc9QQFKenSTrVdZ4iwc6TFnUNu0nZo4S yx7rMxPLdTdW74+fL2dLWdiMF8AHYQGetWpGQrOiZbJMD2sZUsuXY2ldHFlX4c1j0F3c FQoTU5Kdm8IlHqGr7xOxeMv4631aWyFNDW6tBB3TjANOOvbJQbQSLCN4RkZPGMQO0XDg dRcA== X-Gm-Message-State: ABy/qLaY91KAx3eXvw104uj2vp8LiiaNFaCSPXUciBju3TSSjy/xIXJd iR+8+gQfgJPB1CsG8T/i4sQ= X-Google-Smtp-Source: APBJJlE5qewV57NULfRCqX4G0zmEiih+8Pkd4MDIkh7cTLJ8gAdxaCWoyUe5u2Kz5oQEviQ6p89sPg== X-Received: by 2002:a2e:2414:0:b0:2b6:dc55:c3c7 with SMTP id k20-20020a2e2414000000b002b6dc55c3c7mr8190281ljk.20.1691077068278; Thu, 03 Aug 2023 08:37:48 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id r25-20020a2e9959000000b002b9fddc6d85sm19335ljj.62.2023.08.03.08.37.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Aug 2023 08:37:47 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Thu, 3 Aug 2023 17:37:46 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <1D1BB407-D33F-4E55-9F60-7670736CD15D@gmail.com> References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> To: Stefan Monnier X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 3 aug. 2023 kl. 16.43 skrev Stefan Monnier : > (let ((symbols-with-pos-enabled nil)) > (eq f (car cl--labels-convert-cache))) As far as the LAP peephole optimiser is concerned, eq commutes with = unbind so that let-binding will vanish. (I'm more annoyed by the fact that `equal` doesn't work like `eq` for = symbols when symbols-with-pos-enabled is nil.) From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 12:11:42 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 16:11:42 +0000 Received: from localhost ([127.0.0.1]:52794 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRavO-0001pg-4X for submit@debbugs.gnu.org; Thu, 03 Aug 2023 12:11:42 -0400 Received: from mx3.muc.de ([193.149.48.5]:17348) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRavL-0001pQ-MM for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 12:11:40 -0400 Received: (qmail 88643 invoked by uid 3782); 3 Aug 2023 18:11:31 +0200 Received: from acm.muc.de (p4fe157c8.dip0.t-ipconnect.de [79.225.87.200]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 03 Aug 2023 18:11:31 +0200 Received: (qmail 23796 invoked by uid 1000); 3 Aug 2023 16:11:30 -0000 Date: Thu, 3 Aug 2023 16:11:30 +0000 To: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: 65017@debbugs.gnu.org, Stefan Monnier , Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Mattias, and Stefan. On Thu, Aug 03, 2023 at 11:39:33 +0200, Mattias Engdegård wrote: > > Error: invalid-function (#) > That is a symbol-with-position somehow leaking out. > We can simplify your nice little test case to > ------- first file ----------- > (require 'cl-macs) > (defun zeta () (cl-flet () #'equal)) > ------- second file --------- > (defun eta () (cl-flet () (funcall #'equal 12 34))) > ------------------------------ > and indeed, the leak is in cl--labels-convert-cache which will contain `equal` as a symbol-with-pos after byte-compiling the first file, and this causes trouble in the second file. > cl--labels-convert-cache contains > (# function #) > and the function `eta` is consequently defined as > (closure (t) nil (progn (# 12 34))) > where 49 is the position of `equal` in the first file. First thoughts: It would seem cl--labels-convert-cache is failing to get initialised, somewhere. Perhaps this is a problem of cache invalidation. The variable lacks a doc-string, which might otherwise have documented where it is meant to be valid. What does this variable mean? cl--label-convert is defined as "Special macro-expander to rename (function F) references in `cl-labels'.". What does "rename (function F) references" mean? Is the term "name" in this context defined anywhere? > Stefan and Alan should have a word here but I doubt we should hack > this in cl-macs.el somehow, should we? If that's where the bug is, that's what we should fix. > Making Ffuncall (etc) tolerant of symbol-with-pos isn't appealing > either. Definitely not! -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 12:36:15 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 16:36:15 +0000 Received: from localhost ([127.0.0.1]:52806 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRbJ8-0002SO-Oy for submit@debbugs.gnu.org; Thu, 03 Aug 2023 12:36:15 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:6091) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRbJ6-0002SC-U8 for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 12:36:13 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 4F76744467D; Thu, 3 Aug 2023 12:36:07 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691080565; bh=U8xT/xzbFqCvjrryI0DsppXPj1YiIOFVYbIDW9nhf2M=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=E4Oukqd/laNQX1sAvkcX50dCJpbQz8sPwPvSB9GYRqCdCW03pgmndLTrugVX6Ifdu sYFaVtOvE2eAn/NG6KbeZptnYuNVNlHn6lGoQ4yaUJnNRQKvN+WBew5cONAK49CKTY 7NO2DdL18r4ZEBnv/ryEObC+7va42GYeLtdIUkX3vQZ0tYkNFZQ8hMR628ivT29gVd tJKGamC5lIJaUAj5ovOrhTIjlGIzr/NDrVS8wG5bt/KxmjpTNb5+AvHULteWxC/7On iC2v3g5BG6nJL2qSwze1Q0sP8ZErXXKPAgJEP8Y+EYHdwOBgykodQx165MvJY6ZKRc LKJloEYh8RLLQ== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id E019944467B; Thu, 3 Aug 2023 12:36:05 -0400 (EDT) Received: from alfajor (unknown [181.44.118.150]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id B1A6A1201E7; Thu, 3 Aug 2023 12:36:04 -0400 (EDT) From: Stefan Monnier To: Mattias =?windows-1252?Q?Engdeg=E5rd?= Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: <1D1BB407-D33F-4E55-9F60-7670736CD15D@gmail.com> ("Mattias =?windows-1252?Q?Engdeg=E5rd=22's?= message of "Thu, 3 Aug 2023 17:37:46 +0200") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <1D1BB407-D33F-4E55-9F60-7670736CD15D@gmail.com> Date: Thu, 03 Aug 2023 12:36:02 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.015 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> (let ((symbols-with-pos-enabled nil)) >> (eq f (car cl--labels-convert-cache))) > As far as the LAP peephole optimiser is concerned, eq commutes with unbind > so that let-binding will vanish. Hmm... so a bug in the optimizer because the new `eq` semantics breaks a previous assumption :-( The positive side is that this optimization is less important for lexbind code. Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 12:42:13 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 16:42:14 +0000 Received: from localhost ([127.0.0.1]:52811 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRbOv-0002ba-Id for submit@debbugs.gnu.org; Thu, 03 Aug 2023 12:42:13 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:35059) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRbOr-0002bK-RV for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 12:42:12 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 63D6644467F; Thu, 3 Aug 2023 12:42:04 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691080918; bh=WrXrIgyxiONlbP/kQoMg+uwdpPwoq2xLx24syTQdFFU=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=pEZpeMrQyoVO3dFswV2H4WCxmt2vLNaOV+bID4hKbWbeMUK3XoNOTm/z3C2qZvfMe oqMDuNCNUbE53UXVVryJOjfckMg6cW5BsbWK/14RP0XQGz+YZlz3nI+SCr/xCFRZOV 1IRU3i5+5u6fJhWO9V9X38BNlHyAm88uUFfX2RR6WuYhpGe2VqbML57uxW59omgY57 +9Or6HkcTXx29F9WuOKdLDuryuw6TQmgHGlaVrnfF5YfiW0XfB/XpdsU4yk8J9sHTq QNgjjrq0y6P7LyF8SUK9k5iGdoCK2a3TXHdK9OSTm9e/gJKZU6BTuqGRwuXZQhrHFv /FDf6ucSoT6Ug== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id AB77B44467E; Thu, 3 Aug 2023 12:41:58 -0400 (EDT) Received: from alfajor (unknown [181.44.118.150]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 867FC12026E; Thu, 3 Aug 2023 12:41:57 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Thu, 3 Aug 2023 16:11:30 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Thu, 03 Aug 2023 12:41:55 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.014 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > cl--label-convert is defined as "Special macro-expander to rename > (function F) references in `cl-labels'.". What does "rename (function > F) references" mean? Is the term "name" in this context defined > anywhere? `cl--label-convert` is the macro expander for (function F) used inside `cl-labels` so that (cl-labels ((my-foo () toto)) #'my-foo) gets turned into (let ((bar (lambda () toto))) bar) So it "renames" (function my-foo) to the corresponding variable `bar`. But for most (function BLA) the code should be left as-is, which a macro can't do directly, so `cl--labels-convert-cache` is a hack which lets us receive a handle to the overall (function BLA) form so we can return it as-is rather than having to rebuild a *new* (function BLA) which would just make the macro-expander call us right-back. >> Making Ffuncall (etc) tolerant of symbol-with-pos isn't appealing >> either. > Definitely not! I think we all agree on that. Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 12:43:51 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 16:43:51 +0000 Received: from localhost ([127.0.0.1]:52815 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRbQV-0002ds-1G for submit@debbugs.gnu.org; Thu, 03 Aug 2023 12:43:51 -0400 Received: from mx3.muc.de ([193.149.48.5]:18348) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRbQS-0002dc-N0 for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 12:43:49 -0400 Received: (qmail 25077 invoked by uid 3782); 3 Aug 2023 18:43:41 +0200 Received: from acm.muc.de (p4fe157c8.dip0.t-ipconnect.de [79.225.87.200]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 03 Aug 2023 18:43:41 +0200 Received: (qmail 23975 invoked by uid 1000); 3 Aug 2023 16:43:41 -0000 Date: Thu, 3 Aug 2023 16:43:41 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Thu, Aug 03, 2023 at 10:43:16 -0400, Stefan Monnier wrote: > > We can simplify your nice little test case to > > ------- first file ----------- > > (require 'cl-macs) > > (defun zeta () (cl-flet () #'equal)) > > ------- second file --------- > > (defun eta () (cl-flet () (funcall #'equal 12 34))) > > ------------------------------ > > and indeed, the leak is in cl--labels-convert-cache which will contain > > `equal` as a symbol-with-pos after byte-compiling the first file, and this > > causes trouble in the second file. > > cl--labels-convert-cache contains > > (# function #) > > and the function `eta` is consequently defined as > > (closure (t) nil (progn (# 12 34))) > > where 49 is the position of `equal` in the first file. > Thanks Mattias. > AFAICT the problem is that OT1H `symbols-with-pos-enabled` is > non-nil while loading the second file, but OTOH positions aren't > stripped from the resulting code. As I suggested in my other post, it might be a "simple" problem of cache invalidation. Why is the value from the first compilation hanging around until the second one? > So in `cl--labels-convert` when we check > (eq f (car cl--labels-convert-cache)) > we get when `f` is just `equal` whereas the cache contains > the sympos, but that sympos is not stripped later on. > I don't know why `symbols-with-pos-enabled` is non-nil at that point (I > thought we only enabled it wile byte-compiling), .... I believe that is indeed the case. > .... but assuming there's a good reason for it, I tried to work around > the problem with the patch below which is conceptually correct (indeed > we should only return (cdr cl--labels-convert-cache) only in the case > where `f` is exactly the very same object as (car > cl--labels-convert-cache)). > Sadly, it doesn't seem to help for a reason that escapes me. Setting symbols-with-pos-enabled to nil does nothing other than disable the mechanisms which handle symbols with position. Any such symbols which exist will continue to be swp, but will no longer be EQ to the base symbol, or other swp's with the same base symbol. > The *Messages* buffer says: > For information about GNU Emacs and the GNU system, type C-h C-a. > Compiling .../tmp/foo1.el... > Self-rewrite # to #'# (t) > Compiling .../tmp/foo1.el...done > Wrote .../tmp/foo1.elc > Self-rewrite equal to #'# (t) > Self-rewrite # to #'# (t) > where the middle "self-rewrite" is the culprit. > Apparently > (let ((symbols-with-pos-enabled nil)) > (eq f (car cl--labels-convert-cache))) > returned non-nil when `f` was the bare `equal` and the `car` returned > the sympos. How can that be? I thought `eq` should really be the > good old "pointer equality" when `symbols-with-pos-enabled` is nil. > What am I missing? > Stefan > diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el > index 0a3181561bd..bc71f565f3b 100644 > --- a/lisp/emacs-lisp/cl-macs.el > +++ b/lisp/emacs-lisp/cl-macs.el > @@ -2037,7 +2039,12 @@ > ;; *after* handling `function', but we want to stop macroexpansion from > ;; being applied infinitely, so we use a cache to return the exact `form' > ;; being expanded even though we don't receive it. > - ((eq f (car cl--labels-convert-cache)) (cdr cl--labels-convert-cache)) > + ((let ((symbols-with-pos-enabled nil)) > + (eq f (car cl--labels-convert-cache))) > + (let ((print-symbols-bare nil)) > + (message "Self-rewrite %S to %S (%S)" f (cdr cl--labels-convert-cache) > + symbols-with-pos-enabled)) > + (cdr cl--labels-convert-cache)) > (t > (let* ((found (assq f macroexpand-all-environment)) > (replacement (and found -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 12:53:11 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 16:53:11 +0000 Received: from localhost ([127.0.0.1]:52823 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRbZX-0002wu-I8 for submit@debbugs.gnu.org; Thu, 03 Aug 2023 12:53:11 -0400 Received: from mail-lj1-x232.google.com ([2a00:1450:4864:20::232]:55628) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRbZV-0002wh-GX for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 12:53:10 -0400 Received: by mail-lj1-x232.google.com with SMTP id 38308e7fff4ca-2b9ba3d6157so18589971fa.3 for <65017@debbugs.gnu.org>; Thu, 03 Aug 2023 09:53:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691081583; x=1691686383; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=f2RD06PXaj/d2RNs1oq622ZCGIYt+Ibe8Ku+CDkgT7Q=; b=Z1bPWa5JhiySTUcU93urAukMglsXFiK1IV9Km/N3tTJStkyjvJrUpCMr85nTk6d7GP p2wEBWAuvfizTVNOhae9AuwzO9q3a8x6QprJ4Wv7iA91D4Dt0U38iA8788QYTu4tA2ml xkQvEUNzs9fZZt7iATQCNUy+EdALgjhVREMEdl4BmGv2v/BiOu5v2RMVX1ryWTJ5XbPd jWtaRGiL7Ir0DtJCvT/9beEJW+yu4vBX0nufgxKhV6LOMyN5ZTXYjvmalaqy1ILkqzO/ 7WL7QxGUdSMpwk32hJQLOeCSRZBqhipxPeAz4f0thQaSK5tXnJzby2cJRGnfuLgNC00W MzIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691081583; x=1691686383; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=f2RD06PXaj/d2RNs1oq622ZCGIYt+Ibe8Ku+CDkgT7Q=; b=HJdF2t98y+QrVZRB2ELW4k0X1ey+lLpCEnGXIi8GXu9L9dOZ0lU/qB5VpxpX/jyf6Z R7+phxxvWT9SXwCSnwGfba1iTR1ihZQ5csOcj98CLOBC38L+FNGnveWr4MCQatFdKpFv 5J1bCk/7E2ajC27jcweyS5yDjL70WdwEvhUaELIW4EwlJhIoCH6u5+Gs9/Z7FlrLn06G iKZcPNCvtGcBPCv/Gxm2J5Un37sNkNpDkPPd2ZDS6GZkzjDs3Dj0eZ12xCzBNkKtF3y7 Opi40XDq8MzNS1m3ssSJR9KK/Pw3Ufe1EGR9G4d8iprKwCTm5rYhQSwK/4yApF46DcV+ X41w== X-Gm-Message-State: ABy/qLa/bGSpC68tnyfQx2SPG2d2xOvKnNIyseO3M3tgugsQgF1lY2dP NFainvXc41HQAby8AAQZq5A= X-Google-Smtp-Source: APBJJlGTYoF7CVuRgffnYQv6foFLBf+WDrJO1j3Jj1zPgyly+lWHSz4ap8BZ8wWLXacJy8HKrT+pGQ== X-Received: by 2002:a2e:990c:0:b0:2b9:e230:25d0 with SMTP id v12-20020a2e990c000000b002b9e23025d0mr7658344lji.14.1691081583144; Thu, 03 Aug 2023 09:53:03 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id p22-20020a2e93d6000000b002b70a64d4desm42447ljh.46.2023.08.03.09.53.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Aug 2023 09:53:01 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Thu, 3 Aug 2023 18:53:00 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <1D1BB407-D33F-4E55-9F60-7670736CD15D@gmail.com> To: Stefan Monnier X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 3 aug. 2023 kl. 18.36 skrev Stefan Monnier : > Hmm... so a bug in the optimizer because the new `eq` semantics breaks > a previous assumption :-( >=20 > The positive side is that this optimization is less important for > lexbind code. Yes, but as it only applies to one particular variable that isn't bound = very often, I'm loath to remove it. For that matter it's not just about = dynamic variable bindings but other constructs using unbind. From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 13:30:57 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 17:30:57 +0000 Received: from localhost ([127.0.0.1]:52859 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRcA4-0003xe-Sa for submit@debbugs.gnu.org; Thu, 03 Aug 2023 13:30:57 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:23106) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRcA2-0003xR-8Y for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 13:30:55 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id EEECD4446A5; Thu, 3 Aug 2023 13:30:47 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691083846; bh=mvMyv8FwxOLhGTab+L3rP/Juzmx/LYo6ohVso3wMzWU=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=dDQtJKeLvH9mAnY/PnuD9zeXbGa6rpG7lsV5wVqAeTUV3DRmTnAiPlGrXYa6S3c/E vPcABbOReA1Y5M+OWZhpsaioDLBjdHXPbIkH7bz7cXxnFuBIB9Egld3Ahq+aeRfZoN HzguYZSdOygLnpE94kaKTkQyQ7WG4oa1vUNpxM9BPGRjDUtJUskr6ssDzHKfRoAabN wmrPZNuUlbeAeveAYgs99oVGi1mkpexa4OAUZ2v0OF4jXttKhPzvWjNTj1EwP2t/RT Leg9tcCi2KGsPvwwn4Q242p5av7HpUbSLTSvdPpGprzro+Or3iMHZ9gtiB0yVx2NWf R7D3rD+gP+egw== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 747E64446A2; Thu, 3 Aug 2023 13:30:46 -0400 (EDT) Received: from alfajor (unknown [181.44.118.150]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 43F57120153; Thu, 3 Aug 2023 13:30:45 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Thu, 3 Aug 2023 16:43:41 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Thu, 03 Aug 2023 13:30:43 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.013 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > As I suggested in my other post, it might be a "simple" problem of cache > invalidation. Why is the value from the first compilation hanging around > until the second one? That's a side problem. If absolutely needed, we could add some ad-hoc invalidation to work around the core problem, but I'd rather fix the core problem. Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 13:31:08 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 17:31:08 +0000 Received: from localhost ([127.0.0.1]:52863 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRcAG-0003yS-7p for submit@debbugs.gnu.org; Thu, 03 Aug 2023 13:31:08 -0400 Received: from mail-lj1-x22f.google.com ([2a00:1450:4864:20::22f]:55778) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRcAE-0003xq-FL for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 13:31:06 -0400 Received: by mail-lj1-x22f.google.com with SMTP id 38308e7fff4ca-2b9ba3d6157so19245801fa.3 for <65017@debbugs.gnu.org>; Thu, 03 Aug 2023 10:31:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691083860; x=1691688660; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=1APxneY36Zar095Vs8zRhq1Q8oPy+DHJPi/BkheX204=; b=osqqSwDLeWApurQ5/iawbQdKlP9DH/HokwsdpX2q1Lsvzn7Mw13brwjy40zBzv098v SJb/n3CtORmPZfGymWdWypHRR/hChmeyy0YcxvnIbacKU4UORBXyNguAqfnWQN0Tiuff U80PMaqOih8ZgVJRgtc6tr5LaDLQ362xE4H8DnzhoIKefVnL1XBEXtzzDIYiTd0JJXaA baEyMGidIq5Mq+vERrVlglLBNKwP7A3HRcFvewkobORqXxCyLhszkcHdA+g8qQNbRf9k L/87Ycuj7/75iZldOKPsIPm7AdWOtaVELgTBn0/b32ry0VXvsgXm/UbDuQ6MWe8TWeg2 iYew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691083860; x=1691688660; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=1APxneY36Zar095Vs8zRhq1Q8oPy+DHJPi/BkheX204=; b=iAwtgHWFvCzz94G1rEi5rTYwcXlLOGWTJ9KXJnUD0iyYUlA84hkeOmldqokiLcp82F qJNehRy2pfLhedZyHcBiSP3Az+dWWlLNOf461LTvyVCZOqSEprx6G8fDokQ1Tz6/YRPc 5dPW8AioBSdCInnUqJnUQ/5y3NbTnqa5qcM2+vOf8PzY47s8ntKdLDPwX5jcBKLzL4tY vWWKP1kQB3nG0wlYsKLKoa+MdOEOaO72KXm2jqFfhXJYSV+tGDxnSo8CsG7XY5PDQ/5c qEtAdnMtCcetQEulHhrIPpsFu/Bam15UgXAn/AoDzd/FFbq7IbgO9xEWtDJ2ybWRhaXw QS3g== X-Gm-Message-State: ABy/qLbaUunNDg5oB2NufZezWJ8aHnC+piLGg2K2O+iyIoe4iCIi1oAK uL0toGmW/Kqc7o8bR9Ah020= X-Google-Smtp-Source: APBJJlFTxCepQiho0FNtdTQ2bFMiHgILjEpQr118JTwnhSefVAg+dGFirblmpnFZu3C10EB/a7n84Q== X-Received: by 2002:a05:651c:217:b0:2b6:a7dd:e22 with SMTP id y23-20020a05651c021700b002b6a7dd0e22mr7558346ljn.48.1691083860299; Thu, 03 Aug 2023 10:31:00 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id k3-20020a2e8883000000b002b9f1538816sm50723lji.80.2023.08.03.10.30.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Aug 2023 10:30:56 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Thu, 3 Aug 2023 19:30:55 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <1D1BB407-D33F-4E55-9F60-7670736CD15D@gmail.com> To: Stefan Monnier X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 3 aug. 2023 kl. 18.53 skrev Mattias Engdeg=C3=A5rd = : >> Hmm... so a bug in the optimizer because the new `eq` semantics = breaks >> a previous assumption :-( >>=20 >> The positive side is that this optimization is less important for >> lexbind code. >=20 > Yes, but as it only applies to one particular variable that isn't = bound very often, I'm loath to remove it. For that matter it's not just = about dynamic variable bindings but other constructs using unbind. I checked and this particular optimisation (eq-unbind commutation) = doesn't affect the generated bytecode in the Emacs tree anywhere, so it = may be a candidate for removal after all. From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 14:22:41 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 18:22:41 +0000 Received: from localhost ([127.0.0.1]:52907 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRcy9-0005Kr-Fw for submit@debbugs.gnu.org; Thu, 03 Aug 2023 14:22:41 -0400 Received: from mx3.muc.de ([193.149.48.5]:21254) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRcy6-0005Kb-BH for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 14:22:39 -0400 Received: (qmail 40002 invoked by uid 3782); 3 Aug 2023 20:22:31 +0200 Received: from acm.muc.de (p4fe157c8.dip0.t-ipconnect.de [79.225.87.200]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 03 Aug 2023 20:22:31 +0200 Received: (qmail 25139 invoked by uid 1000); 3 Aug 2023 18:22:30 -0000 Date: Thu, 3 Aug 2023 18:22:30 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Thu, Aug 03, 2023 at 13:30:43 -0400, Stefan Monnier wrote: > > As I suggested in my other post, it might be a "simple" problem of cache > > invalidation. Why is the value from the first compilation hanging around > > until the second one? > That's a side problem. If absolutely needed, we could add some ad-hoc > invalidation to work around the core problem, but I'd rather fix the > core problem. Are you sure? What is the core problem? As I see it, if the function cl--labels-convert is called during byte compilation, cl--labels-convert-cache is going to get symbols with position. This is expected and is what we want - such a symbol might easily give the position element of an error occurring in expanded code. What we don't want is this symbol with position hanging around after the end of the byte compilation that gives it its context. If I understood your other, explaining, post right, cl--labels-convert-cache is only valid within a particular invocation of cl-flet or cl-labels. So why would it not be a complete bug fix to initialise this variable to nil at the start of every cl-flet or cl-labels? Or, more likely, bind the variable to nil, to allow it to function better in nested invocations of either of these macros? What am I missing, here? > Stefan -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 14:48:30 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 18:48:30 +0000 Received: from localhost ([127.0.0.1]:52948 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRdN7-00064u-PO for submit@debbugs.gnu.org; Thu, 03 Aug 2023 14:48:30 -0400 Received: from mx3.muc.de ([193.149.48.5]:22005) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRdN5-000646-B2 for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 14:48:28 -0400 Received: (qmail 69118 invoked by uid 3782); 3 Aug 2023 20:48:21 +0200 Received: from acm.muc.de (p4fe157c8.dip0.t-ipconnect.de [79.225.87.200]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 03 Aug 2023 20:48:20 +0200 Received: (qmail 25302 invoked by uid 1000); 3 Aug 2023 18:48:20 -0000 Date: Thu, 3 Aug 2023 18:48:20 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Thu, Aug 03, 2023 at 12:41:55 -0400, Stefan Monnier wrote: > > cl--label-convert is defined as "Special macro-expander to rename > > (function F) references in `cl-labels'.". What does "rename (function > > F) references" mean? Is the term "name" in this context defined > > anywhere? > `cl--label-convert` is the macro expander for (function F) used inside > `cl-labels` so that > (cl-labels ((my-foo () toto)) > #'my-foo) > gets turned into > (let ((bar (lambda () toto))) > bar) > So it "renames" (function my-foo) to the corresponding variable `bar`. > But for most (function BLA) the code should be left as-is, which a macro > can't do directly, so `cl--labels-convert-cache` is a hack which lets us > receive a handle to the overall (function BLA) form so we can return it > as-is rather than having to rebuild a *new* (function BLA) which would > just make the macro-expander call us right-back. OK, thanks. [ .... ] > Stefan -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 17:01:08 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 21:01:09 +0000 Received: from localhost ([127.0.0.1]:53029 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRfRU-000112-Jg for submit@debbugs.gnu.org; Thu, 03 Aug 2023 17:01:08 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:19360) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRfRQ-00010N-Bx for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 17:01:07 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 7795F1000EF; Thu, 3 Aug 2023 17:00:58 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691096457; bh=bLG+mhWE4QciDptmIrcjTZEQGQsPZa/2q0yXyfc3M8s=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=FASSbYfp07Fh+4Lhi6iMFnZIs3Jk3NXzBqUCDkZB04aRbrHqPCqmRj2whHfQi02WC D6jIUH1UAXSK16svuFbk5h2/7aq4ywHI66B46ZBWBRsyciENar6eauEjHGZY21SccF c3WqCoV0W7pImph5wdhwkHB2Lo31GAzM3Cnmb2hm3+C9x6cqj2qJtzfHZFnaac0kgY q+vTOw+DbG2FR/wqu9+xQWMYv7nSE6EAwNC4wKARPoCBInwrEl+qP0iiGtLD7wOrA/ SQ3HXLxRke9o/xxNB6/3xjIGuniqI8OxVkqgYBN1N/8/XlqcXRhKGk2rwlYOoDtvVm KYhzBYBA6pDJA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 33DFC1000BC; Thu, 3 Aug 2023 17:00:57 -0400 (EDT) Received: from alfajor (unknown [181.44.118.150]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 04BFE12034F; Thu, 3 Aug 2023 17:00:55 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Thu, 3 Aug 2023 18:22:30 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Thu, 03 Aug 2023 17:00:53 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.115 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> That's a side problem. If absolutely needed, we could add some ad-hoc >> invalidation to work around the core problem, but I'd rather fix the >> core problem. > > Are you sure? Yes. > What is the core problem? The core problem is that we want to stop infinite macroexpansion of the same term. I.e. we want to stop macroexpansion from turning (function FOO) => (function FOO) => (function FOO) => ... so we tweak the macro-expander for `function` so that if FOO is exaction the same as last time, we return the exact list (function FOO) we built last time, rather than building a new one. If `eq` works as god intended (i.e. it performs a pointer-equality test), then this is perfectly safe because the cache always has the form (#1= . (function #1#)) so using the `cdr` of the cache returns something perfectly equivalent to `(function ,f) just with the added twist that it's the exact same list we return last time so may stop the macroexpansion. So it's safe the use the cache even when it's stale. But with `symbols-with-pos-enabled` the `eq` test can end up using the cache when FOO is not 100% the same as and so we can end up replacing `equal` or # with (function #) neither of which is really correct, the second of which can happen even if we try and flush the cache. > As I see it, if the function cl--labels-convert is called during byte > compilation, cl--labels-convert-cache is going to get symbols with > position. This is expected and is what we want - such a symbol might > easily give the position element of an error occurring in expanded code. Exactly, which is why the `eq` needs to pay attention to the position information. > So why would it not be a complete bug fix to initialise this variable > to nil at the start of every cl-flet or cl-labels? Because it could/would still replace some sympos with others within the same `cl-label`. Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 17:11:08 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 21:11:08 +0000 Received: from localhost ([127.0.0.1]:53033 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRfb9-0001Eq-Ot for submit@debbugs.gnu.org; Thu, 03 Aug 2023 17:11:08 -0400 Received: from mx3.muc.de ([193.149.48.5]:26044) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRfb5-0001EI-ST for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 17:11:05 -0400 Received: (qmail 32255 invoked by uid 3782); 3 Aug 2023 23:10:57 +0200 Received: from acm.muc.de (p4fe157c8.dip0.t-ipconnect.de [79.225.87.200]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 03 Aug 2023 23:10:56 +0200 Received: (qmail 27131 invoked by uid 1000); 3 Aug 2023 21:10:56 -0000 Date: Thu, 3 Aug 2023 21:10:56 +0000 To: Stefan Monnier , Mattias =?iso-8859-1?Q?Engdeg=E5rd?= Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: acm@muc.de, 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan and Mattias. On Thu, Aug 03, 2023 at 13:30:43 -0400, Stefan Monnier wrote: > > As I suggested in my other post, it might be a "simple" problem of cache > > invalidation. Why is the value from the first compilation hanging around > > until the second one? > That's a side problem. If absolutely needed, we could add some ad-hoc > invalidation to work around the core problem, but I'd rather fix the > core problem. Sorry about my last post. I now see what the core problem is, namely that (equal 'equal #) is returning non-nil. I think the cause is in internal_equal in src/fns.c, where the following code appears: /* A symbol with position compares the contained symbol, and is `equal' to the corresponding ordinary symbol. */ if (SYMBOL_WITH_POS_P (o1)) o1 = SYMBOL_WITH_POS_SYM (o1); if (SYMBOL_WITH_POS_P (o2)) o2 = SYMBOL_WITH_POS_SYM (o2); This piece of code converts symbols with positions to ordinary symbols without first checking symbols-with-pos-enabled. I think this is the bug. I will work on a patch to fix it (which shouldn't take long). > Stefan -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 03 17:46:58 2023 Received: (at 65017) by debbugs.gnu.org; 3 Aug 2023 21:46:58 +0000 Received: from localhost ([127.0.0.1]:53070 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRg9q-00029P-HO for submit@debbugs.gnu.org; Thu, 03 Aug 2023 17:46:58 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:25798) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRg9m-00029A-FM for 65017@debbugs.gnu.org; Thu, 03 Aug 2023 17:46:56 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id A21A41000EF; Thu, 3 Aug 2023 17:46:48 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691099207; bh=aq6IjbiAoygtKK8pXr5l1NZlG5sOTWFJ5/Z8ZGnks7I=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=lnBC+MhtAbas+tPQEFsKcHsJbB3fUzMxPcrgM8d/OBj/o1Jmx+OQV9y5OXyJ7ZbPe HbFAKUWG9ISxFm60XIA1gyZ5Ql7w2XoqlWxHnGnGe3eGUdccCXS3O1VLsNCBD587Me PEBnNEaqz7GqIbO0vfm0X18frV8WZjftyFvsOc8VdH5oN0X506CRhykKWzd/6ZOlRx KGZGzlnspUJBbkSyIyciJFOD7dK3u1sdPOMaEf3MfEtXdUDH3Pit8/pomhkKxBuB9A ilIHhskUAlIScDBiooqETAhWWza4bMEMHGgZwqUbMGTzi2QVEtsotf57P3CZtyyzO+ EKcP+eJ2zYl7w== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 8F5701000BC; Thu, 3 Aug 2023 17:46:47 -0400 (EDT) Received: from alfajor (unknown [181.44.118.150]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 5C8BF1202AD; Thu, 3 Aug 2023 17:46:46 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Thu, 3 Aug 2023 21:10:56 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Thu, 03 Aug 2023 17:46:44 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.108 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Sorry about my last post. I now see what the core problem is, namely > that (equal 'equal #) is returning non-nil. This is not really the core problem IIUC since `cl-macs.el` uses `eq` rather than `equal` so changing `equal` won't make much of a difference here. I'm not sure whether the above should return nil, or non-nil, or the value of `symbols-with-pos-enabled`, to be honest, but I guess returning non-nil has worked fine until now, so I think we'd be better off staying with that. I'd even like it to try and replace uses of `eq/eql` with `equal` in those cases where we want to overlook differences in symbol-positions, so that we can eventually get rid of `symbols-with-pos-enabled` which I consider as a wart. Stefan From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 01:35:30 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 05:35:30 +0000 Received: from localhost ([127.0.0.1]:53252 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRnTG-0006C6-HF for submit@debbugs.gnu.org; Fri, 04 Aug 2023 01:35:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40954) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRnTF-0006Bj-9p for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 01:35:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRnT7-0004bl-Su; Fri, 04 Aug 2023 01:35:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=0p2UO8t6lX6YMbOxswbIXtr9Uo217K2VufP6wP7nzFI=; b=c7krAdVdwlK+ fVZqoCwSS6PE7LWfYeB+g+il0iS717JEs/+ZWhk1S65Vp09YvKwBXkew/GcOqhc57jRp8gqYbW7ZI NERFTnUowNmf7gajtLGAScwvZxxfwAnW7yJonkGVamvi3ebgmAi8Anx3noxc+oijOrp1LS3arNZeG v48aVFBBWkPMbkMWwYyCcNOUoOE/8YyEzpI0xwSAEDigPEXrB5qK9m5uty6UsEZRTDHOTayKlnI/p 6jbaBQwQs8NOLWbHorbd9IpDuJTTCwfQKrnu7lK20CJ3gm/QM1VSf7biUl67DaIINKltsI9atbIgt CP8xMyoVx4OKwVMF+RNwfQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRnT5-0003CD-Hr; Fri, 04 Aug 2023 01:35:20 -0400 Date: Fri, 04 Aug 2023 08:35:30 +0300 Message-Id: <83leerwesd.fsf@gnu.org> From: Eli Zaretskii To: Alan Mackenzie In-Reply-To: (message from Alan Mackenzie on Thu, 3 Aug 2023 21:10:56 +0000) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, monnier@iro.umontreal.ca, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: acm@muc.de, 65017@debbugs.gnu.org, > Eric Marsden > Date: Thu, 3 Aug 2023 21:10:56 +0000 > From: Alan Mackenzie > > I think the cause is in internal_equal in src/fns.c, where the following > code appears: > > /* A symbol with position compares the contained symbol, and is > `equal' to the corresponding ordinary symbol. */ > if (SYMBOL_WITH_POS_P (o1)) > o1 = SYMBOL_WITH_POS_SYM (o1); > if (SYMBOL_WITH_POS_P (o2)) > o2 = SYMBOL_WITH_POS_SYM (o2); > > This piece of code converts symbols with positions to ordinary symbols > without first checking symbols-with-pos-enabled. I think this is the > bug. > > I will work on a patch to fix it (which shouldn't take long). Thanks, but when you have a solution in hand, please also check its effect on performance. AFAIR, this part was tuned for optimal performance, back when symbols with positions were introduced; it would be a pity to lose performance due to fixing this bug, if that can be avoided. From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 05:55:56 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 09:55:57 +0000 Received: from localhost ([127.0.0.1]:53447 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRrXI-0004WI-Lc for submit@debbugs.gnu.org; Fri, 04 Aug 2023 05:55:56 -0400 Received: from mx3.muc.de ([193.149.48.5]:48029) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRrXG-0004W3-HJ for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 05:55:55 -0400 Received: (qmail 20404 invoked by uid 3782); 4 Aug 2023 11:55:47 +0200 Received: from acm.muc.de (pd953a874.dip0.t-ipconnect.de [217.83.168.116]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 04 Aug 2023 11:55:47 +0200 Received: (qmail 5837 invoked by uid 1000); 4 Aug 2023 09:55:46 -0000 Date: Fri, 4 Aug 2023 09:55:46 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Thu, Aug 03, 2023 at 17:46:44 -0400, Stefan Monnier wrote: > > Sorry about my last post. I now see what the core problem is, namely > > that (equal 'equal #) is returning non-nil. > This is not really the core problem IIUC since `cl-macs.el` uses `eq` > rather than `equal` so changing `equal` won't make much of > a difference here. No, it wasn't the cause of this bug. It's a separate bug in its own right, though. > I'm not sure whether the above should return nil, or non-nil, or the value > of `symbols-with-pos-enabled`, to be honest, but I guess returning non-nil > has worked fine until now, so I think we'd be better off staying with that. I've lost the context, somewhat, but the key thing is that the notion of symbol with position isn't really defined when symbols-with-pos-enabled is nil. Returning non-nil for (equal 'foo #) in this case is like saying 'foo is equal to an undefined entity. This is asking for the sort of trouble we're seeing in this bug. > I'd even like it to try and replace uses of `eq/eql` with `equal` in > those cases where we want to overlook differences in symbol-positions, so > that we can eventually get rid of `symbols-with-pos-enabled` which > I consider as a wart. OK. I don't think you can do this, because you'd have to replace lots of eq's in macros with equal. We don't control all these macros. That's aside from the massive disruption this would cause to bytecomp.el and friends. I don't think you should do this, since symbols-with-pos-enabled, ugly though it may be, is working. Also, you'd have to be careful not to slow Emacs down. Now, let's diagnose bug#65017! > Stefan -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 06:14:25 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 10:14:26 +0000 Received: from localhost ([127.0.0.1]:53451 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRrpB-0004xB-Gc for submit@debbugs.gnu.org; Fri, 04 Aug 2023 06:14:25 -0400 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]:47246) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRrp8-0004wx-Ne for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 06:14:23 -0400 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-4fe3b86cec1so3237896e87.2 for <65017@debbugs.gnu.org>; Fri, 04 Aug 2023 03:14:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691144057; x=1691748857; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=z1amEirZ6wCr/4HnTRVrVdsB56OjDavGJAwrKWVn/cs=; b=sxnBYYoXNyX7S1uWBav83BEckxEUM8U7RnkUgogmg23AStoNEQ6TICdarePK6W9wsM CMuJxVgi0uUQz44HmuBBhxxcMN4VQXvlnBTeASkXkBhnyXKUwYj+a+gEcxmOBk8wVGIv Z8md13r2JJE+U0D5+A6+sfvYWITrf+1KN5DgziNd9TrAQ35wGkhmsD1VgxNxZlBFcWOW 37+JgsPRCZXk4buugPGILdU9UV0bxNKPpvBTKvyFNRu1Tfc4NtczjzytGa4jP3/ieVEi 4vwcuKjOYQI1hki9bwqUXPrm1/tW6iVrr0t4KyOlXoHO5iet7FcmHStrjv4zV7hY++uB avqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691144057; x=1691748857; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=z1amEirZ6wCr/4HnTRVrVdsB56OjDavGJAwrKWVn/cs=; b=h5jJWgkmBjeT0Magl3jgHZzayaMaYOC31nRNJBVsFKOklqU6pVk/u3EIoKbT5to8iq vzuTgP37mrHFKir4OSP9UTB19B+RKdMC/q0s771Or/MZ+ujGposoOLNtMozpcZX5e6WI xvuS6o7edrcEv4fJqlSrwMteOqn8HGkcpp4FbtrzWk3OokK4KfZ41LGdrrjTQpPZ2UnK 5jBP4jsCD+OFA1X2ri+XgIJ5eAwGLmJW6lojMUfLYZEs9BXBNi/tI2cDeCAVe6UKlmhX s7dhJZi1mqVWL7S2A+pjRW1RZ3iyuJ9qQylDmUXVMbf8d1LHUvDwWxQWQ41tlHZ1aQPO dbow== X-Gm-Message-State: AOJu0YxqQFSUqc7IOkXuUQvGGH7wn1wzVVOmrL9vgENsDbRjchwRbDZN bMCH0YLSKwMgdneIHYdluj8= X-Google-Smtp-Source: AGHT+IEcS880vNIHgHCsUO5Guso10g+k9IOf+yZVWf7QmqfPx3k7aCkdZv8/lEW83ICNKm0RNzNLeQ== X-Received: by 2002:a19:5007:0:b0:4fb:821e:2241 with SMTP id e7-20020a195007000000b004fb821e2241mr865280lfb.23.1691144056431; Fri, 04 Aug 2023 03:14:16 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id a6-20020a19f806000000b004fe27229f55sm312638lff.216.2023.08.04.03.14.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Aug 2023 03:14:16 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Fri, 4 Aug 2023 12:14:14 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> To: Stefan Monnier X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 3 aug. 2023 kl. 23.46 skrev Stefan Monnier : > I'd even like it to try and replace uses of `eq/eql` with `equal` in > those cases where we want to overlook differences in symbol-positions, = so > that we can eventually get rid of `symbols-with-pos-enabled` which > I consider as a wart. I agree it would be wonderful if we could restore `eq` to its former = simplicity and speed but is that easily achievable at this point? For = example, what about macros that compare arguments with `eq`? Separate data structures for locations might be an option worth = exploring, keeping the actual s-expressions unadorned. Consider a reader = mode that also produces a table mapping cons cells read to their = locations, for example. (By the way, I removed the unwind-eq commutation on master for now.) From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 07:11:57 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 11:11:57 +0000 Received: from localhost ([127.0.0.1]:53499 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRsir-0006gT-9v for submit@debbugs.gnu.org; Fri, 04 Aug 2023 07:11:57 -0400 Received: from mx3.muc.de ([193.149.48.5]:50190) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRsio-0006g9-T0 for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 07:11:55 -0400 Received: (qmail 10002 invoked by uid 3782); 4 Aug 2023 13:11:47 +0200 Received: from acm.muc.de (pd953a874.dip0.t-ipconnect.de [217.83.168.116]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 04 Aug 2023 13:11:47 +0200 Received: (qmail 6498 invoked by uid 1000); 4 Aug 2023 11:11:46 -0000 Date: Fri, 4 Aug 2023 11:11:46 +0000 To: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: 65017@debbugs.gnu.org, Stefan Monnier , Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Mattias. On Fri, Aug 04, 2023 at 12:14:14 +0200, Mattias Engdegård wrote: > 3 aug. 2023 kl. 23.46 skrev Stefan Monnier : > > I'd even like it to try and replace uses of `eq/eql` with `equal` in > > those cases where we want to overlook differences in > > symbol-positions, so that we can eventually get rid of > > `symbols-with-pos-enabled` which I consider as a wart. > I agree it would be wonderful if we could restore `eq` to its former > simplicity and speed but is that easily achievable at this point? For > example, what about macros that compare arguments with `eq`? > Separate data structures for locations might be an option worth > exploring, keeping the actual s-expressions unadorned. Consider a > reader mode that also produces a table mapping cons cells read to their > locations, for example. Using a hash table, or something similar? This is all very well, but the garbage collecter, for every collected object, would have to check whether that object is in that table, and if so remove it. This would slow down garbage collection, possibly by a lot. The slow down would be less if there were a variable saying whether or not this table is active. Something very like symbols-with-pos-enabled. ;-) [ .... ] -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 09:23:07 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 13:23:08 +0000 Received: from localhost ([127.0.0.1]:53585 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRuln-00073F-Kr for submit@debbugs.gnu.org; Fri, 04 Aug 2023 09:23:07 -0400 Received: from mx3.muc.de ([193.149.48.5]:53999) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRull-00072S-KA for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 09:23:06 -0400 Received: (qmail 62345 invoked by uid 3782); 4 Aug 2023 15:22:59 +0200 Received: from acm.muc.de (pd953a874.dip0.t-ipconnect.de [217.83.168.116]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 04 Aug 2023 15:22:58 +0200 Received: (qmail 794 invoked by uid 1000); 4 Aug 2023 13:22:58 -0000 Date: Fri, 4 Aug 2023 13:22:58 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan, Mattias, and Eric. On Thu, Aug 03, 2023 at 10:43:16 -0400, Stefan Monnier wrote: [ .... ] > AFAICT the problem is that OT1H `symbols-with-pos-enabled` is > non-nil while loading the second file, but OTOH positions aren't > stripped from the resulting code. > So in `cl--labels-convert` when we check > (eq f (car cl--labels-convert-cache)) [ .... ] > I don't know why `symbols-with-pos-enabled` is non-nil at that point (I > thought we only enabled it wile byte-compiling), .... This is not quite the case. symbols-with-pos-enabled gets erroneously bound to t in internal-macroexpand-for-load (emacs-lisp/macroexp.el). This is the cause of the bug; in cl--labels-convert it causes the first eq to return non-nil when comparing 'equal to #. Removing that binding fixes the problem. This does not affect make bootstrap or make check. The necessary patch is: diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el index b05aba3e1a7..ea838f5b7b2 100644 --- a/lisp/emacs-lisp/macroexp.el +++ b/lisp/emacs-lisp/macroexp.el @@ -799,8 +799,7 @@ macroexp--debug-eager (defun internal-macroexpand-for-load (form full-p) ;; Called from the eager-macroexpansion in readevalloop. - (let ((symbols-with-pos-enabled t) - (print-symbols-bare t)) + (let ((print-symbols-bare t)) (cond ;; Don't repeat the same warning for every top-level element. ((eq 'skip (car macroexp--pending-eager-loads)) form) Stefan, it would still be nice for cl--labels-convert-cache to get initialised each time it gets used. Please try out the patch, and if all is well I can commit it and close the bug. [ .... ] > Stefan -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 09:41:29 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 13:41:29 +0000 Received: from localhost ([127.0.0.1]:53601 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRv3Z-0007bj-05 for submit@debbugs.gnu.org; Fri, 04 Aug 2023 09:41:29 -0400 Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]:54423) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRv3W-0007bT-AZ for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 09:41:27 -0400 Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-4fe11652b64so3505061e87.0 for <65017@debbugs.gnu.org>; Fri, 04 Aug 2023 06:41:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691156480; x=1691761280; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=GJjOMnH80zw8m+vCVejWkkwx2qItIbaAbA7HrYBo8ic=; b=YQ1v00CLJxww7A/MHpfFkH9u6c5qDb/2qGkZbV+aVytcy7GEbZsy/9HwSPE5R60qfq gomSU8mcME56zYWEiONDwl/aiZi1mYGjxPXVJjjgV8hFOY2plIH+HYLl2o/hBhFeTZw2 p5Xkc73NUMSJOPqE6IxegSxxZ21zColjYUV/cXF4+ZNF2+UpLs3/LGrFW7Ng552vyezd W2YmjOzCJcl2BiqGJPQ1Wjxdj4G6I4Putu2fv8OFm1raSL4jcW5QkvEsV4AbGwT2+jDc CJ4ADB+8cEljA2EUi61kslMjPjgRvgbWzD4T7icPOauOTLsGzWBSHvFWsjwReVMXr6pk X1vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691156480; x=1691761280; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=GJjOMnH80zw8m+vCVejWkkwx2qItIbaAbA7HrYBo8ic=; b=d4wglLwAS2rVbwkdYqdbGURyZEaMVPnCRWj1L9yx5ITggU2oefFWaZR0kbzIW/R2jV q0NnKFxQvPxZjqbuDsWDx0jqpIRKjToPThxxGfaI9sZU8BYWK3knEA9NU35UR8iUktNu t/HTDXVtIvEPcIXrk/gyrq8ACaAVqpi5DMgu/Pojvrm3H9KsD3b5AKGW29Fm8lPJUs77 20sUTjzxnjJm8kPOBZ3PJ3tTnza7fk9H0x9rCRY4XXA3rG+htnzVccm4akbLmYOhMM7B MRUaO/Of/oaeGhFNhgjtucPyHehQomo9hnxOiw5VVHwPGxvAiu1bVdHUGFrIbPxHedD1 p+wQ== X-Gm-Message-State: AOJu0YxkD6GteJygAmxMP4iJI123wZoX/Qw50haq9T/A3VKF4E4QEECK zXFGNCcGkqSN0UMKsk+uDUI= X-Google-Smtp-Source: AGHT+IHbEjZSU0ZJ8UP9SI5S4eqebscjUws8s7g1zcq/n30bKeDBA9JpJS1+uaqJOCiaFQLc+Zohgw== X-Received: by 2002:a05:6512:b9c:b0:4fb:820a:f87f with SMTP id b28-20020a0565120b9c00b004fb820af87fmr1848582lfv.10.1691156479457; Fri, 04 Aug 2023 06:41:19 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id y18-20020a197512000000b004fb73bea65esm381063lfe.25.2023.08.04.06.41.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Aug 2023 06:41:18 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Fri, 4 Aug 2023 15:41:17 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <3CB03B6B-8C4F-4A4C-A4C4-A9BECF5EFCEF@gmail.com> References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> To: Alan Mackenzie X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: 65017@debbugs.gnu.org, Stefan Monnier , Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 4 aug. 2023 kl. 13.11 skrev Alan Mackenzie : > Using a hash table, or something similar? This is all very well, but = the > garbage collecter, for every collected object, would have to check > whether that object is in that table, and if so remove it. This would > slow down garbage collection, possibly by a lot. The table could have weak keys, and there are known GC algorithms for = dealing with those. It doesn't actually need to be a hash table at all -- we only need to = optimise for fast insertion and small size, not lookup (for displaying = compiler diagnostics, for example). This is a very half-baked idea; I haven't thought it through at all. For = example, we may need a way to transfer location from one sexp to a newly = built one, and that may complicate the inner workings of the compiler = (or not). It's clear that symbols-with-pos is a neat hack that is surprisingly = effective, with almost no changes to Lisp code needed at all. It = definitely has improved diagnostics. Unfortunately it makes everything a = bit slower, not just the compiler, but symbols-with-pos will serve until = we come up with something better. From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 10:04:34 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 14:04:35 +0000 Received: from localhost ([127.0.0.1]:54458 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRvPu-00005L-JA for submit@debbugs.gnu.org; Fri, 04 Aug 2023 10:04:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50416) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRvPs-000057-MA for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 10:04:33 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRvPl-0002M9-MF; Fri, 04 Aug 2023 10:04:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=30xscrQtxufZ0jXk0fC/+4AzvTX7JKgfSXmGw+xdgPE=; b=QrcDngCu8ssbz9ur4f1B 0BuoWGQsLJb05vpPVlRsLzqoM9h7xhV2OtNfFpk5H8oJrUlBHUCa/4PgNTIQR6FCpG9f8UW0uM/NS j6pNfoj5V9ZLJsgZGe7O5LTNEDe9Rqda9oeRzanNuijEldOeMrhKSJ7WWigDidH76MYzF6cp6yz8w xZ66JCgCZrm5LH0iE+heWiGC3BVPYgksNxe9tvNSzfPd8WToNhwN63Khn5vPz07lxOAlE8NiqSX4t JbRteyHlpUCjTt2hcKllMvoKOTCLG7VG4N47w1Cf+gIn0JQtn82kvoUVWbPQs3XN7qCjARFqshEJL 5tQRiayD9jglrg==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRvPl-0007nC-5t; Fri, 04 Aug 2023 10:04:25 -0400 Date: Fri, 04 Aug 2023 17:04:36 +0300 Message-Id: <83r0oivr7v.fsf@gnu.org> From: Eli Zaretskii To: Alan Mackenzie In-Reply-To: (message from Alan Mackenzie on Fri, 4 Aug 2023 13:22:58 +0000) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, monnier@iro.umontreal.ca, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: Mattias EngdegÃ¥rd , > 65017@debbugs.gnu.org, Eric Marsden > Date: Fri, 4 Aug 2023 13:22:58 +0000 > From: Alan Mackenzie > > symbols-with-pos-enabled gets erroneously > bound to t in internal-macroexpand-for-load (emacs-lisp/macroexp.el). > This is the cause of the bug; in cl--labels-convert it causes the first > eq to return non-nil when comparing 'equal to #. Why "erroneously"? what are the rules for binding that variable to a non-nil value? I don't see any of that documented in the "Symbols with Position" node of the ELisp manual. From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 10:16:32 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 14:16:32 +0000 Received: from localhost ([127.0.0.1]:54484 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRvbU-0000TD-DS for submit@debbugs.gnu.org; Fri, 04 Aug 2023 10:16:32 -0400 Received: from mx3.muc.de ([193.149.48.5]:55566) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRvbR-0000Sv-NO for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 10:16:30 -0400 Received: (qmail 24398 invoked by uid 3782); 4 Aug 2023 16:16:23 +0200 Received: from acm.muc.de (pd953a874.dip0.t-ipconnect.de [217.83.168.116]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 04 Aug 2023 16:16:23 +0200 Received: (qmail 1158 invoked by uid 1000); 4 Aug 2023 14:16:22 -0000 Date: Fri, 4 Aug 2023 14:16:22 +0000 To: Eli Zaretskii Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <83leerwesd.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <83leerwesd.fsf@gnu.org> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, monnier@iro.umontreal.ca, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Eli. On Fri, Aug 04, 2023 at 08:35:30 +0300, Eli Zaretskii wrote: > > Cc: acm@muc.de, 65017@debbugs.gnu.org, > > Eric Marsden > > Date: Thu, 3 Aug 2023 21:10:56 +0000 > > From: Alan Mackenzie > > > > I think the cause is in internal_equal in src/fns.c, where the following > > code appears: > > > > /* A symbol with position compares the contained symbol, and is > > `equal' to the corresponding ordinary symbol. */ > > if (SYMBOL_WITH_POS_P (o1)) > > o1 = SYMBOL_WITH_POS_SYM (o1); > > if (SYMBOL_WITH_POS_P (o2)) > > o2 = SYMBOL_WITH_POS_SYM (o2); > > > > This piece of code converts symbols with positions to ordinary symbols > > without first checking symbols-with-pos-enabled. I think this is the > > bug. > > > > I will work on a patch to fix it (which shouldn't take long). I have raised bug #65051 for this (which already contains a patch). > Thanks, but when you have a solution in hand, please also check its > effect on performance. AFAIR, this part was tuned for optimal > performance, back when symbols with positions were introduced; it > would be a pity to lose performance due to fixing this bug, if that > can be avoided. I don't think we need worry, here. In the generated code, in the normal non-compilation scenario, two tests are being replaced by one test. The current two tests are on variables which will already be in registers, but they perform fairly involved bit twiddling. The new test is a simple zero/non-zero test on a variable which, whilst not yet in a register, will certainly be in the processor's cache. Also, that variable will shortly be needed again, in the Lisp_Cons case. I'll try a few timings, though. I'll be surprised indeed if there're any measurable differences. -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 10:50:05 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 14:50:05 +0000 Received: from localhost ([127.0.0.1]:54502 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRw7w-0001Ya-JU for submit@debbugs.gnu.org; Fri, 04 Aug 2023 10:50:05 -0400 Received: from mx3.muc.de ([193.149.48.5]:56511) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRw7u-0001Xw-GO for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 10:50:03 -0400 Received: (qmail 62456 invoked by uid 3782); 4 Aug 2023 16:49:56 +0200 Received: from acm.muc.de (pd953a874.dip0.t-ipconnect.de [217.83.168.116]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 04 Aug 2023 16:49:56 +0200 Received: (qmail 1388 invoked by uid 1000); 4 Aug 2023 14:49:56 -0000 Date: Fri, 4 Aug 2023 14:49:56 +0000 To: Eli Zaretskii Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <83r0oivr7v.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <83r0oivr7v.fsf@gnu.org> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, monnier@iro.umontreal.ca, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Eli. On Fri, Aug 04, 2023 at 17:04:36 +0300, Eli Zaretskii wrote: > > Cc: Mattias Engdegård , > > 65017@debbugs.gnu.org, Eric Marsden > > Date: Fri, 4 Aug 2023 13:22:58 +0000 > > From: Alan Mackenzie > > symbols-with-pos-enabled gets erroneously > > bound to t in internal-macroexpand-for-load (emacs-lisp/macroexp.el). > > This is the cause of the bug; in cl--labels-convert it causes the first > > eq to return non-nil when comparing 'equal to #. > Why "erroneously"? what are the rules for binding that variable to a > non-nil value? internal-macroexpand-for-load isn't being called in the context of a byte compilation. It might create a symbol with position which wrongly matches, or fails to match, another symbol. This is what has happened in this bug. > I don't see any of that documented in the "Symbols with Position" node > of the ELisp manual. Well, there is the sentence: "These objects are intended for use by the byte compiler, which records in them the position of each symbol occurrence and uses those positions in warning and error messages.". Do you think this should be firmed up to something like: "These objects are for the use of the byte compiler, which records in them the position of each symbol occurrence and uses those positions in warning and error messages. They shouldn't normally be used otherwise."? -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 11:22:29 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 15:22:29 +0000 Received: from localhost ([127.0.0.1]:54570 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRwdJ-0002Xn-2f for submit@debbugs.gnu.org; Fri, 04 Aug 2023 11:22:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45852) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRwdG-0002XX-6I for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 11:22:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRwd8-0001f9-NP; Fri, 04 Aug 2023 11:22:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=1FuwMec55uIVOWoypY/qiyjulftvPsn5VZI4qgoGPLY=; b=XQq0XQRjSBhX sw/2AMOBk71wUCzYK9h/5GoRfOWo62fdoAyCBk0ZuGaIXQWoOJlC4Oq56K4rm3dpkM/3bnzjfDR6Z vrdCQgOKm4etJo0IQgBKH3F5eSksCyXdo8s8E9yNRgr864OLEk2wASTA6YLKxepeBCek/YTcRA6hk 2RFUctdAiLyH4RI1tO/BEzQySIbtmfy+xRXDt7QKpcKwI88bfW5NxZcD1qKN07ogPlWNkG/VhzVW0 wZPqECZhxuRjK71Wp8vhcizBal09xiI3eKq03r4iW8lTnn1p6WNQbKe2f9SOLjfLxYML2AhMaLGLi arqLfFM5byEVcID5B9Q26g==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRwd8-0007NS-6O; Fri, 04 Aug 2023 11:22:18 -0400 Date: Fri, 04 Aug 2023 18:22:29 +0300 Message-Id: <83jzuavnm2.fsf@gnu.org> From: Eli Zaretskii To: Alan Mackenzie In-Reply-To: (message from Alan Mackenzie on Fri, 4 Aug 2023 14:49:56 +0000) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <83r0oivr7v.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, monnier@iro.umontreal.ca, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Fri, 4 Aug 2023 14:49:56 +0000 > Cc: monnier@iro.umontreal.ca, mattias.engdegard@gmail.com, > 65017@debbugs.gnu.org, eric.marsden@risk-engineering.org > From: Alan Mackenzie > > > > symbols-with-pos-enabled gets erroneously > > > bound to t in internal-macroexpand-for-load (emacs-lisp/macroexp.el). > > > This is the cause of the bug; in cl--labels-convert it causes the first > > > eq to return non-nil when comparing 'equal to #. > > > Why "erroneously"? what are the rules for binding that variable to a > > non-nil value? > > internal-macroexpand-for-load isn't being called in the context of a > byte compilation. It might create a symbol with position which wrongly > matches, or fails to match, another symbol. This is what has happened > in this bug. If internal-macroexpand-for-load is "verboten" from being called by the byte-compiler, I'd expect an assertion in it to that effect. Because someone, some day, might easily forget and call that function in the byte-compiler. Btw, why was this binding added there to begin with? > > I don't see any of that documented in the "Symbols with Position" node > > of the ELisp manual. > > Well, there is the sentence: "These objects are intended for use by the > byte compiler, which records in them the position of each symbol > occurrence and uses those positions in warning and error messages.". > > Do you think this should be firmed up to something like: "These objects > are for the use of the byte compiler, which records in them the position > of each symbol occurrence and uses those positions in warning and error > messages. They shouldn't normally be used otherwise."? Something like that, perhaps even stronger. And maybe an explanation what kind of problems could using them outside of the byte compiler cause. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 12:43:22 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 16:43:22 +0000 Received: from localhost ([127.0.0.1]:54651 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRxtZ-0004zT-TS for submit@debbugs.gnu.org; Fri, 04 Aug 2023 12:43:22 -0400 Received: from mx3.muc.de ([193.149.48.5]:59771) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRxtX-0004zC-BH for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 12:43:20 -0400 Received: (qmail 93884 invoked by uid 3782); 4 Aug 2023 18:43:13 +0200 Received: from acm.muc.de (pd953a874.dip0.t-ipconnect.de [217.83.168.116]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 04 Aug 2023 18:43:13 +0200 Received: (qmail 28016 invoked by uid 1000); 4 Aug 2023 16:43:12 -0000 Date: Fri, 4 Aug 2023 16:43:12 +0000 To: Eli Zaretskii Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <83r0oivr7v.fsf@gnu.org> <83jzuavnm2.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <83jzuavnm2.fsf@gnu.org> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, monnier@iro.umontreal.ca, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Eli. On Fri, Aug 04, 2023 at 18:22:29 +0300, Eli Zaretskii wrote: > > Date: Fri, 4 Aug 2023 14:49:56 +0000 > > Cc: monnier@iro.umontreal.ca, mattias.engdegard@gmail.com, > > 65017@debbugs.gnu.org, eric.marsden@risk-engineering.org > > From: Alan Mackenzie > > > > symbols-with-pos-enabled gets erroneously > > > > bound to t in internal-macroexpand-for-load (emacs-lisp/macroexp.el). > > > > This is the cause of the bug; in cl--labels-convert it causes the first > > > > eq to return non-nil when comparing 'equal to #. > > > Why "erroneously"? what are the rules for binding that variable to a > > > non-nil value? > > internal-macroexpand-for-load isn't being called in the context of a > > byte compilation. It might create a symbol with position which wrongly > > matches, or fails to match, another symbol. This is what has happened > > in this bug. > If internal-macroexpand-for-load is "verboten" from being called by > the byte-compiler, I'd expect an assertion in it to that effect. > Because someone, some day, might easily forget and call that function > in the byte-compiler. I don't think there's any such prohibition in this case. The function is called only from readevalloop in src/lread.c as part of loading a .el file. It is probable that an eval-when-compile could cause a .el file to be loaded during a byte compilation. This would call internal-macroexpand-for-load with symbols-with-pos-enabled non-nil, I think. > Btw, why was this binding added there to begin with? A good question. It was back in January 2022, I was getting eager macro expansion failures while trying to bootstrap (my development version of) Emacs. Binding that variable made those failures go away. It turns out, that was not the correct fix. Somehow that erroneous binding stayed in the code and got committed. > > > I don't see any of that documented in the "Symbols with Position" node > > > of the ELisp manual. > > Well, there is the sentence: "These objects are intended for use by the > > byte compiler, which records in them the position of each symbol > > occurrence and uses those positions in warning and error messages.". > > Do you think this should be firmed up to something like: "These objects > > are for the use of the byte compiler, which records in them the position > > of each symbol occurrence and uses those positions in warning and error > > messages. They shouldn't normally be used otherwise."? > Something like that, perhaps even stronger. And maybe an explanation > what kind of problems could using them outside of the byte compiler > cause. OK. Maybe ".... They shouldn't normally be used otherwise. Doing so can cause unexpected results with basic Emacs functions such as `eq' and `equal'."? > Thanks. -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 04 13:54:38 2023 Received: (at 65017) by debbugs.gnu.org; 4 Aug 2023 17:54:38 +0000 Received: from localhost ([127.0.0.1]:54743 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRz0X-0001Ax-KJ for submit@debbugs.gnu.org; Fri, 04 Aug 2023 13:54:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60504) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qRz0V-0001Ai-AL for 65017@debbugs.gnu.org; Fri, 04 Aug 2023 13:54:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRz0O-0004Wl-KL; Fri, 04 Aug 2023 13:54:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=J5JVA1Hlm6xqSeFuQW5JJtwSF/Dor6v90/mLupeRDrs=; b=Iu0usueCoKLB dcRzIk2OwTvjaced3R7aQ51ZCp2pRUx1svBQJjYejSaZ38GSwws3PD3G2RkCOj9kwRfO4Ez0gJ9Yf dPeWiHUcegnP6EYcDBz5ieaqMNlnRxXA4F6X0wW8uOTPzOlqUolSPXbKR91v3E+CuiGheJRKsPWWm X/+FxfKhC2EtYULXTNUpgfUVZOpzQa+tBRkUI3MUbQaVYqF/8y0jLIECSkECjVeexSsXv1bU/N0He IUCmL41dJA7jplhRb1B9sUO2JMNtxt0+RHsjx+8IdTWd5ppyfHVkfsL/KmCFosVwd8Bm+rNvNr/qw LBhPqCGsGHOeSAJVwMvl5Q==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRz0O-0001oS-4X; Fri, 04 Aug 2023 13:54:28 -0400 Date: Fri, 04 Aug 2023 20:54:40 +0300 Message-Id: <83a5v6vgkf.fsf@gnu.org> From: Eli Zaretskii To: Alan Mackenzie In-Reply-To: (message from Alan Mackenzie on Fri, 4 Aug 2023 16:43:12 +0000) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <83r0oivr7v.fsf@gnu.org> <83jzuavnm2.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, monnier@iro.umontreal.ca, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Fri, 4 Aug 2023 16:43:12 +0000 > Cc: monnier@iro.umontreal.ca, mattias.engdegard@gmail.com, > 65017@debbugs.gnu.org, eric.marsden@risk-engineering.org > From: Alan Mackenzie > > > If internal-macroexpand-for-load is "verboten" from being called by > > the byte-compiler, I'd expect an assertion in it to that effect. > > Because someone, some day, might easily forget and call that function > > in the byte-compiler. > > I don't think there's any such prohibition in this case. The function is > called only from readevalloop in src/lread.c as part of loading a .el > file. It is probable that an eval-when-compile could cause a .el file to > be loaded during a byte compilation. This would call > internal-macroexpand-for-load with symbols-with-pos-enabled non-nil, I > think. But then, if the caller binds this variable non-nil, the problem will again rear its ugly head? > > > Do you think this should be firmed up to something like: "These objects > > > are for the use of the byte compiler, which records in them the position > > > of each symbol occurrence and uses those positions in warning and error > > > messages. They shouldn't normally be used otherwise."? > > > Something like that, perhaps even stronger. And maybe an explanation > > what kind of problems could using them outside of the byte compiler > > cause. > > OK. Maybe ".... They shouldn't normally be used otherwise. Doing so can > cause unexpected results with basic Emacs functions such as `eq' and > `equal'."? That's a good start, thanks. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 05 16:22:44 2023 Received: (at 65017) by debbugs.gnu.org; 5 Aug 2023 20:22:44 +0000 Received: from localhost ([127.0.0.1]:58254 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSNnQ-0006AI-FE for submit@debbugs.gnu.org; Sat, 05 Aug 2023 16:22:44 -0400 Received: from mx3.muc.de ([193.149.48.5]:51964) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSNnL-0006A3-MZ for 65017@debbugs.gnu.org; Sat, 05 Aug 2023 16:22:43 -0400 Received: (qmail 99437 invoked by uid 3782); 5 Aug 2023 22:22:33 +0200 Received: from acm.muc.de (p4fe15973.dip0.t-ipconnect.de [79.225.89.115]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 05 Aug 2023 22:22:33 +0200 Received: (qmail 19963 invoked by uid 1000); 5 Aug 2023 20:22:32 -0000 Date: Sat, 5 Aug 2023 20:22:32 +0000 To: Eli Zaretskii Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <83leerwesd.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <83leerwesd.fsf@gnu.org> X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, monnier@iro.umontreal.ca, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Eli On Fri, Aug 04, 2023 at 08:35:30 +0300, Eli Zaretskii wrote: > > Cc: acm@muc.de, 65017@debbugs.gnu.org, > > Eric Marsden > > Date: Thu, 3 Aug 2023 21:10:56 +0000 > > From: Alan Mackenzie > Thanks, but when you have a solution in hand, please also check its > effect on performance. AFAIR, this part was tuned for optimal > performance, back when symbols with positions were introduced; it > would be a pity to lose performance due to fixing this bug, if that > can be avoided. This should really be in bug #65051, but: I've run time src/emacs -Q -batch --eval '(let ((a 1) (b 1) (times 10000000)) (while (> times 0) (equal a b) (setq times (1- times))))' with different values for a and b (and slightly different quoting syntax, sometimes). I get the following results, reporting the "user: time" from GNU/Linux: New code: a, b = 1, 1: 1.760s 1.760s 1,746s a, b = 1, 3: 1.796s 1,772s 1.777s 1.0, 1.0: 1,757s 1.776s 1.751s 1.0, 1.1: 1.792s 1.760s 1.779s '(a b c), '(a b c): 2.041s 2.042s 2.039s '(a b c), '(a b d): 2.096s 2.071s 2.084s "1", "1": 1.841s 1.860s 1.845s "1", "3": 1.865s 1.846s 1.869s Old code: a, b = 1, 1: 1.744s 1.757s 1.762s a, b = 1, 3: 1.755s 1,777s 1.759s 1.0, 1.0: 1.787s 1.748s 1.775s 1.0, 1.1: 1.762s 1.770s 1.774s '(a b c), '(a b c): 2.021s 2.057s 2.019s '(a b c), '(a b d): 2.046s 2.090s 2.100s "1", "1": 1.854s 1.900s 1.884s "1", "3": 1.849s 1.833s 1.838s I think it's fair to say that the new code is not slower than the old code, to within the measuring limits of these simple tests. Any differences, such as they are, are in the second and third decimal places, and vary more between different measurements, than between the Old code and New code. They are surely too small to matter. -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 05 18:40:59 2023 Received: (at 65017) by debbugs.gnu.org; 5 Aug 2023 22:40:59 +0000 Received: from localhost ([127.0.0.1]:58303 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSPxD-0001AB-F9 for submit@debbugs.gnu.org; Sat, 05 Aug 2023 18:40:59 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:3106) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSPxB-00019w-4m for 65017@debbugs.gnu.org; Sat, 05 Aug 2023 18:40:58 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id CC45D444903; Sat, 5 Aug 2023 18:40:50 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691275249; bh=ydMIOY1z5kASmQP1ljp/I3WTjXOkAwmMbue4Z5ni8sY=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=UVfQTnPJLZJ6bt/PvPNZntWlQfeQi3Cb10GPywK1bNvUvlHz2QJsqPsmC/QdQqQKj JQVCvpuKIcKhMfAeX1rMlBf2bTfcnuHvceQ9bJQvvEvzfuyYBLVQ4abRyZ8NjNNlXk dlSObyBoNWO3IBCtxdcidaOToCifaQSh7p5rNkD/H4dj4EcwLRiYfN3V6y3PxgIpwH rXmX9cwVYOOFbvCwbAXWfrf5hh/wfTc84fga7+EqXA9JShLzVegthLy0URw6Eh620I f1v1vW+z0D3jz4KQ2R+dEUH6lFp54dltqsgJZd0YmVe7w8yAXdqKlE2Ken3JMIMgzy QZ+HKNJc9g72Q== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 222A24448FB; Sat, 5 Aug 2023 18:40:49 -0400 (EDT) Received: from alfajor (unknown [190.16.213.142]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 8EBE712032D; Sat, 5 Aug 2023 18:40:47 -0400 (EDT) From: Stefan Monnier To: Mattias =?windows-1252?Q?Engdeg=E5rd?= Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: ("Mattias =?windows-1252?Q?Engdeg=E5rd=22's?= message of "Fri, 4 Aug 2023 12:14:14 +0200") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Sat, 05 Aug 2023 18:40:34 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> I'd even like it to try and replace uses of `eq/eql` with `equal` in >> those cases where we want to overlook differences in symbol-positions, so >> that we can eventually get rid of `symbols-with-pos-enabled` which >> I consider as a wart. > I agree it would be wonderful if we could restore `eq` to its former > simplicity and speed but is that easily achievable at this point? I didn't say it would be easy. But I think it's feasible (tho it will take many years). > For example, what about macros that compare arguments with `eq`? Yes, these are the cases where we need to "replace uses of `eq/eql` with `equal`". > Separate data structures for locations might be an option worth exploring, > keeping the actual s-expressions unadorned. Consider a reader mode that also > produces a table mapping cons cells read to their locations, for example. When Alan looked at it, the cost seemed prohibitive. BTW, a related option would be to develop a new kind of macro-definition along the lines of what's used in Scheme, where the macro author doesn't need to worry about such issues because the macro knows which parts of the data it manipulates are chunks of code (potentially adorned with metainfo) and can take care of extracting the underlying unadorned code. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 05 18:46:00 2023 Received: (at 65017) by debbugs.gnu.org; 5 Aug 2023 22:46:00 +0000 Received: from localhost ([127.0.0.1]:58308 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSQ24-0001Hc-8X for submit@debbugs.gnu.org; Sat, 05 Aug 2023 18:46:00 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:13960) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSQ22-0001HO-2L for 65017@debbugs.gnu.org; Sat, 05 Aug 2023 18:45:58 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id C79111000EF; Sat, 5 Aug 2023 18:45:52 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691275547; bh=o6X44dlQ8e8cUwuWyzhIdLC44P4CoCpHEG1M+Fh2Xr0=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=mMcfWttIpRe9ziW/IgH1YjlEX/KBun9TBJ2Kd0JIG/NWywqE+kqPApe2QZOzwad+5 6PTvWcYU1oWmfQJZJTAWDq8iYS9Q4c3pbOBVFvyrOBdqzAasUWFUUT93paPZsZCuBg Qk9iaAHFBLn9yIJDnkJJnGK9aAgzU94yrtn4A8q6tgzSjSdiPddyZrDgHWnQOEudky aJH8+/qsae0/LsKeMP/37fht4E7T2IMt/sCjdtpfzExxM53eZ5d/bIALRKZkHBKylO qBgrJhCfIDYaPKr6pAz6RZsFA66epSSWDGl5znHdkF8KnGluNFaVgj/JBinY9dhaMg 89mEP7H3o1AbQ== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 847081000AD; Sat, 5 Aug 2023 18:45:47 -0400 (EDT) Received: from alfajor (unknown [190.16.213.142]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id CCD881203DF; Sat, 5 Aug 2023 18:45:45 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Fri, 4 Aug 2023 09:55:46 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Sat, 05 Aug 2023 18:45:32 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.250 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > I've lost the context, somewhat, but the key thing is that the notion of > symbol with position isn't really defined when symbols-with-pos-enabled > is nil. Returning non-nil for (equal 'foo #) in this > case is like saying 'foo is equal to an undefined entity. This is > asking for the sort of trouble we're seeing in this bug. I don't like this idea that we should pretend sympos objects don't exist (or are "undefined") when `symbols-with-pos-enabled` is nil. > and friends. I don't think you should do this, since > symbols-with-pos-enabled, ugly though it may be, is working. As I said, I consider it as a wart, IOW a technical debt. We *should* come up with a plan to reimburse this debt, regardless if it takes a long time to pay it back. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 05 18:54:12 2023 Received: (at 65017) by debbugs.gnu.org; 5 Aug 2023 22:54:12 +0000 Received: from localhost ([127.0.0.1]:58323 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSQA0-0001Zt-2Q for submit@debbugs.gnu.org; Sat, 05 Aug 2023 18:54:12 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:49496) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSQ9x-0001Ze-Qo for 65017@debbugs.gnu.org; Sat, 05 Aug 2023 18:54:10 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 5E853444904; Sat, 5 Aug 2023 18:54:04 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691276042; bh=ep1vweo6jfjlV46RxsM76NAtoNweD6JX9Rchk9AWT5s=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=iTmD9QyKu2cDqM4XGwL41KiqRGh9Ryp7wEv4eLa1AzqSiI7wVejQpyb3IAHjut2si +QFgpFeyU0LH8kixL4mDTMx64FtqnHWycaRtRTPZZHkw8Yt9CnztzAvMdf+L6LujA/ wLP7LLkNAmCzzPwILNcj/m0UTi2YviTMLekCDbsZwRWJgK6n1ORuNGWhOB6Ag36QPM tJX9YoH/mqCd3mo426f0ps7r/sFHVIRHUpWvC6rgmtOHBs/M+yMM4OFhPlFeob+YqR bbWc/2zxjYf1P8yky3SlCYRctr56ZyWVoWZ6kWj7T6rwZ8LGRGZtGtq4A9u5tXTwLd j821+oS+8EN4A== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id C70054448FB; Sat, 5 Aug 2023 18:54:02 -0400 (EDT) Received: from alfajor (unknown [190.16.213.142]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 73FFC120397; Sat, 5 Aug 2023 18:54:01 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Fri, 4 Aug 2023 13:22:58 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Sat, 05 Aug 2023 18:53:48 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> I don't know why `symbols-with-pos-enabled` is non-nil at that point (I >> thought we only enabled it wile byte-compiling), .... > > This is not quite the case. symbols-with-pos-enabled gets erroneously > bound to t in internal-macroexpand-for-load (emacs-lisp/macroexp.el). Aha! > diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el > index b05aba3e1a7..ea838f5b7b2 100644 > --- a/lisp/emacs-lisp/macroexp.el > +++ b/lisp/emacs-lisp/macroexp.el > @@ -799,8 +799,7 @@ macroexp--debug-eager > > (defun internal-macroexpand-for-load (form full-p) > ;; Called from the eager-macroexpansion in readevalloop. > - (let ((symbols-with-pos-enabled t) > - (print-symbols-bare t)) > + (let ((print-symbols-bare t)) > (cond > ;; Don't repeat the same warning for every top-level element. > ((eq 'skip (car macroexp--pending-eager-loads)) form) Looks good to me. AFAICT this binding was added at some point where it seemed like a good idea but we later figured better places to do it, and we just didn't remove it because it seemed "harmless" (or because we just didn't think of it). > Stefan, it would still be nice for cl--labels-convert-cache to get > initialised each time it gets used. No, the problem is not initialization, as I pointed out. The problem is that this `eq` should not consider a symbol equal to a sympos *ever* (contrary to most other uses of `eq` in macros). Stefan From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 05 18:58:24 2023 Received: (at 65017) by debbugs.gnu.org; 5 Aug 2023 22:58:24 +0000 Received: from localhost ([127.0.0.1]:58328 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSQE3-0001h1-Pc for submit@debbugs.gnu.org; Sat, 05 Aug 2023 18:58:24 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:14199) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSQE1-0001gp-Cc for 65017@debbugs.gnu.org; Sat, 05 Aug 2023 18:58:21 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 20B9F1000EF; Sat, 5 Aug 2023 18:58:16 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691276294; bh=K3Wia532RHU0R38VpeV5yFKSs/UpIf47KHAgZ674sDE=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=BYx/J5v9fNAXynKYszCUbwS5ESsmd/R3K7TeVgNe1/VoJCfxym52rQz7ZvkmFXAIR S4wJr9vUUO/McUSgIxUQoF4k+yTj5ejDGD7OwzggLP2Vbfa+kRJTvFQSzLSOR/sauN goiZXeBjG3NTlIbzg2cKaYrrawvVBREpf6YPerxaXfclr/n8luR5StGM0CdyXi39hx 0XtCizLZw/sMQsJDQ9H1GhJ1wfUiWhJ3qyae98Bp1j3D2UKMD6+CXBRbbxlv8giV/8 HYAbJcIqI6EJhgICp0ueXSI9QVsqVfVDx+y1ck8EdmGL3ipm8M3+dr9wYV/TPGEgGi Rd4bpEIuVdwew== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id E2FC91000AD; Sat, 5 Aug 2023 18:58:14 -0400 (EDT) Received: from alfajor (unknown [190.16.213.142]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 611E712032C; Sat, 5 Aug 2023 18:58:13 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: <83jzuavnm2.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 04 Aug 2023 18:22:29 +0300") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <83r0oivr7v.fsf@gnu.org> <83jzuavnm2.fsf@gnu.org> Date: Sat, 05 Aug 2023 18:58:00 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.125 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> internal-macroexpand-for-load isn't being called in the context of a >> byte compilation. It might create a symbol with position which wrongly >> matches, or fails to match, another symbol. This is what has happened >> in this bug. > > If internal-macroexpand-for-load is "verboten" from being called by > the byte-compiler, I'd expect an assertion in it to that effect. It's not "verboten". Removing the binding just says that if you want to pass adorned code to that function, it's the caller's responsability to let-bind the variable around the call. > Because someone, some day, might easily forget and call that function > in the byte-compiler. The byte-compiler already let-binds that variables, so it won't be a problem. > Btw, why was this binding added there to begin with? IIUC it took some trial and error to get to understand where that var needs to be bound (as well as where is the right spot to strip the sympos), and this is just one binding that was never removed after we found a better place for it. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 06 00:49:59 2023 Received: (at 65017) by debbugs.gnu.org; 6 Aug 2023 04:49:59 +0000 Received: from localhost ([127.0.0.1]:58492 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSViI-0002DF-Rh for submit@debbugs.gnu.org; Sun, 06 Aug 2023 00:49:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42862) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSViF-0002D1-6D for 65017@debbugs.gnu.org; Sun, 06 Aug 2023 00:49:57 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qSVi7-0007J6-HP; Sun, 06 Aug 2023 00:49:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=qBQ7nkWgX4eN3hIkt0yTC0K4TY1B+xZbb577NlQ8GkY=; b=S5IOU1h8IxWj cjTRj456ZPx9X3mRFuvzE5z1jY2sWRcPEK4AC0dWwQlOk/JEtH+f3OkOlY8WYlJowVKOmEPCHSBfz UXNhqAqODimcG1dsWGLBYbZHe+jWx+KiwOwQHLyV6zA1NqmWuwq0Yk9KA2AjhwM0WVeXfBcLctiEY l/H4tulUSxMuLII2OT6eg6yNktV2NU4pLT2ICt1N3+sxVf8YHcqEJ2k/yBUcO7Y+aJ7l88zRHkYHu yF9f1ghOkLASLh15cdMGuj45KmFem0kZyNUv6yZJe2iGEQoJToJ8ffcafKX9uyrHnyVbfH12EicAq qVZE25G7NhduxX1SSRt1hg==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qSVi2-0003HD-SR; Sun, 06 Aug 2023 00:49:45 -0400 Date: Sun, 06 Aug 2023 07:49:58 +0300 Message-Id: <83sf8wrczt.fsf@gnu.org> From: Eli Zaretskii To: Alan Mackenzie In-Reply-To: (message from Alan Mackenzie on Sat, 5 Aug 2023 20:22:32 +0000) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <83leerwesd.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: mattias.engdegard@gmail.com, 65017@debbugs.gnu.org, monnier@iro.umontreal.ca, eric.marsden@risk-engineering.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sat, 5 Aug 2023 20:22:32 +0000 > Cc: monnier@iro.umontreal.ca, mattias.engdegard@gmail.com, > 65017@debbugs.gnu.org, eric.marsden@risk-engineering.org > From: Alan Mackenzie > > > Thanks, but when you have a solution in hand, please also check its > > effect on performance. AFAIR, this part was tuned for optimal > > performance, back when symbols with positions were introduced; it > > would be a pity to lose performance due to fixing this bug, if that > > can be avoided. > > This should really be in bug #65051, but: > > I've run > > time src/emacs -Q -batch --eval '(let ((a 1) (b 1) (times 10000000)) (while (> times 0) (equal a b) (setq times (1- times))))' > > with different values for a and b (and slightly different quoting > syntax, sometimes). I get the following results, reporting the "user: > time" from GNU/Linux: > > New code: > a, b = 1, 1: 1.760s 1.760s 1,746s > a, b = 1, 3: 1.796s 1,772s 1.777s > 1.0, 1.0: 1,757s 1.776s 1.751s > 1.0, 1.1: 1.792s 1.760s 1.779s > '(a b c), '(a b c): 2.041s 2.042s 2.039s > '(a b c), '(a b d): 2.096s 2.071s 2.084s > "1", "1": 1.841s 1.860s 1.845s > "1", "3": 1.865s 1.846s 1.869s > > Old code: > a, b = 1, 1: 1.744s 1.757s 1.762s > a, b = 1, 3: 1.755s 1,777s 1.759s > 1.0, 1.0: 1.787s 1.748s 1.775s > 1.0, 1.1: 1.762s 1.770s 1.774s > '(a b c), '(a b c): 2.021s 2.057s 2.019s > '(a b c), '(a b d): 2.046s 2.090s 2.100s > "1", "1": 1.854s 1.900s 1.884s > "1", "3": 1.849s 1.833s 1.838s > > I think it's fair to say that the new code is not slower than the old > code, to within the measuring limits of these simple tests. Any > differences, such as they are, are in the second and third decimal > places, and vary more between different measurements, than between the > Old code and New code. They are surely too small to matter. Thanks, this LGTM. From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 06 06:47:27 2023 Received: (at 65017) by debbugs.gnu.org; 6 Aug 2023 10:47:27 +0000 Received: from localhost ([127.0.0.1]:58708 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSbIF-0006Ev-CZ for submit@debbugs.gnu.org; Sun, 06 Aug 2023 06:47:27 -0400 Received: from mail-lf1-x134.google.com ([2a00:1450:4864:20::134]:62750) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qSbIB-0006Ee-IP for 65017@debbugs.gnu.org; Sun, 06 Aug 2023 06:47:26 -0400 Received: by mail-lf1-x134.google.com with SMTP id 2adb3069b0e04-4fe45da0a89so5653326e87.1 for <65017@debbugs.gnu.org>; Sun, 06 Aug 2023 03:47:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691318837; x=1691923637; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=k23d1CquyqKsQoiy4tVy/HAx8gYSxCctMnvROHpzc9k=; b=Po7g9zJfdFGmL5cJTfHFESUFawSrJ1f+wMnoEuTTa+OVbzIc6Y1HjOwWWUhWEhRY1c UfI0DuALsh2TdzUw/aeMmf6Du4eIHRuaL3DJ+1FQvgkldc/mOoGjctptn2GI5oKZ35FX GVu6whCXGMki2gScuQW/Mum+wdhlv3QKuKcLmodMjPOHOFVfsWp/yq3TVY6yI8tDt8vy t3x2dtRi2K+QmVp3zz9jrCi12Jp9VEOLqT7+VXNlFYdCQ89B8iXtwO650MIwWIqg0eNg dx2uFezOhBTmJ7Pip2tXsWFfsMZKbIberxg11M2ehegFGjZMdFywF9vADkNW0QLowdkH Qm2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691318837; x=1691923637; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=k23d1CquyqKsQoiy4tVy/HAx8gYSxCctMnvROHpzc9k=; b=QlkHLLw/YtvKs65o9t8McpiPGvz2W+Kb4LzYLygcqr/jzn/8r+01uI3+UjjvLQ8Y8n jnXcFXv0XrzkRPVLed5WKVOjGEoYLZtTdwU4faRuk0rABgjqxWZ9BU0EvLT2egsA2uqL n3T1h3qVL6sWadFmcJGJgiWHCjER016HppoPsEVrkfHMK6HVeIgFXI1/kQxqPWT6jxb7 nvc4iFgNgYBaZ1BlqIBr/FUyL+635AzDsKkDiRar5KgAoJikBkOy/xDk89w8lzLjdilP oa+VjZV4N5OPN+l28eHrvs9EswD8sKr4PjQ6RORFGdxvt0SY2bQAbqgVRIOs9XeWqf98 VJyA== X-Gm-Message-State: AOJu0YwLdJDR10WHxCt0uz8oEEGj+qsHtg3DDirggTFiTSoOTt74mqUg XqOyUroePYZ5xTs85XvMkJU= X-Google-Smtp-Source: AGHT+IHw6lnt527WSH5O0529r15j7xJ8AJrRHbRfC27hnh+TB7KlmDMO/X+EZBBdsEVv2KEI83QfTw== X-Received: by 2002:a05:6512:ea3:b0:4fd:daa0:aed8 with SMTP id bi35-20020a0565120ea300b004fddaa0aed8mr1497067lfb.4.1691318837224; Sun, 06 Aug 2023 03:47:17 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id a6-20020a19f806000000b004fe27229f55sm1066673lff.216.2023.08.06.03.47.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Aug 2023 03:47:16 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Sun, 6 Aug 2023 12:47:15 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> To: Stefan Monnier X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 6 aug. 2023 kl. 00.40 skrev Stefan Monnier : >> Separate data structures for locations might be an option worth = exploring, >> keeping the actual s-expressions unadorned. Consider a reader mode = that also >> produces a table mapping cons cells read to their locations, for = example. >=20 > When Alan looked at it, the cost seemed prohibitive. I'm not aware of the details but think that it may be worth revisiting. The only serious difficulty a priori appears to be keeping locations in = transformed code, but most of our diagnostics are issued on code before = Lisp optimisations so this should mostly matter to macro-expansion. > BTW, a related option would be to develop a new kind of = macro-definition > along the lines of what's used in Scheme, where the macro author = doesn't > need to worry about such issues because the macro knows which parts of > the data it manipulates are chunks of code (potentially adorned with > metainfo) and can take care of extracting the underlying unadorned = code. Yes, Scheme doesn't have much problems with user code manipulating Lisp = as data. But what do actual Lisp compilers do? Do they perform a slow re-read = when until they get to the location they want, when they need to issue a = diagnostic? (Relint does something similar, in fact.) From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 06 07:59:47 2023 Received: (at 65017) by debbugs.gnu.org; 6 Aug 2023 11:59:47 +0000 Received: from localhost ([127.0.0.1]:58751 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qScQE-00050J-KL for submit@debbugs.gnu.org; Sun, 06 Aug 2023 07:59:47 -0400 Received: from mx3.muc.de ([193.149.48.5]:23601) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qScQB-000503-EO for 65017@debbugs.gnu.org; Sun, 06 Aug 2023 07:59:45 -0400 Received: (qmail 76216 invoked by uid 3782); 6 Aug 2023 13:59:37 +0200 Received: from acm.muc.de (pd953a3d8.dip0.t-ipconnect.de [217.83.163.216]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 06 Aug 2023 13:59:36 +0200 Received: (qmail 6500 invoked by uid 1000); 6 Aug 2023 11:59:36 -0000 Date: Sun, 6 Aug 2023 11:59:36 +0000 To: Stefan Monnier , Mattias =?iso-8859-1?Q?Engdeg=E5rd?= Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: acm@muc.de, 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan and Mattias. On Sat, Aug 05, 2023 at 18:53:48 -0400, Stefan Monnier wrote: > >> I don't know why `symbols-with-pos-enabled` is non-nil at that point (I > >> thought we only enabled it wile byte-compiling), .... > > This is not quite the case. symbols-with-pos-enabled gets erroneously > > bound to t in internal-macroexpand-for-load (emacs-lisp/macroexp.el). > Aha! > > diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el > > index b05aba3e1a7..ea838f5b7b2 100644 > > --- a/lisp/emacs-lisp/macroexp.el > > +++ b/lisp/emacs-lisp/macroexp.el > > @@ -799,8 +799,7 @@ macroexp--debug-eager > > (defun internal-macroexpand-for-load (form full-p) > > ;; Called from the eager-macroexpansion in readevalloop. > > - (let ((symbols-with-pos-enabled t) > > - (print-symbols-bare t)) > > + (let ((print-symbols-bare t)) > > (cond > > ;; Don't repeat the same warning for every top-level element. > > ((eq 'skip (car macroexp--pending-eager-loads)) form) > Looks good to me. AFAICT this binding was added at some point where it > seemed like a good idea but we later figured better places to do it, > and we just didn't remove it because it seemed "harmless" (or because > we just didn't think of it). There is another unneeded binding of symbols-with-pos-enabled in macroexp.el, and several redundant bindings of print-symbols-bare in bytecomp.el (which are commented as such), and one of print-symbols-bare in macroexp.el. The patch below removes these bindings. make bootstrap and make check still work OK, with it. A compile-defun in a function with an error produces the correct error message. I suggest installing this patch into master. > > Stefan, it would still be nice for cl--labels-convert-cache to get > > initialised each time it gets used. > No, the problem is not initialization, as I pointed out. The problem is > that this `eq` should not consider a symbol equal to a sympos *ever* > (contrary to most other uses of `eq` in macros). Are you sure? Why not? If cl--labels-convert-cache is being used inside the byte compiler, it surely needs to consider # and # as eq? There is no mechanism to make these two SWPs eq whilst excluding their eq with the bare symbol. diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el index ac040799a22..b9d1948e555 100644 --- a/lisp/emacs-lisp/bytecomp.el +++ b/lisp/emacs-lisp/bytecomp.el @@ -489,8 +489,7 @@ byte-compile-recurse-toplevel ;; 3.2.3.1, "Processing of Top Level Forms". The semantics are very ;; subtle: see test/lisp/emacs-lisp/bytecomp-tests.el for interesting ;; cases. - (let ((print-symbols-bare t)) ; Possibly redundant binding. - (setf form (macroexp-macroexpand form byte-compile-macro-environment))) + (setf form (macroexp-macroexpand form byte-compile-macro-environment)) (if (eq (car-safe form) 'progn) (cons (car form) (mapcar (lambda (subform) @@ -568,11 +567,10 @@ byte-compile-initial-macro-environment ;; Don't compile here, since we don't know ;; whether to compile as byte-compile-form ;; or byte-compile-file-form. - (let* ((print-symbols-bare t) ; Possibly redundant binding. - (expanded - (macroexpand--all-toplevel - form - macroexpand-all-environment))) + (let ((expanded + (macroexpand--all-toplevel + form + macroexpand-all-environment))) (eval (byte-run-strip-symbol-positions (bytecomp--copy-tree expanded)) lexical-binding) @@ -2489,8 +2487,7 @@ byte-compile-output-file-form ;; Spill output for the native compiler here (push (make-byte-to-native-top-level :form form :lexical lexical-binding) byte-to-native-top-level-forms)) - (let ((print-symbols-bare t) ; Possibly redundant binding. - (print-escape-newlines t) + (let ((print-escape-newlines t) (print-length nil) (print-level nil) (print-quoted t) @@ -2524,8 +2521,7 @@ byte-compile-output-docform ;; in the input buffer (now current), not in the output buffer. (let ((dynamic-docstrings byte-compile-dynamic-docstrings)) (with-current-buffer byte-compile--outbuffer - (let (position - (print-symbols-bare t)) ; Possibly redundant binding. + (let (position) ;; Insert the doc string, and make it a comment with #@LENGTH. (when (and (>= (nth 1 info) 0) dynamic-docstrings) (setq position (byte-compile-output-as-comment @@ -2621,8 +2617,7 @@ byte-compile-flush-pending byte-compile-jump-tables nil)))) (defun byte-compile-preprocess (form &optional _for-effect) - (let ((print-symbols-bare t)) ; Possibly redundant binding. - (setq form (macroexpand-all form byte-compile-macro-environment))) + (setq form (macroexpand-all form byte-compile-macro-environment)) ;; FIXME: We should run byte-optimize-form here, but it currently does not ;; recurse through all the code, so we'd have to fix this first. ;; Maybe a good fix would be to merge byte-optimize-form into diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el index b05aba3e1a7..47d663b5d4a 100644 --- a/lisp/emacs-lisp/macroexp.el +++ b/lisp/emacs-lisp/macroexp.el @@ -107,8 +107,7 @@ macroexp--all-clauses (defun macroexp--compiler-macro (handler form) (condition-case-unless-debug err - (let ((symbols-with-pos-enabled t)) - (apply handler form (cdr form))) + (apply handler form (cdr form)) (error (message "Warning: Optimization failure for %S: Handler: %S\n%S" (car form) handler err) @@ -799,40 +798,38 @@ macroexp--debug-eager (defun internal-macroexpand-for-load (form full-p) ;; Called from the eager-macroexpansion in readevalloop. - (let ((symbols-with-pos-enabled t) - (print-symbols-bare t)) - (cond - ;; Don't repeat the same warning for every top-level element. - ((eq 'skip (car macroexp--pending-eager-loads)) form) - ;; If we detect a cycle, skip macro-expansion for now, and output a warning - ;; with a trimmed backtrace. - ((and load-file-name (member load-file-name macroexp--pending-eager-loads)) - (let* ((bt (delq nil - (mapcar #'macroexp--trim-backtrace-frame - (macroexp--backtrace)))) - (elem `(load ,(file-name-nondirectory load-file-name))) - (tail (member elem (cdr (member elem bt))))) - (if tail (setcdr tail (list '…))) - (if (eq (car-safe (car bt)) 'macroexpand-all) (setq bt (cdr bt))) - (if macroexp--debug-eager - (debug 'eager-macroexp-cycle) - (error "Eager macro-expansion skipped due to cycle:\n %s" - (mapconcat #'prin1-to-string (nreverse bt) " => "))) - (push 'skip macroexp--pending-eager-loads) - form)) - (t - (condition-case err - (let ((macroexp--pending-eager-loads - (cons load-file-name macroexp--pending-eager-loads))) - (if full-p - (macroexpand--all-toplevel form) - (macroexpand form))) - (error - ;; Hopefully this shouldn't happen thanks to the cycle detection, - ;; but in case it does happen, let's catch the error and give the - ;; code a chance to macro-expand later. - (error "Eager macro-expansion failure: %S" err) - form)))))) + (cond + ;; Don't repeat the same warning for every top-level element. + ((eq 'skip (car macroexp--pending-eager-loads)) form) + ;; If we detect a cycle, skip macro-expansion for now, and output a warning + ;; with a trimmed backtrace. + ((and load-file-name (member load-file-name macroexp--pending-eager-loads)) + (let* ((bt (delq nil + (mapcar #'macroexp--trim-backtrace-frame + (macroexp--backtrace)))) + (elem `(load ,(file-name-nondirectory load-file-name))) + (tail (member elem (cdr (member elem bt))))) + (if tail (setcdr tail (list '…))) + (if (eq (car-safe (car bt)) 'macroexpand-all) (setq bt (cdr bt))) + (if macroexp--debug-eager + (debug 'eager-macroexp-cycle) + (error "Eager macro-expansion skipped due to cycle:\n %s" + (mapconcat #'prin1-to-string (nreverse bt) " => "))) + (push 'skip macroexp--pending-eager-loads) + form)) + (t + (condition-case err + (let ((macroexp--pending-eager-loads + (cons load-file-name macroexp--pending-eager-loads))) + (if full-p + (macroexpand--all-toplevel form) + (macroexpand form))) + (error + ;; Hopefully this shouldn't happen thanks to the cycle detection, + ;; but in case it does happen, let's catch the error and give the + ;; code a chance to macro-expand later. + (error "Eager macro-expansion failure: %S" err) + form))))) ;; ¡¡¡ Big Ugly Hack !!! ;; src/bootstrap-emacs is mostly used to compile .el files, so it needs > Stefan -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 07 22:33:59 2023 Received: (at 65017) by debbugs.gnu.org; 8 Aug 2023 02:33:59 +0000 Received: from localhost ([127.0.0.1]:34624 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTCXm-0001qi-Qn for submit@debbugs.gnu.org; Mon, 07 Aug 2023 22:33:59 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:22819) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTCXh-0001qR-Pc for 65017@debbugs.gnu.org; Mon, 07 Aug 2023 22:33:57 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id E14C980699; Mon, 7 Aug 2023 22:33:47 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691462026; bh=5p/flkQvkyY6YSvRPjJLLxflXtV/tGzfKNG6c0HkgZ4=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=ZX6Wuoz/l1JzvVs5MRoN9a045yDliK6vqEB2dtRGZkkn/LrCeIo6embCcicXVStRu QNxQ9Zm7+XhGsWc+PQ0vPUOJgt8cwAXWsKGHmCwVJj5s0xlA9GmHkd5x/Qjc6vZbZz nxr70UuK7syVmj7uqTbOIQl5Fx4mBFEekNm599w7Z8RtogWh1zs6kOqnN7wQAAauJk 7zdLKQ7sph8G/P1ulAW5w28797vU+633duXr8+5ShPJWRpwT0rcXYcq76L5BDJ5g8D 9tJHcx/mrSK9r6h9o9F2orkO8qFBbiY2iOeeFD24O8xO1SGt0yT+jdZDGwbK+1a8uw N86AALszoP6Cw== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 5793C80058; Mon, 7 Aug 2023 22:33:46 -0400 (EDT) Received: from alfajor (host56.201-252-107.telecom.net.ar [201.252.107.56]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 169271202EA; Mon, 7 Aug 2023 22:33:44 -0400 (EDT) From: Stefan Monnier To: Mattias =?windows-1252?Q?Engdeg=E5rd?= Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: ("Mattias =?windows-1252?Q?Engdeg=E5rd=22's?= message of "Sun, 6 Aug 2023 12:47:15 +0200") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Mon, 07 Aug 2023 22:33:31 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >>> Separate data structures for locations might be an option worth exploring, >>> keeping the actual s-expressions unadorned. Consider a reader mode that also >>> produces a table mapping cons cells read to their locations, for example. >> When Alan looked at it, the cost seemed prohibitive. > I'm not aware of the details but think that it may be worth revisiting. Feel free :-) > The only serious difficulty a priori appears to be keeping locations in > transformed code, but most of our diagnostics are issued on code before Lisp > optimisations so this should mostly matter to macro-expansion. We shouldn't emit errors/warnings from byte-opt, indeed, so that part doesn't need to preserve the info, but we still need to preserve it at least through macro-expansion and cconv. > Yes, Scheme doesn't have much problems with user code manipulating Lisp as data. > But what do actual Lisp compilers do? Do they perform a slow re-read when > until they get to the location they want, when they need to issue > a diagnostic? (Relint does something similar, in fact.) Scheme's macro system has a notion of "syntactic object" (i.e. an sexp combined with metainfo), and the macros make it clear when you're looking inside a syntactic object or when you're building a new syntactic object. Don't know what Common Lisp systems do with `defmacro`, sorry. I know Gambit Scheme has "defmacro" (under the name `define-macro`) but it treats it as "second class citizen" in the sense that when using a macro defined with `define-macro`, the metainfo is just lost. Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 07 22:44:52 2023 Received: (at 65017) by debbugs.gnu.org; 8 Aug 2023 02:44:52 +0000 Received: from localhost ([127.0.0.1]:34639 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTCiJ-00027k-Pg for submit@debbugs.gnu.org; Mon, 07 Aug 2023 22:44:52 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:6490) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTCiI-00027Y-Fg for 65017@debbugs.gnu.org; Mon, 07 Aug 2023 22:44:51 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id A2C1C444AC6; Mon, 7 Aug 2023 22:44:44 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691462683; bh=W2qkgMyTWYLtqBr146S8I+38Sin7rbOI61nt4pUJhLo=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=WUAJvUAkSMGlOVfMKd5sspBXlNTnN4Cv20qd8TJmm98hkY4pGUc3qyVNuInjZPb7q Vj/HBPMqbD/ciBXSCW2b/FEQUUX0mvT8xaExll4+6vKbeLOIIhsI5/y+UtI7VRdJvO tQDb4u3whhNWAnYyr9/ZDwy+8Dv1KyLsL82NKyNFuAJ2m6oLWKN9ZGAnBoWhrQ8zfG fo08SVFcdY/jiDL8Eg6TESEBBZodeEL1KOkfMxpKSYzFDs8o7FlDvzh17iZTaW0CbK E2bOg1vPpy0cYOBopjx/BXjF9R76ETvCZIbwQXsNM2vUSjfEdwO6i0hEL+2/ZSI4oF TMXkgs6/Z+a1g== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 4A1D9444AC8; Mon, 7 Aug 2023 22:44:43 -0400 (EDT) Received: from alfajor (host56.201-252-107.telecom.net.ar [201.252.107.56]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 015FE1202BD; Mon, 7 Aug 2023 22:44:41 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Sun, 6 Aug 2023 11:59:36 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Mon, 07 Aug 2023 22:44:29 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > I suggest installing this patch into master. LGTM, thanks. >> > Stefan, it would still be nice for cl--labels-convert-cache to get >> > initialised each time it gets used. >> No, the problem is not initialization, as I pointed out. The problem is >> that this `eq` should not consider a symbol equal to a sympos *ever* >> (contrary to most other uses of `eq` in macros). > > Are you sure? Yes. > Why not? Not sure how to explain it any further than what I already described. > If cl--labels-convert-cache is being used > inside the byte compiler, it surely needs to consider # 42> and # as eq? No, it should not treat them equal (when it does, it introduces an incorrect sympos and can thus lead to error messages pointing at the wrong place). > There is no mechanism to make these two SWPs eq whilst excluding their > eq with the bare symbol. We luckily don't need such a mechanism here: we just need to use "BASE_EQ" :-) Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 08 12:56:12 2023 Received: (at 65017) by debbugs.gnu.org; 8 Aug 2023 16:56:12 +0000 Received: from localhost ([127.0.0.1]:37630 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTQ0C-00069n-Ev for submit@debbugs.gnu.org; Tue, 08 Aug 2023 12:56:12 -0400 Received: from mx3.muc.de ([193.149.48.5]:59683) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTQ0A-00069Z-Ij for 65017@debbugs.gnu.org; Tue, 08 Aug 2023 12:56:11 -0400 Received: (qmail 30350 invoked by uid 3782); 8 Aug 2023 18:56:03 +0200 Received: from acm.muc.de (pd953ac8f.dip0.t-ipconnect.de [217.83.172.143]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Tue, 08 Aug 2023 18:56:03 +0200 Received: (qmail 8451 invoked by uid 1000); 8 Aug 2023 16:56:03 -0000 Date: Tue, 8 Aug 2023 16:56:03 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: acm@muc.de, Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Mon, Aug 07, 2023 at 22:44:29 -0400, Stefan Monnier wrote: > > I suggest installing this patch into master. > LGTM, thanks. > >> > Stefan, it would still be nice for cl--labels-convert-cache to get > >> > initialised each time it gets used. > >> No, the problem is not initialization, as I pointed out. The problem is > >> that this `eq` should not consider a symbol equal to a sympos *ever* > >> (contrary to most other uses of `eq` in macros). > > Are you sure? > Yes. What about two SWPs with the same symbol but different positions? If they aren't considered EQ here, there will never be a match for the first arm of the cond form in cl--labels-convert; then cl--labels-convert-cache will get written, but never used. And if, somehow, it does get used (the current code, I think), then (as you write below) the argument F will get replaced by an F with the wrong position. Am I right, here? Why must the F get replaced by a different F? There must surely be a way, a simpler way than the current cl--labels-convert, to retain the current F (hence, not corrupting its position)? [ .... ] > > If cl--labels-convert-cache is being used > > inside the byte compiler, it surely needs to consider # > 42> and # as eq? > No, it should not treat them equal (when it does, it introduces an > incorrect sympos and can thus lead to error messages pointing at the > wrong place). Then isn't what is wrong here the introduction of the incorrect SWP rather than treating the two SWPs as EQ? This is obscure, difficult code. :-( We should think about committing a fix to the original bug, sometime, too. [ .... ] > Stefan -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 09 08:27:16 2023 Received: (at 65017-done) by debbugs.gnu.org; 9 Aug 2023 12:27:16 +0000 Received: from localhost ([127.0.0.1]:38450 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTiHT-0003Fh-Px for submit@debbugs.gnu.org; Wed, 09 Aug 2023 08:27:16 -0400 Received: from mx3.muc.de ([193.149.48.5]:37994) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTiHP-0003FQ-Gz for 65017-done@debbugs.gnu.org; Wed, 09 Aug 2023 08:27:14 -0400 Received: (qmail 75214 invoked by uid 3782); 9 Aug 2023 14:27:05 +0200 Received: from acm.muc.de (pd953a736.dip0.t-ipconnect.de [217.83.167.54]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 09 Aug 2023 14:27:04 +0200 Received: (qmail 29574 invoked by uid 1000); 9 Aug 2023 12:27:04 -0000 Date: Wed, 9 Aug 2023 12:27:04 +0000 To: Eric Marsden Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017-done Cc: acm@muc.de, 65017-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Eric. On Wed, Aug 02, 2023 at 12:28:24 +0200, Eric Marsden wrote: > The byte-compiler seems to erroneously remove the symbol-function for > equal in the > code show below. > --- file "perturb.el" --- > (require 'cl-lib) > (defun foo () >   (cl-flet ((bar (v) (list v))) >     (make-hash-table :test #'equal))) > --- > --- file "use.el" --- > (require 'cl-lib) > (require 'ert) > (defun test () >   (cl-flet ((foo (x) (list x))) >     (should (equal nil 42)))) > --- > % emacs -Q --batch --eval '(byte-compile-file "perturb.el")' -l use.el > -f test > Error: invalid-function (#) >   mapbacktrace(#f(compiled-function (evald func args flags) # -0x84e95e6e2517821>)) >   debug-early-backtrace() >   debug-early(error (invalid-function #)) >   #(nil 42) >   apply(# (nil 42)) >   (setq value-2 (apply fn-0 args-1)) >   (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq > form-description-4 (nconc (list '(should (equal nil 42))) (list :form > (cons fn-0 args-1)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil > (list :value value-2)) (if (eql value-2 'ert-form-evaluation-aborted-3) > nil (let* ((-explainer- (and t (ert--get-explainer 'equal)))) (if > -explainer- (list :explanation (apply -explainer- args-1)) nil))))) > (ert--signal-should-execution form-description-4)) >   (if (unwind-protect (setq value-2 (apply fn-0 args-1)) (setq > form-description-4 (nconc (list '(should (equal nil 42))) (list :form > (cons fn-0 args-1)) (if (eql value-2 'ert-form-evaluation-aborted-3) nil > (list :value value-2)) (if (eql value-2 'ert-form-evaluation-aborted-3) > nil (let* ((-explainer- (and t (ert--get-explainer 'equal)))) (if > -explainer- (list :explanation (apply -explainer- args-1)) nil))))) > (ert--signal-should-execution form-description-4)) nil (ert-fail > form-description-4)) [ .... ] >   test() >   command-line-1(("--eval" "(byte-compile-file \"perturb.el\")" "-l" > "use.el" "-f" "test")) >   command-line() >   normal-top-level() > Invalid function: # > The byte-compiler seems to have erroneously removed the symbol-function > for equal. The bug is now fixed in the master branch, and I'm closing the bug with this post. > In GNU Emacs 29.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.38, >  cairo version 1.16.0) of 2023-08-01, modified by Debian built on >  x86-ubc-02 > Windowing system distributor 'The X.Org Foundation', version 11.0.12201009 > System Description: Debian GNU/Linux trixie/sid The patch I posted on Sunday, although correct, doesn't apply cleanly to Emacs 29.1. So I'm giving you a version of the patch which does work on Emacs-29, and will enable you to fix the bug in your own copy of Emacs. After applying the patch, it will be advisable/necessary to rebuild your Emacs with $ make -j16 bootstrap # adjust the -j16 for your processor. .. I think you'll know how to do this, but if you want any help with applying the patch or running the bootstrap, feel free to send me private email. Here's the patch: diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el index d093d95a775..a9deb53db03 100644 --- a/lisp/emacs-lisp/bytecomp.el +++ b/lisp/emacs-lisp/bytecomp.el @@ -483,8 +483,7 @@ byte-compile-recurse-toplevel ;; 3.2.3.1, "Processing of Top Level Forms". The semantics are very ;; subtle: see test/lisp/emacs-lisp/bytecomp-tests.el for interesting ;; cases. - (let ((print-symbols-bare t)) ; Possibly redundant binding. - (setf form (macroexp-macroexpand form byte-compile-macro-environment))) + (setf form (macroexp-macroexpand form byte-compile-macro-environment)) (if (eq (car-safe form) 'progn) (cons (car form) (mapcar (lambda (subform) @@ -526,12 +525,11 @@ byte-compile-initial-macro-environment ;; Don't compile here, since we don't know ;; whether to compile as byte-compile-form ;; or byte-compile-file-form. - (let* ((print-symbols-bare t) ; Possibly redundant binding. - (expanded - (byte-run-strip-symbol-positions - (macroexpand--all-toplevel - form - macroexpand-all-environment)))) + (let ((expanded + (byte-run-strip-symbol-positions + (macroexpand--all-toplevel + form + macroexpand-all-environment)))) (eval expanded lexical-binding) expanded))))) (with-suppressed-warnings @@ -2436,8 +2434,7 @@ byte-compile-output-file-form ;; Spill output for the native compiler here (push (make-byte-to-native-top-level :form form :lexical lexical-binding) byte-to-native-top-level-forms)) - (let ((print-symbols-bare t) ; Possibly redundant binding. - (print-escape-newlines t) + (let ((print-escape-newlines t) (print-length nil) (print-level nil) (print-quoted t) @@ -2471,8 +2468,7 @@ byte-compile-output-docform ;; in the input buffer (now current), not in the output buffer. (let ((dynamic-docstrings byte-compile-dynamic-docstrings)) (with-current-buffer byte-compile--outbuffer - (let (position - (print-symbols-bare t)) ; Possibly redundant binding. + (let (position) ;; Insert the doc string, and make it a comment with #@LENGTH. (when (and (>= (nth 1 info) 0) dynamic-docstrings) (setq position (byte-compile-output-as-comment @@ -2568,8 +2564,7 @@ byte-compile-flush-pending byte-compile-jump-tables nil)))) (defun byte-compile-preprocess (form &optional _for-effect) - (let ((print-symbols-bare t)) ; Possibly redundant binding. - (setq form (macroexpand-all form byte-compile-macro-environment))) + (setq form (macroexpand-all form byte-compile-macro-environment)) ;; FIXME: We should run byte-optimize-form here, but it currently does not ;; recurse through all the code, so we'd have to fix this first. ;; Maybe a good fix would be to merge byte-optimize-form into diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el index 168de1bf180..6c604a75a33 100644 --- a/lisp/emacs-lisp/macroexp.el +++ b/lisp/emacs-lisp/macroexp.el @@ -107,8 +107,7 @@ macroexp--all-clauses (defun macroexp--compiler-macro (handler form) (condition-case-unless-debug err - (let ((symbols-with-pos-enabled t)) - (apply handler form (cdr form))) + (apply handler form (cdr form)) (error (message "Warning: Optimization failure for %S: Handler: %S\n%S" (car form) handler err) @@ -787,40 +786,38 @@ macroexp--debug-eager (defun internal-macroexpand-for-load (form full-p) ;; Called from the eager-macroexpansion in readevalloop. - (let ((symbols-with-pos-enabled t) - (print-symbols-bare t)) - (cond - ;; Don't repeat the same warning for every top-level element. - ((eq 'skip (car macroexp--pending-eager-loads)) form) - ;; If we detect a cycle, skip macro-expansion for now, and output a warning - ;; with a trimmed backtrace. - ((and load-file-name (member load-file-name macroexp--pending-eager-loads)) - (let* ((bt (delq nil - (mapcar #'macroexp--trim-backtrace-frame - (macroexp--backtrace)))) - (elem `(load ,(file-name-nondirectory load-file-name))) - (tail (member elem (cdr (member elem bt))))) - (if tail (setcdr tail (list '…))) - (if (eq (car-safe (car bt)) 'macroexpand-all) (setq bt (cdr bt))) - (if macroexp--debug-eager - (debug 'eager-macroexp-cycle) - (error "Eager macro-expansion skipped due to cycle:\n %s" - (mapconcat #'prin1-to-string (nreverse bt) " => "))) - (push 'skip macroexp--pending-eager-loads) - form)) - (t - (condition-case err - (let ((macroexp--pending-eager-loads - (cons load-file-name macroexp--pending-eager-loads))) - (if full-p - (macroexpand--all-toplevel form) - (macroexpand form))) - (error - ;; Hopefully this shouldn't happen thanks to the cycle detection, - ;; but in case it does happen, let's catch the error and give the - ;; code a chance to macro-expand later. - (error "Eager macro-expansion failure: %S" err) - form)))))) + (cond + ;; Don't repeat the same warning for every top-level element. + ((eq 'skip (car macroexp--pending-eager-loads)) form) + ;; If we detect a cycle, skip macro-expansion for now, and output a warning + ;; with a trimmed backtrace. + ((and load-file-name (member load-file-name macroexp--pending-eager-loads)) + (let* ((bt (delq nil + (mapcar #'macroexp--trim-backtrace-frame + (macroexp--backtrace)))) + (elem `(load ,(file-name-nondirectory load-file-name))) + (tail (member elem (cdr (member elem bt))))) + (if tail (setcdr tail (list '…))) + (if (eq (car-safe (car bt)) 'macroexpand-all) (setq bt (cdr bt))) + (if macroexp--debug-eager + (debug 'eager-macroexp-cycle) + (error "Eager macro-expansion skipped due to cycle:\n %s" + (mapconcat #'prin1-to-string (nreverse bt) " => "))) + (push 'skip macroexp--pending-eager-loads) + form)) + (t + (condition-case err + (let ((macroexp--pending-eager-loads + (cons load-file-name macroexp--pending-eager-loads))) + (if full-p + (macroexpand--all-toplevel form) + (macroexpand form))) + (error + ;; Hopefully this shouldn't happen thanks to the cycle detection, + ;; but in case it does happen, let's catch the error and give the + ;; code a chance to macro-expand later. + (error "Eager macro-expansion failure: %S" err) + form))))) ;; ¡¡¡ Big Ugly Hack !!! ;; src/bootstrap-emacs is mostly used to compile .el files, so it needs -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 09 23:42:12 2023 Received: (at 65017) by debbugs.gnu.org; 10 Aug 2023 03:42:12 +0000 Received: from localhost ([127.0.0.1]:40932 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTwYt-00059m-LB for submit@debbugs.gnu.org; Wed, 09 Aug 2023 23:42:12 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:53325) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qTwYq-00059N-EC for 65017@debbugs.gnu.org; Wed, 09 Aug 2023 23:42:09 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 7C54E1000DB; Wed, 9 Aug 2023 23:42:02 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691638921; bh=yUY7whvsZTT2fh5Af8910LJB8RKzEySI8fnRTt0BkXs=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=EtPNojFSM7BfhssccKZ6WIgQVCxB509lLMlLZbDu0xpYSpMHZiH5e2P6sRshztJDW K8e+pgx4Ukzat4v9habDuGkzQkmuT9g0G9ih/M/HInaP5kMWnCzMiB58bH22+/oJ6P VzT69FTxh98uIxEJjXVLuqm2QalT9IKlI86X9LTNYXWTxTF2OBPoKCzn/8GOy/EwjU qfGchNN8lIw92ndCUFeoXRsVpARqwwxkdgtvYBxEmukrhazwifThN4CfZ48Ubr6COg 5fZkMgY20Rdk1cRlQaoJdyP1xn5cbw83GmmDLjcn8A8WcEVGL1w4ugjMaW5mf3+lTS 3mzd1IR5HffLQ== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 5D5C8100068; Wed, 9 Aug 2023 23:42:01 -0400 (EDT) Received: from alfajor (unknown [190.190.107.145]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 274BD1202DD; Wed, 9 Aug 2023 23:41:59 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Tue, 8 Aug 2023 16:56:03 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Wed, 09 Aug 2023 23:41:56 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> > Are you sure? >> Yes. > What about two SWPs with the same symbol but different positions? If > they aren't considered EQ here, there will never be a match for the > first arm of the cond form in cl--labels-convert; then > cl--labels-convert-cache will get written, but never used. Nope: when it gets written, the `function` macro returns: (function ) so the macro is immediately called again with the *exact* same , so the second time the `function` macro is called, the cache does hit (and it's the only case where it should hit), making the second call to the macro return the *exact* `eq`-same (function ) list which is the trick that stops the infinite macroexpansion loop. Next time the `function` macro is invoked with a "similar" sympos the cache should *not* match because we don't want to accidentally replace (function ) with (function ) > And if, somehow, it does get used (the current code, I think), then (as > you write below) the argument F will get replaced by an F with the wrong > position. Am I right, here? That's right. > Why must the F get replaced by a different F? There must surely be a > way, a simpler way than the current cl--labels-convert, to retain the > current F (hence, not corrupting its position)? There might. The current hack is the best I could come up with. >> > If cl--labels-convert-cache is being used >> > inside the byte compiler, it surely needs to consider #> > 42> and # as eq? >> No, it should not treat them equal (when it does, it introduces an >> incorrect sympos and can thus lead to error messages pointing at the >> wrong place). > Then isn't what is wrong here the introduction of the incorrect SWP > rather than treating the two SWPs as EQ? I can't see how to separate one from the other here: the introduction of the incorrect SWP is due to treating the two SWP as `eq`. > This is obscure, difficult code. :-( Agreed. > We should think about committing a fix to the original bug, > sometime, too. I'm not completely sure we agree yet on what is "the original bug", but obviously I agree with your sentence :-) Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 10 10:50:15 2023 Received: (at 65017) by debbugs.gnu.org; 10 Aug 2023 14:50:15 +0000 Received: from localhost ([127.0.0.1]:43818 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qU6zP-00072l-20 for submit@debbugs.gnu.org; Thu, 10 Aug 2023 10:50:15 -0400 Received: from mx3.muc.de ([193.149.48.5]:27702) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qU6zK-00072P-VN for 65017@debbugs.gnu.org; Thu, 10 Aug 2023 10:50:13 -0400 Received: (qmail 2114 invoked by uid 3782); 10 Aug 2023 16:50:04 +0200 Received: from acm.muc.de (pd953a568.dip0.t-ipconnect.de [217.83.165.104]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 10 Aug 2023 16:50:03 +0200 Received: (qmail 5309 invoked by uid 1000); 10 Aug 2023 14:50:03 -0000 Date: Thu, 10 Aug 2023 14:50:03 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Wed, Aug 09, 2023 at 23:41:56 -0400, Stefan Monnier wrote: > >> > Are you sure? > >> Yes. > > What about two SWPs with the same symbol but different positions? If > > they aren't considered EQ here, there will never be a match for the > > first arm of the cond form in cl--labels-convert; then > > cl--labels-convert-cache will get written, but never used. > Nope: when it gets written, the `function` macro returns: > (function ) > so the macro is immediately called again with the *exact* same > , so the second time the `function` macro is called, the > cache does hit (and it's the only case where it should hit), making the > second call to the macro return the *exact* `eq`-same > (function ) > list which is the trick that stops the infinite macroexpansion loop. OK, thanks, I think I've got that now. > Next time the `function` macro is invoked with a "similar" sympos the > cache should *not* match because we don't want to accidentally replace > (function ) > with > (function ) > > And if, somehow, it does get used (the current code, I think), then (as > > you write below) the argument F will get replaced by an F with the wrong > > position. Am I right, here? > That's right. OK. So perhaps binding symbols-with-pos-enabled to nil around that eq call could be the way to go. > > Why must the F get replaced by a different F? There must surely be a > > way, a simpler way than the current cl--labels-convert, to retain the > > current F (hence, not corrupting its position)? > There might. The current hack is the best I could come up with. I'm not criticising the hack, not at all! But it could be better commented, and the doc string for cl--labels-convert could be more informative. The "why" is missing - why is necessary to handle `function' as a macro? I think it's to inhibit the processing of `function' as function somewhere else, but where and why? I think you explained it, at least partly, in an earlier post on this thread. > >> > If cl--labels-convert-cache is being used > >> > inside the byte compiler, it surely needs to consider # >> > 42> and # as eq? > >> No, it should not treat them equal (when it does, it introduces an > >> incorrect sympos and can thus lead to error messages pointing at the > >> wrong place). > > Then isn't what is wrong here the introduction of the incorrect SWP > > rather than treating the two SWPs as EQ? > I can't see how to separate one from the other here: the introduction of > the incorrect SWP is due to treating the two SWP as `eq`. > > This is obscure, difficult code. :-( > Agreed. > > We should think about committing a fix to the original bug, > > sometime, too. > I'm not completely sure we agree yet on what is "the original bug", but > obviously I agree with your sentence :-) I meant bug #65017. I committed a fix for it yesterday using the patch I posted here on Sunday, and closed the bug. > Stefan -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 11 23:28:19 2023 Received: (at 65017) by debbugs.gnu.org; 12 Aug 2023 03:28:19 +0000 Received: from localhost ([127.0.0.1]:48328 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUfIZ-0002Ts-BW for submit@debbugs.gnu.org; Fri, 11 Aug 2023 23:28:19 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:27020) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUfIU-0002TT-Kc for 65017@debbugs.gnu.org; Fri, 11 Aug 2023 23:28:18 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id EA6124423D2; Fri, 11 Aug 2023 23:28:08 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691810882; bh=grHNf1l7+fhBPhaX4S3i+wfl3ztFxZiFNYvlVI6Ouxw=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=N2yZkyukCIjICxniLgSsu7v4RWQifvD2wE8fJ9JYC5OvZujKR941zltuHXC0mOx72 +WV0t3B7Gzmu9DnW6PTBNGH+i0yDomokeM3ZMqorsAKvZWxEVkpgxwpohRwHTEQH2K GPQZuXDBfGTc7KKpn5r2PZmMU7BT6vFmws7Kda86fGmkxJ4MJ5WYB4oh6JQ1qEun6y rK2RWswwK7+D/cv/VsLoEzyk7Eya9I5ImCE+7LQ4aWHQTfq8FSUWdSU7xJ4D/vAISi x5nPS7q4Gyscym9ooJJF9J87yZM9LUQMvEfTiZrtVS5QxVjHwpXtk5hVWbgk1xRy5N yfV3ZhCBHD69Q== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id C16F64423CF; Fri, 11 Aug 2023 23:28:02 -0400 (EDT) Received: from pastel (69-165-141-248.dsl.teksavvy.com [69.165.141.248]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9428812019A; Fri, 11 Aug 2023 23:28:02 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Thu, 10 Aug 2023 14:50:03 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Fri, 11 Aug 2023 23:28:01 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.086 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> > And if, somehow, it does get used (the current code, I think), then (as >> > you write below) the argument F will get replaced by an F with the wrong >> > position. Am I right, here? >> That's right. > OK. So perhaps binding symbols-with-pos-enabled to nil around that eq > call could be the way to go. Indeed, that's the patch I suggested. I think it's fundamentally right, but it doesn't work (yet) because it bumps into the optimization bug introduced by making `eq` depend on `symbols-with-pos-enabled` :-( >> > Why must the F get replaced by a different F? There must surely be a >> > way, a simpler way than the current cl--labels-convert, to retain the >> > current F (hence, not corrupting its position)? >> There might. The current hack is the best I could come up with. > I'm not criticising the hack, not at all! But it could be better > commented, and the doc string for cl--labels-convert could be more > informative. It would help if you could send a (rough) patch showing what comment you'd have liked to see. > The "why" is missing - why is necessary to handle `function' as > a macro? I couldn't really think of any alternative for it ("it" being to implement `cl-labels` and `cl-flet`). FWIW, the pre-cl-lib code did it differently by duplicating `macroexp--expand-all` wholesale and then tweaking its handling of `function` in an ad-hoc way. BTW, there's a standard solution from Common Lisp to get rid of this hack: implement the `&whole` macro argument. [Macro Lambda Lists](http://clhs.lisp.se/Body/03_dd.htm) > I think it's to inhibit the processing of `function' as function > somewhere else, but where and why? It's not a function but a special operator, which is thus handled in a hard-coded way by `macroexp--expand-all`. >> I'm not completely sure we agree yet on what is "the original bug", but >> obviously I agree with your sentence :-) > I meant bug #65017. I committed a fix for it yesterday using the patch > I posted here on Sunday, and closed the bug. FWIW my notion of "the original bug" should be fixed by the patch below (modulo the above-mentioned optimization bug which makes the patch ineffective). BTW, I pushed an extended version of that patch which additionally does what you suggested we should do (and which I claimed wouldn't work), i.e. flush the cache (I originally couldn't see where to flush it). I believe this does fix the original problem even in the remaining corner cases, despite the optimization bug. Stefan diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el index 0a3181561bd..a405ae67691 100644 --- a/lisp/emacs-lisp/cl-macs.el +++ b/lisp/emacs-lisp/cl-macs.el @@ -2037,7 +2037,9 @@ cl--labels-convert ;; *after* handling `function', but we want to stop macroexpansion from ;; being applied infinitely, so we use a cache to return the exact `form' ;; being expanded even though we don't receive it. - ((eq f (car cl--labels-convert-cache)) (cdr cl--labels-convert-cache)) + ((let ((symbols-with-pos-enabled nil)) + (eq f (car cl--labels-convert-cache))) + (cdr cl--labels-convert-cache)) (t (let* ((found (assq f macroexpand-all-environment)) (replacement (and found From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 12 05:59:50 2023 Received: (at 65017) by debbugs.gnu.org; 12 Aug 2023 09:59:50 +0000 Received: from localhost ([127.0.0.1]:48744 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUlPR-0002i8-NW for submit@debbugs.gnu.org; Sat, 12 Aug 2023 05:59:50 -0400 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]:55347) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUlPN-0002hc-FY for 65017@debbugs.gnu.org; Sat, 12 Aug 2023 05:59:48 -0400 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-4fe61ae020bso4250851e87.2 for <65017@debbugs.gnu.org>; Sat, 12 Aug 2023 02:59:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691834379; x=1692439179; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=1wJyM21Sl6Sy2PcRN520KFJ4PIZWp9l3z7oLcmzBIgE=; b=pzyidWshtZmLVEbUtvCAOYFeZWZTMyhXoq7aeHj+nGCJNFCGOyc47C9ckLzWH2EMYl DPbF3sxHlseeqQ8D2lYQUmMG1GXCDxfxT4wkXz52bKrSkZDIDxUuFVtMLQ/N5fXn0rJ5 ovBd/99dBaUYFBqkC4xoNXOHjjX7OByqQ/Ebi90+W42btbagYHqcOGGjpADD+0U5ZTVO 5+TiweLJkftTovndRjGpvLXg1rqG3sBS/aQZOcVc+3RY6nwF/eAXkEEzc4oMrX0UXY7Y hbFTBdrKXjDVXqHi/qf3oLHM0N0P9M3izVRgCjkSuPZ9r0dc+gYtQfduwa1gAltylvz0 N08w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691834379; x=1692439179; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=1wJyM21Sl6Sy2PcRN520KFJ4PIZWp9l3z7oLcmzBIgE=; b=RXk3nz7cziQRv+s7+m7ZOQWmPCKzR5ecHiAR3LVLatP5DWG/u+p7VSBo9Cdq8XQi5M RTWSEzDin8iiNK+c4qDjGm2BuAMYdfdQJSAG8wkCqjjsaBQgGODTvH2VsRlajkOOI+kK uGgUB3jYf6mx/uDwDk/Y9RiMCNpviAFzAvNBJa69yJYitLpIrOdIzefpKldkPxPnWXVT +ggR5ocXte3CF+6YpTD4Lt3Bszux7Xc8OCuWjcJ+Wcbe9ODb6eB9DqyFce2g/Qbsot3F UzYDUL68bye+XjKokip/5GQ1BH5xVqq36pBFX2OFpCb/7Dg7z9CsU1356gwfP47f+11c MgEQ== X-Gm-Message-State: AOJu0YzbT3he1cutvy1jG8wELASftbOLwHMngSA7t+/l/DM9tLDwERc/ MZyMKKxvKbbgRW+JHfUkQDk= X-Google-Smtp-Source: AGHT+IEefUCLWGaZyQq3J40Fnfm9+V2F8rWR0xf9zFdVNKsV0+E4E4Dc+vik1sAj6JE0L2j2Hh9cew== X-Received: by 2002:a05:6512:2202:b0:4fe:4e2c:8e52 with SMTP id h2-20020a056512220200b004fe4e2c8e52mr3735830lfu.42.1691834378383; Sat, 12 Aug 2023 02:59:38 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id q5-20020ac24a65000000b004fbae25fcc4sm1085839lfp.61.2023.08.12.02.59.36 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 12 Aug 2023 02:59:36 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Sat, 12 Aug 2023 11:59:35 +0200 Content-Transfer-Encoding: 7bit Message-Id: <95AC3DDE-BB5F-4FA3-A7F0-66C1B441D0CA@gmail.com> References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> To: Stefan Monnier X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 12 aug. 2023 kl. 05.28 skrev Stefan Monnier : > Indeed, that's the patch I suggested. I think it's fundamentally right, > but it doesn't work (yet) because it bumps into the optimization bug > introduced by making `eq` depend on `symbols-with-pos-enabled` :-( The one disabled in 44d7fd3805? From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 12 06:40:16 2023 Received: (at 65017) by debbugs.gnu.org; 12 Aug 2023 10:40:16 +0000 Received: from localhost ([127.0.0.1]:48791 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUm2Z-0005Ia-QM for submit@debbugs.gnu.org; Sat, 12 Aug 2023 06:40:16 -0400 Received: from mail-lj1-x22a.google.com ([2a00:1450:4864:20::22a]:49230) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUm2V-0005IJ-Li for 65017@debbugs.gnu.org; Sat, 12 Aug 2023 06:40:14 -0400 Received: by mail-lj1-x22a.google.com with SMTP id 38308e7fff4ca-2b9cdba1228so44055191fa.2 for <65017@debbugs.gnu.org>; Sat, 12 Aug 2023 03:40:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691836805; x=1692441605; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:sender:from:to:cc:subject:date:message-id:reply-to; bh=JuJgGXjfktVEf20BEZ+KhRKrJ/1aN2YQ9qQblCGk8yQ=; b=nkCzWjyj3K7NzBLIOrQ+q4jlLAChLNWiRLmE+jhyvFJs42ELYtUKbEOcsv5XNzJnli XtOWy79yHQCeasGdLIZqTvFFykdluObvxTJjBJ9B6u7lhyegcTETa4y7EcQ+xwmtXJkQ +pjHz0Xg1Vcfjr5SSMJiqVkUNiCeBQA3FJqOR9LVTRQ/SVjRUjO0UYzwCkUpqvFfdNBN zjUaK8Mj/U07uJw62n/a9jH4AQ9Qxb2Q4FIgWdnx8BJsyhmoT6mbZ9GL2Rdw9ZvBsv7d GfFPAADRDJ4ltXZ0hArX6sSrhLcfHBxMX0I8ButpDud7wHcZb1o3ZQXD82Mb8y0rlIfx GKtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691836805; x=1692441605; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=JuJgGXjfktVEf20BEZ+KhRKrJ/1aN2YQ9qQblCGk8yQ=; b=K0MPYZFOY8WueOvwpAGExZtjsMHI+UFNtOj9jXFqlCUFcV1VGpNTArHR4b7EhZt1m/ PDrO5UECTtpUDMN7ULemvf3uMqJhZbT0iPWJDSnZgXGdwpmSVIc7WpCSEf0ZkYsRCmf3 Ass/SLRcl4SBy6T8h47ahFBBtoKhV8E16d6eGke700yDw6Ksh+7I7uaHcUvHjGVFLksj qWPUXIzSM7rb5clOfVooI/iT22aausAlBY361bHwUVkzLR+LLEYyhvviccDdQAMGUUW+ 3tQuoi+wT+gX/8BdFNKghwEGCLFh/LbwYjFGcKfBYaQemBKw7EhvmKdDhYbKNtePbm7T rscg== X-Gm-Message-State: AOJu0Yy2lbq0v6yNwsn0AGdIgENKsd/QbZWgJR07qHj0uje1uQphrAcF vBzhgORPsCZ/DA/TdH7Bp6w= X-Google-Smtp-Source: AGHT+IE7o3UI0Cm6bkg3cM3SxIkJ50x/y/mxUczp7uDvjFi3F3aEr0+9fbLsq4aP7kztGxAP6RhDnA== X-Received: by 2002:a05:6512:3193:b0:4ff:95c:e158 with SMTP id i19-20020a056512319300b004ff095ce158mr147401lfe.64.1691836805057; Sat, 12 Aug 2023 03:40:05 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id u7-20020ac243c7000000b004fddb0eb961sm1089445lfl.18.2023.08.12.03.40.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 12 Aug 2023 03:40:04 -0700 (PDT) From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= Message-Id: Content-Type: multipart/mixed; boundary="Apple-Mail=_CDCDA28F-2083-4B0B-811B-19539BE888AC" Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Date: Sat, 12 Aug 2023 12:40:03 +0200 In-Reply-To: To: Stefan Monnier References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --Apple-Mail=_CDCDA28F-2083-4B0B-811B-19539BE888AC Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii 12 aug. 2023 kl. 05.28 skrev Stefan Monnier : > there's a standard solution from Common Lisp to get rid of this > hack: implement the `&whole` macro argument. If we can live without the special `&whole` syntax then it seems rather = straightforward (patch), although I'd rather not add the extra specbind = until needed for a concrete purpose. One such purpose, apart from removing the hack, would be better = diagnostics in some cases. --Apple-Mail=_CDCDA28F-2083-4B0B-811B-19539BE888AC Content-Disposition: attachment; filename=macroexp-whole.diff Content-Type: application/octet-stream; x-unix-mode=0644; name="macroexp-whole.diff" Content-Transfer-Encoding: 7bit diff --git a/lisp/emacs-lisp/macroexp.el b/lisp/emacs-lisp/macroexp.el index 6d5cf8723f9..18eaf005cb4 100644 --- a/lisp/emacs-lisp/macroexp.el +++ b/lisp/emacs-lisp/macroexp.el @@ -199,6 +199,15 @@ macroexp--obsolete-warning (instead (format-message "; use `%s' instead." instead)) (t "."))))) +(defvar macroexp--whole-form + ;; The current form being macroexpanded. + ) + +(defun macroexp-whole-form () + "The current form being macro-expanded. +Can only be called during macro-expansion (ie, inside a macro)." + macroexp--whole-form) + (defun macroexpand-1 (form &optional environment) "Perform (at most) one step of macroexpansion." (cond @@ -219,7 +228,8 @@ macroexpand-1 ((not (consp def)) form) (t (if (eq 'macro (car def)) - (apply (cdr def) (cdr form)) + (let ((macroexp--whole-form form)) + (apply (cdr def) (cdr form))) form)))))))) (t form))) diff --git a/src/eval.c b/src/eval.c index 9e54d489a3b..0d32b63bf1d 100644 --- a/src/eval.c +++ b/src/eval.c @@ -1161,7 +1161,9 @@ DEFUN ("macroexpand", Fmacroexpand, Smacroexpand, 1, 2, 0, break; } { - Lisp_Object newform = apply1 (expander, XCDR (form)); + specpdl_ref pdl = SPECPDL_INDEX (); + specbind (Qmacroexp__whole_form, form); + Lisp_Object newform = unbind_to (pdl, apply1 (expander, XCDR (form))); if (EQ (form, newform)) break; else @@ -4462,4 +4464,5 @@ syms_of_eval (void) defsubr (&Sspecial_variable_p); DEFSYM (Qfunctionp, "functionp"); defsubr (&Sfunctionp); + DEFSYM (Qmacroexp__whole_form, "macroexp--whole-form"); } --Apple-Mail=_CDCDA28F-2083-4B0B-811B-19539BE888AC-- From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 12 12:46:56 2023 Received: (at 65017) by debbugs.gnu.org; 12 Aug 2023 16:46:56 +0000 Received: from localhost ([127.0.0.1]:56742 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUrlQ-0001kD-6L for submit@debbugs.gnu.org; Sat, 12 Aug 2023 12:46:56 -0400 Received: from mx3.muc.de ([193.149.48.5]:57604) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUrlK-0001jv-IS for 65017@debbugs.gnu.org; Sat, 12 Aug 2023 12:46:54 -0400 Received: (qmail 74004 invoked by uid 3782); 12 Aug 2023 18:46:43 +0200 Received: from acm.muc.de (p4fe1539a.dip0.t-ipconnect.de [79.225.83.154]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 12 Aug 2023 18:46:43 +0200 Received: (qmail 1784 invoked by uid 1000); 12 Aug 2023 16:46:42 -0000 Date: Sat, 12 Aug 2023 16:46:42 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Fri, Aug 11, 2023 at 23:28:01 -0400, Stefan Monnier wrote: [ .... ] > >> > Why must the F get replaced by a different F? There must surely be a > >> > way, a simpler way than the current cl--labels-convert, to retain the > >> > current F (hence, not corrupting its position)? > >> There might. The current hack is the best I could come up with. > > I'm not criticising the hack, not at all! But it could be better > > commented, and the doc string for cl--labels-convert could be more > > informative. > It would help if you could send a (rough) patch showing what comment > you'd have liked to see. I'll come up with something as soon as I understand it enough. > > The "why" is missing - why is necessary to handle `function' as > > a macro? > I couldn't really think of any alternative for IT ("it" being to > implement `cl-labels` and `cl-flet`). FWIW, the pre-cl-lib code did IT > differently by duplicating `macroexp--expand-all` wholesale and then > tweaking its handling of `function` in an ad-hoc way. Your reply doesn't address my question. It is not clear to me what the IT in your previous paragraph is. You may or may not have thought of some alternative for IT, and previous code may have done IT differently, but that doesn't help me understand what IT is. What was the difficulty in cl-labels and cl-flet for which IT and cl--labels-convert was the solution? The code is substituting (function F) with a non-eq (function F). You're saying this has some effect in macroexp--expand-all. I can't see that, yet. All I see is FORM, (function F), being substituted by a different (function F) in L327 of macroexp.el. Then there are the pcase arms for (function (lambda ....)) and for (function ....). Are either of these pcase arms affected by the "expansion" of FORM? If so, how? Or am I looking at the wrong place entirely? [ .... ] > > I think it's to inhibit the processing of `function' as function > > somewhere else, but where and why? > It's not a function but a special operator, which is thus handled in > a hard-coded way by `macroexp--expand-all`. Is it the case that this hard-coded handling for function is prevented by the macro "expansion" of (function F)? Otherwise, what is the relevance of this hard-coded handling to cl--labels-convert? [ .... ] > Stefan [ .... ] -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 12 14:21:29 2023 Received: (at 65017) by debbugs.gnu.org; 12 Aug 2023 18:21:29 +0000 Received: from localhost ([127.0.0.1]:57320 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUtEv-0004ZI-2z for submit@debbugs.gnu.org; Sat, 12 Aug 2023 14:21:29 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:30731) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUtEr-0004Yy-Kn for 65017@debbugs.gnu.org; Sat, 12 Aug 2023 14:21:27 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 3478444107B; Sat, 12 Aug 2023 14:21:20 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691864474; bh=vTsuiYa/9yRNqjb59GUWmSxZ6evbQLM+bMqN23khZl4=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=dCU2E1m3Uxr0xD1dBL18AZm1DgvsoCwiwOsZKxLIaR97BozdYvn1hf30phzDR9moa IVwwvVj9GtqsJ11THoa4eC55as3rgWAiRN7HGnTp1areJhc9c+ts2lFAOrR9at6HeU DWRLzsc4tIpk/lEUjVTeLGiAJj5WqL0I3sipiuWl7gd9Mc6jfr0L14a+ai5m7LlJiK Hs4ParMsEYNVY8dDvPm2sN3bkVC6bwI80LxKpUz7AzofcLuLDXPh/KG7gTnAqjL1+G LFmXXPs8M717k50E/3OBcr+/QnH5bov0QXFJ7ryX35t2H991WQNf6aAccL+nwa2pQa 3NvuaRHqhMqLQ== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id AB30D4410F3; Sat, 12 Aug 2023 14:21:14 -0400 (EDT) Received: from pastel (69-165-141-248.dsl.teksavvy.com [69.165.141.248]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 776C7120388; Sat, 12 Aug 2023 14:21:14 -0400 (EDT) From: Stefan Monnier To: Mattias =?windows-1252?Q?Engdeg=E5rd?= Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: <95AC3DDE-BB5F-4FA3-A7F0-66C1B441D0CA@gmail.com> ("Mattias =?windows-1252?Q?Engdeg=E5rd=22's?= message of "Sat, 12 Aug 2023 11:59:35 +0200") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> <95AC3DDE-BB5F-4FA3-A7F0-66C1B441D0CA@gmail.com> Date: Sat, 12 Aug 2023 14:21:13 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.149 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Alan Mackenzie , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> Indeed, that's the patch I suggested. I think it's fundamentally right, >> but it doesn't work (yet) because it bumps into the optimization bug >> introduced by making `eq` depend on `symbols-with-pos-enabled` :-( > The one disabled in 44d7fd3805? Ah, sorry, I didn't notice it, thanks for the heads up. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 12 14:28:54 2023 Received: (at 65017) by debbugs.gnu.org; 12 Aug 2023 18:28:54 +0000 Received: from localhost ([127.0.0.1]:57421 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUtM5-0004o9-Tk for submit@debbugs.gnu.org; Sat, 12 Aug 2023 14:28:54 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:41455) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qUtM4-0004nt-FU for 65017@debbugs.gnu.org; Sat, 12 Aug 2023 14:28:53 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id B90D11000EF; Sat, 12 Aug 2023 14:28:46 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691864925; bh=Wzlbu/+9kdyJ1D+wMcgOW7iaF38j9CCYY4db0FQSSHE=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=enPKWvX6K7+x4zE1sJLn6WJO9NzAToh+X2E8C0FES6aZSdLa+iDRCW3INMh8bLExu QRRxZBgGv+dWgePvWgvWQBxBZ56Tlt4QWwPZdDPD9PEBO81/DMbJ5nhzFJ4TzwzCc7 KPW7/XttzEX2RYCjJ/sKzHLiclO5uTZoOb3yz7Zg+L4IT1YkpiA/OayNY0HC4fuZNi 8vMERsW5Wyqyvn8TpbJ2AE17Dylje59BtnPdenFJcW0W82b9Pjf9Q737yLSCVkXyVp tum7eXr90OnMX7MtmxJzT6Sb/3z8wb4o8MO8SCtqMot8Et+KUPQo36/gvYpXUwukYq XmBR2voztWMRA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id AA39E1000AD; Sat, 12 Aug 2023 14:28:45 -0400 (EDT) Received: from pastel (69-165-141-248.dsl.teksavvy.com [69.165.141.248]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 7CB2D120281; Sat, 12 Aug 2023 14:28:45 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Sat, 12 Aug 2023 16:46:42 +0000") Message-ID: References: <8B08E514-B2D5-48F5-BA90-4F5A9492F8F9@gmail.com> Date: Sat, 12 Aug 2023 14:28:44 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.132 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > The code is substituting (function F) with a non-eq (function F). > You're saying this has some effect in macroexp--expand-all. I can't see > that, yet. All I see is FORM, (function F), being substituted by a > different (function F) in L327 of macroexp.el. Then there are the pcase > arms for (function (lambda ....)) and for (function ....). Are either > of these pcase arms affected by the "expansion" of FORM? If so, how? > Or am I looking at the wrong place entirely? `cl-flet` needs to replace (function LOCALFUN) with LOCALVAR within the body of the let, for those LOCALFUNs defined in the `cl-flet`. That's easy to do with a macro. But it also should leave all other uses of `function` untouched. That's the part that does not map well to macros since macros are repeatedly expanded until they return something that's not a macro call. >> It's not a function but a special operator, which is thus handled in >> a hard-coded way by `macroexp--expand-all`. > Is it the case that this hard-coded handling for function is prevented > by the macro "expansion" of (function F)? Yes, we first expand the macros and then try to handle the result which should be one of the hard-coded cases (or is otherwise assumed to be a function call). Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 13 06:10:40 2023 Received: (at 65017) by debbugs.gnu.org; 13 Aug 2023 10:10:40 +0000 Received: from localhost ([127.0.0.1]:58294 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qV83T-0006QC-Si for submit@debbugs.gnu.org; Sun, 13 Aug 2023 06:10:40 -0400 Received: from mx3.muc.de ([193.149.48.5]:30935) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qV83R-0006Pw-Qx for 65017@debbugs.gnu.org; Sun, 13 Aug 2023 06:10:39 -0400 Received: (qmail 90089 invoked by uid 3782); 13 Aug 2023 12:10:31 +0200 Received: from acm.muc.de (p4fe152a2.dip0.t-ipconnect.de [79.225.82.162]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 13 Aug 2023 12:10:31 +0200 Received: (qmail 17114 invoked by uid 1000); 13 Aug 2023 10:10:30 -0000 Date: Sun, 13 Aug 2023 10:10:30 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Sat, Aug 12, 2023 at 14:28:44 -0400, Stefan Monnier wrote: > >> I couldn't really think of any alternative for IT ("it" being to > >> implement `cl-labels` and `cl-flet`). FWIW, the pre-cl-lib code > >> did IT differently by duplicating `macroexp--expand-all` wholesale > >> and then tweaking its handling of `function` in an ad-hoc way. > > Your reply doesn't address my question. It is not clear to me what > > the IT in your previous paragraph is. You may or may not have > > thought of some alternative for IT, and previous code may have done > > IT differently, but that doesn't help me understand what IT is. > > What was the difficulty in cl-labels and cl-flet for which IT and > > cl--labels-convert was the solution? > > The code is substituting (function F) with a non-eq (function F). > > You're saying this has some effect in macroexp--expand-all. I can't see > > that, yet. All I see is FORM, (function F), being substituted by a > > different (function F) in L327 of macroexp.el. Then there are the pcase > > arms for (function (lambda ....)) and for (function ....). Are either > > of these pcase arms affected by the "expansion" of FORM? If so, how? > > Or am I looking at the wrong place entirely? > `cl-flet` needs to replace (function LOCALFUN) with LOCALVAR within the > body of the let, for those LOCALFUNs defined in the `cl-flet`. > That's easy to do with a macro. > But it also should leave all other uses of `function` untouched. > That's the part that does not map well to macros since macros are > repeatedly expanded until they return something that's not a macro call. Thanks, that's useful information. But it doesn't address my questions in the slightest. This is the third time I'm asking you for help. I thought it would be quicker than figuring everything out on my own. You wrote the code, I think. I don't understand how cl--labels-convert works, down at the car and cdr level. I'm asking you to help me, but I'm not sure how sensible it is to carry on asking repeatedly. I've replaced two paragraphs you snipped from your last reply. Would you please answer these specific questions, now, to help me understand this difficult mechanism. Thanks! > >> It's not a function but a special operator, which is thus handled in > >> a hard-coded way by `macroexp--expand-all`. > > Is it the case that this hard-coded handling for function is prevented > > by the macro "expansion" of (function F)? > Yes, we first expand the macros and then try to handle the result > which should be one of the hard-coded cases (or is otherwise assumed to > be a function call). Are you talking about the code in macroexp--expand-all, here? By "macros", do you mean cl-flet and cl-labels here (as opposed to function)? What do you mean by "hard-coded cases"? Do you mean the pcase arms in macroexp--expand-all? If so, which ones? > Stefan -- Alan Mackenzie (Nuremberg, Germany). From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 13 12:12:16 2023 Received: (at 65017) by debbugs.gnu.org; 13 Aug 2023 16:12:16 +0000 Received: from localhost ([127.0.0.1]:60129 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qVDhN-0005hD-9C for submit@debbugs.gnu.org; Sun, 13 Aug 2023 12:12:16 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:19786) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qVDhK-0005gu-Vh for 65017@debbugs.gnu.org; Sun, 13 Aug 2023 12:12:12 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id E3C38441D03; Sun, 13 Aug 2023 12:12:04 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1691943123; bh=d+vPrFLNJGNbzqds59YkCXWQYNAVJjygIyx7H6daRIc=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=UTcZbqB/HynTvT7jyET7gX8LyKMDslEYUo1KUzpeYLGOFMpoFOkexbL/nWFhVbec5 h1fKT418hben0i6UzxWTtkuD4LQW95kQEgqk/Q3VrSsWXjcUFe2p1cA7DQl+58i5gl 9Okpe6wV+PDW7yGxfBv1XrO0yZf+l7mpZb2opADBLpwo4U+DoDUlPHr6J2ggE9DGBI +/2EWNiFqGlJw0rUr7UAr8nNQD9ElLOlmpCb6HqXDim5RCc2glaOnMRNUD2yz5oPuT wY9LOqUhK54uQz884hUJJxKwB4AFMi328CSM4e5jexHRDsyeF5nyeVBJumgJHygrWy FHBReo/RPzhGA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 6EEB2441CFE; Sun, 13 Aug 2023 12:12:03 -0400 (EDT) Received: from pastel (unknown [45.72.228.154]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3C33612020D; Sun, 13 Aug 2023 12:12:03 -0400 (EDT) From: Stefan Monnier To: Alan Mackenzie Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function In-Reply-To: (Alan Mackenzie's message of "Sun, 13 Aug 2023 10:10:30 +0000") Message-ID: References: Date: Sun, 13 Aug 2023 12:12:02 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.039 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Thanks, that's useful information. But it doesn't address my questions > in the slightest. I'm sorry. I guess I still haven't figured what it is that I assume as known but which you don't actually know. > Would you please answer these specific questions, now, to help me > understand this difficult mechanism. Thanks! [ I assume you're talking about the questions below. ] >> >> It's not a function but a special operator, which is thus handled in >> >> a hard-coded way by `macroexp--expand-all`. >> > Is it the case that this hard-coded handling for function is prevented >> > by the macro "expansion" of (function F)? >> Yes, we first expand the macros and then try to handle the result >> which should be one of the hard-coded cases (or is otherwise assumed to >> be a function call). > Are you talking about the code in macroexp--expand-all, here? Yes. > By "macros", do you mean cl-flet and cl-labels here (as opposed to > function)? I'm talking about any call to an identifier that is "currently" defined as a macro. This can be either because the `symbol-function` holds something of the form `(macro . )` or because `macroexpand-all-environment` has an entry for that identifier. > What do you mean by "hard-coded cases"? The face that `macroexp--expand-all` handles the `function` identifier as follows: (pcase form [...] (`(,(or 'function 'quote) . ,_) form) [...] -- Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 14 13:10:38 2023 Received: (at 65017) by debbugs.gnu.org; 14 Aug 2023 17:10:39 +0000 Received: from localhost ([127.0.0.1]:34339 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qVb5N-0005U5-AK for submit@debbugs.gnu.org; Mon, 14 Aug 2023 13:10:38 -0400 Received: from mx3.muc.de ([193.149.48.5]:29272) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qVb5G-0005Ti-8L for 65017@debbugs.gnu.org; Mon, 14 Aug 2023 13:10:32 -0400 Received: (qmail 72490 invoked by uid 3782); 14 Aug 2023 19:10:19 +0200 Received: from acm.muc.de (pd953a69e.dip0.t-ipconnect.de [217.83.166.158]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Mon, 14 Aug 2023 19:10:19 +0200 Received: (qmail 29030 invoked by uid 1000); 14 Aug 2023 17:10:17 -0000 Date: Mon, 14 Aug 2023 17:10:17 +0000 To: Stefan Monnier Subject: Re: bug#65017: 29.1; Byte compiler interaction with cl-lib function objects, removes symbol-function Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65017 Cc: Mattias =?iso-8859-1?Q?Engdeg=E5rd?= , 65017@debbugs.gnu.org, Eric Marsden X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Sun, Aug 13, 2023 at 12:12:02 -0400, Stefan Monnier wrote: > > Thanks, that's useful information. But it doesn't address my questions > > in the slightest. > I'm sorry. I guess I still haven't figured what it is that I assume as > known but which you don't actually know. I didn't, and don't know the answers to the questions I asked. > > Would you please answer these specific questions, now, to help me > > understand this difficult mechanism. Thanks! > [ I assume you're talking about the questions below. ] I was actually talking about those strings of characters which I had terminated with the '?' character. All you've done with them is to snip them from your reply. I'm not going to ask you a fourth time. You clearly don't want to answer these questions, for some reason. Maybe I'll get around to working out for myself how this code works, maybe I won't. But if it's up to me to fix the broken commenting/doc strings associated with cl--labels-convert, it's not looking like it'll get done any time soon. Thnks for the answers that you did give me, below. > >> >> It's not a function but a special operator, which is thus handled in > >> >> a hard-coded way by `macroexp--expand-all`. > >> > Is it the case that this hard-coded handling for function is prevented > >> > by the macro "expansion" of (function F)? > >> Yes, we first expand the macros and then try to handle the result > >> which should be one of the hard-coded cases (or is otherwise assumed to > >> be a function call). > > Are you talking about the code in macroexp--expand-all, here? > Yes. > > By "macros", do you mean cl-flet and cl-labels here (as opposed to > > function)? > I'm talking about any call to an identifier that is "currently" defined > as a macro. This can be either because the `symbol-function` holds > something of the form `(macro . )` or because > `macroexpand-all-environment` has an entry for that identifier. > > What do you mean by "hard-coded cases"? > The face that `macroexp--expand-all` handles the `function` identifier > as follows: > (pcase form > [...] > (`(,(or 'function 'quote) . ,_) form) > [...] > -- Stefan -- Alan Mackenzie (Nuremberg, Germany). From unknown Fri Aug 15 04:04:25 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 12 Sep 2023 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator