From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 03 21:06:22 2020 Received: (at submit) by debbugs.gnu.org; 4 Sep 2020 01:06:22 +0000 Received: from localhost ([127.0.0.1]:36575 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kE0BI-0008PU-9Y for submit@debbugs.gnu.org; Thu, 03 Sep 2020 21:06:22 -0400 Received: from lists.gnu.org ([209.51.188.17]:36450) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kE0BE-0008PH-Cy for submit@debbugs.gnu.org; Thu, 03 Sep 2020 21:06:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54184) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kE0BE-00055U-7g for bug-gnu-emacs@gnu.org; Thu, 03 Sep 2020 21:06:16 -0400 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]:50379) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kE0BB-0002yP-Be for bug-gnu-emacs@gnu.org; Thu, 03 Sep 2020 21:06:15 -0400 Received: by mail-wm1-x32f.google.com with SMTP id e17so4567205wme.0 for ; Thu, 03 Sep 2020 18:06:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=ECll1aaavKUUl50un30zgtLlTssRolPHyxY8UOjZkZc=; b=RqjPTeXxZ7czDC12WrZ8naZcOP0yoD4hZU48XwmShpSpq+uF5BKZ+rCVD+p3QmeB8q ic81vIDvB395cgqZ3InepPtdJH2IkOUEwd17HMPUHVc0bqhUAnCZUojf0FVSS7RKzHJ4 Y67R5DUB/tuNKq/Wl/eVa/I6frQsXqVr0LV/LxsLS9R645H98uo8zOzLuIbqLjszWF+G 6ddLdVES/Di2lJUDNzTv5qOG98IIot95aOEBlW3GuDwT/cBfaJQhJNjsrCojKCdsm29u gHuoySeiiABaZzRJ0U+r8H5SjKZi3DVyxeq/7KmWde4ucHlkJ2BYVKWJtcnkad7tanDX Yubw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=ECll1aaavKUUl50un30zgtLlTssRolPHyxY8UOjZkZc=; b=NFUfcv7oPPAxGRqx+6YiMBIOXpMCnBtfH+hWi7qCxhL5HHq8abD8vhL+xbA96C1YHd KHuIXUTygfuqr70nhu2aQQYFpHuBYYGCC5XNLUMLqj5Hn0bkFp3m30uuUB/zVfhFdn7H KIxxvNJEGR0gJeSH3C7eczoi5mn8WyyMAquwkIDWAewWphmgAzh8GoH0R9qrgkrVv7RI QBbR8sJglN26NqpnTtNvcpXU1iPTEWCvnq/MEMhXjNivtKZqGtjhFZWUEEzRT5+2oVf/ vO112nut0xbzsFfgGf3vxMVIcb/Hspn4U9lmORLj+PAGEThEbAp0h7ORfmvLXzIZzFZR R4Lg== X-Gm-Message-State: AOAM530EPHqAdzWuqKvyhht5cvBn21wZ63sXG3xEkUo1tM7m5UNw9qAO 56p4nn+k1LIoScfBXOa40em/XIOItUHQ83veNh9Pxc+Hiet1iw== X-Google-Smtp-Source: ABdhPJyQiZbycWDfLpmkGBiT4vE9tspHdg9vA9f/t/qrTYfbX8ZCNRn+R4ekKdnPInhlRVMxSY5askKAyafSm8xCJn0= X-Received: by 2002:a1c:3d44:: with SMTP id k65mr4767279wma.132.1599181571023; Thu, 03 Sep 2020 18:06:11 -0700 (PDT) MIME-Version: 1.0 From: Tom Gillespie Date: Thu, 3 Sep 2020 18:06:00 -0700 Message-ID: Subject: feature/native-comp; failure to compile json-mode leaves Emacs broken To: Emacs Bug Report Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::32f; envelope-from=tgbugs@gmail.com; helo=mail-wm1-x32f.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: submit Cc: Andrea Corallo X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Hi Andrea, Found another one. I don't know what is going wrong on the compiler side that induces it, but hopefully the repro can get you pointed in the right direction. Also sent to the bug tracker so it should show up there shortly. I have left the section on issues with kill-buffer in the report so that anyone encountering the effect of the bug won't have to spend the same amount of time I did figuring out what is happening. Best! Tom This issue occurs at 3023eb569213a3dd5183640f6e322acd00ea536a. It does not happen in emacs-27 nor on master at ae6daa680a5f5f5fb9c6a15296e5e88c97cd770a. When trying to load json-mode for the first time it fails to compile due to some issue in json-snatcher, the error that is displayed is json-mode.el:33:1: Error: Symbol=E2=80=99s function definition is void: jsons-remove-buffer I suspect that something goes wrong inside json-snatcher.el and the end result is that =3D(add-hook 'kill-buffer-hook 'jsons-remove-buffer)=3D on line json-snatcher.el:85 manages to run, but then the defun for that function is never called. The fact that that function cannot be found means that anything that calls kill-buffer will be broken, which includes a terrifyingly large portion of Emacs' debugging functionality (see footnote 1). On a bad day this also breaks things like the ability to exit Emacs via =3DC-x C-c=3D. Fortunately in the repro below this state is not triggered so you can at least exit. Just in case I included the escape hatches that can be evaluated using =3DC-x C-e=3D in the scratch buffer most relevantly =3D(pop kill-buffer-hook)=3D. Here is a simple reproduction that is hopefully enough to track down what is going wrong with native compile. I have not been able to produce a simple visible repro using --batch, but I think the problem is clear enough. #+begin_src bash [[ -d "/tmp/test/.emacs.d/" ]] && rm -r /tmp/test/.emacs.d/; \ emacs -q \ --eval "(setq user-emacs-directory \"/tmp/test/.emacs.d/\")" \ --eval "(require 'package)" \ --eval "(add-to-list 'package-archives '(\"melpa\" . \"https://melpa.org/packages/\") t)" \ --eval "(package-refresh-contents)" \ --eval "(package-install 'json-mode)" #+end_src The repro for emacs-27 without the --eval options. #+begin_src bash [[ -d "/tmp/test/.emacs.d/" ]] && rm -r /tmp/test/.emacs.d/; \ emacs-27 -q \ #+end_src The repro for the master branch without the --eval options. #+begin_src bash ./autogen.sh ./configure make -j 8 [[ -d "/tmp/test/.emacs.d/" ]] && rm -r /tmp/test/.emacs.d/; \ src/emacs -q \ #+end_src A longer reproduction that controls for a number of variables and shows the larger issues with kill-buffer is below. Run this and hit enter in the ielm buffer and you will experience and odd hang. Hit enter again and then switch to the scratch buffer if you want to play around with the additional issues. #+begin_src bash [[ -d "/tmp/test/.emacs.d/" ]] && rm -r /tmp/test/.emacs.d/; \ emacs -q \ --eval "(setq user-emacs-directory \"/tmp/test/.emacs.d/\")" \ --eval "(when (boundp 'comp-eln-load-path) (pop comp-eln-load-path) (add-to-list 'comp-eln-load-path (concat user-emacs-directory \"eln-cache\")))" \ --eval "(require 'package)" \ --eval "(add-to-list 'package-archives '(\"melpa\" . \"https://melpa.org/packages/\") t)" \ --eval "(package-refresh-contents)" \ --eval "(ignore-errors (package-install 'json-mode))" \ --eval "(ielm)" \ --eval "(with-current-buffer \"*ielm*\" (insert \"(setq asdf (+ 1 2))\"))" = \ --eval "(with-current-buffer \"*scratch*\" (insert \"(pop kill-buffer-hook) ; calling this will revert the massive bre= akage asdf ; C-x C-e this to see more brokeness with native comp or working with= 27 (fmakunbound 'jsons-remove-buffer) ; use to repro the effects without native comp C-x-C-e-me-i-am-not-bound ; asdf will be bound if testing with 27 ;; run these blocks to be able to safely enter the debugger (setq debug-on-message \\\"break\\\") (defvar break-message \\\"break\\\") (defun jsons-remove-buffer () (let ((bm break-message)) (setq break-message \\\"\\\") (message bm)))\"))" #+end_src Backtraces. #+begin_example elisp Debugger entered--Lisp error: "break" message("break") (let ((bm break-message)) (setq break-message "") (message bm)) jsons-remove-buffer() kill-buffer(#) ielm-eval-input(#("(+ 1 2)" 0 7 (fontified t)) nil) ielm-send-input(nil) ielm-return() funcall-interactively(ielm-return) command-execute(ielm-return) #+end_example #+begin_example elisp Debugger entered--Lisp error: "break" message("break") (let ((bm break-message)) (setq break-message "") (message bm)) jsons-remove-buffer() kill-buffer(#) #f(compiled-function () #)() cl-print-to-string-with-limit(backtrace--print (void-variable asdf) 5000) backtrace--print-to-string((void-variable asdf) nil) backtrace-print-to-string((void-variable asdf)) debugger--insert-header((error (void-variable asdf))) #f(compiled-function () #)() backtrace-print() debugger-setup-buffer((error (void-variable asdf))) debug(error (void-variable asdf)) (progn asdf) elisp--eval-last-sexp(nil) eval-last-sexp(nil) funcall-interactively(eval-last-sexp nil) command-execute(eval-last-sexp) #+end_example Thoughts on the presence of a larger issue. While this is clearly a bug in the way that json-snatcher is written (the symbol should never have been added to the hook before the function was defined), it is only causing an issue with the new native comp code. Furthermore this example reveals a number of systematic issues with when and where kill-buffer is being called inside core Emacs code, and with how hook failures on kill-buffer are being handled. This was an absolute nightmare to debug and I think I will submit a separate issue about the utterly undebuggable nature of issues with kill-buffer-hook. Thoughts on potential solutions while they are fresh in my mind. It might be possible to add a check in add-hook that could be invoked for hooks that want to require any symbol added to the hook list to already be bound at the time that it is added to the hook list. This would prevent the issue that we see with json-snatcher in the future. However this does not prevent issues if someone was doing something evil like using push to add to kill-buffer-hook. The error message that is currently emitted for these kinds of failures is almost good enough, I think that it just needs a note that the symbol was originally found the kill-buffer-hook list and maybe provide instructions on how to remove the symbol from the hook. For example "kill-buffer: Symbol=E2=80=99s function definition is void: jsons-remove-bu= ffer Symbol jsons-remove-buffer in kill-buffer-hook list is void you can run (remove-hook 'kill-buffer-hook 'jsons-remove-buffer) to restore kill-buffer functionality" Footnotes. 1. For example toggle-debug-on-error completely fails to function correctly in this case, as do things like ielm-return. Anything which uses with-temp-buffer and tries to print, such as cl-print-to-string-with-limit will have its output silenced or ignored due to the failure of kill-buffer in the unwind forms. Normal approaches to debugging nearly all wind up stuck in infinite loops since most of them make a call to kill-buffer somewhere. From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 04 03:56:32 2020 Received: (at submit) by debbugs.gnu.org; 4 Sep 2020 07:56:32 +0000 Received: from localhost ([127.0.0.1]:37163 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kE6aG-0006yI-Ju for submit@debbugs.gnu.org; Fri, 04 Sep 2020 03:56:32 -0400 Received: from lists.gnu.org ([209.51.188.17]:52490) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kE6aE-0006yB-Py for submit@debbugs.gnu.org; Fri, 04 Sep 2020 03:56:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40726) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kE6aE-0007AQ-Hl for bug-gnu-emacs@gnu.org; Fri, 04 Sep 2020 03:56:30 -0400 Received: from mx.sdf.org ([205.166.94.24]:64062) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kE6aC-0000SF-GX for bug-gnu-emacs@gnu.org; Fri, 04 Sep 2020 03:56:30 -0400 Received: from mab (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTP id 0847uLFc009204; Fri, 4 Sep 2020 07:56:21 GMT From: Andrea Corallo To: Tom Gillespie Subject: Re: feature/native-comp; failure to compile json-mode leaves Emacs broken References: Date: Fri, 04 Sep 2020 07:56:21 +0000 In-Reply-To: (Tom Gillespie's message of "Thu, 3 Sep 2020 18:06:00 -0700") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=205.166.94.24; envelope-from=akrl@sdf.org; helo=mx.sdf.org X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/04 03:56:21 X-ACL-Warn: Detected OS = ??? X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: Emacs Bug Report X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Hi Tom, thanks for the detailed report. Tom Gillespie writes: > While this is clearly a bug in the way that json-snatcher is written > (the symbol should never have been added to the hook before the > function was defined), it is only causing an issue with the new native > comp code. had no time to reproduce but to a very first look I'm quite unconvinced this is a native-comp bug. I think you'd get the same exact error byte-compiling the file from shell using a vanilla Emacs. The fact is that native async compilations are child processes so they start with a clean environment. This makes the native compiler a little more picky on bugs when the compilation is relaying silently on some definition sneaked in from the environment. This lead already to a number of packages to fix their requires. The proof is typically trying to byte-compile from shell, this give systematically the same error, this because the front-end of the native compiler *is* the byte-compiler :) Each compilation unit should be consistent and compilable autonomously without relaying on previous modifications of the environment. Thanks Andrea From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 17 13:49:45 2020 Received: (at submit) by debbugs.gnu.org; 17 Sep 2020 17:49:45 +0000 Received: from localhost ([127.0.0.1]:38945 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kIy2S-0001g6-Pu for submit@debbugs.gnu.org; Thu, 17 Sep 2020 13:49:45 -0400 Received: from lists.gnu.org ([209.51.188.17]:39990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kIy2R-0001fw-2Y for submit@debbugs.gnu.org; Thu, 17 Sep 2020 13:49:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45964) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kIy2Q-0004sX-Pt for bug-gnu-emacs@gnu.org; Thu, 17 Sep 2020 13:49:42 -0400 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:36208) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kIy2O-0003ue-Qj for bug-gnu-emacs@gnu.org; Thu, 17 Sep 2020 13:49:42 -0400 Received: by mail-wr1-x443.google.com with SMTP id z1so3005298wrt.3 for ; Thu, 17 Sep 2020 10:49:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=19Yz+trWa8pKSE5GDGoQht8tnxtPMuQ2BGt75NxW6eA=; b=sJI9uxgpfEz+kty010fQasutK34tEux3MMaIU2J8ftV10d7JpVqH9OTD1CM3/K+BKy gx3zE9ZM2uU/3bETv81aDQGCF5p/avLlU4rawfz3MlTAQk4qHAFh7ZPcKRE6rwf7X/wI bJ77P9sDNpxorYSLWBBhmDXLS2/7Abs/UgCjcqpUKqX95alLF5clt8AE7Uv3ge0FfWLg IAr7m7ugGIuyahUFHEXmng2NjH1Iep8Yrvj1Bk4OSGjtKdr1POvxWUZycEqlRhpc/zDj pOxnArSflQRdxJFGzvy3Nvi0JXm9Wg3v332ekN+usvlDHRCi4KDygzyW82cSbbpxK8zy 0kMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=19Yz+trWa8pKSE5GDGoQht8tnxtPMuQ2BGt75NxW6eA=; b=CArLZ02+JkIpbNM+wv+pbAwH0smgDwmc7MBssu2oRZtY1lF12bqIEaJmSnhR5rcqSu La9Z3/evE94RzO0MDAQqLdMoZT3P7E8lag2cL5Qo4KuARwgN9TE+LGwATJ/+uOFp/Ixt v91BPkrl7uzf7wkZc+TEZLusfyr2XTMRGP0c/npVmh/tgwnsX14vopYgnbxtMujHBLgZ QJ8mT2oxk/rN0LrlTuQ76mwFsq2HVt9XoBONpoYYmnsiigMPnupEjc3c+XC//q8fVpHv dF3Fj944nC8/ujSGlchBjqewFBfqqbqJwKG1kvcxmOfSeVxYDGKEyx+aGzEQvnZ1UWac jrOg== X-Gm-Message-State: AOAM533UKs4DCfxEg6Acuco4FLvz+8TEbFp5n+hPhn1te7QNFHMq9NTA 1mF8hYzR6f1FjMxIZcmM/w/9lXBCyo1z01rUVsk= X-Google-Smtp-Source: ABdhPJyFCGmQFNApvFqjjB4r2ASMw94mOKFGjBevTn8rqB8+rSiQ7aQS2rNvp5aWyOslQF3vmBwS45R8QbbI7AA8k24= X-Received: by 2002:adf:e44b:: with SMTP id t11mr10925240wrm.101.1600364979173; Thu, 17 Sep 2020 10:49:39 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Tom Gillespie Date: Thu, 17 Sep 2020 13:49:28 -0400 Message-ID: Subject: Re: feature/native-comp; failure to compile json-mode leaves Emacs broken To: Andrea Corallo Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2a00:1450:4864:20::443; envelope-from=tgbugs@gmail.com; helo=mail-wr1-x443.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: Emacs Bug Report X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Hi Andrea, Having poked about a bit more I agree. I don't think it has anything to do with the native compilation directly. I suspect that it has to do with how or exactly when the native compiled code is swapped in. When byte compiling on the command line I get an error at the same place that I get one when native compiling stating that json-snatcher could not be required. However when byte compiling the file, the code to add to kill-buffer-hook is not run. I'm wondering whether the load starts, native comp kicks in, succeeds on the first couple of forms, then fails for the same reason as bytecomp, except that the first forms were successfully loaded? This assumes that native-comp compiles and loads single expressions at a time. The other possibility that comes to mind is that maybe the regular el file forms are loaded, and then there is a step where the forms that would be replaced are unloaded? My ignorance of how you do the swap doesn't help here. Note that if you try to repro this now you will have to install an older version of json-snacher since the patch to fix the issue has been merged. Best, Tom On Fri, Sep 4, 2020 at 12:56 AM Andrea Corallo wrote: > > Hi Tom, > > thanks for the detailed report. > > Tom Gillespie writes: > > > While this is clearly a bug in the way that json-snatcher is written > > (the symbol should never have been added to the hook before the > > function was defined), it is only causing an issue with the new native > > comp code. > > had no time to reproduce but to a very first look I'm quite unconvinced > this is a native-comp bug. I think you'd get the same exact error > byte-compiling the file from shell using a vanilla Emacs. > > The fact is that native async compilations are child processes so they > start with a clean environment. This makes the native compiler a little > more picky on bugs when the compilation is relaying silently on some > definition sneaked in from the environment. > > This lead already to a number of packages to fix their requires. > > The proof is typically trying to byte-compile from shell, this give > systematically the same error, this because the front-end of the native > compiler *is* the byte-compiler :) > > Each compilation unit should be consistent and compilable autonomously > without relaying on previous modifications of the environment. > > Thanks > > Andrea From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 18 15:24:03 2020 Received: (at submit) by debbugs.gnu.org; 18 Sep 2020 19:24:03 +0000 Received: from localhost ([127.0.0.1]:44441 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLzH-0004Rr-B5 for submit@debbugs.gnu.org; Fri, 18 Sep 2020 15:24:03 -0400 Received: from lists.gnu.org ([209.51.188.17]:57572) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLzF-0004RW-6a for submit@debbugs.gnu.org; Fri, 18 Sep 2020 15:24:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58138) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJLzE-0007Xf-UY for bug-gnu-emacs@gnu.org; Fri, 18 Sep 2020 15:24:00 -0400 Received: from mx.sdf.org ([205.166.94.24]:59492) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJLzB-00042W-9n for bug-gnu-emacs@gnu.org; Fri, 18 Sep 2020 15:24:00 -0400 Received: from mab (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTP id 08IJNqKl001250; Fri, 18 Sep 2020 19:23:52 GMT From: Andrea Corallo To: Tom Gillespie Subject: Re: feature/native-comp; failure to compile json-mode leaves Emacs broken References: Date: Fri, 18 Sep 2020 19:23:52 +0000 In-Reply-To: (Tom Gillespie's message of "Thu, 17 Sep 2020 13:49:28 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=205.166.94.24; envelope-from=akrl@sdf.org; helo=mx.sdf.org X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/18 15:09:10 X-ACL-Warn: Detected OS = ??? X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: Emacs Bug Report X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Hi Tom, Tom Gillespie writes: > Hi Andrea, > Having poked about a bit more I agree. I don't think it has > anything to do with the native compilation directly. I suspect that it > has to do with how or exactly when the native compiled code is swapped > in. When byte compiling on the command line I get an error at the same > place that I get one when native compiling stating that json-snatcher > could not be required. As I mentioned what you have seen is 99% due to the fact that the compiled package is missing to require some feature explicitly. For some reason this feature in a interactive session in loaded and the byte-compiler is able to compile without complaining of this missing definition. The native compiler is picky as the byte compiler invoked in the command line (actually because the byte-compiler is its front-end). If for example the native compiler compiles a file using a macro but this is not defined it will assume this is a function. As a result the produced code is wrong and once loaded will not behave correctly. If a require is missing the code is not correct and we cannot garantee to compile such a code correctly, as the package has been fixed and there's no evidence of a native compiler bug I'd be for closing this bug :) Andrea From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 18 22:42:39 2020 Received: (at submit) by debbugs.gnu.org; 19 Sep 2020 02:42:40 +0000 Received: from localhost ([127.0.0.1]:45416 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJSpj-0006l7-M8 for submit@debbugs.gnu.org; Fri, 18 Sep 2020 22:42:39 -0400 Received: from lists.gnu.org ([209.51.188.17]:41606) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJSpg-0006ky-8E for submit@debbugs.gnu.org; Fri, 18 Sep 2020 22:42:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48604) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJSpg-0005Ex-06 for bug-gnu-emacs@gnu.org; Fri, 18 Sep 2020 22:42:36 -0400 Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d]:34160) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kJSpe-000743-1t for bug-gnu-emacs@gnu.org; Fri, 18 Sep 2020 22:42:35 -0400 Received: by mail-wr1-x42d.google.com with SMTP id t10so7369653wrv.1 for ; Fri, 18 Sep 2020 19:42:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=R9YwVfI+ArwFFEPSkm5oxHqpRX5DkWCOy6xVMXGiA5Y=; b=TDETVkYXQHaGsTRANgqdbr4faMXdI7DqePySgmS31m9agBswfZ4nNkbQg/8fB2++81 i6pZm8PM0aAfpJEMs562/J6UCXGi09kDVRoMKNfjs2cFyhLkPGoEXP3USe4ypj9jV2g7 riv0VO2tHeMJm15diR/KxdL5LPecwyH8AUMaP0GIwzt07Xm56xu2KMb3TelAdG/D5NBD ABk/dBBpmkx8Z2mqonJW4E+/4UQr9KpadSj2n3US/vhZWyWcP7B8IOKq7TUV8dDREJlm bcmsq9qDM4QBUhDC5BpOAESf/+hwMNrWOardv7azNn+z6fAG+IO6A5VGHlyyp/Hi7iqC Eorg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=R9YwVfI+ArwFFEPSkm5oxHqpRX5DkWCOy6xVMXGiA5Y=; b=qZdV9yyOsIcYBaFu+YQ/IradJOosKMATwwP1Co+ujBlzfOddrusjye7jDTOB1k/k8u UuM48KhByRafuCdfUnbAJgboUH2Tzsrfw0lYVMGccXqYaqO+GKDUwoHP7K4xKOU6LyrV EzeUxcRsRXZ/7mnexynP1XMm2RBwsa75F7mZvDtct+IaQViRNVlOsHxXmIEt2pejZHeF WZh3oielOXwPGGy8KnZfIev2cKxlrzd3Fm6+qoJ792M2eC5IMVFOU+X6Pft+0duRGEOJ RGhr15kSgOJ0GSjdFCVKPHTzQAh9Uao/gfOiFn1HvdjSNBjjnwy7c3DVkLDgpJahq8Sq DE9A== X-Gm-Message-State: AOAM531Ao2zRvzwXqGn00p/FHvKHHgMflR2AkOAsprqcMa3iQ+6vRjrz qcfPG3qQNT/ohDC6crmfpZYrfll2vnQt1V2wjTo= X-Google-Smtp-Source: ABdhPJxRRlH9tvK2fnUR0YouFL9H0Y4lwmj6+vI4JPwDvQJ5R+bbcdACKcgcXaJpRbGkmUI1W6GQMOEsorguNj55CXE= X-Received: by 2002:adf:912b:: with SMTP id j40mr40657064wrj.42.1600483351875; Fri, 18 Sep 2020 19:42:31 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Tom Gillespie Date: Fri, 18 Sep 2020 22:42:20 -0400 Message-ID: Subject: Re: feature/native-comp; failure to compile json-mode leaves Emacs broken To: Andrea Corallo Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2a00:1450:4864:20::42d; envelope-from=tgbugs@gmail.com; helo=mail-wr1-x42d.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: Emacs Bug Report X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) I think we are ok to close. If a similar issue shows up in the future we will have this as a reference. Best! Tom On Fri, Sep 18, 2020 at 3:23 PM Andrea Corallo wrote: > > Hi Tom, > > Tom Gillespie writes: > > > Hi Andrea, > > Having poked about a bit more I agree. I don't think it has > > anything to do with the native compilation directly. I suspect that it > > has to do with how or exactly when the native compiled code is swapped > > in. When byte compiling on the command line I get an error at the same > > place that I get one when native compiling stating that json-snatcher > > could not be required. > > As I mentioned what you have seen is 99% due to the fact that the > compiled package is missing to require some feature explicitly. For > some reason this feature in a interactive session in loaded and the > byte-compiler is able to compile without complaining of this missing > definition. The native compiler is picky as the byte compiler invoked > in the command line (actually because the byte-compiler is its > front-end). > > If for example the native compiler compiles a file using a macro but > this is not defined it will assume this is a function. As a result the > produced code is wrong and once loaded will not behave correctly. > > If a require is missing the code is not correct and we cannot garantee > to compile such a code correctly, as the package has been fixed and > there's no evidence of a native compiler bug I'd be for closing this bug > :) > > Andrea From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 19 02:38:47 2020 Received: (at 43190-done) by debbugs.gnu.org; 19 Sep 2020 06:38:47 +0000 Received: from localhost ([127.0.0.1]:45551 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJWWF-000423-5w for submit@debbugs.gnu.org; Sat, 19 Sep 2020 02:38:47 -0400 Received: from mx.sdf.org ([205.166.94.24]:61097) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJWWA-00041j-5v for 43190-done@debbugs.gnu.org; Sat, 19 Sep 2020 02:38:45 -0400 Received: from mab (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTP id 08J6cf3i025090; Sat, 19 Sep 2020 06:38:41 GMT From: Andrea Corallo To: Tom Gillespie Subject: Re: feature/native-comp; failure to compile json-mode leaves Emacs broken References: Date: Sat, 19 Sep 2020 06:38:41 +0000 In-Reply-To: (Tom Gillespie's message of "Fri, 18 Sep 2020 22:42:20 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 43190-done Cc: 43190-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Tom Gillespie writes: > I think we are ok to close. If a similar issue shows up in the future > we will have this as a reference. Best! > Tom Great thanks! closing Andrea From unknown Sun Jun 22 17:13:38 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 17 Oct 2020 11:24:10 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator