GNU bug report logs -
#31118
27.0.50; Can't load/compile websocket in 32bit master
Previous Next
Reported by: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Date: Tue, 10 Apr 2018 03:04:01 UTC
Severity: normal
Found in version 27.0.50
Done: Paul Eggert <eggert <at> cs.ucla.edu>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 31118 in the body.
You can then email your comments to 31118 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Tue, 10 Apr 2018 03:04:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Stefan Monnier <monnier <at> IRO.UMontreal.CA>
:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org
.
(Tue, 10 Apr 2018 03:04:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Package: Emacs
Version: 27.0.50
The websocket package available from GNU ELPA can't be used in a 32bit
build of master any more because of the #xffffffff constant in its code:
(defun websocket-to-bytes (val nbytes)
"Encode the integer VAL in NBYTES of data.
NBYTES much be a power of 2, up to 8.
This supports encoding values up to 536870911 bytes (2^29 - 1),
approximately 537M long."
(when (and (< nbytes 8)
(> val (expt 2 (* 8 nbytes))))
;; not a user-facing error, this must be caused from an error in
;; this library
(error "websocket-to-bytes: Value %d could not be expressed in %d bytes"
val nbytes))
(if (= nbytes 8)
(progn
(let ((hi-32bits (lsh val -32))
;; Test for systems that don't have > 32 bits, and
;; for those systems just return the value.
(low-32bits (if (= 0 (expt 2 32))
val
(logand #xffffffff val))))
(when (or (> hi-32bits 0) (> (lsh low-32bits -29) 0))
(signal 'websocket-frame-too-large val))
(bindat-pack `((:val vec 2 u32))
`((:val . [,hi-32bits ,low-32bits])))))
(bindat-pack
`((:val ,(cond ((= nbytes 1) 'u8)
((= nbytes 2) 'u16)
((= nbytes 4) 'u32)
;; Library error, not system error
(t (error "websocket-to-bytes: Unknown NBYTES: %S" nbytes)))))
`((:val . ,val)))))
The code used to compile&run correctly before both on 32bit and 64bit
systems (but admittedly, it failed to run correctly on 64bit systems
after having been compiled on a 32bit system).
Stefan
Reply sent
to
Paul Eggert <eggert <at> cs.ucla.edu>
:
You have taken responsibility.
(Tue, 10 Apr 2018 22:42:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Stefan Monnier <monnier <at> IRO.UMontreal.CA>
:
bug acknowledged by developer.
(Tue, 10 Apr 2018 22:42:02 GMT)
Full text and
rfc822 format available.
Message #10 received at 31118-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
I fixed that by installing the attached into ELPA.
I can't build ELPA from scratch on master now, for other reasons. The build
fails as follows:
In toplevel form:
packages/counsel-ebdb/counsel-ebdb.el:29:1:Error: Cannot open load file: No such
file or directory, ebdb
make: *** [GNUmakefile:171: packages/counsel-ebdb/counsel-ebdb.elc] Error 1
Is anybody building ELPA regularly? It looks like not.
[0001-Port-websocket-to-bleeding-edge-32-bit-Emacs.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 01:57:01 GMT)
Full text and
rfc822 format available.
Message #13 received at 31118 <at> debbugs.gnu.org (full text, mbox):
> I fixed that by installing the attached into ELPA.
>
> I can't build ELPA from scratch on master now, for other reasons. The build
> fails as follows:
>
> In toplevel form:
> packages/counsel-ebdb/counsel-ebdb.el:29:1:Error: Cannot open load file: No
> such file or directory, ebdb
> make: *** [GNUmakefile:171: packages/counsel-ebdb/counsel-ebdb.elc] Error 1
>
> Is anybody building ELPA regularly?
Yes, I do. Not sure why you can't find ebdb since it's in elpa.git.
Have you added the .../elpa/packages to your package-directory-list?
But I can tell you that the build fails a bit later because
grep '#xffffffff' elpa/**/*.el
still finds other cases.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 08:08:02 GMT)
Full text and
rfc822 format available.
Message #16 received at 31118 <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier wrote:
>> I fixed that by installing the attached into ELPA.
>>
>> I can't build ELPA from scratch on master now, for other reasons. The build
>> fails as follows:
>>
>> In toplevel form:
>> packages/counsel-ebdb/counsel-ebdb.el:29:1:Error: Cannot open load file: No
>> such file or directory, ebdb
>> make: *** [GNUmakefile:171: packages/counsel-ebdb/counsel-ebdb.elc] Error 1
>>
>> Is anybody building ELPA regularly?
>
> Yes, I do. Not sure why you can't find ebdb since it's in elpa.git.
> Have you added the .../elpa/packages to your package-directory-list?
No, I tried to follow the instructions in README
<http://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/README>. They say that to
install all the packages "in place" (which I assume is the simplest way to build
ELPA) one should run "make externals" first. This fails from a fresh elpa
checkout as follows:
$ make externals
emacs --batch -l admin/archive-contents.el \
-f archive-add/remove/update-externals
No "emacs" subdir: will skip :core packages
Cloning branch auctex:
fatal: Not a valid object name: 'origin/externals/auctex'.
...
Cloning branch ebdb:
fatal: Not a valid object name: 'origin/externals/ebdb'.
...
and no doubt this explains the problem. I charged ahead and typed plain 'make'
which failed as I mentioned earlier. What am I doing wrong? Should the README be
updated? (As you can probably tell I've never used ELPA, and I must say that the
startup overhead is offputting.)
> But I can tell you that the build fails a bit later because
>
> grep '#xffffffff' elpa/**/*.el
>
> still finds other cases.
That command doesn't work for me:
$ grep '#xffffffff' elpa/**/*.el
grep: elpa/**/*.el: No such file or directory
And this one doesn't find anything for me, in the elpa directory:
grep -r '#xffffffff' .
Are we talking about the same repository? I'm using the Savannah ELPA repository
under the emacs tree, as in the abovementioned URL.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 08:24:01 GMT)
Full text and
rfc822 format available.
Message #19 received at 31118 <at> debbugs.gnu.org (full text, mbox):
On Apr 11 2018, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
> $ make externals
> emacs --batch -l admin/archive-contents.el \
> -f archive-add/remove/update-externals
> No "emacs" subdir: will skip :core packages
> Cloning branch auctex:
> fatal: Not a valid object name: 'origin/externals/auctex'.
> ...
> Cloning branch ebdb:
> fatal: Not a valid object name: 'origin/externals/ebdb'.
> ...
How did you clone the repository?
Andreas.
--
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 08:36:02 GMT)
Full text and
rfc822 format available.
Message #22 received at 31118 <at> debbugs.gnu.org (full text, mbox):
Andreas Schwab wrote:
> How did you clone the repository?
I followed my usual practice:
git clone eggert <at> git.sv.gnu.org:/srv/git/emacs/elpa.git elpa-savannah
git clone elpa-savannah elpa
I do this so that I have a relatively-pristine repository (elpa-savannah) and a
dirtier working one (elpa). Hmm, I now see that this practice does not work with
ELPA for some reason. Too bad. I guess I'll not bother with the
relatively-pristine repository for ELPA. Thanks for the help.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 12:20:01 GMT)
Full text and
rfc822 format available.
Message #25 received at 31118 <at> debbugs.gnu.org (full text, mbox):
> $ make externals
> emacs --batch -l admin/archive-contents.el \
> -f archive-add/remove/update-externals
> No "emacs" subdir: will skip :core packages
> Cloning branch auctex:
> fatal: Not a valid object name: 'origin/externals/auctex'.
> ...
> Cloning branch ebdb:
> fatal: Not a valid object name: 'origin/externals/ebdb'.
> ...
Indeed, "make externals" assumes that the other branches are in your
repository, in the remote called "origin". For some reason your
local repository doesn't seem to be setup this way.
> That command doesn't work for me:
>
> $ grep '#xffffffff' elpa/**/*.el
> grep: elpa/**/*.el: No such file or directory
The "**/" glob is a zsh extension (supported also by Eshell, AFAIK),
whose meaning is clear enough.
> And this one doesn't find anything for me, in the elpa directory:
> grep -r '#xffffffff' .
This is because of the above error (the other matches are in exwm,
i.e. in the origin/externals/exwm branch).
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 12:48:02 GMT)
Full text and
rfc822 format available.
Message #28 received at 31118 <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier <monnier <at> IRO.UMontreal.CA> writes:
>> That command doesn't work for me:
>>
>> $ grep '#xffffffff' elpa/**/*.el
>> grep: elpa/**/*.el: No such file or directory
>
> The "**/" glob is a zsh extension (supported also by Eshell, AFAIK),
> whose meaning is clear enough.
FWIW, Bash 4.0 can also interpret "**" recursively via the globstar
shell option; see https://stackoverflow.com/a/28199633/3084001.
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 16:24:01 GMT)
Full text and
rfc822 format available.
Message #31 received at 31118 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 04/11/2018 05:19 AM, Stefan Monnier wrote:
> "make externals" assumes that the other branches are in your
> repository, in the remote called "origin". For some reason your
> local repository doesn't seem to be setup this way.
Yes, Andreas put me on the right track. I installed the attached to help
others who might have a similar problem.
However, even when I clone directly from Savannah so that 'make
externals' works, the resulting 'make' still fails from a fresh
checkout. Here's the output of redoing the 'make'. (I can resend the log
of the full 'make -j5' if you like, but it's 11,000 lines long....) Are
the people doing periodic builds starting from a clean sheet, or is
there some other mysterious thing that I am doing "wrong"?
$ make
Byte compiling packages/hyperbole/hinit.el
Unable to activate package ‘helm-ebdb’.
Required package ‘helm-1.0’ is unavailable
Unable to activate package ‘ebdb-i18n-chn’.
Required package ‘pyim-1.6.0’ is unavailable
Loading /home/eggert/src/gnu/elpa/packages/hyperbole/hui-window.el
(source)...
In toplevel form:
packages/hyperbole/hinit.el:26:1:Error: "hui-window.el": ‘hmouse-alist’
must be defined before loading this.
make: *** [GNUmakefile:171: packages/hyperbole/hinit.elc] Error 1
Compilation exited abnormally with code 2 at Wed Apr 11 08:56:13
[0001-README-Don-t-clone-a-clone.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 16:42:02 GMT)
Full text and
rfc822 format available.
Message #34 received at 31118 <at> debbugs.gnu.org (full text, mbox):
> From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
> Date: Wed, 11 Apr 2018 08:19:36 -0400
> Cc: 31118 <at> debbugs.gnu.org
>
> > $ make externals
> > emacs --batch -l admin/archive-contents.el \
> > -f archive-add/remove/update-externals
> > No "emacs" subdir: will skip :core packages
> > Cloning branch auctex:
Btw, this part of archive-contents.el:
(setq archive--use-worktree
(list
(ignore-errors
(zerop (call-process "git" nil nil nil "worktree" "--help"))))))
is sub-optimal: on MS-Windows, Git help commands invoke the system Web
browser to show the HTML version of the docs, so the browser suddenly
pops up when this code runs.
I actually don't understand why --help is the best idea here; is there
something wrong with running "worktree list" instead? I tested that
with Git v2.3.5, which doesn't support worktrees, and it exited with
non-zero code.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 18:38:02 GMT)
Full text and
rfc822 format available.
Message #37 received at 31118 <at> debbugs.gnu.org (full text, mbox):
> if you like, but it's 11,000 lines long....) Are the people doing periodic
> builds starting from a clean sheet, or is there some other mysterious thing
> that I am doing "wrong"?
I don't re-compile everything all the time, no. It would be good to
have such a thing in a CI somewhere, ideally with some way to
automatically send the build failure notices to the right package maintainer.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 18:39:01 GMT)
Full text and
rfc822 format available.
Message #40 received at 31118 <at> debbugs.gnu.org (full text, mbox):
> I actually don't understand why --help is the best idea here; is there
> something wrong with running "worktree list" instead?
I use "--help" because I tried it and it worked for me.
Feel free to replace it with "list".
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 11 Apr 2018 19:49:01 GMT)
Full text and
rfc822 format available.
Message #43 received at 31118 <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier wrote:
> I don't re-compile everything all the time, no. It would be good to
> have such a thing in a CI somewhere, ideally with some way to
> automatically send the build failure notices to the right package maintainer.
Such a setup was posted once:
http://lists.gnu.org/archive/html/emacs-devel/2013-07/msg00305.html
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Thu, 12 Apr 2018 12:51:01 GMT)
Full text and
rfc822 format available.
Message #46 received at 31118 <at> debbugs.gnu.org (full text, mbox):
> From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
> Cc: eggert <at> cs.ucla.edu, 31118 <at> debbugs.gnu.org
> Date: Wed, 11 Apr 2018 14:38:45 -0400
>
> > I actually don't understand why --help is the best idea here; is there
> > something wrong with running "worktree list" instead?
>
> I use "--help" because I tried it and it worked for me.
> Feel free to replace it with "list".
Done.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Sun, 15 Apr 2018 22:13:02 GMT)
Full text and
rfc822 format available.
Message #49 received at 31118 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Although I still can't build ELPA (see Bug#31118#31), I took a shot at
making ELPA build for 32-bit Emacs on master by installing the attached
patches into the xelb and exwm packages, respectively.
As I understand it, some parts of ELPA still assume 64-bit Emacs, but
that's lower priority as it's not a regression. Also, one still cannot
reliably build ELPA on 64-bit Emacs and run it on 32-bit Emacs (or vice
versa), but people don't do that much and I suspect it wasn't really
working either.
[0001-Port-to-32-bit-Emacs-master.patch (text/x-patch, attachment)]
[0001-Port-to-32-bit-Emacs-on-master-branch.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Mon, 16 Apr 2018 01:28:02 GMT)
Full text and
rfc822 format available.
Message #52 received at 31118 <at> debbugs.gnu.org (full text, mbox):
> Although I still can't build ELPA (see Bug#31118#31), I took a shot at
> making ELPA build for 32-bit Emacs on master by installing the attached
> patches into the xelb and exwm packages, respectively.
I think a major problem remains: these problems in GNU ELPA packages
probably also affect other packages that we can't patch so easily.
So I think we should reconsider whether reading something like
#xffffffff in a 32bit build should signal an error.
More specifically, I think we should first make it emit a warning, while
keeping the old broken behavior. Then in a later release we can make it
signal an error.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Tue, 17 Apr 2018 07:52:01 GMT)
Full text and
rfc822 format available.
Message #55 received at 31118 <at> debbugs.gnu.org (full text, mbox):
On 04/15/2018 06:26 PM, Stefan Monnier wrote:
> I think we should first make it emit a warning, while
> keeping the old broken behavior.
Do you mean 'read' should call 'display-warning' for now? I suppose we could
make read-integer-overflow-as-float be a three-state variable: either (1) signal
an error, or (2) call display-warning and yield a float, or (3) silently yield a
float, and have (2) be the default for now. However, I worry that (2) might lead
to further problems, e.g., display-warning is Lisp code that might in turn call
'read' and loop recursively.
The code we're talking about had nonportable code like (eq desktop #xffffffff)
that doesn't work on platforms with 30-bit fixnums anyway. If the goal is to
make Elisp code safer on 32-bit platforms, --with-wide-int is a simpler and more
reliable way to get there (of course bignums would be better but that's a much
bigger project). Most Emacs development occurs on 64-bit platforms now, and it's
becoming more and more of a pain to insist that programmers must hack on code to
make it portable to platforms with 30-bit fixnums.
Admittedly --with-wide-hit is a ~30% CPU performance hit on my circa-2010 AMD
desktop. If defaulting to --with-wide-int is too drastic, I hope that the
already-existing read-integer-overflow-as-float flag is enough backstop for
people who want to run nonportable code on platforms with 30-bit fixnums.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Tue, 17 Apr 2018 12:19:02 GMT)
Full text and
rfc822 format available.
Message #58 received at 31118 <at> debbugs.gnu.org (full text, mbox):
> Do you mean 'read' should call 'display-warning' for now? I suppose we could
> make read-integer-overflow-as-float be a three-state variable: either (1)
> signal an error, or (2) call display-warning and yield a float, or (3)
> silently yield a float, and have (2) be the default for now. However,
> I worry that (2) might lead to further problems, e.g., display-warning is
> Lisp code that might in turn call 'read' and loop recursively.
We already solved this problem for old-style backquotes: don't emit the
warning directly, instead set a variable and let the caller check the
value of the variable to emit a warning if needed.
> The code we're talking about had nonportable code like (eq desktop
> #xffffffff) that doesn't work on platforms with 30-bit fixnums anyway.
I don't disagree with your change. But the problems encountered with
code in GNU ELPA makes me think that we want to do it in two steps to
reduce the backward compatibility pain.
> Admittedly --with-wide-hit is a ~30% CPU performance hit on my circa-2010
> AMD desktop.
4 out of 5 of my machines predate your AMD desktop, and I use a 32bit
build on them.
> If defaulting to --with-wide-int is too drastic, I hope that the
> already-existing read-integer-overflow-as-float flag is enough
> backstop for people who want to run nonportable code on platforms with
> 30-bit fixnums.
If we want to signal an error by default, then I think it's important
for this error to give clear steps to set read-integer-overflow-as-float
(and in a message that's not too prone to disappear before the user
gets to see and act on it).
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Tue, 17 Apr 2018 23:54:02 GMT)
Full text and
rfc822 format available.
Message #61 received at 31118 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 04/17/2018 05:18 AM, Stefan Monnier wrote:
> If we want to signal an error by default, then I think it's important
> for this error to give clear steps to set read-integer-overflow-as-float
Something like the attached patch, perhaps?
[0001-Tell-user-about-read-integer-overflow-as-float.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 18 Apr 2018 12:24:02 GMT)
Full text and
rfc822 format available.
Message #64 received at 31118 <at> debbugs.gnu.org (full text, mbox):
> Something like the attached patch, perhaps?
That might be good enough.
Stefan
> From eb2fa1a00c5c85aced40900b59183dc03d822958 Mon Sep 17 00:00:00 2001
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Tue, 17 Apr 2018 16:52:04 -0700
> Subject: [PATCH] Tell user about read-integer-overflow-as-float
>
> * src/lread.c (string_to_number): Suggest
> read-integer-overflow-as-float in signal message.
> Suggested by Stefan Monnier (Bug#31118#58).
> ---
> src/lread.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/src/lread.c b/src/lread.c
> index 6eda740540..16e34aff0c 100644
> --- a/src/lread.c
> +++ b/src/lread.c
> @@ -3797,7 +3797,12 @@ string_to_number (char const *string, int base, int flags)
> value = n;
>
> if (! (state & DOT_CHAR) && ! (flags & S2N_OVERFLOW_TO_FLOAT))
> - xsignal1 (Qoverflow_error, build_string (string));
> + {
> + AUTO_STRING (fmt, ("%s is out of fixnum range; "
> + "maybe set `read-integer-overflow-as-float'?"));
> + xsignal1 (Qoverflow_error,
> + CALLN (Fformat_message, fmt, build_string (string)));
> + }
> }
>
> /* Either the number uses float syntax, or it does not fit into a fixnum.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#31118
; Package
emacs
.
(Wed, 18 Apr 2018 19:51:01 GMT)
Full text and
rfc822 format available.
Message #67 received at 31118 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 04/18/2018 05:22 AM, Stefan Monnier wrote:
> That might be good enough.
OK, I installed the attached, which is slightly improved from what I
sent earlier.
[0001-Tell-user-about-read-integer-overflow-as-float.patch (text/x-patch, attachment)]
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Thu, 17 May 2018 11:24:04 GMT)
Full text and
rfc822 format available.
This bug report was last modified 7 years and 28 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.