Package: emacs;
Reported by: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Date: Wed, 15 Feb 2017 19:20:02 UTC
Severity: normal
Found in version 26.0.50
Done: Stefan Monnier <monnier <at> iro.umontreal.ca>
Bug is archived. No further changes may be made.
View this message in rfc822 format
From: Stefan Monnier <monnier <at> IRO.UMontreal.CA> To: 25743 <at> debbugs.gnu.org Subject: bug#25743: rehash-size ignored Date: Wed, 15 Feb 2017 14:18:54 -0500
Package: Emacs
Version: 26.0.50
Looking accidentally at maybe_resize_hash_table, I noticed that we don't
obey resize_hash, although we do all the work needed for that.
Worse, we make dangerous assumptions about the behavior of
larger_vector:
maybe_resize_hash_table takes `old_size' from `ASIZE (h->next)' and then
uses rehash_size to compute the desired new_size. The problem comes
here:
set_hash_key_and_value (h, larger_vector (h->key_and_value,
2 * (new_size - old_size), -1));
set_hash_next (h, larger_vector (h->next, new_size - old_size, -1));
This says, that h->next and h->key_and_value are replaced by new vectors
that are larger than the previous one so that they are large enough to
accomodate new_size.
The problem is that larger_vector's promise is to return a vector at
least as large as requested, but sometimes larger. In practice the new
vector is twice the size of the old vector. This means that rehash_size
is ignored when it means to something smaller than twice the size.
It *also* means that h->next may end up larger than planned. But `ASIZE
(h->next)' is the definition of the hash-table's size, so if it's larger
than planned, then it's indispensible to make sure that all other
related tables are equally larger: h->key_and_value must always be at
least twice the size of h->next. But the call above only makes sure the
new h->key_and_value is at least twice the desired new_size but not
twice the size of the new h->next.
I think the right fix is to change the calls to larger_vector so that
instead of requesting vectors "at least as large as new_size", we should
request vectors "of size exactly new_size". That will fix both problems
at the same time. The patch below implements it.
Any objection?
Stefan
diff --git a/src/fns.c b/src/fns.c
index 760a017dedf..62c834fac60 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -3536,19 +3535,19 @@ get_key_arg (Lisp_Object key, ptrdiff_t nargs, Lisp_Object *args, char *used)
Lisp_Object
larger_vector (Lisp_Object vec, ptrdiff_t incr_min, ptrdiff_t nitems_max)
{
+ eassert (VECTORP (vec));
struct Lisp_Vector *v;
- ptrdiff_t incr, incr_max, old_size, new_size;
+ ptrdiff_t old_size = ASIZE (vec);
ptrdiff_t C_language_max = min (PTRDIFF_MAX, SIZE_MAX) / sizeof *v->contents;
- ptrdiff_t n_max = (0 <= nitems_max && nitems_max < C_language_max
+ ptrdiff_t n_max = (nitems_max == 0 ? old_size + incr_min
+ : 0 < nitems_max && nitems_max < C_language_max
? nitems_max : C_language_max);
- eassert (VECTORP (vec));
eassert (0 < incr_min && -1 <= nitems_max);
- old_size = ASIZE (vec);
- incr_max = n_max - old_size;
- incr = max (incr_min, min (old_size >> 1, incr_max));
+ ptrdiff_t incr_max = n_max - old_size;
+ ptrdiff_t incr = max (incr_min, min (old_size >> 1, incr_max));
if (incr_max < incr)
memory_full (SIZE_MAX);
- new_size = old_size + incr;
+ ptrdiff_t new_size = old_size + incr;
v = allocate_vector (new_size);
memcpy (v->contents, XVECTOR (vec)->contents, old_size * sizeof *v->contents);
memclear (v->contents + old_size, incr * word_size);
@@ -3828,10 +3827,10 @@ maybe_resize_hash_table (struct Lisp_Hash_Table *h)
message ("Growing hash table to: %"pI"d", new_size);
#endif
- set_hash_key_and_value (h, larger_vector (h->key_and_value,
- 2 * (new_size - old_size), -1));
- set_hash_next (h, larger_vector (h->next, new_size - old_size, -1));
- set_hash_hash (h, larger_vector (h->hash, new_size - old_size, -1));
+ set_hash_key_and_value (h, larger_vector (h->key_and_value,
+ 2 * (new_size - old_size), 0));
+ set_hash_next (h, larger_vector (h->next, new_size - old_size, 0));
+ set_hash_hash (h, larger_vector (h->hash, new_size - old_size, 0));
set_hash_index (h, Fmake_vector (make_number (index_size), Qnil));
/* Update the free list. Do it so that new entries are added at
In GNU Emacs 26.0.50.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw3d scroll bars)
of 2017-01-30 built on lechazo
Repository revision: 5ddca31bf707686aafaf891c216702b64657c516
Windowing system distributor 'The X.Org Foundation', version 11.0.11900000
System Description: Debian GNU/Linux 9.0 (stretch)
Recent messages:
Wrote /home/monnier/src/emacs/work/src/fns.c
Saving file /home/monnier/src/emacs/work/src/fns.c...
Wrote /home/monnier/src/emacs/work/src/fns.c
previous-line: Beginning of buffer [2 times]
Hunk already applied
Mark set
Saving file /home/monnier/src/emacs/work/src/fns.c...
Wrote /home/monnier/src/emacs/work/src/fns.c
Saving file /home/monnier/src/emacs/work/src/fns.c...
Wrote /home/monnier/src/emacs/work/src/fns.c
Configured using:
'configure -C --enable-checking --enable-check-lisp-object-type
'CFLAGS=-Wall -g3 -Og -Wno-pointer-sign'
PKG_CONFIG_PATH=/home/monnier/lib/pkgconfig'
Configured features:
XAW3D XPM JPEG TIFF GIF PNG SOUND GPM DBUS GSETTINGS NOTIFY GNUTLS
LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11
Important settings:
value of $LANG: fr_CH.UTF-8
locale-coding-system: utf-8-unix
Major mode: InactiveMinibuffer
Minor modes in effect:
csv-field-index-mode: t
diff-auto-refine-mode: t
shell-dirtrack-mode: t
electric-pair-mode: t
global-reveal-mode: t
reveal-mode: t
auto-insert-mode: t
savehist-mode: t
minibuffer-electric-default-mode: t
global-compact-docstrings-mode: t
url-handler-mode: t
global-eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
global-prettify-symbols-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
line-number-mode: t
transient-mark-mode: t
Load-path shadows:
/home/monnier/src/emacs/elpa/packages/svg/svg hides /home/monnier/src/emacs/work/lisp/svg
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-xref hides /home/monnier/src/emacs/work/lisp/progmodes/ada-xref
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-stmt hides /home/monnier/src/emacs/work/lisp/progmodes/ada-stmt
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-prj hides /home/monnier/src/emacs/work/lisp/progmodes/ada-prj
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-mode hides /home/monnier/src/emacs/work/lisp/progmodes/ada-mode
/home/monnier/src/emacs/elpa/packages/landmark/landmark hides /home/monnier/src/emacs/work/lisp/obsolete/landmark
/home/monnier/src/emacs/elpa/packages/crisp/crisp hides /home/monnier/src/emacs/work/lisp/obsolete/crisp
Features:
(shadow mail-extr emacsbug message puny rfc822 mml mml-sec epa derived
epg mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail grep csv-mode smerge-mode whitespace gud
debug completion tar-mode dabbrev vc vc-dispatcher caml tuareg_indent
tuareg caml-help caml-types caml-emacs map sm-c-mode smie ox-latex
ox-icalendar ox-html ox-ascii ox-publish ox org-protocol org-mouse
org-mobile org-agenda org-indent org-feed org-crypt org-capture
org-attach org-id org-element org-rmail org-mhe org-irc org-info
org-gnus gnus-util rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils org-docview org-bibtex org-bbdb org-w3m
org org-macro org-footnote org-pcomplete org-list org-faces org-entities
org-version ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp
org-src ob-keys ob-comint ob-core ob-eval org-compat org-macs
org-loaddefs bibtex-style bibtex rect skeleton eieio-opt speedbar
sb-image ezimage dframe find-func help-fns radix-tree html5-schema
rng-xsd xsd-regexp rng-cmpct rng-nxml rng-valid rng-loc rng-uri
rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns
nxml-mode nxml-outln nxml-rap sgml-mode dom nxml-util nxml-enc xmltok
sort mpc view cal-china lunar solar cal-dst cal-bahai cal-islam
cal-hebrew holidays hol-loaddefs warnings cal-french diary-lib
diary-loaddefs cal-move cal-menu calendar cal-loaddefs misearch
multi-isearch cus-edit cus-start cus-load wid-edit autorevert filenotify
doc-view subr-x jka-compr image-mode dired dired-loaddefs format-spec
executable copyright vc-git diff-mode filecache reftex-dcr reftex
reftex-loaddefs reftex-vars tex-mode compile shell pcomplete comint
ansi-color ring latexenc server time-date noutline outline easy-mmode
flyspell ispell checkdoc thingatpt load-dir elec-pair reveal autoinsert
proof-site proof-autoloads cl pg-vars savehist minibuf-eldef disp-table
compact-docstrings kotl-loaddefs advice info realgud-recursive-autoloads
finder-inf url-auth package epg-config url-handlers url-parse
auth-source eieio eieio-core cl-macs eieio-loaddefs password-cache
url-vars seq byte-opt gv bytecomp byte-compile cl-extra help-mode
easymenu cconv cl-loaddefs pcase cl-lib bbdb-loaddefs mule-util tooltip
eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript case-table epa-hook jka-cmpr-hook help
simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button
faces cus-face macroexp files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote dbusbind inotify dynamic-setting system-font-setting
font-render-setting x-toolkit x multi-tty make-network-process emacs)
Memory information:
((conses 8 523925 99482)
(symbols 24 37808 0) (miscs 20 5334 863) (strings 16 112634 17461)
(string-bytes 1 4377921)
(vectors 8 68327) (vector-slots 4 2220644 87010) (floats 8 1387 1336)
(intervals 28 30482 0)
(buffers 520 88))
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.