From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 07:18:05 2019 Received: (at submit) by debbugs.gnu.org; 29 Jun 2019 11:18:05 +0000 Received: from localhost ([127.0.0.1]:43523 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhBMq-0002UM-Ps for submit@debbugs.gnu.org; Sat, 29 Jun 2019 07:18:05 -0400 Received: from lists.gnu.org ([209.51.188.17]:45562) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhBMo-0002UD-4B for submit@debbugs.gnu.org; Sat, 29 Jun 2019 07:18:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54377) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhBMm-0004uE-Mg for bug-gnu-emacs@gnu.org; Sat, 29 Jun 2019 07:18:02 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_50,RCVD_IN_DNSWL_MED, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hhBMk-0003Zm-Ve for bug-gnu-emacs@gnu.org; Sat, 29 Jun 2019 07:18:00 -0400 Received: from mout02.posteo.de ([185.67.36.66]:37295) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hhBMj-0003Ra-RG for bug-gnu-emacs@gnu.org; Sat, 29 Jun 2019 07:17:58 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 89AC72400FB for ; Sat, 29 Jun 2019 13:17:47 +0200 (CEST) Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 45bWMG3XmSz9rxP for ; Sat, 29 Jun 2019 13:17:46 +0200 (CEST) Date: Sat, 29 Jun 2019 13:17:34 +0200 (CEST) Message-Id: <20190629.131734.877718102639559715.wl@gnu.org> To: bug-gnu-emacs@gnu.org Subject: Crash in marker.c:337 From: Werner LEMBERG X-Mailer: Mew version 6.8 on Emacs 27.0.50 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Mixed; boundary="--Next_Part(Sat_Jun_29_13_17_34_2019_521)--" Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 185.67.36.66 X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) ----Next_Part(Sat_Jun_29_13_17_34_2019_521)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Dear Emacs team, while sending an e-mail with `mew', I get the attached crash. This is something new, which I haven't experienced before. I can reliably repeat it. Werner ====================================================================== In GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.30, cairo version 1.15.10) of 2019-06-29 built on linux Repository revision: 67b50770c050c55a26cd13b9568b01a80a449885 Repository branch: master Windowing system distributor 'The X.Org Foundation', version 11.0.12003000 System Description: openSUSE Leap 15.1 Recent messages: Quit Setting up Mew world... Updating status...done Setting up Mew world...done Scanning +inbox...done Quit Type C-x 1 to remove help window. Mark saved where search started Quit Making completion list... Configured using: 'configure --with-x-toolkit=gtk --with-cairo --enable-checking=yes,glyphs --enable-check-lisp-object-type 'CC=ccache gcc' 'CFLAGS=-O0 -g3 -gdwarf-4'' Configured features: XPM JPEG TIFF GIF PNG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM THREADS PDUMPER GMP Important settings: value of $LANG: en_US.UTF-8 value of $XMODIFIERS: @im=none locale-coding-system: utf-8-unix Major mode: Summary Minor modes in effect: TeX-PDF-mode: t tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t buffer-read-only: t column-number-mode: t transient-mark-mode: t Load-path shadows: /usr/local/share/emacs/site-lisp/thai-word hides /usr/local/share/emacs/27.0.50/lisp/language/thai-word Features: (shadow emacsbug message rmc puny dired dired-loaddefs format-spec rfc822 mml mml-sec epa derived epg gnus-util rmail rmail-loaddefs text-property-search time-date mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils misearch multi-isearch apropos pp mew-varsx mew-unix elec-pair edmacro kmacro rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns nxml-mode nxml-outln nxml-rap nxml-util nxml-enc xmltok sgml-mode dom hideshow cal-menu calendar cal-loaddefs mew-auth mew-config mew-imap2 mew-imap mew-nntp2 mew-nntp mew-pop mew-smtp mew-ssl mew-ssh mew-net mew-highlight mew-sort mew-fib mew-ext mew-refile mew-demo mew-attach mew-draft mew-message mew-thread mew-virtual mew-summary4 mew-summary3 mew-summary2 mew-summary mew-search mew-pick mew-passwd mew-scan mew-syntax mew-bq mew-smime mew-pgp mew-header mew-exec mew-mark mew-mime mew-edit mew-decode mew-encode mew-cache mew-minibuf mew-complete mew-addrbook mew-local mew-vars3 mew-vars2 mew-vars mew-env mew-mule3 mew-mule mew-gemacs mew-key mew-func mew-blvs mew-const mew tex dbus xml crm advice auto-loads tex-site quail help-mode cjktilde mm-util mail-prsvr disp-table finder-inf mule-util package easymenu epg-config url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 155296 8644) (symbols 48 16475 1) (strings 32 59212 1207) (string-bytes 1 1586649) (vectors 16 23308) (vector-slots 8 379039 13042) (floats 8 56 42) (intervals 56 1877 161) (buffers 992 14)) ----Next_Part(Sat_Jun_29_13_17_34_2019_521)-- Content-Type: Application/Octet-Stream Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="gdb.txt.xz" /Td6WFoAAATm1rRGAgAhARYAAAB0L+Wj4ZpLGeBdABGMADiqi2QIh2amU6cDyw+cBZ/PH9rxhemb aDhviSWsKR7NeK+sHoz8c8g/hG9oXNAkVHpvgUcazsDlS6np/2b9WoRe2VTACtFmPw4f7/JOU8mw BafsDhjL1HEDbNy1/CS++9yz+5ysVW38Gqq3TwCpk9sh3p/pIDldGr7k+PpuuqKg4czqVf4wCEt5 KtZCKpstv78GBlsVR/TVO7XH2DFSZda9H+q/GGYx8YABk4IvDnin81GpXaZSEIkdfKc3h1dtufTc 7n+/e68YmMWozwqhZJfONQc7Vw3+bobOAzyLc2Xz7wegslcGuHYvsVqk+K8F8qAphHgUYeLHfQQP h3wu0dyZORWCK2EYr600Kb3kHudpREqLKE9IcsAaU+paVB8DFFkmPDEjAnDy5VzYvj6XSEAZrqPc uVCBDgXIisZe1gjhgmfN6OC5ojWSn31PyUdrnf7MTc7SxhgFT25D6zK3zd6ln9Ixl3qs1gfkiFDm TVboCEYR7Vh1pz6a4VjPHqJeGO2BfJX3d/Csafa1PPrKId9KwVf6mmN6L6C0E/6EV5Ea1q7EMlhP z4c4+5NjzKdYgx46h3xLwILoBf5mL/nQapjNrI3nnd2mX2t9iawM/ROKx7u5xAv2RhCuuTsPWOVu gUN/5FglxlVEO4IuqvhimqIO5DzC6Hr9yBJV9jqKHuyiRsGC0d9WIkzmYRrgH9FSvSv5rqfEk/ku NT5cr6nwyFWABDy82ISv37fZpi5ARrAoNanxsqxaidmOVZrY7KvLmP6kz95qQo8sCQkfcfL6IAfA JGK8kWlP2lO28d5/Wf8pR9SihuQsvHB6wO4NkNGnWavUT4D9T5edCg3S5sReSMnS9Gyx+toELnoW jK3hSNEtjbBw0CSZQl54gEU6W4DsMIuSOqe6MVexfWHnujKJ1pzWUSZI4zgGh+A3k5Y7Nr+RR7U9 VGBC3gr854zarlDdtBcK2QMQCEaYoazrFWkYSOqvTv8WVoaNroF7OTfbi1v/k/fA6Jq9hU6ghdcg Bi7Z9E4E3/+VmBvtOv7ogJxrsaScoyuLZ3bu/XaYqKSpbPn4QQLweS6iKSq0FLHOCxi7T5lb7gJU y0lOyxdTGXXImAtmXSQinoqxCkNtAiwLqD0DxEj8mWO4OZEAalA+AHL3VM4TFbhOsu3VINbJhpi+ KFBObY5J1JNXTIXl0efX+w7qoYKoSkxSS20WjDJFzpy9Eijfqu0SmB+ZA2W4L459ysNfxDx/QsyT 21I3BGaI27h04C/Am6dLqbTjzr/9czQr3NvFrQ2fV+9QQNSmehwEbdo+RsuX3M/ECKhOX74uEJY7 QalC/p61RGBG9DsptLUcSMBZVtn/CqhQZLaUWVQGdIRuMSORpsRie7K0iJa0FS3Ov+eZCrlJH+Dz N7ts5j6prRFlX9XblH1Qfr9k3w2hTSFrFd2L+Mux7XFQvtQaWK6jgwUmfbIVoV+GM8RaQS4riPnR bdU5LknZAWfwmNBbZnCNlpDDFlB2Ke8TIslk3EpGi4U+UAEnvW0fJrEDYubIZOEi4Qy16PmlcqSR e2RsRx6B1SWuGD1qr9IS2qhiHlBi6sMLTGmEmG9QVjii5AR27McQQQMCzpqz9J8LvXVERXFsTgrB m0TfMEPcBuWy82e+ppGAz/o9LFU7UEk9R08vwk428kT6/aIDttIUypz3z0rjiAg+QnNX1tJWeeul YQdoAdzxqzYVb/taySX0BBYfbW5gvQZQ63Iph2eyBMy/lVE8qDERKymJ68T+hMnD0gAm46HC6nLe 2l4kiLGEvAOPDCHOLoui0GhC4RMQ8AoerHjznX6mhL1bYySKFnLX+1JIg2vc8iu+vkeCpZm6+Zr/ qkqVzDT6hzDR+TbprdBeytzIiQ1WqbGbq5O7A6xxZFMjKe6Q6abQMRrLrLLCd1GHzxANIHZ0+eKJ jhCMNc0RenZ04g+n6XcPhJktUnj8mXrlcCd4ZLjIUgEpSrjW2PAGbGmSGgg3x9ufZAyp/t4IO5QA is9yKShI/cpbOJ6X6wFeZmW8DUtHhQyaHk96IdI8qETdDPzd+flUUhGgaCSj6rx0mjy2Uc+0N6YL zb8fkHRiHske5Ayp16G2l3Ehbze3QHPsf0diYVQN8/sIniG3uh95kqwF3JcJno2fF9PtGj6pH4ml LYO8HEn32WCAZmsi0C92yFygN7/2if9xXp5ZNXoKm/W1TUCJ7QTr40ZbAZu8kNCtBo2NpUD5sMVc RUjE0WNme/pw+xOPzqqj0XoRKWrXlY0j8cVlBy7mBixOaMxtFYo0kXjoCUGTr25X5qwijDCT7DIG GErTI0yaesjaeL2bM9f13fr+F2HeUb6wGw/ugHeGulJ6V+GZzPBe5zBLeXRrEXXHkzECTNhYrrDN ctshvz9y5VP/4cII4DkFcp0RKLYtHblARnxdAqi7kcTpgPEURXdHHOloVPbSjkf6MNli6JHPNxAR Aqxt0KmDKtUTZxFaU3D7RP1qj8tWgQPRkhYIHOsRVsdqz0xr5a6wVB0BDFbpah5DDSfwQLTdFEMb p9uAJYpMFnVob5kkg6ZP1TU9lh2rfHfCTtTQiTEvZ2MyZZ8iB7171vq6iAaSvd9ij4qSZZgfeEn7 fIKMw5FC6hD6BPo8wIHW+TWTMFhHeRPsog5u4R/e8Hel6bSoBkUKF7nuCMafaEeVePVNE0CgyPSU rlEvXNs7/i9W+ZcVLN2lZ+wOn/JJhrpePR7W1fHt88AEuL9f+gWsgwvntaMreXgV04/55phfzl+N KL4d3DazmYMup5c5M1nkptJAkV/pYfCvazUKeZZNKnwLzlFSpptbsu1TspM4351sbAxzPrvCyhbr lvO1W2kdlvc9GdV+GhSE8e8M710QGjoxtpYfkgtFZ4s+EsqLrYx+2Q0iuOLgbs2WXB53Gv8Qzc5T yh5s5V2PBUQcR8sgQRA4CzZ0qxft9oVRpvXzoqs0vQkpVuW2I9SzY3QW2mHBW2fA3XgiEwD7b3cA Hm7ZsUVfGfa8DGLMgPLJGIbMWMNSz2p+Jp/gNLZfZRqx7WFO4u/B4bQYSn9cplieN5vul1KhtC9U gAeM+L/wk86LlxtKEl9e3TGOblZmQF3TAExjbve4a6etaxjLNWdAXGvmvxioUw2FDg9mvUCiQa5R sz8grnWLqoynth2MYeiqI+1dRKqpgebSfdCzwYA8Up3d/U6RY9KL3mTuCPmO6+yZ+izbmSWNRafT itJTr4w3v8fuyAwAIgxh9h1QPX7HvxDd5RwzLfsy6ndigJM1xhSmX37/HAqSr2tjYY+LZ5XDvT52 b5fM/wRdFOxdEIClLyaMKY56B7tuQouVRHddTAJb3QPc/CC7HrHRilN6D69FAVWYvlvxWb0rDulM Kk93ddXts+f6AvujD8q74gWGDp88L1yDSPuwCG19iZu/xF5eNgnWtddl1T3dmRsc1uUNsUfh6GP6 pbQ/u0RmhE3VYTN0DpPE45OMwwiYyHiGKN8FyBe7lG5rp9oPqnRJNpgviW/QVOtOOe0alDKJKltn 5AbCamKsB+LgrK/Q4tt0tFywsNxHh4RfqtHA2j0VQ1IQ6U7b/Ubxp3LmuytE3JshZETU66sUejSh a7CVZeV1rIuIUkz1EQuXAkh4lWW30Swoko4IxmOgjiuzL/CILgB7xx8xssX1RYGeVqe0sfCN+zQm YCxbiF+6I56sDPaldpdrw4erEELlEVaqKUS8adPoNekiuz58LrZeK2AQ1vouyDe5d+43SMGUeEAm JczSmYidXH4Nkz+bGlceY87k3KQsttaHfV7QcbaENWLtpuOC96+BPTse3rXwNpJUaWm7nXejmwcP /7f6RuV4jSYJxp3jH3HoaH5yCxrNAJ/HoAPbakhVICX0mA04GNetOdMRtHuzdCno8ipsXqkh3ynt MhVDL/EoOQmowirJu+m9Nfz1IcmuLhaVIBZEAwlrHYfsn8xhOzt8zctZUXsJPWnbTo88n1c4xmoc anVXEOXcdBNeXI8rjRt/24a7J7czL1QD2TFk6AD8AAnFLia8ykrgwIx53ZuppGap9PbOd6ARmcay hjLvQ2+L1XN6FYDxX9CmcVPqV//h6IIZ/HjtOs/pMA0Yx8js1ZBaQyjhRNfqXg7czhKhlJ7Co7zv z1sedgK3tOCj9g9hcL9zEWzOV+/ujyVDvgNggLR7emnltUcKmoKILLbGhq65dvkS4ZctBxDe5vFR QVGcuPdSeAUpSAMsJvenRs56diRv6ZmJPgSph6kKSNxeoOJwRKu1k/Ekw3mYCmK7CYubk1J6oODN 2XZsFTkiEbXvIGTpsqDBygBRJvpmL1jjRaxO6WK6W5OeKKu/5hDbu/qIETpSupNFhpf8fXwlxm73 F2KnZFPwwN+mirHs4MikX7gQgjT9iI9kY6qLAhyIqnIHMCFwln1847iJ7lpoIAlzHnWl8dMysoJr QS4rEHwrbtCbWPMEb1QR1rPOpLAobd/DXE5vqOWVjvtQIA/7O4G/R0zMHhxJos3f0NgcX4tWoGeQ VS57QnzdBBuKjZjzkcEOPorOcn5B+f/hjvHqrblvjzLo28DAjo/17fdkleMfmk0qOtPJSZywVq+6 Fl22W29TivitYg1mDGxPvrT6Nl2aLjCkL5+LvW2z0/EfvY4O4mW3wuPGI7Y7BmnHx8XMDs4Vy2ew FgUnvWqemYd/3RNwLirtIlDPKMq95bNGd2ZZ5ZaP2a8SUFHtBsfPJ2k4lXcQewRMxOdzH1QsaM/w cwjDhEtuFCd/GVKZo0ZYr7KzoVkx1fSj26SouHlEpm2AfgDQLpkXQraBsn05Qz/rlqhtyCMfVD0l UtQqB24SFlVLkI16CxzB55Fc3UdjNIC5gwta5SAFwvOXbI7glzRWwS4/5qFJJXKy1iZPqqVMVkdP G6QfpP7z37tYyP5NrvgYxrT4m9MlxqBe2oZ/jrxCF6JFlDV2QDdrEneVCA18hDSm/vLAahAxxNqF vM71Hu0nIc6mHpE1BVwMNLY3s25+lA1LSA3Q2JZLJ0mYeMqFOlk8jxJASEJMziYJ7s1aUk5cbsT+ yxmZPEyALmBcMJZoG/gYBPv+7ghkE2AuJAdMf016Z8jeQ+teZcbY5bq6pLne7bBqQyNEKBqSSSAy LgsHgXg+rrkJ/A9dLIZNK06HsVmM6zKHkxV7EGKJ6JPr5jjzEF+Ipn+kzvyaxYapeJ0ZXPc/WvyG LBISGICzEykKHGfGyYIAIg8E1jLWTvjIfJScVvHMALTT44SHVV6ZOvN8TuILKJx1eYJlPqImCOZt V/ua9owfBlOA8tZLlfirdoUXd0M5zW708DFhtGymu0bFfiZFgFhS6n3hbdnimY5SR3dqnNNZCoyF fjBm5lniyRs4zJTwRkVYmHoZBKrdJgSBMu2S9Ufe2EFjpFdl/rQ4+Mu1NUkjAR6amdSmbcqIGd8Q I1XC3kJB7v3Ja6XHGQY1hjQKVBCQhktFtZqTFXeZ7GmyurYhU+8vzvbdm6gXUNVIUl9RyuY6x+9a 8+70L1g44eWiZTlblhzRTXgo74d7To6XOlfOT+/Wyj0A32+fD8KfcPmAu8Fm3IWkx65ogo650SqP eU1aXJTvRqa885Cnwput0iHgp4r1yB1f26J+CtUjsH31Jx/KCzOGkfxtOC3AWsTgYWsFfniv7UIM yedFBZ1EDaugarDAM/j5fcDYEQXffzwm1AlHH2/iB5JSTQsnAhyi1j4FbTrnDR7dGW+HUmmHr7Mb Qs03vrIaJs5uZsMWHnxpkcjMLTUO3xbf4cEuHlZWCNlJlNawjt0pxlBAuKkBwJug25sAL8jIXQGl c2IV7b3CHVjQM1yJmBLGKGhnHejAOa09UvckqsyQINp0CI5AuUOWVXE0ARKWRo6h+cw0RIKceV7m CI47kF5mG81SYqz+SJmTJj5G/eBVXC6K97rmmWGF+TIyAs5AQpU6aEFrY15jXXNC11jiU2Ywju7M N0v5TqI+IVBVtP4sMTbr7qF9hXbdIYvu0Rs50RWBsOPQMSmMvQtRy1ijEwxjRBDOXTBEk0B8XZpQ 5/r0IoVT1Iur4jYMxiM4VQ0SJ2hs29nhiG+4CIaxzs+xwHfZQ1Dvt7ONNgmeiChWEzEFV2W9TFhN 9cRE1DJstEsHyknoBF+O68RrFYVWX/AIrCVKZg/F/iD6Zq7V7wJSvScyJijnH9r1n29p+Jqfxbuo zhfyJltq3/LEslvgloPlQRH0dzisA50YnyAyP3yLKo5NFKLfjU6tdmjuxXbCFOyXfJqmE/TflitM 5IoUnQwqS2DWWLE02OFhZU/okc/kA5eS/s1hLzy90/YNYipCmOarxesnWfQB0BFcMJ9rrZaX0Q4f hQqSMf0Z9lOkJAsRb+JEzrZcb9+rBSDGZ9Obj9T/v70qqWFzCL3DPolxxOm5RZXo2YLLhM7M48kk qfql/NOjCnRAFOsbuAVcygSxqJYLAZ1P93PMXMPHh3e9LcM2+RI13/SLpX9JrLH0Y7hELgWl5nGZ UXevNwZtbGiZ9lS3j1PXSVy1OTAO27IoQoeMhlFIG9vWw7gOGXrQNo5DrGpCQ76xcpWnbuIpLV2Y 2GLRb9NxKsmahV7BJ+Et1EgvWPRXFCvpZm1htqGOf2pc04YbgIm7oL+9AzTMP50OQ5NCrp5gilYH znbcZh5CO40B+PaWlJqZ80V41WYLu4OUapAYFnlsCWzbzbJOwfkyn8hC5YoAq/IyfhZe47Y3YkGQ 1426GSRSxkpIdfBDcYH4sMPDBrAYZmPO4mKdEaR7qDnYH5tFORNIuMHX8gAmhdpM8XDHg0KYPHYu XxAhHCtqhM4OZDYq741qr01THnAGBNtSQTnUX2R8mRIHuf/M0vIYrJfkktCdXsiD9s9GKy707hmJ Msl10tcAz5URAk/flH3tKkFEAX2J4pw/cATmAbeW6FLciNYYd5M/HWP7sFseaAXmMX9lccYeXDaE oO3P62yvkxFAKt8dGbXBlXFjm4RS9KHAY/pjAAJVmQe0nxlSPNft2lnYV5Pm67dUcr3RxYDMfnBz tjTKcJ695XpBf/ajIm/ZdbnOb0C+BnAgrWAQF7jwKz53mFnxRGNRqUS/NahDIJLcV0ZT55gN8Nws Z3JNIWm5PnS8pRVErBBMS7Bz2AjDpPIUIFxVaTGZKmxw9rKvBsUeDXRW0xmQBuEQlgubDbd3zEYS Cok30sU7eI2jQ2EnSaIbBpnZzEZ2K2wHKAef9lBEZJJs8+hN/KVBvxzLJWJXAh7alew6f7cZ0BaG GHPSbogvlCGoLdButvswpQq4K28LG01GyNmCdJpRszBu7NcCuZRGFKpA2gLl8NxsJOJqOUfJ0I3G 1gcR2eab/nneJS4Xes1vlgoZjLbhKx1YPOGVlkVGpHYb831+UpuzuKEaf2EOvYOXocrEjLfYKi4I RmU7f2L6ksV9wHLL3Tpd3K9Xz/Wu76pFledQTETErlclLnxW2pd5EZA+EpHzrA3humUNkun38dFA HKh0koafWRMtPQlUDd0gTUK0hfh/Lb/uX161VtiXE1q32i7sU2w5KGX2QYTsKnQ6iZQARm2HyJtj S19Fv+JJXgymeXaXaZYMotkYAv3ZhZkrKmhlmKMOBNeLepmB57T1vAo6AWi8bMzCmKLZ6pyzA8vO hMAy5QWwvARArSIUIvcERhdsz6/4rjnAq6ViqNrDyoQapQg3z7eVsuYkXaXo1j2GOHJIYoyJ0CxY FdIXoKHOoPFsoZ3AYNA4+4ttsHcYMIlUGeLtq8rAVL5Yjj+0vt49A7qKwRkbKgeWRrWolUkmfHTB W+S5AQL2MLQVAeYpuzU1MNVDUVU/fBAFRjvBAgJakJHAL4zVn7Ot91ImRj1XWVMjuevPWOUz+THE ZmfTOYvzQU+wmasDqD6JCi7Zkz4i9kWlCdJ1Y96j0BnXWyCU8g2j3gD+nwVqeBaiiSUkieg2r4U7 F4YnO4oWJgaBzBYbKatgr0+JEsVC+Mtm+lTsXXNkYYXgvHsC/Kh4C2na633Q1jL5Bs4ccs6AaHMk v3hoBy5asw4yzU+O3Qpzcgca84k+CzgGr1W13teXnzF1vuAt6t2S5M2h3He47fe6LJfvfSXc5voU LPicQV9boewh6SV1Fr6AGgAii75RoQyjIEM8uTE9tLfytCkvmKZl75gv/76GtjudlJG4vIuUugWQ QCSlYV5F3WLLtFMqQsb+6f7X65RWuPI89ljJvv3qAuBML8EYDkEHpUvZh4LfkjJT3GQivJCPe6Q+ nro+gFnxhObmPFLGZGTXSRJzI5Utm0VmLRM4nIaAGvnGezFVgmnfVaGcOuh3/5cluwRouieAPs54 QrgcCQjM0QO4Qj7fBpKspWqoNLy+0QbKzMhFy/PEkEG8urO14EhcaKNm8F2dgyp5X0VpSUkp+Qlt wt8PEdlQrMHvWUnSNtQZ/Hum9t96Qhmx8zgTjnoOZ1UxyAnA+gvCbAJyph/ciIWdDL7XwC/N7CRy /6/wRhIdBGwOu5BFOMiPwM2Z67tYKY7VbY8FGeXYa0m7UxaxNGnJL7INN9r9nji8KbHAO5i4edsl j9anc9+mw4S7Pvu6nrp2T4leqr9LzoorZmyaZlwzyLglrP0tWQKfoK1MlfZiuoP1OzmZgahorjLI yxIBiRobEq71+pT3/pNIAWcDFY2sNYM5KIkZgxhK0LbXbVRvJ/VwDEL9Eg77wG9AXufPNdFj71Ri DQrv6WpCtCLFuk2OkgYStc6jzGK9rr/3YwyuqCpXZQrbl4zcoaZNxRqug8Jql3nQfB+jJqLn38n7 yMLiuiyLy+ztTPt6AVGbC1SMpMf9am3kwXBWhVhS5UYMs4txkk+8/1lOAACAcglaQRMyDQAB/DPM tAYAHW8S77HEZ/sCAAAAAARZWg== ----Next_Part(Sat_Jun_29_13_17_34_2019_521)---- From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 08:13:44 2019 Received: (at 36431) by debbugs.gnu.org; 29 Jun 2019 12:13:44 +0000 Received: from localhost ([127.0.0.1]:43548 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhCEi-0005sQ-4z for submit@debbugs.gnu.org; Sat, 29 Jun 2019 08:13:44 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57186) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhCEh-0005sB-8X for 36431@debbugs.gnu.org; Sat, 29 Jun 2019 08:13:43 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47480) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hhCEc-0007me-5Y; Sat, 29 Jun 2019 08:13:38 -0400 Received: from [176.228.60.248] (port=4756 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hhCEb-0008Gg-4e; Sat, 29 Jun 2019 08:13:37 -0400 Date: Sat, 29 Jun 2019 15:13:26 +0300 Message-Id: <831rzch9nd.fsf@gnu.org> From: Eli Zaretskii To: Werner LEMBERG In-reply-to: <20190629.131734.877718102639559715.wl@gnu.org> (message from Werner LEMBERG on Sat, 29 Jun 2019 13:17:34 +0200 (CEST)) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sat, 29 Jun 2019 13:17:34 +0200 (CEST) > From: Werner LEMBERG > > while sending an e-mail with `mew', I get the attached crash. This is > something new, which I haven't experienced before. I can reliably > repeat it. Please send the reproduction recipe, preferably starting from "emacs -Q", including any files required for the reproduction. I don't think it's possible to debug this otherwise, as the problem is clearly data-dependent (somehow we end up requesting a byte-to-character position conversion for a byte position that is not on character boundary). Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 08:21:15 2019 Received: (at 36431) by debbugs.gnu.org; 29 Jun 2019 12:21:15 +0000 Received: from localhost ([127.0.0.1]:43559 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhCLz-00064L-Ax for submit@debbugs.gnu.org; Sat, 29 Jun 2019 08:21:15 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58289) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhCLx-000645-IS for 36431@debbugs.gnu.org; Sat, 29 Jun 2019 08:21:13 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47586) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hhCLq-00055w-AP; Sat, 29 Jun 2019 08:21:06 -0400 Received: from [176.228.60.248] (port=1231 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hhCLp-0000LV-CT; Sat, 29 Jun 2019 08:21:06 -0400 Date: Sat, 29 Jun 2019 15:20:55 +0300 Message-Id: <83zhm0fuqg.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: <831rzch9nd.fsf@gnu.org> (message from Eli Zaretskii on Sat, 29 Jun 2019 15:13:26 +0300) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sat, 29 Jun 2019 15:13:26 +0300 > From: Eli Zaretskii > Cc: 36431@debbugs.gnu.org > > > Date: Sat, 29 Jun 2019 13:17:34 +0200 (CEST) > > From: Werner LEMBERG > > > > while sending an e-mail with `mew', I get the attached crash. This is > > something new, which I haven't experienced before. I can reliably > > repeat it. > > Please send the reproduction recipe, preferably starting from "emacs > -Q", including any files required for the reproduction. I don't think > it's possible to debug this otherwise, as the problem is clearly > data-dependent (somehow we end up requesting a byte-to-character > position conversion for a byte position that is not on character > boundary). CC'ing Stefan, as it turns out he added that assertion not long ago. From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 29 18:57:04 2019 Received: (at 36431) by debbugs.gnu.org; 29 Jun 2019 22:57:04 +0000 Received: from localhost ([127.0.0.1]:45048 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhMHH-0006z8-PA for submit@debbugs.gnu.org; Sat, 29 Jun 2019 18:57:04 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:50394) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhMHG-0006yf-KG for 36431@debbugs.gnu.org; Sat, 29 Jun 2019 18:57:03 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 18340444304; Sat, 29 Jun 2019 18:56:57 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id A95C04442E3; Sat, 29 Jun 2019 18:56:55 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1561849015; bh=aN8kPCM7Tn8/npQ16FFXMvqft2yezn6VSkGfdiErTTs=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=pmggYLsluMzwDsOKFX+dSOsCExxmaJ8PSS0ux85ZGzqeXicc8ZTDca1lQOFYiPgaJ 4fBxr8zewwpItzsvi4iQ/wyS1g3C8RKFCxFWrut3bybA//J3OK2wZvMwyC8LOVy0BI UUiX9kvWAswdWU20HpKgOPzIZS1iGxTIwmQhNWEUn87FjIfQmMKzGFzw1Q7g0kAce4 mCxcN1TlHL+onPF2C/JBnOXwuoNPhW2vk2I+DPzE7N+bl0WkHM4U0mP9Pzik10E/mO 2KGMrEmm7E607zGBoeWrxiRELIf1FOdzK1C5RiYHIIO9/k9NqNm54OrjYb8IiSP/YD WEIbBOzkDxPSA== Received: from alfajor (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 50F99120C68; Sat, 29 Jun 2019 18:56:55 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> Date: Sat, 29 Jun 2019 18:56:53 -0400 In-Reply-To: <83zhm0fuqg.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 29 Jun 2019 15:20:55 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.115 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >> > while sending an e-mail with `mew', I get the attached crash. This is >> > something new, which I haven't experienced before. I can reliably >> > repeat it. Hmm... indeed insert-file-contents takes some shortcuts when inserting the new text into the buffer which can break the expected invariants. I don't really know how to reproduce your bug, but I think I have an idea of what might be going on. Can you try the patch below, to see if it fixes your problem? Stefan diff --git a/src/fileio.c b/src/fileio.c index ed1d2aedf3..a7be05ef5c 100644 --- a/src/fileio.c +++ b/src/fileio.c @@ -3741,6 +3741,7 @@ because (1) it preserves some marker positions and (2) it puts less data CHECK_CODING_SYSTEM (Vcoding_system_for_read); Fset (Qbuffer_file_coding_system, Vcoding_system_for_read); } + eassert (inserted == 0); goto notfound; } @@ -3767,7 +3768,10 @@ because (1) it preserves some marker positions and (2) it puts less data not_regular = 1; if (! NILP (visit)) - goto notfound; + { + eassert (inserted == 0); + goto notfound; + } if (! NILP (replace) || ! NILP (beg) || ! NILP (end)) xsignal2 (Qfile_error, @@ -4435,19 +4439,6 @@ because (1) it preserves some marker positions and (2) it puts less data if (how_much < 0) report_file_error ("Read error", orig_filename); - /* Make the text read part of the buffer. */ - GAP_SIZE -= inserted; - GPT += inserted; - GPT_BYTE += inserted; - ZV += inserted; - ZV_BYTE += inserted; - Z += inserted; - Z_BYTE += inserted; - - if (GAP_SIZE > 0) - /* Put an anchor to ensure multi-byte form ends at gap. */ - *GPT_ADDR = 0; - notfound: if (NILP (coding_system)) @@ -4457,6 +4448,7 @@ because (1) it preserves some marker positions and (2) it puts less data Note that we can get here only if the buffer was empty before the insertion. */ + eassert (Z == BEG); if (!NILP (Vcoding_system_for_read)) coding_system = Vcoding_system_for_read; @@ -4477,6 +4469,16 @@ because (1) it preserves some marker positions and (2) it puts less data bset_undo_list (current_buffer, Qt); record_unwind_protect (decide_coding_unwind, unwind_data); + /* Make the text read part of the buffer. */ + eassert (NILP (BVAR (current_buffer, enable_multibyte_characters))); + GAP_SIZE -= inserted; + GPT += inserted; + GPT_BYTE += inserted; + ZV += inserted; + ZV_BYTE += inserted; + Z += inserted; + Z_BYTE += inserted; + if (inserted > 0 && ! NILP (Vset_auto_coding_function)) { coding_system = call2 (Vset_auto_coding_function, @@ -4493,8 +4495,18 @@ because (1) it preserves some marker positions and (2) it puts less data if (CONSP (coding_system)) coding_system = XCAR (coding_system); } - unbind_to (count1, Qnil); + + /* Move the text back into the gap. Do it now, before we set the + buffer back to multibyte, since the bytes may very well not be + valid for a multibyte buffer. */ + set_point_both (BEG, BEG_BYTE); + move_gap_both (BEG, BEG_BYTE); inserted = Z_BYTE - BEG_BYTE; + GAP_SIZE += inserted; + ZV = Z = BEG; + ZV_BYTE = Z_BYTE = BEG_BYTE; + + unbind_to (count1, Qnil); } if (NILP (coding_system)) @@ -4528,22 +4540,28 @@ because (1) it preserves some marker positions and (2) it puts less data } } + eassert (PT == GPT); + coding.dst_multibyte = ! NILP (BVAR (current_buffer, enable_multibyte_characters)); if (CODING_MAY_REQUIRE_DECODING (&coding) && (inserted > 0 || CODING_REQUIRE_FLUSHING (&coding))) { - move_gap_both (PT, PT_BYTE); - GAP_SIZE += inserted; - ZV_BYTE -= inserted; - Z_BYTE -= inserted; - ZV -= inserted; - Z -= inserted; decode_coding_gap (&coding, inserted, inserted); inserted = coding.produced_char; coding_system = CODING_ID_NAME (coding.id); } else if (inserted > 0) { + /* Make the text read part of the buffer. */ + eassert (NILP (BVAR (current_buffer, enable_multibyte_characters))); + GAP_SIZE -= inserted; + GPT += inserted; + GPT_BYTE += inserted; + ZV += inserted; + ZV_BYTE += inserted; + Z += inserted; + Z_BYTE += inserted; + invalidate_buffer_caches (current_buffer, PT, PT + inserted); adjust_after_insert (PT, PT_BYTE, PT + inserted, PT_BYTE + inserted, inserted); From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 03:26:44 2019 Received: (at 36431) by debbugs.gnu.org; 30 Jun 2019 07:26:44 +0000 Received: from localhost ([127.0.0.1]:45310 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhUEW-0000ad-9g for submit@debbugs.gnu.org; Sun, 30 Jun 2019 03:26:44 -0400 Received: from mout02.posteo.de ([185.67.36.66]:54985) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhUEU-0000aP-6F for 36431@debbugs.gnu.org; Sun, 30 Jun 2019 03:26:42 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 1189B2400FC for <36431@debbugs.gnu.org>; Sun, 30 Jun 2019 09:26:35 +0200 (CEST) Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 45c2B25SXqz9rxH; Sun, 30 Jun 2019 09:26:34 +0200 (CEST) Date: Sun, 30 Jun 2019 09:26:26 +0200 (CEST) Message-Id: <20190630.092626.1377512789964814286.wl@gnu.org> To: monnier@iro.umontreal.ca Subject: Re: bug#36431: Crash in marker.c:337 From: Werner LEMBERG In-Reply-To: References: <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> X-Mailer: Mew version 6.8 on Emacs 27.0.50 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Text/Plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: eliz@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Pj4+ID4gd2hpbGUgc2VuZGluZyBhbiBlLW1haWwgd2l0aCBgbWV3JywgSSBnZXQgdGhlIGF0dGFj aGVkIGNyYXNoLg0KPj4+ID4gVGhpcyBpcyBzb21ldGhpbmcgbmV3LCB3aGljaCBJIGhhdmVuJ3Qg ZXhwZXJpZW5jZWQgYmVmb3JlLiAgSQ0KPj4+ID4gY2FuIHJlbGlhYmx5IHJlcGVhdCBpdC4NCj4g DQo+IEhtbS4uLiBpbmRlZWQgaW5zZXJ0LWZpbGUtY29udGVudHMgdGFrZXMgc29tZSBzaG9ydGN1 dHMgd2hlbg0KPiBpbnNlcnRpbmcgdGhlIG5ldyB0ZXh0IGludG8gdGhlIGJ1ZmZlciB3aGljaCBj YW4gYnJlYWsgdGhlIGV4cGVjdGVkDQo+IGludmFyaWFudHMuDQo+IA0KPiBJIGRvbid0IHJlYWxs eSBrbm93IGhvdyB0byByZXByb2R1Y2UgeW91ciBidWcsIGJ1dCBJIHRoaW5rIEkgaGF2ZSBhbg0K PiBpZGVhIG9mIHdoYXQgbWlnaHQgYmUgZ29pbmcgb24uICBDYW4geW91IHRyeSB0aGUgcGF0Y2gg YmVsb3csIHRvIHNlZQ0KPiBpZiBpdCBmaXhlcyB5b3VyIHByb2JsZW0/DQoNCk5vLCBpdCBkb2Vz bid0IGZpeCB0aGUgcHJvYmxlbSwgdW5mb3J0dW5hdGVseSDigJMgbWV3IGRvZXNuJ3Qgc3RhcnQg YXQNCmFsbCB3aXRoIHRoaXMgcGF0Y2g7IHRoZSBlcnJvciBtZXNzYWdlIHNob3dzIGEgc3RyaW5n IHRoYXQgaXMNCmRlZmluaXRlbHkgY3JvcHBlZC4NCg0KTXkgcHJvYmxlbSBpcyB0aGF0IEkgaGF2 ZSBubyBpZGVhIGhvdyB0byByZXByb2R1Y2UgdGhlIHByb2JsZW0gd2l0aG91dA0KbWV3LiAgV2hh dCBhcHByb2FjaCBkbyB5b3Ugc3VnZ2VzdD8NCg0KDQogICAgV2VybmVyDQo= From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 09:15:13 2019 Received: (at 36431) by debbugs.gnu.org; 30 Jun 2019 13:15:13 +0000 Received: from localhost ([127.0.0.1]:45486 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhZfi-0002bD-8S for submit@debbugs.gnu.org; Sun, 30 Jun 2019 09:15:13 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:20597) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhZfg-0002b0-PK for 36431@debbugs.gnu.org; Sun, 30 Jun 2019 09:15:09 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id D4F2A81157; Sun, 30 Jun 2019 09:15:02 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 82A3980D83; Sun, 30 Jun 2019 09:14:57 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1561900497; bh=J4dEwU2XCrcTcxhpJtJCUN07J27ciFYpXt34GShFblc=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=Y2yANIgIEWuIalJ39XfaM+TEnRD+KjZiz2kNiGigg3y0a1VjCVlLcK9hTHkovqMGE oWYZPhrumwoOHii6RYlLhmxUOZQSZQjPMaTAWv4n0+dYZDJeA90xfDtFFkUe1c2iF7 1A5OyTKkRM+IbrIrAcQCC/jISjit43m8tqT7P3ebPj97tcrtpa6cqVV9GhXF27eTyy +QmY/EKTm/S7E9wOlUhcwynO99PLNejjbdLkz0T4EQTgyivKP3AukIOo6MhI6WmtmF Mprybadx1DPwMBCU8zZ6ew8BHkRhbfjB9KkZEQYSY7a0bhEeBSEUWhI9f5zg0jjRPn w3hWpP3JV/5kg== Received: from pastel (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 47708120592; Sun, 30 Jun 2019 09:14:57 -0400 (EDT) From: Stefan Monnier To: Werner LEMBERG Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <20190630.092626.1377512789964814286.wl@gnu.org> Date: Sun, 30 Jun 2019 09:14:51 -0400 In-Reply-To: <20190630.092626.1377512789964814286.wl@gnu.org> (Werner LEMBERG's message of "Sun, 30 Jun 2019 09:26:26 +0200 (CEST)") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.114 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: eliz@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > No, it doesn't fix the problem, unfortunately =E2=80=93 mew doesn't start= at > all with this patch; the error message shows a string that is > definitely cropped. Yes, I also saw other problems with the patch. I'll send an updated patch later. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 10:40:10 2019 Received: (at 36431) by debbugs.gnu.org; 30 Jun 2019 14:40:10 +0000 Received: from localhost ([127.0.0.1]:47078 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhazy-0004zV-2t for submit@debbugs.gnu.org; Sun, 30 Jun 2019 10:40:10 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50919) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhazv-0004zF-Bx for 36431@debbugs.gnu.org; Sun, 30 Jun 2019 10:40:08 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:34546) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hhazp-0006MN-69; Sun, 30 Jun 2019 10:40:01 -0400 Received: from [176.228.60.248] (port=3197 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hhazl-0008WL-JM; Sun, 30 Jun 2019 10:39:58 -0400 Date: Sun, 30 Jun 2019 17:39:49 +0300 Message-Id: <83ftnrf87e.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Sat, 29 Jun 2019 18:56:53 -0400) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: wl@gnu.org, 36431@debbugs.gnu.org > Date: Sat, 29 Jun 2019 18:56:53 -0400 > > I don't really know how to reproduce your bug, but I think I have an > idea of what might be going on. > Can you try the patch below, to see if it fixes your problem? AFAICT, this patch moves the call to move_gap_both from a fragment where we must decode the inserted text to a fragment where such a decoding might not be necessary. If I'm right, then this makes insert-file-contents slower in some cases, because moving the gap might be very expensive with large buffers. More generally, I'd be leery to make significant changes ion insert-file-contents just to placate that single assertion. What do we gain with that assertion except some theoretical correctness? OTOH, the losses, in stability, performance, and not least our time and energy is (or at least might be) real and tangible. So why bother? From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 10:52:28 2019 Received: (at 36431) by debbugs.gnu.org; 30 Jun 2019 14:52:29 +0000 Received: from localhost ([127.0.0.1]:47086 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhbBs-0005H0-Jn for submit@debbugs.gnu.org; Sun, 30 Jun 2019 10:52:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52855) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhbBq-0005Go-E8 for 36431@debbugs.gnu.org; Sun, 30 Jun 2019 10:52:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:34696) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hhbBl-0003Jx-7A; Sun, 30 Jun 2019 10:52:21 -0400 Received: from [176.228.60.248] (port=3994 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hhbBk-00079m-Jw; Sun, 30 Jun 2019 10:52:21 -0400 Date: Sun, 30 Jun 2019 17:52:13 +0300 Message-Id: <83ef3bf7mq.fsf@gnu.org> From: Eli Zaretskii To: Werner LEMBERG In-reply-to: <20190630.092626.1377512789964814286.wl@gnu.org> (message from Werner LEMBERG on Sun, 30 Jun 2019 09:26:26 +0200 (CEST)) Subject: Re: bug#36431: Crash in marker.c:337 References: <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <20190630.092626.1377512789964814286.wl@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: monnier@iro.umontreal.ca, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sun, 30 Jun 2019 09:26:26 +0200 (CEST) > Cc: eliz@gnu.org, 36431@debbugs.gnu.org > From: Werner LEMBERG > > My problem is that I have no idea how to reproduce the problem without > mew. What approach do you suggest? >From the backtrace, I see that the chain of calls is this: "insert-file-contents" (0xffff8ff8) "apply" (0xffff8ff0) "mew-insert-file-contents2" (0xffff95e8) "apply" (0xffff95e0) "mew-insert-file-contents" (0xffff9a40) "mew-encode-mime-body" (0xffffa250) "mew-encode-singlepart" (0xffffa830) "mew-encode-multipart" (0xffffadb0) "mew-encode-make-multi" (0xffffb2f0) "mew-smtp-encode-message" (0xffffb870) "mew-smtp-encode" (0xffffbd40) "mew-draft-smtp-process-message" (0xffffc2a0) "mew-draft-process-message" (0xffffc790) "mew-draft-send-message" (0xffffce80) So my suggestion would be to try to find out what should be the buffer contents before the call to insert-file-contents, and what should be in the file being inserted, to reproduce the problem, and then post the data needed for the reproduction. If you cannot figure this out on the level of insert-file-contents, try going up the call stack, and find the lowest level where you can figure out the arguments and their values, including the contents of any files needed for the reproduction. Then show the results, and we will try to use the relevant mew code to fill in the blanks. One caveat: your mew customizations might be part of the puzzle, so be sure to spell them out. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 10:59:26 2019 Received: (at 36431) by debbugs.gnu.org; 30 Jun 2019 14:59:26 +0000 Received: from localhost ([127.0.0.1]:47106 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhbIb-0005Rm-Vm for submit@debbugs.gnu.org; Sun, 30 Jun 2019 10:59:26 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:9070) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhbIZ-0005RX-JF for 36431@debbugs.gnu.org; Sun, 30 Jun 2019 10:59:24 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 1AD6181157; Sun, 30 Jun 2019 10:59:18 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id D3C0F81152; Sun, 30 Jun 2019 10:59:12 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1561906752; bh=GCFdPsYDv/My2hrvot7/vqjui97Ij2bkhY4ZHTuo7VE=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=dI6rTP38d035tjUpeNcsDPjmnrKU8fkCAueW2ai2ml1zshSMBhJ5qWkd0nnPRNSQ3 H+gopZyLEHvK08+5CzO8QTZ+I50JdE5FCPy3ga7Aml1n9HI2gevn2UklvbqL4wt/JN Od2mzw9u05ohBgyLWknFQQIiCThc+rh1N9Mr+oYHMsi4qTUMgh95C4wcCa7AvIBKfH Ct9BOEb/CdU3t/6jCarbzItvx4qapoFmz7JwfFlVSC9/zUaqJFMSCrCSynxHDdH/RZ wHLDWqvgp8MCXtYPxl6fm/WZqcEvJUo78N1JhAIGL6bsrU9OZf/lnBc3M+1dcKywtR 4VPyXmqOsSDHQ== Received: from pastel (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 1EA5912020E; Sun, 30 Jun 2019 10:59:12 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> Date: Sun, 30 Jun 2019 10:59:09 -0400 In-Reply-To: <83ftnrf87e.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 30 Jun 2019 17:39:49 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.116 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > AFAICT, this patch moves the call to move_gap_both from a fragment > where we must decode the inserted text to a fragment where such a > decoding might not be necessary. If I'm right, then this makes > insert-file-contents slower in some cases, because moving the gap > might be very expensive with large buffers. Indeed, that's one of the problems, but there are more serious (correctness) problems with it anyway. > More generally, I'd be leery to make significant changes ion > insert-file-contents just to placate that single assertion. I'm still trying to really understand what the code is doing, but so far I get the impression that there are real bugs there, so I'm not really concerned with resolving the assertion but with fixing those bugs. Of course, to do that, I first need to understand the code (which maybe will convince me that there aren't any bugs after all). Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 11:17:07 2019 Received: (at 36431) by debbugs.gnu.org; 30 Jun 2019 15:17:07 +0000 Received: from localhost ([127.0.0.1]:47128 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhbZg-0005tD-GW for submit@debbugs.gnu.org; Sun, 30 Jun 2019 11:17:04 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56647) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhbZe-0005sg-9Q for 36431@debbugs.gnu.org; Sun, 30 Jun 2019 11:17:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35037) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hhbZW-0006dF-Qt; Sun, 30 Jun 2019 11:16:56 -0400 Received: from [176.228.60.248] (port=1517 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hhbZS-0000Uz-8d; Sun, 30 Jun 2019 11:16:54 -0400 Date: Sun, 30 Jun 2019 18:16:43 +0300 Message-Id: <838stjf6hw.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Sun, 30 Jun 2019 10:59:09 -0400) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: wl@gnu.org, 36431@debbugs.gnu.org > Date: Sun, 30 Jun 2019 10:59:09 -0400 > > > More generally, I'd be leery to make significant changes ion > > insert-file-contents just to placate that single assertion. > > I'm still trying to really understand what the code is doing, but so far > I get the impression that there are real bugs there, so I'm not really > concerned with resolving the assertion but with fixing those bugs. > Of course, to do that, I first need to understand the code (which maybe > will convince me that there aren't any bugs after all). I'm not sure those are bugs. When we manipulate the gap, the buffer is left in inconsistent state for short periods of time, and that is normal AFAIU. From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 30 11:54:05 2019 Received: (at 36431) by debbugs.gnu.org; 30 Jun 2019 15:54:05 +0000 Received: from localhost ([127.0.0.1]:47164 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhc9V-0000Rr-Ba for submit@debbugs.gnu.org; Sun, 30 Jun 2019 11:54:05 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:60404) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhc9T-0000RN-LU for 36431@debbugs.gnu.org; Sun, 30 Jun 2019 11:54:04 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id B709481157; Sun, 30 Jun 2019 11:53:56 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id A91FC80D83; Sun, 30 Jun 2019 11:53:50 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1561910030; bh=+A/CcTwT7eHtf+p0C4x9VZ+E1cJ0TX7VJ5JdIrijeOo=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=pqfd/k2iFsZ3zYWH64spBR61NZ+GSSO5a2YEElmZNfY3znR9gTERVegvbWkz/JbgI 3qKWNpeVVt30cATECDJ85jsI1m1Rs36IbV2q+c1bBgivYVJo/luoyMaXOVNDyFxSuN SB1sK6btBJsd6M1rF0uA5Fw9Ii5G6OcmvOauKMBT+WMdQJBdHkLrPxhScRUkU5rot6 IRKaj8BfJKjutOoHR2fOExFd9+rYvw28MyxgoCw0A2mmrKHISfjbTwWFvxXuT3lyug 3T980M+PAAWngKHqWQH/+iGKXEQUV6diTzuiDKD2AK3dONVgzVhfrgId5Q8pVwrO0m R4AaWtOKvDK/g== Received: from alfajor (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 25582120BEE; Sun, 30 Jun 2019 11:53:50 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <838stjf6hw.fsf@gnu.org> Date: Sun, 30 Jun 2019 11:53:41 -0400 In-Reply-To: <838stjf6hw.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 30 Jun 2019 18:16:43 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.109 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > I'm not sure those are bugs. Neither am I. > When we manipulate the gap, the buffer is left in inconsistent state > for short periods of time, and that is normal AFAIU. Yes, it's OK as long as it doesn't "escape". That's what I'm not quite sure of yet. Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 12:29:47 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 16:29:47 +0000 Received: from localhost ([127.0.0.1]:47776 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiLf8-0003Pd-MJ for submit@debbugs.gnu.org; Tue, 02 Jul 2019 12:29:47 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:16966) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiLf6-0003PO-4T for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 12:29:44 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id DC61744445C; Tue, 2 Jul 2019 12:29:37 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id E191344445A; Tue, 2 Jul 2019 12:29:35 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562084975; bh=MALSdmTD8IuF81nVMjWxkQjt9qlXNJIyqIuxw0vYfXA=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=PwHwdqxZrB0FDsoMLYGKudLvxr7YSfJCxtq4ZRpsR0vXguGg3HieEggzhaQD4SuHA pYvdGVOL+HeFsnCbJat8HXVyRgLUOkYa6xfS2eUNCwBE7a4zjK0oaoSYBjf2+zyQiZ 8bQ5cUQuziS55C3NRYg+3H+YVjKv4JuwjKdMgwiYFAKnc08O3Bu25YTHmO12IaJamX c7zIx1C6V3P8GhX2Umk/2BqA7H1ReNSqqI3mXY5R1I6pOz6fwuEl+DmTz6JaKiqLnC +IM0S921FMMO7oxmw7wTkfQtOkOZyebqDcT6wzQZEGJtxQncGo0AO/dSo0f8qrkAkv f28Qt09Z3LljA== Received: from pastel (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 6FC4E120BC2; Tue, 2 Jul 2019 12:29:35 -0400 (EDT) From: Stefan Monnier To: Werner LEMBERG Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <20190630.092626.1377512789964814286.wl@gnu.org> Date: Tue, 02 Jul 2019 12:29:34 -0400 In-Reply-To: (Stefan Monnier's message of "Sun, 30 Jun 2019 09:14:51 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.261 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: eliz@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Stefan Monnier writes: >> No, it doesn't fix the problem, unfortunately =E2=80=93 mew doesn't star= t at >> all with this patch; the error message shows a string that is >> definitely cropped. > > Yes, I also saw other problems with the patch. > I'll send an updated patch later. Can you try the patch below? Stefan =20=20=20=20=20=20=20=20 diff --git a/src/fileio.c b/src/fileio.c index ed1d2aedf3..1fea93fa8e 100644 --- a/src/fileio.c +++ b/src/fileio.c @@ -3741,6 +3741,7 @@ by calling `format-decode', which see. */) CHECK_CODING_SYSTEM (Vcoding_system_for_read); Fset (Qbuffer_file_coding_system, Vcoding_system_for_read); } + eassert (inserted =3D=3D 0); goto notfound; } =20 @@ -3767,7 +3768,10 @@ by calling `format-decode', which see. */) not_regular =3D 1; =20 if (! NILP (visit)) - goto notfound; + { + eassert (inserted =3D=3D 0); + goto notfound; + } =20 if (! NILP (replace) || ! NILP (beg) || ! NILP (end)) xsignal2 (Qfile_error, @@ -4435,19 +4439,6 @@ by calling `format-decode', which see. */) if (how_much < 0) report_file_error ("Read error", orig_filename); =20 - /* Make the text read part of the buffer. */ - GAP_SIZE -=3D inserted; - GPT +=3D inserted; - GPT_BYTE +=3D inserted; - ZV +=3D inserted; - ZV_BYTE +=3D inserted; - Z +=3D inserted; - Z_BYTE +=3D inserted; - - if (GAP_SIZE > 0) - /* Put an anchor to ensure multi-byte form ends at gap. */ - *GPT_ADDR =3D 0; - notfound: =20 if (NILP (coding_system)) @@ -4457,6 +4448,7 @@ by calling `format-decode', which see. */) =20 Note that we can get here only if the buffer was empty before the insertion. */ + eassert (Z =3D=3D BEG); =20 if (!NILP (Vcoding_system_for_read)) coding_system =3D Vcoding_system_for_read; @@ -4477,6 +4469,10 @@ by calling `format-decode', which see. */) bset_undo_list (current_buffer, Qt); record_unwind_protect (decide_coding_unwind, unwind_data); =20 + /* Make the text read part of the buffer. */ + eassert (NILP (BVAR (current_buffer, enable_multibyte_characters= ))); + insert_from_gap_1 (inserted, inserted, false); + if (inserted > 0 && ! NILP (Vset_auto_coding_function)) { coding_system =3D call2 (Vset_auto_coding_function, @@ -4493,8 +4489,22 @@ by calling `format-decode', which see. */) if (CONSP (coding_system)) coding_system =3D XCAR (coding_system); } - unbind_to (count1, Qnil); + + /* Move the text back to the beginning of the gap. + Do it now, before we set the buffer back to multibyte, since = the + bytes may very well not be valid for a multibyte buffer. */ + set_point_both (BEG, BEG_BYTE); + /* In general this may have to move all the bytes, but here + this can't move more bytes than were moved during the executi= on + of Vset_auto_coding_function, which is normally 0 (because it + normally doesn't modify the buffer). */ + move_gap_both (Z, Z_BYTE); inserted =3D Z_BYTE - BEG_BYTE; + GAP_SIZE +=3D inserted; + ZV =3D Z =3D GPT =3D BEG; + ZV_BYTE =3D Z_BYTE =3D GPT_BYTE =3D BEG_BYTE; + + unbind_to (count1, Qnil); } =20 if (NILP (coding_system)) @@ -4528,22 +4538,29 @@ by calling `format-decode', which see. */) } } =20 - coding.dst_multibyte =3D ! NILP (BVAR (current_buffer, enable_multibyte_= characters)); + eassert (PT =3D=3D GPT); + + coding.dst_multibyte + =3D ! NILP (BVAR (current_buffer, enable_multibyte_characters)); if (CODING_MAY_REQUIRE_DECODING (&coding) && (inserted > 0 || CODING_REQUIRE_FLUSHING (&coding))) { - move_gap_both (PT, PT_BYTE); - GAP_SIZE +=3D inserted; - ZV_BYTE -=3D inserted; - Z_BYTE -=3D inserted; - ZV -=3D inserted; - Z -=3D inserted; + /* Now we have all the new bytes at the beginning of the gap, + but `decode_coding_gap` needs them at the end of the gap, so + we need to move them. + FIXME: We should arrange for the bytes to be already at the right + place so we don't need to memmove them in the common case! */ + memmove (GAP_END_ADDR - inserted, GPT_ADDR, inserted); decode_coding_gap (&coding, inserted, inserted); inserted =3D coding.produced_char; coding_system =3D CODING_ID_NAME (coding.id); } else if (inserted > 0) { + /* Make the text read part of the buffer. */ + eassert (NILP (BVAR (current_buffer, enable_multibyte_characters))); + insert_from_gap_1 (inserted, inserted, false); + invalidate_buffer_caches (current_buffer, PT, PT + inserted); adjust_after_insert (PT, PT_BYTE, PT + inserted, PT_BYTE + inserted, inserted); diff --git a/src/insdel.c b/src/insdel.c index 85fffd8fd1..51371ee13c 100644 --- a/src/insdel.c +++ b/src/insdel.c @@ -115,7 +115,7 @@ gap_left (ptrdiff_t charpos, ptrdiff_t bytepos, bool ne= wgap) i =3D GPT_BYTE; to =3D GAP_END_ADDR; from =3D GPT_ADDR; - new_s1 =3D GPT_BYTE; + new_s1 =3D GPT_BYTE; /* May point in the middle of multibyte sequences. = */ =20 /* Now copy the characters. To move the gap down, copy characters up. */ @@ -133,6 +133,7 @@ gap_left (ptrdiff_t charpos, ptrdiff_t bytepos, bool ne= wgap) make_gap_smaller set inhibit-quit. */ if (QUITP) { + /* FIXME: This can point in the middle of a multibyte character.= */ bytepos =3D new_s1; charpos =3D BYTE_TO_CHAR (bytepos); break; @@ -164,7 +165,7 @@ gap_right (ptrdiff_t charpos, ptrdiff_t bytepos) { register unsigned char *to, *from; register ptrdiff_t i; - ptrdiff_t new_s1; + ptrdiff_t new_s1; /* May point in the middle of multibyte sequences. */ =20 BUF_COMPUTE_UNCHANGED (current_buffer, charpos, GPT); =20 @@ -189,6 +190,7 @@ gap_right (ptrdiff_t charpos, ptrdiff_t bytepos) make_gap_smaller set inhibit-quit. */ if (QUITP) { + /* FIXME: This can point in the middle of a multibyte character.= */ bytepos =3D new_s1; charpos =3D BYTE_TO_CHAR (bytepos); break; @@ -1072,6 +1074,31 @@ insert_from_string_1 (Lisp_Object string, ptrdiff_t = pos, ptrdiff_t pos_byte, /* Insert a sequence of NCHARS chars which occupy NBYTES bytes starting at GAP_END_ADDR - NBYTES (if text_at_gap_tail) and at + GPT_ADDR (if not text_at_gap_tail). + Contrary to insert_from_gap, this does not invalidate any cache, + nor update any markers, nor record any buffer modification information + of any sort. */ +void +insert_from_gap_1 (ptrdiff_t nchars, ptrdiff_t nbytes, bool text_at_gap_ta= il) +{ + GAP_SIZE -=3D nbytes; + if (! text_at_gap_tail) + { + GPT +=3D nchars; + GPT_BYTE +=3D nbytes; + } + ZV +=3D nchars; + Z +=3D nchars; + ZV_BYTE +=3D nbytes; + Z_BYTE +=3D nbytes; + + /* Put an anchor to ensure multi-byte form ends at gap. */ + if (GAP_SIZE > 0) *(GPT_ADDR) =3D 0; + eassert (GPT <=3D GPT_BYTE); +} + +/* Insert a sequence of NCHARS chars which occupy NBYTES bytes + starting at GAP_END_ADDR - NBYTES (if text_at_gap_tail) and at GPT_ADDR (if not text_at_gap_tail). */ =20 void @@ -1090,19 +1117,7 @@ insert_from_gap (ptrdiff_t nchars, ptrdiff_t nbytes,= bool text_at_gap_tail) record_insert (GPT, nchars); modiff_incr (&MODIFF); =20 - GAP_SIZE -=3D nbytes; - if (! text_at_gap_tail) - { - GPT +=3D nchars; - GPT_BYTE +=3D nbytes; - } - ZV +=3D nchars; - Z +=3D nchars; - ZV_BYTE +=3D nbytes; - Z_BYTE +=3D nbytes; - if (GAP_SIZE > 0) *(GPT_ADDR) =3D 0; /* Put an anchor. */ - - eassert (GPT <=3D GPT_BYTE); + insert_from_gap_1 (nchars, nbytes, text_at_gap_tail); =20 adjust_overlays_for_insert (ins_charpos, nchars); adjust_markers_for_insert (ins_charpos, ins_bytepos, diff --git a/src/lisp.h b/src/lisp.h index a0619e64f2..1a1d8ee7e4 100644 --- a/src/lisp.h +++ b/src/lisp.h @@ -3667,6 +3667,7 @@ extern void insert (const char *, ptrdiff_t); extern void insert_and_inherit (const char *, ptrdiff_t); extern void insert_1_both (const char *, ptrdiff_t, ptrdiff_t, bool, bool, bool); +extern void insert_from_gap_1 (ptrdiff_t, ptrdiff_t, bool text_at_gap_tail= ); extern void insert_from_gap (ptrdiff_t, ptrdiff_t, bool text_at_gap_tail); extern void insert_from_string (Lisp_Object, ptrdiff_t, ptrdiff_t, ptrdiff_t, ptrdiff_t, bool); From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:04:50 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 17:04:50 +0000 Received: from localhost ([127.0.0.1]:47820 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMD4-0006XF-E1 for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:04:50 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:32776) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMD2-0006Wg-IE for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 13:04:49 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 5E532100B3B; Tue, 2 Jul 2019 13:04:42 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id B289110087A; Tue, 2 Jul 2019 13:04:40 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562087080; bh=d3NWP9hWyyv4t7QV++Ay9ve7Z0MhHrJuSMgCiZ5Yj5A=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=eU+19RA/L6DUHpWU71+LHT83HB/UuuePu2qqWnZ+HFhtPLK7O75u85E+XseHGKri3 pmJL6M1F8v498i5+TGUOsx38Sm7+IEvtO5+8Lojy0EmnTzz2SemmzkFq8M2syIC6xo ycKBNMV8+d2SEXiuzU7fd9NGgIVMlJahmp2y/vywmRiwpZQE5r5lttt9QLh7K0ADnx B9hParGlNphgw5tPe6NISvSkKvuwSDDXATDRqDl4Rlze3riyToZq8wMV0Vhzqo2ME0 Et09m2JioOphB9IuAWIXRicf0bGIBQA/rIZWHlbSrdR8lXDCUh+NML/+W3iVEKPRiC TwvVVD87HzRDw== Received: from pastel (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 7813C1201A4; Tue, 2 Jul 2019 13:04:40 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> Date: Tue, 02 Jul 2019 13:04:38 -0400 In-Reply-To: <83ftnrf87e.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 30 Jun 2019 17:39:49 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.096 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >> I don't really know how to reproduce your bug, but I think I have an >> idea of what might be going on. >> Can you try the patch below, to see if it fixes your problem? > AFAICT, this patch moves the call to move_gap_both from a fragment > where we must decode the inserted text to a fragment where such a > decoding might not be necessary. If I'm right, then this makes > insert-file-contents slower in some cases, because moving the gap > might be very expensive with large buffers. Indeed. It also removed the move_gap_both from the case where we need to decode and we already know the coding-system to use. So in some cases it made it faster (these are the cases where it misbehaved). The new version of the code shouldn't suffer from this performance problem (it still calls move_gap_both in the set-auto-coding part of the code, but this call should have a cost proportional to the amount of buffer modification performed by set-auto-coding, i.e. it should be a nop in pretty much all cases). Looking at this aspect (i.e. not directly related to this bug) I'm wondering why the code works this way: We start by inserting the new bytes at the *beginning* of the gap, but when we do the move_gap_both this moves those bytes to the *end* of the gap (where decode_coding_gap expects them, apparently), so when we decode we always have to move all the inserted bytes, right? > More generally, I'd be leery to make significant changes ion > insert-file-contents just to placate that single assertion. What do > we gain with that assertion except some theoretical correctness? Again, I'm just trying to understand the code at this point. The part that worries me is the following: In the current code, we read the raw bytes to the beginning of the gap, then (when Vset_auto_coding_function needs to be called), we (virtually) move them into the current buffer, which is usually multibyte. AFAICT at this point we have a buffer in a transiently inconsistent state since it's multibyte but it can contain arbitrary byte sequences, hence invalid byte sequences. Before calling Vset_auto_coding_function we make this buffer unibyte, which brings us back to a consistent state, but I wonder if/how/why making the buffer unibyte and then back to multibyte always preserves the original byte sequence, since AFAICT set-buffer-multibyte will always make the effort to bring the buffer to a consistent state, so if the state is inconsistent before the pair of calls to set-buffer-multibyte, either we changed the byte sequence or set-buffer-multibyte doesn't always result in a consistent state. What am I missing? Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:22:35 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 17:22:35 +0000 Received: from localhost ([127.0.0.1]:47829 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMUD-0007MN-AZ for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:22:34 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:16732) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMUB-0007M0-3q for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 13:22:31 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id E47B181176; Tue, 2 Jul 2019 13:22:24 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id B36AA80B8C; Tue, 2 Jul 2019 13:22:23 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562088143; bh=HnZZxyITLvE/oUTIoxpzwpXpyW+2aLtqbjR63kyhMmI=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=mMxe75zoTgEpvy/BdtweRne/9+VfkjnsVLZfyFmNz/T2d2VFjo+s9sgIt0ekl7UPk AAVvQDzjLv+Ob1rB+3SlmIM1dQQQmM1JTkkWvsQysimj8+N51TeLazkoouH985yaL0 BMdWavu10lpIIyMcI+JPg+Aj1iJViorbiR77yPYZ4Du8zL8/2Sb6b9ox443GG4PuDU gxkEo3oB05smZ+Gs80aBzTHrv1q9CKDUoOgqPK5dCIKg/oclvERNe3OqgB3PywqIwZ i/kaHp7JBg9x6LUHRoqnWdvfI69kBD2p361i3r676OzWsXFpE19ejU9aQbkTiC1z6Y /S98k/nTi6g4g== Received: from pastel (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 810E0120B3F; Tue, 2 Jul 2019 13:22:23 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> Date: Tue, 02 Jul 2019 13:22:22 -0400 In-Reply-To: (Stefan Monnier's message of "Tue, 02 Jul 2019 13:04:38 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.108 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > In the current code, we read the raw bytes to the beginning of the gap, > then (when Vset_auto_coding_function needs to be called), we (virtually) > move them into the current buffer, which is usually multibyte. > AFAICT at this point we have a buffer in a transiently inconsistent > state since it's multibyte but it can contain arbitrary byte sequences, > hence invalid byte sequences. Before calling Vset_auto_coding_function > we make this buffer unibyte, which brings us back to a consistent state, > but I wonder if/how/why making the buffer unibyte and then back to > multibyte always preserves the original byte sequence, since AFAICT > set-buffer-multibyte will always make the effort to bring the buffer to > a consistent state, so if the state is inconsistent before the pair of > calls to set-buffer-multibyte, either we changed the byte sequence or > set-buffer-multibyte doesn't always result in a consistent state. > What am I missing? OK, I finally saw that we don't actually call set-buffer-multibyte but instead we just set bset_enable_multibyte_characters. I'm beginning to understand better. Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:38:02 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 17:38:02 +0000 Received: from localhost ([127.0.0.1]:47842 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMjB-00081f-Mm for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:38:01 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:21698) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMj7-00081M-RG for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 13:38:00 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 422C1100A32; Tue, 2 Jul 2019 13:37:50 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 6F660100B4D; Tue, 2 Jul 2019 13:37:48 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562089068; bh=1A5yO6CHsUYDm209Jd8SnReUPEY9F4I0jsR4KX1IHXg=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=pNphPGpseRpNPCFvSNYSpFGwP4RZrNWaoxqJkcmykHGgKXvjVLqxinli4vZyKjYBS WUFzi98fm1B4MDFmy2uB5n7Zo/B7YRwIrVO/VaCK9y/beBNs5TVdmAkNZoQQYPkCte s0ucl151w9n4mcNTeIK0voyIVfLGFzI22DS3XpZ9K9AGn5YrjhbDo0Pj3T8B+4lt8N 832rXRT6JRYYNy38PW/+nUCu3PaFVnkIF2MsuIt8KETFHn5gBXn+P+ZOCE0em8YwwP gTGL4gPulEgBt0vX7J9cqEmWghM3yE86LNl/q6Ud1/mR8lxM3K1Y7+Q2nzIAoJULVY +T3OF1Q3XZMqA== Received: from pastel (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id E8CB3120BFC; Tue, 2 Jul 2019 13:37:47 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> Date: Tue, 02 Jul 2019 13:37:44 -0400 In-Reply-To: (Stefan Monnier's message of "Tue, 02 Jul 2019 13:22:22 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.092 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > OK, I finally saw that we don't actually call set-buffer-multibyte but > instead we just set bset_enable_multibyte_characters. I'm beginning to > understand better. Alright, so here one way to make this "transient" inconsistent state escape its transientness (should it be "transienticity"?) and crash Emacs: (catch 'toto (let ((set-auto-coding-function (lambda (rest _) (throw 'toto nil)))) (erase-buffer) (insert-file-contents "~/tmp/foo-c3c3.txt"))) where `~/tmp/foo-c3c3.txt` is a file that contains some invalid utf-8 byte sequence (in my case I used two bytes C3 together). Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:40:11 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 17:40:11 +0000 Received: from localhost ([127.0.0.1]:47846 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMlH-000855-5g for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:40:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55780) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMlE-00084s-L9 for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 13:40:09 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52949) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hiMl7-0004Kr-NF; Tue, 02 Jul 2019 13:40:03 -0400 Received: from [176.228.60.248] (port=4302 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hiMl5-0007ne-HR; Tue, 02 Jul 2019 13:40:01 -0400 Date: Tue, 02 Jul 2019 20:39:44 +0300 Message-Id: <83v9wkcp3z.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Tue, 02 Jul 2019 13:04:38 -0400) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: wl@gnu.org, 36431@debbugs.gnu.org > Date: Tue, 02 Jul 2019 13:04:38 -0400 > > We start by inserting the new bytes at the *beginning* of the gap, but > when we do the move_gap_both this moves those bytes to the *end* of the > gap (where decode_coding_gap expects them, apparently), so when we > decode we always have to move all the inserted bytes, right? Yes. This is so we don't need to know up front how many bytes will be read from the file. From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:42:52 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 17:42:53 +0000 Received: from localhost ([127.0.0.1]:47850 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMns-00088u-Ms for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:42:52 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56608) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMnr-00088i-67 for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 13:42:51 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52971) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hiMnj-00065v-U6; Tue, 02 Jul 2019 13:42:45 -0400 Received: from [176.228.60.248] (port=4467 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hiMni-00081F-Ia; Tue, 02 Jul 2019 13:42:43 -0400 Date: Tue, 02 Jul 2019 20:42:27 +0300 Message-Id: <83tvc4cozg.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Tue, 02 Jul 2019 13:37:44 -0400) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Alright, so here one way to make this "transient" inconsistent state > escape its transientness (should it be "transienticity"?) and crash Emacs: > > (catch 'toto > (let ((set-auto-coding-function (lambda (rest _) (throw 'toto nil)))) > (erase-buffer) > (insert-file-contents "~/tmp/foo-c3c3.txt"))) > > where `~/tmp/foo-c3c3.txt` is a file that contains some invalid > utf-8 byte sequence (in my case I used two bytes C3 together). You are saying that decide_coding_unwind should do a better job? From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:51:40 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 17:51:40 +0000 Received: from localhost ([127.0.0.1]:47872 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMwO-0008Mv-I3 for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:51:40 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:15450) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiMwN-0008Mi-Gd for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 13:51:39 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 1FCE5444462; Tue, 2 Jul 2019 13:51:33 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id C78E8441B83; Tue, 2 Jul 2019 13:51:31 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562089891; bh=3NHfX4Vw93MTGiiRvaWWfSmCqD75bDdHS+a+XYpCSwE=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=L9n7Rtmulwq/xREiGs2y5V80WyP3KfF9zhvOrJzIfYPE80kvQsHgQKhWvDbF5Os3g KXflePaxeCzwwJaiVv94lYs2F3TecExEk1m/gxOSpvo2SYzL3Ex1P8Ky9cCMC4eXhJ fNbIxOJPis+AQdwK4P1smrowDgROvE32qT0ciKn3lxbdrhOWnWSxICPfJ60mHxlJFh qp2Q5fPkeR7v9vNMLOIXWfLtgj5GDZThqg93ke1hdEYE5PAZzOqhUpnQ6S/KyJzgWe KPUX591M8HKiK+E/tmCrfJ3Lhppcawk8+X5qKAw+06CLYZ+1EuDPiBVyZtLQjw3me1 0H7xSuy4/b/uw== Received: from pastel (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 8DFD11205E0; Tue, 2 Jul 2019 13:51:31 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <83v9wkcp3z.fsf@gnu.org> Date: Tue, 02 Jul 2019 13:51:30 -0400 In-Reply-To: <83v9wkcp3z.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 02 Jul 2019 20:39:44 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.245 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >> We start by inserting the new bytes at the *beginning* of the gap, but >> when we do the move_gap_both this moves those bytes to the *end* of the >> gap (where decode_coding_gap expects them, apparently), so when we >> decode we always have to move all the inserted bytes, right? > > Yes. This is so we don't need to know up front how many bytes will be > read from the file. OK, so IIUC: - we insert the new bytes at the beginning of the gap, in order to have room to grow if there are more bytes than expected, and also in case there are fewer bytes than expected (in which case we'd otherwise have to move the bytes we just read so they properly end at the end of the gap). - decode_coding_gap wants the new input bytes to be at the end of the gap, so that we can put the decoded chars at the beginning of the gap and as one grows the other shrinks, so we don't need space for "IN + OUT" bytes but only for "OUT" bytes. Is that right (I'm trying to find some comment or other evidence that this is the case, but haven't found it yet). IOW, it should be possible to optimize the common case by reading the new bytes into the end of the gap to avoid moving everything in the common case (if the number of bytes read is different from originally expected, we'll have to do extra work, but for the common case where we know the file size upfront and it doesn't change while we read it, this will save us some work). But the effort is probably not worth the trouble: a memmove of a few gigabytes costs relatively little compared to the cost of actually decoding those same gigabytes. Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 13:55:36 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 17:55:36 +0000 Received: from localhost ([127.0.0.1]:47880 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN0C-0008T7-JJ for submit@debbugs.gnu.org; Tue, 02 Jul 2019 13:55:36 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:10621) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiN0B-0008St-8a for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 13:55:35 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 28FDF100B3B; Tue, 2 Jul 2019 13:55:29 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id EECFE1009BB; Tue, 2 Jul 2019 13:55:27 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562090128; bh=FZuWYVKiojycsN+kQxgtPPsUCKG6oGn5XLYhtjrPypk=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=HZ9zW3148tg6dZMis1swsGAsJ7AiY9FRkrORQ8rOksx8M8YBAkzPHIhrZnBSlcwtG zrIOMB3ud8jyvgEXtk1dM55YgFasQsjWqofWYN9GPyLe9/+dSS5Vje44Av1pyiFQ8Z +tGqwpyPGz0rVoWOnyleceCIIGUjPvW8o9bnC0c6LdZyLHsbTzJHagIuDpxWHn3RYT GKKeET9hAJLgfu6Ag9UWE1RH15S9DR4sqQYSlDtEry0n0wpTOhchHTeif8Gw8PpeTJ JTVgHncQq3BSS+56GtXKZwFF6y00uE0nmUEpK2IaXhngfAo6bkkh9YEOv9gupvBxYh zGiXDDxUBKsGQ== Received: from pastel (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id C1BD51205E0; Tue, 2 Jul 2019 13:55:27 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <83tvc4cozg.fsf@gnu.org> Date: Tue, 02 Jul 2019 13:55:26 -0400 In-Reply-To: <83tvc4cozg.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 02 Jul 2019 20:42:27 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.095 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >> Alright, so here one way to make this "transient" inconsistent state >> escape its transientness (should it be "transienticity"?) and crash Emacs: >> >> (catch 'toto >> (let ((set-auto-coding-function (lambda (rest _) (throw 'toto nil)))) >> (erase-buffer) >> (insert-file-contents "~/tmp/foo-c3c3.txt"))) >> >> where `~/tmp/foo-c3c3.txt` is a file that contains some invalid >> utf-8 byte sequence (in my case I used two bytes C3 together). > You are saying that decide_coding_unwind should do a better job? Not sure yet. In any case, it's a rather contrived situation. Stefan From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 14:27:52 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 18:27:53 +0000 Received: from localhost ([127.0.0.1]:47918 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiNVO-0000os-47 for submit@debbugs.gnu.org; Tue, 02 Jul 2019 14:27:50 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45360) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiNVL-0000og-BQ for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 14:27:47 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53556) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hiNVF-0003Ax-6N; Tue, 02 Jul 2019 14:27:41 -0400 Received: from [176.228.60.248] (port=3212 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hiNVA-0004rT-N0; Tue, 02 Jul 2019 14:27:38 -0400 Date: Tue, 02 Jul 2019 21:27:15 +0300 Message-Id: <83sgrocmws.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Tue, 02 Jul 2019 13:51:30 -0400) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <83v9wkcp3z.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: wl@gnu.org, 36431@debbugs.gnu.org > Date: Tue, 02 Jul 2019 13:51:30 -0400 > > - we insert the new bytes at the beginning of the gap, in order to have > room to grow if there are more bytes than expected, and also in case > there are fewer bytes than expected (in which case we'd otherwise > have to move the bytes we just read so they properly end at the end > of the gap). Also, you will see in insert-file-contents that it supports quitting while reading a huge file, and also the REPLACE argument, where we detect the same contents at beginning and end of the file and the buffer. > - decode_coding_gap wants the new input bytes to be at the end of the > gap, so that we can put the decoded chars at the beginning of the gap > and as one grows the other shrinks, so we don't need space for "IN + > OUT" bytes but only for "OUT" bytes. Is that right (I'm trying to > find some comment or other evidence that this is the case, but > haven't found it yet). That's right. The comment you are looking for (well, at least part of it) is in the commentary before decode_coding, where it explains the semantics of CODING->src_pos. You will see at the beginning of decode_coding_gap how it sets things up according to that hairy protocol. > IOW, it should be possible to optimize the common case by reading the > new bytes into the end of the gap to avoid moving everything in the > common case (if the number of bytes read is different from originally > expected, we'll have to do extra work, but for the common case where we > know the file size upfront and it doesn't change while we read it, this > will save us some work). > > But the effort is probably not worth the trouble: a memmove of a few > gigabytes costs relatively little compared to the cost of actually > decoding those same gigabytes. Right. Also, there are the other subtle issues with quitting, the REPLACE argument, special files, etc. From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 15:44:20 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 19:44:20 +0000 Received: from localhost ([127.0.0.1]:48001 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiOhP-0006yc-Gz for submit@debbugs.gnu.org; Tue, 02 Jul 2019 15:44:19 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:11048) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiOhN-0006yO-PF for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 15:44:18 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 9B78A81176; Tue, 2 Jul 2019 15:44:10 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 1C8D380B8C; Tue, 2 Jul 2019 15:44:09 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562096649; bh=Cuc8CapowAddY23zo6kPbHsg1GGCkP8lOE3/Oqe2c6M=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=ewRPjei+FONtmJYkImeu4HfsCmpKIIihZupKJlALK+53qcPfU19DDalrJ0vb/8AZc 6Q+kfiVj72NnqvjjIBcV8MI8pS8sndNk9pKmk73CqkjJEATbew6DQJW+qwWImkAAzk SGoSdGd7vVXkkXc7V5SlvlcacPd+XZfxo2nqBi32Tm2K8Fv+ys7fRvpfN18VtABaAZ ezzJpbWLJVeb8XkmVSbgDF8ruXRsqfP/UzdbkndE/bMg7CZl7h7KVlRoQ9R7E1RjUA JaueepOe2ko5DiXLMw1p31GfpgVXOgAgXNDHZ40YZ4xhrzXdGsj7EC16DrKKeKtz4H 6BCIfM80Z4Y8A== Received: from pastel (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id B8834120B88; Tue, 2 Jul 2019 15:44:08 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <83v9wkcp3z.fsf@gnu.org> <83sgrocmws.fsf@gnu.org> Date: Tue, 02 Jul 2019 15:44:07 -0400 In-Reply-To: <83sgrocmws.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 02 Jul 2019 21:27:15 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.603 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years SCC_5_SHORT_WORD_LINES 1 5 lines with many short words X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >> - we insert the new bytes at the beginning of the gap, in order to have >> room to grow if there are more bytes than expected, and also in case >> there are fewer bytes than expected (in which case we'd otherwise >> have to move the bytes we just read so they properly end at the end >> of the gap). > > Also, you will see in insert-file-contents that it supports quitting > while reading a huge file, and also the REPLACE argument, where we > detect the same contents at beginning and end of the file and the > buffer. Right, tho the end result is the same (e.g. when we quit, we can either abort the whole operation and trow away the bytes we read, or we can keep going with the bytes we did read which is simply another case of reading less than expected). >> - decode_coding_gap wants the new input bytes to be at the end of the >> gap, so that we can put the decoded chars at the beginning of the gap >> and as one grows the other shrinks, so we don't need space for "IN + >> OUT" bytes but only for "OUT" bytes. Is that right (I'm trying to >> find some comment or other evidence that this is the case, but >> haven't found it yet). > > That's right. The comment you are looking for (well, at least part of > it) is in the commentary before decode_coding, where it explains the > semantics of CODING->src_pos. You will see at the beginning of > decode_coding_gap how it sets things up according to that hairy > protocol. IIUC you're referring to this comment: Decode the data at CODING->src_object into CODING->dst_object. CODING->src_object is a buffer, a string, or nil. CODING->dst_object is a buffer. If CODING->src_object is a buffer, it must be the current buffer. In this case, if CODING->src_pos is positive, it is a position of the source text in the buffer, otherwise, the source text is in the gap area of the buffer, and CODING->src_pos specifies the offset of the text from GPT (which must be the same as PT). If this is the same buffer as CODING->dst_object, CODING->src_pos must be negative. If CODING->src_object is a string, CODING->src_pos is an index to that string. If CODING->src_object is nil, CODING->source must already point to the non-relocatable memory area. In this case, CODING->src_pos is an offset from CODING->source. The decoded data is inserted at the current point of the buffer CODING->dst_object. but this doesn't say if the bytes are to be found originally at the beginning of the gap or its end, nor whether they finish at the beginning or the end, nor what happens in the middle and why it's been designed this way. Is the patch below correct? >> IOW, it should be possible to optimize the common case by reading the >> new bytes into the end of the gap to avoid moving everything in the >> common case (if the number of bytes read is different from originally >> expected, we'll have to do extra work, but for the common case where we >> know the file size upfront and it doesn't change while we read it, this >> will save us some work). >> >> But the effort is probably not worth the trouble: a memmove of a few >> gigabytes costs relatively little compared to the cost of actually >> decoding those same gigabytes. > > Right. Also, there are the other subtle issues with quitting, the > REPLACE argument, special files, etc. I think the crash-example I sent can probably be made less esoteric by making it use "quit" instead of catch/throw. I'm beginning to think that when we quit (or signal an error) from within set-auto-coding-function, we simply shouldn't revert the buffer to multibyte. Stefan diff --git a/src/coding.c b/src/coding.c index 5b9bfa17dd..218d69e2e7 100644 --- a/src/coding.c +++ b/src/coding.c @@ -7322,11 +7322,16 @@ produce_annotation (struct coding_system *coding, ptrdiff_t pos) If CODING->src_object is a buffer, it must be the current buffer. In this case, if CODING->src_pos is positive, it is a position of - the source text in the buffer, otherwise, the source text is in the - gap area of the buffer, and CODING->src_pos specifies the offset of - the text from GPT (which must be the same as PT). If this is the - same buffer as CODING->dst_object, CODING->src_pos must be - negative. + the source text in the buffer, otherwise, the source text is at the + end of the gap area of the buffer, and CODING->src_pos specifies the + offset of the text from the end of the gap (which must be the at PT). + If this is the same buffer as CODING->dst_object, CODING->src_pos must + be negative. + + When the text is taken from the gap, it needs to be at the end of + the gap so that we can produce the decoded text at the beginning of + the gap: this way, as the output grows, the input shrinks, so we only + need to allocate enough space for `max(IN, OUT)` instead of `IN + OUT`. If CODING->src_object is a string, CODING->src_pos is an index to that string. From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 16:16:13 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 20:16:13 +0000 Received: from localhost ([127.0.0.1]:48020 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiPCG-0007kB-QS for submit@debbugs.gnu.org; Tue, 02 Jul 2019 16:16:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55186) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiPCC-0007jw-VB for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 16:16:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55649) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hiPC4-0006ls-N9; Tue, 02 Jul 2019 16:16:01 -0400 Received: from [176.228.60.248] (port=1854 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hiPC4-00014j-0x; Tue, 02 Jul 2019 16:16:00 -0400 Date: Tue, 02 Jul 2019 23:15:45 +0300 Message-Id: <83pnmschvy.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Tue, 02 Jul 2019 15:44:07 -0400) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <83v9wkcp3z.fsf@gnu.org> <83sgrocmws.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: wl@gnu.org, 36431@debbugs.gnu.org > Date: Tue, 02 Jul 2019 15:44:07 -0400 > > Decode the data at CODING->src_object into CODING->dst_object. > CODING->src_object is a buffer, a string, or nil. > CODING->dst_object is a buffer. > > If CODING->src_object is a buffer, it must be the current buffer. > In this case, if CODING->src_pos is positive, it is a position of > the source text in the buffer, otherwise, the source text is in the > gap area of the buffer, and CODING->src_pos specifies the offset of > the text from GPT (which must be the same as PT). If this is the > same buffer as CODING->dst_object, CODING->src_pos must be > negative. > [...] > The decoded data is inserted at the current point of the buffer > CODING->dst_object. > > but this doesn't say if the bytes are to be found originally at the > beginning of the gap or its end, nor whether they finish at the beginning or > the end, nor what happens in the middle and why it's been designed this way. It says that (a) CODING->src_pos is the negative of the offset from GPT of where the bytes are in the gap (they don't have to be "at the end", AFAIU, just not "at the beginning"); and (b) that the decoded text is inserted at point. And those are, AFAIK, the only real conditions, all the rest is not necessary, it's just what's convenient. As for why this was designed like that -- where else did you see comments in Emacs that answer this kind of questions? > Is the patch below correct? I think it describes conditions that don't need to exist. > I think the crash-example I sent can probably be made less esoteric by > making it use "quit" instead of catch/throw. I'm beginning to think > that when we quit (or signal an error) from within > set-auto-coding-function, we simply shouldn't revert the buffer > to multibyte. We have code whose purpose is to recover from such calamities, so if it doesn't do its job in all cases, we need to augment it. From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 02 17:00:43 2019 Received: (at 36431) by debbugs.gnu.org; 2 Jul 2019 21:00:43 +0000 Received: from localhost ([127.0.0.1]:48067 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiPtL-0002Wy-3O for submit@debbugs.gnu.org; Tue, 02 Jul 2019 17:00:43 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:20412) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiPtH-0002Wi-Nz for 36431@debbugs.gnu.org; Tue, 02 Jul 2019 17:00:40 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 91544100B3B; Tue, 2 Jul 2019 17:00:32 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 1A9A51009BB; Tue, 2 Jul 2019 17:00:31 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562101231; bh=bfFJ8askGBpx65ZREkyDepBaopvf4JQDmKjiLs+yJQ0=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=E5Va/pVAR+55Nwbw6MKumJpN8FmBYTzdW5ojCgXhDdqA1dcqyMd4uMbpo834Rfu47 ZcfbO18a92TrwxNEHgPxBXCWm27A+LwT/TVz2qm3SG2ypX0Bm5Ac2C3QOmkgjMH4XG jcxBZ5udhMrHmpi4C/hiNzTc5AMdEeGGDEQDBBQk2jU0lg4tb/QUGcJe0qSPZfQ2f0 3R8ARWnW7K158JVeBfZF9cUKfR+ncEurc3mXEwP11M641Z6yuX8aJ6WQ49fKxZluiw nJkE4LYVsertLPgwfmPmm1HDYsg7x+5z28hGDQSLOsWRtOuvfn5lgCkaHJmUauFAtK kVJ0dlohOQcxA== Received: from alfajor (76-10-151-214.dsl.teksavvy.com [76.10.151.214]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9F9491201F1; Tue, 2 Jul 2019 17:00:30 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <83v9wkcp3z.fsf@gnu.org> <83sgrocmws.fsf@gnu.org> <83pnmschvy.fsf@gnu.org> Date: Tue, 02 Jul 2019 17:00:21 -0400 In-Reply-To: <83pnmschvy.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 02 Jul 2019 23:15:45 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.091 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >> Decode the data at CODING->src_object into CODING->dst_object. >> CODING->src_object is a buffer, a string, or nil. >> CODING->dst_object is a buffer. >> >> If CODING->src_object is a buffer, it must be the current buffer. >> In this case, if CODING->src_pos is positive, it is a position of >> the source text in the buffer, otherwise, the source text is in the >> gap area of the buffer, and CODING->src_pos specifies the offset of >> the text from GPT (which must be the same as PT). If this is the >> same buffer as CODING->dst_object, CODING->src_pos must be >> negative. >> [...] >> The decoded data is inserted at the current point of the buffer >> CODING->dst_object. >> >> but this doesn't say if the bytes are to be found originally at the >> beginning of the gap or its end, nor whether they finish at the beginning or >> the end, nor what happens in the middle and why it's been designed this way. > > It says that (a) CODING->src_pos is the negative of the offset from > GPT of where the bytes are in the gap Yes, I think this is actually wrong. E.e. decode_coding_gap does: coding->src_chars = chars; [...] coding->src_pos = -chars; so nowhere does it account for unspecified number of bytes between the beginning of the gap and the beginning of the source text. Here, `src_pos` is the offset from the end of the gap. > (they don't have to be "at the > end", AFAIU, just not "at the beginning"); Oh, indeed, src_pos doesn't need to start as the negation of src_chars. > As for why this was designed like that -- where else did you see > comments in Emacs that answer this kind of questions? There are such design comments at various places. Usually added after the fact, sometimes added as part of a commit-reversal to make sure someone else doesn't fall into the same trap ;-) >> Is the patch below correct? > I think it describes conditions that don't need to exist. How 'bout this new version. Stefan diff --git a/src/coding.c b/src/coding.c index 59589caee6..fd7e7aca0f 100644 --- a/src/coding.c +++ b/src/coding.c @@ -7323,10 +7323,15 @@ produce_annotation (struct coding_system *coding, ptrdiff_t pos) If CODING->src_object is a buffer, it must be the current buffer. In this case, if CODING->src_pos is positive, it is a position of the source text in the buffer, otherwise, the source text is in the - gap area of the buffer, and CODING->src_pos specifies the offset of - the text from GPT (which must be the same as PT). If this is the - same buffer as CODING->dst_object, CODING->src_pos must be - negative. + gap area of the buffer, and CODING->src_pos specifies the + offset of the text from the end of the gap (which must be at PT). + If this is the same buffer as CODING->dst_object, CODING->src_pos must + be negative. + + When the text is taken from the gap, it can't be at the beginning of + the gap so that we can produce the decoded text at the beginning of + the gap: this way, as the output grows, the input shrinks, so we only + need to allocate enough space for `max(IN, OUT)` instead of `IN + OUT`. If CODING->src_object is a string, CODING->src_pos is an index to that string. From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 00:22:04 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 04:22:05 +0000 Received: from localhost ([127.0.0.1]:48288 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiWmS-0006p8-DA for submit@debbugs.gnu.org; Wed, 03 Jul 2019 00:22:04 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:52166) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiWmR-0006of-Ai for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 00:22:03 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 01FF74444A2; Wed, 3 Jul 2019 00:21:57 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 318224444A0; Wed, 3 Jul 2019 00:21:55 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562127715; bh=GPeNsEL01yCDHpLb/O33WIcbjlzE9sREOSGVIibELZM=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=juR8JAAOCYQ+0EZgrr8Cg+yQjiJ1m2Ogq7fTm6+FHvoAEhHs1I41ZLcc1l1Ve57jy F8YY5q+rD2bEOolwmxSKW51Hmd2sOjxYz18j6YCDUfLhr2RMHZ6m7caBMOavI5eRdF d3cbCUv2mXkEgn0BiNRPrnq86gf3KowGAKV4qK2UyuAbGOHl680IAEo3B7EW1QlWL5 oFWKmQ4v2axHLdtwnjLFRl0YY+e516GjqD7GFMqagu0yX1qlGhO6vuXomHyZD0TYF5 QDcCMiXvK4h749Dx34Txo6MK3IehbouYdtNk6dOsfiDuPV4KoAz7EGvO8wvzvGcojm L7rfPX7gJwBpQ== Received: from alfajor (76-10-141-139.dsl.teksavvy.com [76.10.141.139]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id BF6EA120BF4; Wed, 3 Jul 2019 00:21:54 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> Date: Wed, 03 Jul 2019 00:21:54 -0400 In-Reply-To: <83ftnrf87e.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 30 Jun 2019 17:39:49 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.231 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > AFAICT, this patch moves the call to move_gap_both from a fragment > where we must decode the inserted text to a fragment where such a > decoding might not be necessary. If I'm right, then this makes > insert-file-contents slower in some cases, because moving the gap > might be very expensive with large buffers. Here's an alternative patch which doesn't suffer from this problem but also eliminates the transiently-inconsistent multibyte buffer situation. Stefan diff --git a/src/fileio.c b/src/fileio.c index 2825c1b54c..9ed1fcf8ca 100644 --- a/src/fileio.c +++ b/src/fileio.c @@ -3705,6 +3705,7 @@ because (1) it preserves some marker positions and (2) it puts less data CHECK_CODING_SYSTEM (Vcoding_system_for_read); Fset (Qbuffer_file_coding_system, Vcoding_system_for_read); } + eassert (inserted == 0); goto notfound; } @@ -3731,7 +3732,10 @@ because (1) it preserves some marker positions and (2) it puts less data not_regular = 1; if (! NILP (visit)) - goto notfound; + { + eassert (inserted == 0); + goto notfound; + } if (! NILP (replace) || ! NILP (beg) || ! NILP (end)) xsignal2 (Qfile_error, @@ -4399,10 +4403,10 @@ because (1) it preserves some marker positions and (2) it puts less data if (how_much < 0) report_file_error ("Read error", orig_filename); - /* Make the text read part of the buffer. */ - insert_from_gap_1 (inserted, inserted, false); - - notfound: + notfound: ; + Lisp_Object multibyte + = BVAR (current_buffer, enable_multibyte_characters); + bool ingap = true; /* Bytes are currently in the gap. */ if (NILP (coding_system)) { @@ -4411,6 +4415,7 @@ because (1) it preserves some marker positions and (2) it puts less data Note that we can get here only if the buffer was empty before the insertion. */ + eassert (Z == BEG); if (!NILP (Vcoding_system_for_read)) coding_system = Vcoding_system_for_read; @@ -4421,8 +4426,6 @@ because (1) it preserves some marker positions and (2) it puts less data enable-multibyte-characters directly here without taking care of marker adjustment. By this way, we can run Lisp program safely before decoding the inserted text. */ - Lisp_Object multibyte - = BVAR (current_buffer, enable_multibyte_characters); Lisp_Object undo_list = BVAR (current_buffer, undo_list); ptrdiff_t count1 = SPECPDL_INDEX (); @@ -4430,6 +4433,10 @@ because (1) it preserves some marker positions and (2) it puts less data bset_undo_list (current_buffer, Qt); record_unwind_protect (restore_buffer, Fcurrent_buffer ()); + /* Make the text read part of the buffer. */ + insert_from_gap_1 (inserted, inserted, false); + ingap = false; + if (inserted > 0 && ! NILP (Vset_auto_coding_function)) { coding_system = call2 (Vset_auto_coding_function, @@ -4455,15 +4462,10 @@ because (1) it preserves some marker positions and (2) it puts less data adjust_overlays_for_delete (BEG, Z - BEG); set_buffer_intervals (current_buffer, NULL); TEMP_SET_PT_BOTH (BEG, BEG_BYTE); - - /* Change the buffer's multibyteness directly. We used to do this - from within unbind_to, but it was unsafe since the bytes - may contain invalid sequences for a multibyte buffer (which is OK - here since we'll decode them before anyone else gets to see - them, but is dangerous when we're doing a non-local exit). */ - bset_enable_multibyte_characters (current_buffer, multibyte); bset_undo_list (current_buffer, undo_list); inserted = Z_BYTE - BEG_BYTE; + /* The bytes may be invalid for a multibyte buffer, so we can't + restore the multibyteness yet. */ } if (NILP (coding_system)) @@ -4471,7 +4473,7 @@ because (1) it preserves some marker positions and (2) it puts less data else CHECK_CODING_SYSTEM (coding_system); - if (NILP (BVAR (current_buffer, enable_multibyte_characters))) + if (NILP (multibyte)) /* We must suppress all character code conversion except for end-of-line conversion. */ coding_system = raw_text_coding_system (coding_system); @@ -4490,33 +4492,51 @@ because (1) it preserves some marker positions and (2) it puts less data { /* Visiting a file with these coding system makes the buffer unibyte. */ - if (inserted > 0) + if (!ingap) + multibyte = Qnil; + else if (inserted > 0) bset_enable_multibyte_characters (current_buffer, Qnil); - else + else Fset_buffer_multibyte (Qnil); } } - coding.dst_multibyte = ! NILP (BVAR (current_buffer, enable_multibyte_characters)); + coding.dst_multibyte = !NILP (multibyte); if (CODING_MAY_REQUIRE_DECODING (&coding) && (inserted > 0 || CODING_REQUIRE_FLUSHING (&coding))) { - move_gap_both (PT, PT_BYTE); - GAP_SIZE += inserted; - ZV_BYTE -= inserted; - Z_BYTE -= inserted; - ZV -= inserted; - Z -= inserted; + if (ingap) + { /* Text is at beginning of gap, move it to the end. */ + memmove (GAP_END_ADDR - inserted, GPT_ADDR, inserted); + } + else + { /* Text is inside the buffer; move it to end of the gap. */ + move_gap_both (PT, PT_BYTE); + eassert (inserted == Z_BYTE - BEG_BYTE); + GAP_SIZE += inserted; + ZV = Z = GPT = BEG; + ZV_BYTE = Z_BYTE = GPT_BYTE = BEG_BYTE; + /* Now we are safe to change the buffer's multibyteness directly. */ + bset_enable_multibyte_characters (current_buffer, multibyte); + } + decode_coding_gap (&coding, inserted); inserted = coding.produced_char; coding_system = CODING_ID_NAME (coding.id); } - else if (inserted > 0) + else if (inserted > 0 && ingap) { + /* Make the text read part of the buffer. */ + eassert (NILP (BVAR (current_buffer, enable_multibyte_characters))); + insert_from_gap_1 (inserted, inserted, false); invalidate_buffer_caches (current_buffer, PT, PT + inserted); adjust_after_insert (PT, PT_BYTE, PT + inserted, PT_BYTE + inserted, inserted); } + else if (!ingap) + { /* Apparently, no decoding needed, so just set the bytenesss. */ + bset_enable_multibyte_characters (current_buffer, multibyte); + } /* Call after-change hooks for the inserted text, aside from the case of normal visiting (not with REPLACE), which is done in a new buffer From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 00:49:34 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 04:49:34 +0000 Received: from localhost ([127.0.0.1]:48296 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiXD3-0007Rc-VZ for submit@debbugs.gnu.org; Wed, 03 Jul 2019 00:49:34 -0400 Received: from eggs.gnu.org ([209.51.188.92]:46116) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiXD2-0007RP-JC for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 00:49:33 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:34155) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hiXCw-0004ay-Aq; Wed, 03 Jul 2019 00:49:26 -0400 Received: from [176.228.60.248] (port=1214 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hiXCv-0006Uy-SM; Wed, 03 Jul 2019 00:49:26 -0400 Date: Wed, 03 Jul 2019 07:49:13 +0300 Message-Id: <83lfxfd8om.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Tue, 02 Jul 2019 17:00:21 -0400) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <83v9wkcp3z.fsf@gnu.org> <83sgrocmws.fsf@gnu.org> <83pnmschvy.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: wl@gnu.org, 36431@debbugs.gnu.org > Date: Tue, 02 Jul 2019 17:00:21 -0400 > > > It says that (a) CODING->src_pos is the negative of the offset from > > GPT of where the bytes are in the gap > > Yes, I think this is actually wrong. You are right. > E.e. decode_coding_gap does: > > coding->src_chars = chars; > [...] > coding->src_pos = -chars; > > so nowhere does it account for unspecified number of bytes between the > beginning of the gap and the beginning of the source text. > Here, `src_pos` is the offset from the end of the gap. Yes, the clear evidence is in coding_set_source: if (coding->src_pos < 0) coding->source = BUF_GAP_END_ADDR (buf) + coding->src_pos_byte; (Note that it actually only uses the byte offset's numerical value. I couldn't find any place where it actually uses the character offset in src_pos, it only checks its sign.) > > As for why this was designed like that -- where else did you see > > comments in Emacs that answer this kind of questions? > > There are such design comments at various places. > Usually added after the fact, sometimes added as part of > a commit-reversal to make sure someone else doesn't fall into the same > trap ;-) Interesting. Can you point me to a couple of such design comments? > If CODING->src_object is a buffer, it must be the current buffer. > In this case, if CODING->src_pos is positive, it is a position of > the source text in the buffer, otherwise, the source text is in the > - gap area of the buffer, and CODING->src_pos specifies the offset of > - the text from GPT (which must be the same as PT). If this is the > - same buffer as CODING->dst_object, CODING->src_pos must be > - negative. > + gap area of the buffer, and CODING->src_pos specifies the > + offset of the text from the end of the gap (which must be at PT). The "which must be at PT" part is ambiguous. I suggest to say explicitly that the gap must be at PT (although decode-coding doesn't really blindly assume that, as you can see from its calls to move_gap_both). > + If this is the same buffer as CODING->dst_object, CODING->src_pos must > + be negative. I don't see the condition of the same buffer in the code. What did I miss? > + When the text is taken from the gap, it can't be at the beginning of > + the gap so that we can produce the decoded text at the beginning of > + the gap: this way, as the output grows, the input shrinks, so we only > + need to allocate enough space for `max(IN, OUT)` instead of `IN + OUT`. I think this should also tell that decoding in this setup takes bytes from encoded text at the end of the gap and inserts the decoded text starting at PT, which is the same as the beginning of the gap. Without saying that, the above paragraph might not be as clear as it should be. From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 00:55:40 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 04:55:40 +0000 Received: from localhost ([127.0.0.1]:48302 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiXIx-0007aj-Pg for submit@debbugs.gnu.org; Wed, 03 Jul 2019 00:55:40 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47231) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiXIw-0007aV-6k for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 00:55:38 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:34206) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hiXIq-0001ic-OF; Wed, 03 Jul 2019 00:55:32 -0400 Received: from [176.228.60.248] (port=1591 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hiXIp-000715-PH; Wed, 03 Jul 2019 00:55:32 -0400 Date: Wed, 03 Jul 2019 07:55:18 +0300 Message-Id: <83k1czd8eh.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Wed, 03 Jul 2019 00:21:54 -0400) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: wl@gnu.org, 36431@debbugs.gnu.org > Date: Wed, 03 Jul 2019 00:21:54 -0400 > > - move_gap_both (PT, PT_BYTE); > - GAP_SIZE += inserted; > - ZV_BYTE -= inserted; > - Z_BYTE -= inserted; > - ZV -= inserted; > - Z -= inserted; > + if (ingap) > + { /* Text is at beginning of gap, move it to the end. */ > + memmove (GAP_END_ADDR - inserted, GPT_ADDR, inserted); > + } > + else > + { /* Text is inside the buffer; move it to end of the gap. */ > + move_gap_both (PT, PT_BYTE); > + eassert (inserted == Z_BYTE - BEG_BYTE); > + GAP_SIZE += inserted; > + ZV = Z = GPT = BEG; > + ZV_BYTE = Z_BYTE = GPT_BYTE = BEG_BYTE; > + /* Now we are safe to change the buffer's multibyteness directly. */ > + bset_enable_multibyte_characters (current_buffer, multibyte); > + } > + Why did you prefer to use memmove instead of move_gap_both? AFAIK the latter actually does the former under the hood, but why expose that implementation detail outside of insdel.c? From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 02:20:42 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 06:20:42 +0000 Received: from localhost ([127.0.0.1]:48318 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiYdF-0001J2-OV for submit@debbugs.gnu.org; Wed, 03 Jul 2019 02:20:41 -0400 Received: from mout02.posteo.de ([185.67.36.66]:57843) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiYdB-0001Ie-IB for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 02:20:38 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id C4C692400E6 for <36431@debbugs.gnu.org>; Wed, 3 Jul 2019 08:20:30 +0200 (CEST) Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 45drZP4G71z9rxL; Wed, 3 Jul 2019 08:20:29 +0200 (CEST) Date: Wed, 03 Jul 2019 08:20:23 +0200 (CEST) Message-Id: <20190703.082023.255697981040411264.wl@gnu.org> To: monnier@iro.umontreal.ca Subject: Re: bug#36431: Crash in marker.c:337 From: Werner LEMBERG In-Reply-To: References: <83ftnrf87e.fsf@gnu.org> X-Mailer: Mew version 6.8 on Emacs 27.0.50 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Mixed; boundary="--Next_Part(Wed_Jul__3_08_20_23_2019_256)--" Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: eliz@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) ----Next_Part(Wed_Jul__3_08_20_23_2019_256)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit > Here's an alternative patch which doesn't suffer from this problem > but also eliminates the transiently-inconsistent multibyte buffer > situation. Thanks for the patch. It partially works: One crash is avoided that I get without it, but another crash appears every time I want to send an e-mail with `mew'; see the attached backtrace. Werner ----Next_Part(Wed_Jul__3_08_20_23_2019_256)-- Content-Type: Application/Octet-Stream Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="gdb.txt.xz" /Td6WFoAAATm1rRGAgAhARYAAAB0L+Wj4RADGgddABGMADiqi2QIh2amU8HLG9nMxuEVBq8AyRe5 Wk7EmR5WOq0tFom6txG7wWCD9hZWPykHtlT+TfrtmWLm4jfM2E2stuWPYsKQ8Ubv4SClZe5xYyld j5cNwq2iC+Ux6SDkfmF8ExPnaAKO5LHz3d8+b2WILdvkay+ETY+Iekga0CRRbjClv0lrRU+I2Glg a+Y/yIOqjfFsOYdNPdFlfai/QF2aoZbf3LqObVOf+8vNlx7W6gfGbEfzs9a59v6hRSJtmGNhWR4E 6Rx166DfC/CwIM8VFcZ6F/8zmq7BWPvCRNNyTCCP2KTOxuIErTkMBi8GmAPE3OgO6klHFpf9a4E0 m+xh6smXOFl2HDKjH6WwiEe2xE3kvbczVklbeQxffj2qxKHjePIHG8VFguVPFzdFBKArEfI5SfLn wYbq3Tg0mdzMT+sKxQZCwtiBma6eq+AaOGxOHpZO3pFajBVuwvhVFMw/sNLzkQuWyB2EMdrJRusY ZmszqfviBjT0gb5zYzyGB6J/dodH7SivH1EsMtfOvPdynIB5FRCvYgFOcuElBvCYnPbPFESnG8DY oH6DX+KQSFuwc04cmO+q4fId71HHk5lIzFPfikolLlAKYBrIOdezxqcHI4dnhvLpdRXN/NvmrEIm saWSBcFbZ6CZLoKnpp7OBfwI3aSYYwOLEdbpOftW0ua5satIlIM2aZE+aJ5GjZHnYRxmuV4N4aQw JbmTDaMKo5GCzAIyfnDffKepJfRYrTDNN7TIuWzuHiqHLcCZbnlp472hfrAPCaj5RsH5oQWGBwds RBoGp5m3tc2/r4jVcDsvc/gE/K+tBLGbhF6tg8ZWZYfF1x7KmEt9TCaA6Wl20LpDbx+5BFGF5lCK 47RNomDX8zu+Y9K+b1kBNY3kDNqEVFLxVm3zeJBdAl3XHTQIRn2UefaZIGOiKi0/A23tIQmOuIVu MfYNku4N/29CC5jYeFY8XIg1d2KwVzDFXm8L+xFV1qgFdfdk7T1g/n8PD1fAVSPNddwvSvLZDOsC 3kA/9OB+/H3l29Pvg6DBRE+jFG8VAKgwAuP2AldR+ALN0vNDEoAquiow5rfcmJwBt2wXGQhuWGY8 u+gKXQXVyM2DK3kRe2WzOZ0Eyz9ywRRCdZO7iQ01QXcABNEEqZkzVyTtGcEquFbE+vDdDfmvkDoJ Ks0OGjmdo/UIJqiZ8GzeZv0dxUkL2Zehx1zGB5cpfSOxeCGOAtuadvgMa6EWajIbQfWwsF/HuYCn q7Bp3R/FrOzMFzKReXgExNwlnQNPbPnGWFx6O3PVRjYrFfFiCV9BmLreTZ8QSuuRZPrrgSPRR7JO GIA38kpPjHq3AdsgJyCkpuSUUIOuP/4WJ5S0QxvOKYwLnpWtSqez+Ef+3UEy8jGeVTKJRrJoBFRT DouWeZF2rPfVzhDdQyUC/v3xKU1n5L5btv2rpN+To8R7zfiB8q7dWNKCrRZB6O77zmFTgmXMS+QG tCN43y1ZRiALkQu3FBWpJ3WmLbVk6jlI6WCt5mS5BkNuXV0yrUA9fXPClqFvvROsd1PkIyTs/A2y M3gaN6rpmz3c7jo50MQJoJp9dYwACFdBJUsLlI3DTba9os781RjmC74Sp0squpo2OAhtshVZ/Twy kEhThfCXND0u1II8dTnnQri4Bq8sujxfvqKpfEk+eY/kp5WamfrwqS0QE7VY1APNrvDsY/qRP9Ia jtjSfok0AMoZSc/+We5Rh84b/HjqqSfEn7WmBEn54xYgPrQ+3NxjAozQxu0SGIVMUhL+xcBKAx4u KDZs/5U4QFdNlcJe8ZdLerHpLuPHX6YtUUX7i2/nP2SUzPoGXRIQWvVP1ltsvoL9tEWv9P05rNvx w3z7CD3b7ETO5xST/0tJxVc+HOa9cjg0op6leakrtzCL+8LPFr6jXgvBBu5zbCstA1+attxgK39h OMhTR3pRQ3SxFlZxSMAKWX6tKx1uMgMDqo+z1GS3ra+uzU4zHixxx3oyrGsU1ec6zhod/aAtzf5K MrJGI41RVVq3OAnhHl3frdWyT6FO2g20fw0wG54Lqvq+rk9blh7YRSrmC37vY06NjGw2cwUnxDJ/ /3DtXyvuGTnhYCARODNJoCT19N+7GFEQvxzOXG/EHxTqWXAzup74F/oYrCQiS1n34Yfrp9sO0UYK wXcohdGOzk0oAamyLgmYR6c6q18kMCX7+PXAw4YyYlo5hRHGR2ogQesJox13B0pCIt4XnxayOqp0 8SobgtfWhJHldYwAa9zJKOBEagqScy0BpsIoSOhiDJvRsiET43eej1CHdyfJGi6CJIze3IyO+bju pzC8WKzW2fgBvGqYikLLn4lnF8VOmqSFhjcJ7ornMF2J/+y1eF8Ndi2ifxSmY0fPnM5X7eeZB7CH 33Sx/LCB+9EqePGiiLaE2JQXSptDPJeZuciq1wfWw9uWwxF8Ns2LvefYJ57/IxDSSkUSV3q93XeX OeKbPM0GxUlHFmVAc3orpVi0FzmGOIlogaLbZkSEcGya1GyaPqvdd3+e0vG8az4cTZOepMhLsVPb /E6d4utuw67Kf6vAsgoTOnCSSR/8qRzZ7Cg4pGlyWcUuZSq/bnt549VchkAVpijiCSooEI6m1BI5 YneMMzTUbTK42S0OvzGQOGL/F2cV9xwHTlMGvk6+8sEvCaNLo39cujaROLCLlJzedVNJ+N700yZr NUtCfe3kSSqcQRQ3V9p93py1NihpypYLKjBFZI+kA4I/RC1C7FVix03abWR9cEQKQz1HucC/8Scp e/kmIipDX5o1AbHHDrQsoFX3MimTAxMVRN1uhTHBpffqII9gcCp7cvBMtl0pakFnG1e6Fj5q1vJ+ kEZa0Z1QFbcmIDo6QqQ/oydZWz9jbAcg60kXWL8d2zHvhXpKvSUsHCwi9/V7Mh3hUhwxh+BRweFZ W/Mr6tjaFMQ9cL7s0CNDDHqSPsDqaxcvf1UbYO/ZWv0tLi0IcOmKnLgLL5gcnI0fPZ3tpUE3CMRO KYkmb1a7MRrSKAE+EKj3zSl0RXJ/l/E8+ixWkZbj1+JYi5E17riMzXUiRYHlBc94HxFb7H1rEdAu r64eLvXZGAfXqE7PmYJs+Yq+A0qbXLViDk/ikwjJMCthi4ONBVA6ctzK3cdbxi5/2cd0K1CvDClG fIVMdPnqBcVdATCn0b+JQlooHyUZeWkU3/BakZqZwvQZQK3fTP95oYwkA/9cA6/C211M31l2CCtl NZ9sBNljijABMRSdebiVQtXoUj79vXuWIM+tNKgiAwQKOd8iCPyrIzWN7hjXJa4bjaVo2O0om8yv RRscWqgS/qO60wQnNVZomkwKQ1ICjXMfGxFUv5Es6m4EJav9QE49BWU7C6NZjz70p2jgcwjr679s G6PfUChME6Ycdl9sFay4iFLe2YlHNVyYrH3bkR1XMQA2sFXdPDn13azx9U32cedr9BVMu2ueS2OB z1b849WwSn8Sqk1zNCUF5L14hXNts2gF0GGjyhM/sA/QPToNh7tEkyE9k19SrkJcE8NmusUaJ0Ib PD6i3qzc0RD6CrxCpmGhyGCj5re5B102b87z0Op4gl/q3GyONARGjFLDaTVClsi+CS+/xfLaMEKi lfm2PjIwCbEBwntJdCUMP6apuH2GJ/rA3YSudmkSDZMSbSVkwQZU1RYbNVZ5qSDVdNg32DvF0gMu zzthjdTs0zVTg6yLJQfmeMCH0d3p+afZ8ZE1+cMHQUmUXM0NLs7BjjlPwq/UXKBVhyK9ISz8+3Jm atApBlXxU9Ewo7XSrTH6j9RGh6mXrhWqrbLWWOzivddo7NUPgjM+9cfZnnTTZtahbx+O9CeQiL80 j+4kUArSWNyDrmou17LmRwqPFwbMGZN2yr7tNVYGZVKm5NPWCHNdighCq3HjOSAN3PAGwdw/Zzti j+JL91ZFAsJftIbzrr+v+RKaXMyk+nS+1L+gAfzUGk2DIEAtZ550+lUoCmTJieliHV/tJ43M1ogx gdaAVzA5nkoG3ASSEZ5Y8PfkVG2lGwFrarHRzTZPPqLVjtNNFcQbZh5mOjbyZMpgM3FmuOZg63Nk DjrZw3L1S7EsTd2zyPddwejkxBWKU49DyyCQ+X8zzNNmmKvhyImsS0sxYOlWz51qNpB+LLm5S/hq 63GzY+qs+WUGGpcR/ixTgNte+xXKGPnZUc7TeU3C29H+hezTDk0IazJBggs7GfbmR1JYbro3SMHM c54GLkFwlgjvrNTNbta4nqSrN3l4qnn6/3qndw8QCFJ1lDhybDJRU32JfM5KKSfdluup/KCJ6Tur 87hN10sw0Sey7sfeO6APzQRwMV3Ngaatzys1OVbxc08mS4P8Cs5+cis6lZVc6c9o8/7E5cvMpoRs sQmB8HxGL8wtEKg3FieM/9VCviH7MBmamH08tHuATVQ7RUrLEOs+HXk4lAJhahpUvf8s4hurQB++ B+EAg9gwKF4W9YiO6WMGtiIi9S5DTO8nfc7Kbz3+88pBS/kwvQvfY7RQJamq4asKV9AkYDuch0ih nmmCICKjwfMSeajSx+SL5cqfEiNrWO5k73HYTV660kaImOr100LdWeCbOoHXIAyfOPmHuv+/r3HL /+MA7zOxfdpKuMY/MEuvjAYM7MGrz8+7XShP8qMnaxxpAvo0od9sXHTws08kduMbaKZR1PPWap2Y 5+EUhwgM9AGbHZDDbPMjx9jGGlQZqnVH/lutiliB1CqRWlEqUTzWUyNa2MnV0K6qlIxSQwo0zkoO 9SWYD6yuTkFtXelP8yS9o0uCv13Hu9dBWkEs+mAU+zt/iSYdSlZbbozX2rup7N2YQol3F7sNY7CH o3ZwmJoLRibMejLUBlnxTc4YfFz2MPELcMUvaUWSxkLnPyUNjinbXGp9GXbg74CEbQPwwvms/v/o TjYw5VrydDj5bd9sQovIT7m7xvf1Od3HVDJkc+GCJyp7W/JwzADJB/MF8YbT50w6VOGCPGc6Ytfv cqznFM6A/RiiLy4gCB0mFdaKvDrJqQO7PcJMjzQwyyt+7yIHaw4kDa1CmVzDnjzSwDss8u+a3aKt XXAmyFqOTt1RAi01iLJ3GgceLptoy/TX7ZXM4BUrkPmDmsiOL/4LGEr8d6pS8ispG2OQsxcKWc8r kFkOwSWSCzw5iV0896YyXzZPsQuqY/bYBbFwvKakAJjLtxTW6WHLomkQCxqQPLNkPepUB7E97ElI BA4OZDECz4q+EKeae3jzbl/PsbkVBdh9cM8Gm8eGtuyu70+9cu+XkzvI7sp5x637SoIdbH/BzEB+ pQFb4tRklvgKc1uVHSnBkc/QAsQxuKP1NRoVGIqQniVkKvxWAOzkSp+/0RmDYgDlspZUosUjvHRn BmI/BmR9CQPED6BLZgPcxBRV64feXleV3ACtjpomrJCgyy8ZgGAwyPygx420UNfRO5kOmKdFyeDM 0FweDXx9uhc5PNsm+AJcEE31SpS8p2K57UaavodeiKQZPxnjbomV8UtvTiljU0cc8u5BFiY6EKXK uPTYF9v9dSMhZWF2Z01bcTv+ZTW/gAgWezpvxhLjP7G0Rk8rOuZN9p3gRdi7VMzOkR2momYP4QK/ 2uEavqwNcFsSOJBVNzyYzhXz504gsX9XbK93BwArA9c9e//cZLz8clCbES+O/xluG/s4pw9UtaJg c8qovIwIv0kTcpI3KkdOQB9uX0bX6mN+dUZsmfC8uva7SCWJIfzxFfxgWgLFwQKEyDsIF+AWBrTp 6YRZMVlTWgqVxVJWfG+DCQvQvy94/PzKSNe+a+abgARxEDiPPEs4ZjT4FEDaWsUt4QNt3rIAgPMm IKyvoXv8HcRj1tbJ41s5Sz2tdwYe6yq01+ibiHfRxizFGIA5bu1hiZZpayLxgnxi/2SpaQrpl7+l a9daDi1dDR0uHsOawP0v61GZ/U7kOunCHEiYu09rWMkgfo7AYWI/B/U7zg4XKz9yRRQzWVaiftb7 sQTmg8D6iz2G9Lvhh7jZTZnuDbqC5HOfX/1afSSEeP9+T6beD1lzn0iBdUkyDEI7LCzymzuZSDZ1 rhjznm3obIQjo6CNmOUbzbFbO6bd/ugqOEHT9NB0UHk2/0a3hs5ygve62gqja6uy+y6PSN/AcWH6 I07ATQyrmdYtr45NcGF8KikNiD52uOQpX3Tgkx1I0GSGYbBpdIH0k8/HLvlakraZjsecrT2ig+WR pqGT7Is6YZDzJqQ71FHcyTjYF9AQMcMORlEut8YikcR3MjkYWjCD26gg3kEmCM50xqBbJME9Pytq 1o9hgAUHMYHJlCPhajK1z06j3SAw+PFEXHWKJenjXp8TEpm9LwXol0WbyB5hI7VeuZYDnLEdQ4Km PGxY3zaJhJUT3WEIA9Z5L3GEh1Rlcg2LqkpJT9rvy8yGF49T9FPJoZjrIk+lPueXUZ9PVGvqRp4M M5OQ41RS9PR2rXh8zpP4/WizpAnnWg486E8wLFbzlcKPbl1UHeRKEto0qTFjERSnlNBPX4b31A+0 FHLAMZsK/CA4ZSJUOFrDu0IaGQ5VTVE2og9lysecq1RVS8Bw4bAmCduXH1fjJPqonsYtEbxY2Mwi Sz4q+B87RdMpkjYHHHRD8lfy38KLMGP3IaJOMe9H2fBmZE+u0MJI767Lc/ZVOxVR0lXji8/LXqAA nKt3g82GlktQcmoKD4umQeQbUiEqqZgmRkfvCH0WjDWHCXDCRzCqW/jTmWnNZ+tajBZR2/SXVuiT Ii79cBMBIwQ5WLIBCw6hyu7oXxqLzUE4aQH85E5ugw4pQzUuzO2IND/gUxvP3SwML+JrfksKglEO 0nsboeWre52zH5JvOdHNQysjFoP7PgQbmbM9qIen/C6+7PbUdxZTc1RhxU9N7uZ5FKedFJByYOpX RGYYUA27EZZHmGkr2S750cxAQNGH4hQREOv+Z7rx6A5PAqIKsWwPxvthMA6erlo4aKPCDnGWw57R +chUJ0xopfc0NG9TKeXHnPLaUwzN6UuYM1BpwCHnyzt1KrlWH42ZTnKgF/tEYcz5S/AxvMXpLmDy T9oa1Dmu200j95l4bIc8ZRPfb1Qlc79tgKKHO9AgrnmqclrjFyH7AawKAS3i5juj++Br5p1tYNs/ bUxOxWnzp+dLdDRgOuNNo3SIMXrU+ZZ0bhDmufGNGXF34bKxbq5PkeLs+7MEme9w7WWp8KbBsLLq IyVOIYxKR2nQzP8+PrhicZTckpCkiKv0L31B5xM3G4Pk2lKQXfG67kSfer/iB7drwNAO8y/RyX4t lMShKkd+8s5GYZPLm7m/iw5COGxkNnJZKMv0R0TPReAOXmM9tcfzy7fNx2Wdaxev4Qvvuk0x9dqF MZzIgi2Cy5J/tfJDy/ecsW09IE1KIqquLPXRZi3eZgOG8czbjTHGVMAKQecSRRVYS9/Mk67p9k3K YGJ7mxgpHnSDbZA/+fQq6DMCRdm+8mpz8NowolyYpiX144K+vdE1F4kXvymIzDm3T7ngZS5lqaIG C53iVhgywUP0Xl9Vjo5mrq0aSDtvkSMrYHfm4hRvCauh7He5YhN5kV5Wl0AEGtF3t7fpJagZTs8K SOks9MwY7Od5jBdmAJCB8AOzrq/uX6t5mWslT6avjCTNLauubwbpOHSMe7abD4taRlhAhjOayvEQ 4evhsJf8Y0Jqsa1fuUyzuEFxQ8Fa+zeI+JxsLwPncgazBxC8lMihy+21sWtcGIkFUqwZJMG7FSXd XalQYsQhfUPN6gtPqCVnWgg44AOsKYnlSJzCj8PCiIo40jPRJ7kkwucgksdIz1LKXfNSaZFoiwxC kvFt7HseSx/qfAZTG5CSemBSIUHSGrZpoEFgutguj3ka9CP0Olkm/6jEH1ATsN6diIzhCCd5ZMkt 9EMIbm7Wgk3N2Y4/eG0wQ0oYNrM/d5VJlQkccdL5Rk7VehUVvQjqVMBurSgHni2EncMfA43ldW52 P7mTIsDNdMZazDgPv60LfsogxtB7y/KMZ07KGJe2/y+y4qbAJyT7unB3Mme8dgc95z1qoqLh0n6j 9944w6V4Jluef+EU4zHIG+NExP8PGzcPAviVOgqSjqToEJvJ2/eqwP+3Xlr6sjksGasVIDBVvGxe Lh9XWQDQARTeuomXHwIeY+/wQrJtWA7X+Iy/6PHNifuQraMe/igQ3pO1PbSc/grihBB13j/jYL0v WUskr8+bu6o2gjYOKpOIout/QGPpBsXLuPBcdNm783e8/lpB1P2E25osKQERfwncWN87RddoVAoy ZG0O4wJjPk4xKZPEdZdPYD43rWpw7oBFa2Pt5OSXK9zE1OF5BJ8pC5Hvgx8r038lxT5eqB9epsxH of59eUcbnbZqO0uGQKyMuB2YOPkCx85MZFduPwlIIGu2mTy/K7th1eF771I4rNpwfyG3hblZBu2S bKRqBO8Ly055wa3EUOPyFHbXvDtflGzE6lsyBragudJMJREz3IyWbk54rCD/qXIj+DQlGpE2pctn 2Gtv6E7nvGx4kl8F/BLReLGrhaLonAwLibQqsXAGy7bRo0e/jLsAuMA7uJ6Tc6Js73U2uD9GEswC dYftCR5pPwzPd/dLBS1Zun7wSnaoUCX9vhsrzFQfB7dt2V8sdyQZWCYgBN30qJZeRrqUE06pDIGU vqmrFtje+qaBbvczHmqAdF1GfWs6QEckV4IK2dThxxx8zYAxFqaVO0iGhrHcASAjWyp4h8+19XYB pO0J3uN6rQtZ35BV0CR0DecKH32DejtDq+UldGjnNlKCgwc6bwl4eqgrmnZoQEVaBUy+y8aEjKk2 1MHDK52GHymDp/27f94Wz4VQRZl3m0zgxR4da1eMz3lRgOZIubAAxKcTmhRcJYgIw1ncUizmaPzf RcsBgXMWFE976XOnBatbZLjRkQu4JFqsewAApZ97urPylz4AAaM0hKAEANMQU9exxGf7AgAAAAAE WVo= ----Next_Part(Wed_Jul__3_08_20_23_2019_256)---- From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 02:29:40 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 06:29:40 +0000 Received: from localhost ([127.0.0.1]:48324 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiYlv-0001W9-Nz for submit@debbugs.gnu.org; Wed, 03 Jul 2019 02:29:39 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37414) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiYlu-0001Vx-86 for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 02:29:38 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35008) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hiYln-0000L8-AV; Wed, 03 Jul 2019 02:29:31 -0400 Received: from [176.228.60.248] (port=3660 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hiYlm-0008Kq-8o; Wed, 03 Jul 2019 02:29:30 -0400 Date: Wed, 03 Jul 2019 09:29:16 +0300 Message-Id: <83ef37d41v.fsf@gnu.org> From: Eli Zaretskii To: Werner LEMBERG In-reply-to: <20190703.082023.255697981040411264.wl@gnu.org> (message from Werner LEMBERG on Wed, 03 Jul 2019 08:20:23 +0200 (CEST)) Subject: Re: bug#36431: Crash in marker.c:337 References: <83ftnrf87e.fsf@gnu.org> <20190703.082023.255697981040411264.wl@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: monnier@iro.umontreal.ca, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Wed, 03 Jul 2019 08:20:23 +0200 (CEST) > Cc: eliz@gnu.org, 36431@debbugs.gnu.org > From: Werner LEMBERG > > > Here's an alternative patch which doesn't suffer from this problem > > but also eliminates the transiently-inconsistent multibyte buffer > > situation. > > Thanks for the patch. It partially works: One crash is avoided that I > get without it, but another crash appears every time I want to send an > e-mail with `mew'; see the attached backtrace. Werner, PLEASE try to prepare a reproduction recipe as I suggested earlier. Without that, debugging this problem is going to be very inefficient, and we will not be able to tell for sure we fixed it even if you report that your particular use case no longer triggers assertions. TIA P.S. I also think your sources are out of sync with the current master, which is another factor that makes debugging harder and more error-prone. Can you update from master, please? From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 02:46:35 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 06:46:35 +0000 Received: from localhost ([127.0.0.1]:48339 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiZ2I-0001x0-TV for submit@debbugs.gnu.org; Wed, 03 Jul 2019 02:46:35 -0400 Received: from mout02.posteo.de ([185.67.36.66]:35267) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiZ2E-0001wk-Tp for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 02:46:31 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id C56AE2400E5 for <36431@debbugs.gnu.org>; Wed, 3 Jul 2019 08:46:23 +0200 (CEST) Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 45ds8G3CxKz9rxQ; Wed, 3 Jul 2019 08:46:22 +0200 (CEST) Date: Wed, 03 Jul 2019 08:46:21 +0200 (CEST) Message-Id: <20190703.084621.438958346025193738.wl@gnu.org> To: eliz@gnu.org Subject: Re: bug#36431: Crash in marker.c:337 From: Werner LEMBERG In-Reply-To: <83ef37d41v.fsf@gnu.org> References: <20190703.082023.255697981040411264.wl@gnu.org> <83ef37d41v.fsf@gnu.org> X-Mailer: Mew version 6.8 on Emacs 27.0.50 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: monnier@iro.umontreal.ca, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > PLEASE try to prepare a reproduction recipe as I suggested earlier. > Without that, debugging this problem is going to be very > inefficient, and we will not be able to tell for sure we fixed it > even if you report that your particular use case no longer triggers > assertions. I know, I know, however, this needs *a lot* of time which I don't have right now, sorry. Will try to come up with something in the future. > P.S. I also think your sources are out of sync with the current > master, which is another factor that makes debugging harder and more > error-prone. Can you update from master, please? Nope, I'm *always* updating to master (this time it is commit 20c1406c), then applying the patch. Werner PS: I do `git pull', apply the patch, then running `make && make install' from the top-level directory, re-using the stuff from previous compilations. In case this isn't sufficient please tell me. From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 03:15:02 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 07:15:02 +0000 Received: from localhost ([127.0.0.1]:48355 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiZTq-0002d4-Hq for submit@debbugs.gnu.org; Wed, 03 Jul 2019 03:15:02 -0400 Received: from eggs.gnu.org ([209.51.188.92]:46186) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiZTo-0002cS-Hs for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 03:15:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35422) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hiZTj-0005J9-E1; Wed, 03 Jul 2019 03:14:55 -0400 Received: from [176.228.60.248] (port=2487 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hiZTi-0002SF-Uy; Wed, 03 Jul 2019 03:14:55 -0400 Date: Wed, 03 Jul 2019 10:14:42 +0300 Message-Id: <83a7dvd1y5.fsf@gnu.org> From: Eli Zaretskii To: Werner LEMBERG In-reply-to: <20190703.084621.438958346025193738.wl@gnu.org> (message from Werner LEMBERG on Wed, 03 Jul 2019 08:46:21 +0200 (CEST)) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190703.082023.255697981040411264.wl@gnu.org> <83ef37d41v.fsf@gnu.org> <20190703.084621.438958346025193738.wl@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: monnier@iro.umontreal.ca, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Wed, 03 Jul 2019 08:46:21 +0200 (CEST) > Cc: monnier@iro.umontreal.ca, 36431@debbugs.gnu.org > From: Werner LEMBERG > > > P.S. I also think your sources are out of sync with the current > > master, which is another factor that makes debugging harder and more > > error-prone. Can you update from master, please? > > Nope, I'm *always* updating to master (this time it is commit > 20c1406c), then applying the patch. Then maybe I got confused by the time sequence of changes Stefan made in the repository and the patches he sent to you. Sorry. From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 12:08:22 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 16:08:22 +0000 Received: from localhost ([127.0.0.1]:49932 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hihnx-00038R-OR for submit@debbugs.gnu.org; Wed, 03 Jul 2019 12:08:21 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:56249) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hihnw-00038E-DN for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 12:08:20 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 4BAAE100A37; Wed, 3 Jul 2019 12:08:13 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id E258A100964; Wed, 3 Jul 2019 12:08:11 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562170091; bh=AL9T+gG3MF1QioL5IGZLftEGLmB6zb0IiepHeBOAjuE=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=nuAUl/riqSyMz1mGrdojuEguiI6EhPwl2eOjW/UCCA3Mhs+IqkbK7zHzTIAObTevI agzueVJlFcCHMbJbybhr2pvMfHIvfy4DdIhUHPIcFPP0cBpN4V0VaH69ZQr4kL8SX0 LoiHbZDZGdtS+uv4frKTlDMUuQ2sFpkwerH2OVhneOou9+OaUjJtplD6fv6/u6cKxX SqA/1/9WQlg2rIf2RhoxQMapIyG7DuqQLHY85P8Knz7FigYA3n0brUhuJHVhmjimsN T3c6pIMBhnQ+hJAR733mAapDiad6XqHquPcBC2DSUmQaYN35teoVekKeUbnFRv3j7o b/dTo6I+U/F1Q== Received: from pastel (76-10-141-139.dsl.teksavvy.com [76.10.141.139]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 31516120B84; Wed, 3 Jul 2019 12:08:11 -0400 (EDT) From: Stefan Monnier To: Werner LEMBERG Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <83ftnrf87e.fsf@gnu.org> <20190703.082023.255697981040411264.wl@gnu.org> Date: Wed, 03 Jul 2019 12:08:04 -0400 In-Reply-To: <20190703.082023.255697981040411264.wl@gnu.org> (Werner LEMBERG's message of "Wed, 03 Jul 2019 08:20:23 +0200 (CEST)") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.088 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: eliz@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Thanks for the patch. It partially works: One crash is avoided that I > get without it, but another crash appears every time I want to send an > e-mail with `mew'; see the attached backtrace. Thanks, that should be easy to fix, I'll send another patch soon, Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 12:19:38 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 16:19:38 +0000 Received: from localhost ([127.0.0.1]:49942 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hihys-0003O2-9d for submit@debbugs.gnu.org; Wed, 03 Jul 2019 12:19:38 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:7375) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hihyq-0003Nq-4P for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 12:19:36 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id A99F181153; Wed, 3 Jul 2019 12:19:30 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 322FF80D52; Wed, 3 Jul 2019 12:19:25 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562170765; bh=jHp6Y69Sw8RtRfe+ST9beYzN9ovR9Kv2HtLoYw0+XFU=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=WI9hK7+dOXtSitB+57iZV2Qq1fE0Q96+EmojuZBzZQWzYhrqnY+GRIEaS5D4Q6S7L ENihcrjOqUCit4NDO+MEc/IuFLUPLhmbv2wd+dNXPSZNs47uGlIoSbjrxYeLbizKQa iNwzRLxxfuaKS7/Fz6oL19/P4Cb8SXeWdumAaRSENvTpcC7h8Mi1EPNYjM1V5bs/5Z qelFUlQecFRpXl2fMSr2C31m2UKOr3z2zP5rji3VCxEK9rCVpCA3okX+rTjnuzVG/q FgV6XmtvQfPhFzw2g7fSM9IoQxlLSqGT2D2CjwMJUa7rdKIqMMTntBgKrYkat7BiPd RvHKcPuTJ2klg== Received: from pastel (76-10-141-139.dsl.teksavvy.com [76.10.141.139]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 54CBA12082E; Wed, 3 Jul 2019 12:19:23 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <83v9wkcp3z.fsf@gnu.org> <83sgrocmws.fsf@gnu.org> <83pnmschvy.fsf@gnu.org> <83lfxfd8om.fsf@gnu.org> Date: Wed, 03 Jul 2019 12:19:22 -0400 In-Reply-To: <83lfxfd8om.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 03 Jul 2019 07:49:13 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.086 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > (Note that it actually only uses the byte offset's numerical value. I > couldn't find any place where it actually uses the character offset in > src_pos, it only checks its sign.) The source is required to be unibyte, so src_pos and src_pos_byte have to be equal at start and one of the two is redundant. >> There are such design comments at various places. Usually added >> after the fact, sometimes added as part of a commit-reversal to make >> sure someone else doesn't fall into the same trap ;-) > Interesting. Can you point me to a couple of such design comments? Not off-hand, no, but I know I added such comments every once in a while. >> If CODING->src_object is a buffer, it must be the current buffer. >> In this case, if CODING->src_pos is positive, it is a position of >> the source text in the buffer, otherwise, the source text is in the >> - gap area of the buffer, and CODING->src_pos specifies the offset of >> - the text from GPT (which must be the same as PT). If this is the >> - same buffer as CODING->dst_object, CODING->src_pos must be >> - negative. >> + gap area of the buffer, and CODING->src_pos specifies the >> + offset of the text from the end of the gap (which must be at PT). > > The "which must be at PT" part is ambiguous. I suggest to say > explicitly that the gap must be at PT AFAICT that's exactly what it is saying, so I'm not sure what ambiguity you're thinking of. > (although decode-coding doesn't really blindly assume that, as you can > see from its calls to move_gap_both). [ FWIW, this part of the text is mostly preserved from the old text. ] I think the issue is that decode_coding's calls to move_gap_both *must* be no-ops when the source is in the gap otherwise it'll destroy the source-text. >> + If this is the same buffer as CODING->dst_object, CODING->src_pos must >> + be negative. > I don't see the condition of the same buffer in the code. What did I > miss? This one I don't know: I just preserved this part of the text. >> + When the text is taken from the gap, it can't be at the beginning of >> + the gap so that we can produce the decoded text at the beginning of >> + the gap: this way, as the output grows, the input shrinks, so we only >> + need to allocate enough space for `max(IN, OUT)` instead of `IN + OUT`. > > I think this should also tell that decoding in this setup takes bytes > from encoded text at the end of the gap and inserts the decoded text > starting at PT, which is the same as the beginning of the gap. [ PT is both the beginning and the end of the gap ;-) ] OK, I'll clarify a bit, thanks. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 03 12:33:49 2019 Received: (at 36431) by debbugs.gnu.org; 3 Jul 2019 16:33:49 +0000 Received: from localhost ([127.0.0.1]:49948 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiiCa-0003jB-M8 for submit@debbugs.gnu.org; Wed, 03 Jul 2019 12:33:49 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51750) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiiCY-0003iy-Dx for 36431@debbugs.gnu.org; Wed, 03 Jul 2019 12:33:47 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44244) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hiiCO-0006AX-DR; Wed, 03 Jul 2019 12:33:38 -0400 Received: from [176.228.60.248] (port=2491 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hiiCN-00071X-K7; Wed, 03 Jul 2019 12:33:36 -0400 Date: Wed, 03 Jul 2019 19:33:22 +0300 Message-Id: <83o92baxil.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Wed, 03 Jul 2019 12:19:22 -0400) Subject: Re: bug#36431: Crash in marker.c:337 References: <20190629.131734.877718102639559715.wl@gnu.org> <831rzch9nd.fsf@gnu.org> <83zhm0fuqg.fsf@gnu.org> <83ftnrf87e.fsf@gnu.org> <83v9wkcp3z.fsf@gnu.org> <83sgrocmws.fsf@gnu.org> <83pnmschvy.fsf@gnu.org> <83lfxfd8om.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 36431 Cc: wl@gnu.org, 36431@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: wl@gnu.org, 36431@debbugs.gnu.org > Date: Wed, 03 Jul 2019 12:19:22 -0400 > > > (Note that it actually only uses the byte offset's numerical value. I > > couldn't find any place where it actually uses the character offset in > > src_pos, it only checks its sign.) > > The source is required to be unibyte, so src_pos and src_pos_byte have > to be equal at start and one of the two is redundant. My point was that the code uses the sign of one of them and the value of the other. > >> + gap area of the buffer, and CODING->src_pos specifies the > >> + offset of the text from the end of the gap (which must be at PT). > > > > The "which must be at PT" part is ambiguous. I suggest to say > > explicitly that the gap must be at PT > > AFAICT that's exactly what it is saying, so I'm not sure what ambiguity > you're thinking of. The text could be interpreted as meaning that the end of the gap must be at PT. IOW, "which" could either allude to the gap or to the end of the gap. > > (although decode-coding doesn't really blindly assume that, as you can > > see from its calls to move_gap_both). > > [ FWIW, this part of the text is mostly preserved from the old text. ] I assumed that you wanted to clarify the comment, so preserving old text sounds like a non-goal ;-) > I think the issue is that decode_coding's calls to move_gap_both *must* > be no-ops when the source is in the gap otherwise it'll destroy the > source-text. No, I meant the code there actually tests these conditions, and if they are not true, it moves the gap as needed to make them true. > >> + If this is the same buffer as CODING->dst_object, CODING->src_pos must > >> + be negative. > > I don't see the condition of the same buffer in the code. What did I > > miss? > > This one I don't know: I just preserved this part of the text. I don't think it's true. CODING->src_pos must be negative if the source text is in the gap, period. > >> + When the text is taken from the gap, it can't be at the beginning of > >> + the gap so that we can produce the decoded text at the beginning of > >> + the gap: this way, as the output grows, the input shrinks, so we only > >> + need to allocate enough space for `max(IN, OUT)` instead of `IN + OUT`. > > > > I think this should also tell that decoding in this setup takes bytes > > from encoded text at the end of the gap and inserts the decoded text > > starting at PT, which is the same as the beginning of the gap. > > [ PT is both the beginning and the end of the gap ;-) ] Not in what I wrote, AFAICT. From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 09 17:04:57 2019 Received: (at 36431-done) by debbugs.gnu.org; 9 Jul 2019 21:04:57 +0000 Received: from localhost ([127.0.0.1]:34374 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hkxIG-0000jh-3C for submit@debbugs.gnu.org; Tue, 09 Jul 2019 17:04:56 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:30639) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hkxIC-0000jQ-4B for 36431-done@debbugs.gnu.org; Tue, 09 Jul 2019 17:04:52 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 954FC80D83; Tue, 9 Jul 2019 17:04:46 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 548F880D7B; Tue, 9 Jul 2019 17:04:45 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1562706285; bh=riC1DULDi0s5GdQ6uU30L9pmlfvSdKjxqdxKb+dFm9c=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=mpEfq3YTgZeOTBGeUSPqrlP134x6Ht7nnQLgGKUCZVjXmrUpoEsmBYDNi7PhDp0il ec8boHptvX7xKD6yN+OB2zsxOJB5J1KEI7EpOsThmDh+VRlhrTzWA3HzbVHQFw5pFG iF7GS19oC1G+EWqYiH1ux+DEQINZFLs0ncXUqjIPXILUZo9IUmDT74hf2qb/4Zo/TN /Lg9/TicefxSYseXLqhpV9ZKjO6imwH2XLgKvEYg7rExP9t0CNd7uftLzAeMBm+MB6 8ORrbjybHQrjLbqgUUQoOXHAnImEsal4kLGxaltWh0rUyk50gyB6gmGCxJ/uLcp4/2 N/qPTAW3kikUA== Received: from alfajor (modemcable157.163-203-24.mc.videotron.ca [24.203.163.157]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3730712059A; Tue, 9 Jul 2019 17:04:45 -0400 (EDT) From: Stefan Monnier To: Werner LEMBERG Subject: Re: bug#36431: Crash in marker.c:337 Message-ID: References: <83ftnrf87e.fsf@gnu.org> <20190703.082023.255697981040411264.wl@gnu.org> Date: Tue, 09 Jul 2019 17:04:44 -0400 In-Reply-To: <20190703.082023.255697981040411264.wl@gnu.org> (Werner LEMBERG's message of "Wed, 03 Jul 2019 08:20:23 +0200 (CEST)") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.421 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36431-done Cc: eliz@gnu.org, 36431-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Thanks for the patch. It partially works: One crash is avoided that I > get without it, but another crash appears every time I want to send an > e-mail with `mew'; see the attached backtrace. I installed the patch below into master. Eli said: > I think we should do 2. I also did that. Stefan From unknown Fri Jun 13 10:53:52 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 07 Aug 2019 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator