From unknown Sat Aug 09 04:56:40 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#23595 <23595@debbugs.gnu.org> To: bug#23595 <23595@debbugs.gnu.org> Subject: Status: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) Reply-To: bug#23595 <23595@debbugs.gnu.org> Date: Sat, 09 Aug 2025 11:56:40 +0000 retitle 23595 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, = Git, RCS) reassign 23595 emacs submitter 23595 Uwe Brauer severity 23595 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Sat May 21 09:02:54 2016 Received: (at submit) by debbugs.gnu.org; 21 May 2016 13:02:55 +0000 Received: from localhost ([127.0.0.1]:59595 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b46YP-0001vU-B1 for submit@debbugs.gnu.org; Sat, 21 May 2016 09:02:54 -0400 Received: from eggs.gnu.org ([208.118.235.92]:38258) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b46YM-0001vG-Kr for submit@debbugs.gnu.org; Sat, 21 May 2016 09:02:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b46YC-00031I-AZ for submit@debbugs.gnu.org; Sat, 21 May 2016 09:02:45 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:42893) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b46YC-000319-6S for submit@debbugs.gnu.org; Sat, 21 May 2016 09:02:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57450) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b46Y6-0002k9-6J for bug-gnu-emacs@gnu.org; Sat, 21 May 2016 09:02:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b46Y0-0002zp-AB for bug-gnu-emacs@gnu.org; Sat, 21 May 2016 09:02:33 -0400 Received: from mail-wm0-x235.google.com ([2a00:1450:400c:c09::235]:36629) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b46Xz-0002zl-Em for bug-gnu-emacs@gnu.org; Sat, 21 May 2016 09:02:28 -0400 Received: by mail-wm0-x235.google.com with SMTP id n129so17596047wmn.1 for ; Sat, 21 May 2016 06:02:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mat.ucm.es; s=google; h=from:to:subject:date:message-id:user-agent:mime-version; bh=fDlde/LYJmDjt/UX5VJ3dsiVdNxM/yFrSr0llpKB88E=; b=KHZVAhVnHt5l0tYq73bLG3jSaWNGt6NERxYmiiu3Ic+9FyEYjKABIzuF6Se0ZReQCB fI/pTIM39yG3BITyJhaOBLVLemVyuMlRJr8tvSx+KUI9ROmyx9+UOmOTks0jgeuF9a4s FQPr0nRf5oYMUXijJE/f1kLv0kzmKo4LSunKk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:user-agent :mime-version; bh=fDlde/LYJmDjt/UX5VJ3dsiVdNxM/yFrSr0llpKB88E=; b=C3jOxZgVP3bF3eUirrjPqixwxCGTau6S8b368xz8DtLHTlCcSmbxIkmIxPOJCVrfrA Bpy9Lmkay4Gz57ulNk49mdxurNVm/wuHXQhNfb/v632XLFsPTOA/T5h7VliHSj6LHLBb q2U25Zp/njpUdjNAahZmAUB9MFTmc2FI6qoc435Uip9S6Z3A4LC1fkIMKusTdhYHZ+2g cbyagVgqnuIRaPqvzcYuqkWl7TyQjITXYpXqGfSwk8NBdJRqgKc5C/tZewsFv++db8zd qSDJG7lU/LFNh9VZ+Z8Q9zx6iXUHJLxKE3XKbZzrAMP1KPk0Se8EyUE/MHWt5G0G054T UYFQ== X-Gm-Message-State: AOPr4FWTNp4K/fCVUdeL22EjA65vmLhdYDeNTNS1R0IfiLHCFmajwt+jgs91d3unSNU/AwPP X-Received: by 10.28.234.130 with SMTP id g2mr8981860wmi.42.1463835745124; Sat, 21 May 2016 06:02:25 -0700 (PDT) Received: from Burrurr (84.red-79-153-137.dynamicip.rima-tde.net. [79.153.137.84]) by smtp.gmail.com with ESMTPSA id kc2sm24921606wjb.5.2016.05.21.06.02.22 for (version=TLS1 cipher=AES128-SHA bits=128/128); Sat, 21 May 2016 06:02:22 -0700 (PDT) X-Mailer: emacs 25.1.50.1 (via feedmail 11-beta-1 I) From: Uwe Brauer To: bug-gnu-emacs@gnu.org Subject: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) X-Hashcash: 1:20:160521:bug-gnu-emacs@gnu.org::fPuxEHRKA4CB+9uD:00000000000000000000000000000000000000004OBT Date: Sat, 21 May 2016 13:02:17 +0000 Message-ID: <87bn3z4l9i.fsf@mat.ucm.es> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) --=-=-= Content-Type: text/plain The LaTeX file attached, which contains Chinese/Japanese chars in UTF-8(16) coding, can be compiled successfully with LaTeX (texlive 2015). The following fails: emacs -Q Register the file (either with GIT, HG, RCS) modify commit and then run vc-diff The resulting diff contains either rubbish or fails to run. Files attached. In GNU Emacs 25.1.50.1 (i686-pc-linux-gnu, GTK+ Version 2.20.1) of 2016-01-19 built on Gilgamesch Repository revision: e2203fb3260d959661eba307db0e289143698c5e Windowing system distributor 'The X.Org Foundation', version 11.0.10706000 System Description: Ubuntu 10.04.4 LTS Configured using: 'configure --prefix=/opt/emacs25/' Configured features: XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS NOTIFY GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK2 X11 Important settings: value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix Major mode: Group Minor modes in effect: gnus-topic-mode: t cursor-sensor-mode: t TeX-PDF-mode: t TeX-source-correlate-mode: t global-diff-hl-mode: t diff-auto-refine-mode: t display-time-mode: t global-orglink-mode: t async-bytecomp-package-mode: t better-registers: t recentf-mode: t gnus-undo-mode: t shell-dirtrack-mode: t tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t buffer-read-only: t line-number-mode: t transient-mark-mode: t abbrev-mode: t Recent messages: nnimap read 175k from imap.gmail.com nnimap read 180k from imap.gmail.com nnimap read 193k from imap.gmail.com nnimap read 202k from imap.gmail.com nnimap read 206k from imap.gmail.com nnimap read 225k from imap.gmail.com Reading active file from archive via nnml...done Reading active file via nndraft...done Checking new news...done Invalid face reference: font-lock-comment-warn-face [18 times] Load-path shadows: /home/oub/emacs/site-lisp/packages/vm-8.2.0b/lisp/vm-pcrisis hides /home/oub/emacs/site-lisp/versch/vm-pcrisis /home/oub/emacs/site-lisp/packages/auctex-git/auctex/texmathp hides /home/oub/emacs/site-lisp/versch/texmathp /home/oub/emacs/site-lisp/packages/personal-lisp/test hides /home/oub/emacs/site-lisp/versch/test /home/oub/emacs/site-lisp/packages/remember-2.0/remember hides /home/oub/emacs/site-lisp/versch/remember /home/oub/emacs/site-lisp/packages/personal-lisp/extra hides /home/oub/emacs/site-lisp/versch/extra /home/oub/emacs/site-lisp/babel hides /home/oub/emacs/site-lisp/versch/babel /home/oub/emacs/site-lisp/versch/org-addons hides /home/oub/emacs/init/org-addons /home/oub/emacs/site-lisp/versch/bm hides /home/oub/.emacs.d/elpa/bm-20151222.1603/bm /home/oub/emacs/site-lisp/versch/latex-pretty-symbols hides /home/oub/.emacs.d/elpa/latex-pretty-symbols-20151112.244/latex-pretty-symbols /home/oub/emacs/site-lisp/versch/markdown-mode hides /home/oub/.emacs.d/elpa/markdown-mode-20160513.618/markdown-mode /home/oub/.emacs.d/elpa/helm-20160520.1124/helm-multi-match hides /home/oub/.emacs.d/elpa/helm-core-20160519.2213/helm-multi-match /home/oub/emacs/site-lisp/versch/matlab-publish hides /home/oub/.emacs.d/elpa/matlab-mode-20160416.34/matlab-publish /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-mac-link hides /home/oub/.emacs.d/elpa/org-mac-link-20160109.1443/org-mac-link /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox hides /home/oub/.emacs.d/elpa/org-20160516/ox /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-texinfo hides /home/oub/.emacs.d/elpa/org-20160516/ox-texinfo /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-publish hides /home/oub/.emacs.d/elpa/org-20160516/ox-publish /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-org hides /home/oub/.emacs.d/elpa/org-20160516/ox-org /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-odt hides /home/oub/.emacs.d/elpa/org-20160516/ox-odt /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-md hides /home/oub/.emacs.d/elpa/org-20160516/ox-md /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-man hides /home/oub/.emacs.d/elpa/org-20160516/ox-man /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-latex hides /home/oub/.emacs.d/elpa/org-20160516/ox-latex /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-icalendar hides /home/oub/.emacs.d/elpa/org-20160516/ox-icalendar /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-html hides /home/oub/.emacs.d/elpa/org-20160516/ox-html /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-beamer hides /home/oub/.emacs.d/elpa/org-20160516/ox-beamer /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-ascii hides /home/oub/.emacs.d/elpa/org-20160516/ox-ascii /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org hides /home/oub/.emacs.d/elpa/org-20160516/org /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-w3m hides /home/oub/.emacs.d/elpa/org-20160516/org-w3m /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-version hides /home/oub/.emacs.d/elpa/org-20160516/org-version /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-timer hides /home/oub/.emacs.d/elpa/org-20160516/org-timer /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-table hides /home/oub/.emacs.d/elpa/org-20160516/org-table /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-src hides /home/oub/.emacs.d/elpa/org-20160516/org-src /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-rmail hides /home/oub/.emacs.d/elpa/org-20160516/org-rmail /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-protocol hides /home/oub/.emacs.d/elpa/org-20160516/org-protocol /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-plot hides /home/oub/.emacs.d/elpa/org-20160516/org-plot /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-pcomplete hides /home/oub/.emacs.d/elpa/org-20160516/org-pcomplete /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-mouse hides /home/oub/.emacs.d/elpa/org-20160516/org-mouse /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-mobile hides /home/oub/.emacs.d/elpa/org-20160516/org-mobile /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-mhe hides /home/oub/.emacs.d/elpa/org-20160516/org-mhe /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-macs hides /home/oub/.emacs.d/elpa/org-20160516/org-macs /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-macro hides /home/oub/.emacs.d/elpa/org-20160516/org-macro /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-loaddefs hides /home/oub/.emacs.d/elpa/org-20160516/org-loaddefs /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-list hides /home/oub/.emacs.d/elpa/org-20160516/org-list /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-lint hides /home/oub/.emacs.d/elpa/org-20160516/org-lint /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-irc hides /home/oub/.emacs.d/elpa/org-20160516/org-irc /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-install hides /home/oub/.emacs.d/elpa/org-20160516/org-install /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-inlinetask hides /home/oub/.emacs.d/elpa/org-20160516/org-inlinetask /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-info hides /home/oub/.emacs.d/elpa/org-20160516/org-info /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-indent hides /home/oub/.emacs.d/elpa/org-20160516/org-indent /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-id hides /home/oub/.emacs.d/elpa/org-20160516/org-id /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-habit hides /home/oub/.emacs.d/elpa/org-20160516/org-habit /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-gnus hides /home/oub/.emacs.d/elpa/org-20160516/org-gnus /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-footnote hides /home/oub/.emacs.d/elpa/org-20160516/org-footnote /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-feed hides /home/oub/.emacs.d/elpa/org-20160516/org-feed /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-faces hides /home/oub/.emacs.d/elpa/org-20160516/org-faces /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-eshell hides /home/oub/.emacs.d/elpa/org-20160516/org-eshell /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-entities hides /home/oub/.emacs.d/elpa/org-20160516/org-entities /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-element hides /home/oub/.emacs.d/elpa/org-20160516/org-element /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-docview hides /home/oub/.emacs.d/elpa/org-20160516/org-docview /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-datetree hides /home/oub/.emacs.d/elpa/org-20160516/org-datetree /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-ctags hides /home/oub/.emacs.d/elpa/org-20160516/org-ctags /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-crypt hides /home/oub/.emacs.d/elpa/org-20160516/org-crypt /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-compat hides /home/oub/.emacs.d/elpa/org-20160516/org-compat /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-colview hides /home/oub/.emacs.d/elpa/org-20160516/org-colview /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-clock hides /home/oub/.emacs.d/elpa/org-20160516/org-clock /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-capture hides /home/oub/.emacs.d/elpa/org-20160516/org-capture /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-bibtex hides /home/oub/.emacs.d/elpa/org-20160516/org-bibtex /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-bbdb hides /home/oub/.emacs.d/elpa/org-20160516/org-bbdb /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-attach hides /home/oub/.emacs.d/elpa/org-20160516/org-attach /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-archive hides /home/oub/.emacs.d/elpa/org-20160516/org-archive /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-agenda hides /home/oub/.emacs.d/elpa/org-20160516/org-agenda /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob hides /home/oub/.emacs.d/elpa/org-20160516/ob /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-tangle hides /home/oub/.emacs.d/elpa/org-20160516/ob-tangle /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-table hides /home/oub/.emacs.d/elpa/org-20160516/ob-table /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-stan hides /home/oub/.emacs.d/elpa/org-20160516/ob-stan /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-sqlite hides /home/oub/.emacs.d/elpa/org-20160516/ob-sqlite /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-sql hides /home/oub/.emacs.d/elpa/org-20160516/ob-sql /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-shen hides /home/oub/.emacs.d/elpa/org-20160516/ob-shen /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-shell hides /home/oub/.emacs.d/elpa/org-20160516/ob-shell /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-sed hides /home/oub/.emacs.d/elpa/org-20160516/ob-sed /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-screen hides /home/oub/.emacs.d/elpa/org-20160516/ob-screen /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-scheme hides /home/oub/.emacs.d/elpa/org-20160516/ob-scheme /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-scala hides /home/oub/.emacs.d/elpa/org-20160516/ob-scala /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-sass hides /home/oub/.emacs.d/elpa/org-20160516/ob-sass /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ruby hides /home/oub/.emacs.d/elpa/org-20160516/ob-ruby /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ref hides /home/oub/.emacs.d/elpa/org-20160516/ob-ref /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-python hides /home/oub/.emacs.d/elpa/org-20160516/ob-python /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-processing hides /home/oub/.emacs.d/elpa/org-20160516/ob-processing /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-plantuml hides /home/oub/.emacs.d/elpa/org-20160516/ob-plantuml /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-picolisp hides /home/oub/.emacs.d/elpa/org-20160516/ob-picolisp /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-perl hides /home/oub/.emacs.d/elpa/org-20160516/ob-perl /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-org hides /home/oub/.emacs.d/elpa/org-20160516/ob-org /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-octave hides /home/oub/.emacs.d/elpa/org-20160516/ob-octave /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ocaml hides /home/oub/.emacs.d/elpa/org-20160516/ob-ocaml /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-mscgen hides /home/oub/.emacs.d/elpa/org-20160516/ob-mscgen /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-maxima hides /home/oub/.emacs.d/elpa/org-20160516/ob-maxima /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-matlab hides /home/oub/.emacs.d/elpa/org-20160516/ob-matlab /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-makefile hides /home/oub/.emacs.d/elpa/org-20160516/ob-makefile /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-lob hides /home/oub/.emacs.d/elpa/org-20160516/ob-lob /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-lisp hides /home/oub/.emacs.d/elpa/org-20160516/ob-lisp /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-lilypond hides /home/oub/.emacs.d/elpa/org-20160516/ob-lilypond /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ledger hides /home/oub/.emacs.d/elpa/org-20160516/ob-ledger /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-latex hides /home/oub/.emacs.d/elpa/org-20160516/ob-latex /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-keys hides /home/oub/.emacs.d/elpa/org-20160516/ob-keys /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-js hides /home/oub/.emacs.d/elpa/org-20160516/ob-js /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-java hides /home/oub/.emacs.d/elpa/org-20160516/ob-java /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-io hides /home/oub/.emacs.d/elpa/org-20160516/ob-io /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-haskell hides /home/oub/.emacs.d/elpa/org-20160516/ob-haskell /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-groovy hides /home/oub/.emacs.d/elpa/org-20160516/ob-groovy /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-gnuplot hides /home/oub/.emacs.d/elpa/org-20160516/ob-gnuplot /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-fortran hides /home/oub/.emacs.d/elpa/org-20160516/ob-fortran /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-forth hides /home/oub/.emacs.d/elpa/org-20160516/ob-forth /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-exp hides /home/oub/.emacs.d/elpa/org-20160516/ob-exp /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-eval hides /home/oub/.emacs.d/elpa/org-20160516/ob-eval /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-emacs-lisp hides /home/oub/.emacs.d/elpa/org-20160516/ob-emacs-lisp /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ebnf hides /home/oub/.emacs.d/elpa/org-20160516/ob-ebnf /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-dot hides /home/oub/.emacs.d/elpa/org-20160516/ob-dot /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ditaa hides /home/oub/.emacs.d/elpa/org-20160516/ob-ditaa /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-css hides /home/oub/.emacs.d/elpa/org-20160516/ob-css /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-core hides /home/oub/.emacs.d/elpa/org-20160516/ob-core /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-coq hides /home/oub/.emacs.d/elpa/org-20160516/ob-coq /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-comint hides /home/oub/.emacs.d/elpa/org-20160516/ob-comint /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-clojure hides /home/oub/.emacs.d/elpa/org-20160516/ob-clojure /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-calc hides /home/oub/.emacs.d/elpa/org-20160516/ob-calc /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-awk hides /home/oub/.emacs.d/elpa/org-20160516/ob-awk /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-asymptote hides /home/oub/.emacs.d/elpa/org-20160516/ob-asymptote /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-abc hides /home/oub/.emacs.d/elpa/org-20160516/ob-abc /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-R hides /home/oub/.emacs.d/elpa/org-20160516/ob-R /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-J hides /home/oub/.emacs.d/elpa/org-20160516/ob-J /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-C hides /home/oub/.emacs.d/elpa/org-20160516/ob-C /home/oub/emacs/site-lisp/versch/json hides /opt/emacs25/share/emacs/25.1.50/lisp/json /home/oub/emacs/site-lisp/versch/ffap hides /opt/emacs25/share/emacs/25.1.50/lisp/ffap /home/oub/emacs/site-lisp/versch/abbrev hides /opt/emacs25/share/emacs/25.1.50/lisp/abbrev /home/oub/ALLES/vc-emacs-neu/vc hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc /home/oub/ALLES/vc-emacs-neu/vc-svn hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-svn /home/oub/ALLES/vc-emacs-neu/vc-src hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-src /home/oub/ALLES/vc-emacs-neu/vc-sccs hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-sccs /home/oub/ALLES/vc-emacs-neu/vc-rcs hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-rcs /home/oub/ALLES/vc-emacs-neu/vc-mtn hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-mtn /home/oub/ALLES/vc-emacs-neu/vc-hooks hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-hooks /home/oub/ALLES/vc-emacs-neu/vc-hg hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-hg /home/oub/ALLES/vc-emacs-neu/vc-git hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-git /home/oub/ALLES/vc-emacs-neu/vc-filewise hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-filewise /home/oub/ALLES/vc-emacs-neu/vc-dispatcher hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-dispatcher /home/oub/ALLES/vc-emacs-neu/vc-dir hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-dir /home/oub/ALLES/vc-emacs-neu/vc-dav hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-dav /home/oub/ALLES/vc-emacs-neu/vc-cvs hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-cvs /home/oub/ALLES/vc-emacs-neu/vc-bzr hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-bzr /home/oub/ALLES/vc-emacs-neu/vc-annotate hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/vc-annotate /home/oub/ALLES/vc-emacs-neu/smerge-mode hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/smerge-mode /home/oub/ALLES/vc-emacs-neu/pcvs hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/pcvs /home/oub/ALLES/vc-emacs-neu/pcvs-util hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/pcvs-util /home/oub/ALLES/vc-emacs-neu/pcvs-parse hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/pcvs-parse /home/oub/ALLES/vc-emacs-neu/pcvs-info hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/pcvs-info /home/oub/ALLES/vc-emacs-neu/pcvs-defs hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/pcvs-defs /home/oub/ALLES/vc-emacs-neu/log-view hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/log-view /home/oub/ALLES/vc-emacs-neu/log-edit hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/log-edit /home/oub/ALLES/vc-emacs-neu/emerge hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/emerge /home/oub/ALLES/vc-emacs-neu/ediff hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff /home/oub/ALLES/vc-emacs-neu/ediff-wind hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-wind /home/oub/ALLES/vc-emacs-neu/ediff-vers hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-vers /home/oub/ALLES/vc-emacs-neu/ediff-util hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-util /home/oub/ALLES/vc-emacs-neu/ediff-ptch hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-ptch /home/oub/ALLES/vc-emacs-neu/ediff-mult hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-mult /home/oub/ALLES/vc-emacs-neu/ediff-merg hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-merg /home/oub/ALLES/vc-emacs-neu/ediff-init hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-init /home/oub/ALLES/vc-emacs-neu/ediff-hook hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-hook /home/oub/ALLES/vc-emacs-neu/ediff-help hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-help /home/oub/ALLES/vc-emacs-neu/ediff-diff hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/ediff-diff /home/oub/ALLES/vc-emacs-neu/diff hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/diff /home/oub/ALLES/vc-emacs-neu/diff-mode hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/diff-mode /home/oub/ALLES/vc-emacs-neu/cvs-status hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/cvs-status /home/oub/ALLES/vc-emacs-neu/compare-w hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/compare-w /home/oub/ALLES/vc-emacs-neu/add-log hides /opt/emacs25/share/emacs/25.1.50/lisp/vc/add-log /home/oub/emacs/site-lisp/packages/remember-2.0/remember hides /opt/emacs25/share/emacs/25.1.50/lisp/textmodes/remember /home/oub/emacs/site-lisp/packages/personal-lisp/refill hides /opt/emacs25/share/emacs/25.1.50/lisp/textmodes/refill /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-texinfo hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-texinfo /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-publish hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-publish /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-org hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-org /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-odt hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-odt /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-md hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-md /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-man hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-man /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-latex hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-latex /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-icalendar hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-icalendar /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-html hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-html /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-beamer hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-beamer /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ox-ascii hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ox-ascii /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-w3m hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-w3m /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-version hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-version /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-timer hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-timer /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-table hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-table /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-src hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-src /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-rmail hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-rmail /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-protocol hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-protocol /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-plot hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-plot /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-pcomplete hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-pcomplete /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-mouse hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-mouse /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-mobile hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-mobile /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-mhe hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-mhe /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-macs hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-macs /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-macro hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-macro /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-loaddefs hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-loaddefs /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-list hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-list /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-irc hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-irc /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-install hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-install /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-inlinetask hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-inlinetask /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-info hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-info /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-indent hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-indent /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-id hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-id /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-habit hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-habit /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-gnus hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-gnus /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-footnote hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-footnote /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-feed hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-feed /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-faces hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-faces /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-eshell hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-eshell /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-entities hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-entities /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-element hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-element /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-docview hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-docview /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-datetree hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-datetree /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-ctags hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-ctags /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-crypt hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-crypt /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-compat hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-compat /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-colview hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-colview /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-clock hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-clock /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-capture hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-capture /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-bibtex hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-bibtex /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-bbdb hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-bbdb /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-attach hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-attach /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-archive hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-archive /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/org-agenda hides /opt/emacs25/share/emacs/25.1.50/lisp/org/org-agenda /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-tangle hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-tangle /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-table hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-table /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-sqlite hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-sqlite /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-sql hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-sql /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-shen hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-shen /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-screen hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-screen /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-scheme hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-scheme /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-scala hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-scala /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-sass hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-sass /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ruby hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-ruby /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ref hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-ref /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-python hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-python /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-plantuml hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-plantuml /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-picolisp hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-picolisp /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-perl hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-perl /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-org hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-org /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-octave hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-octave /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ocaml hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-ocaml /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-mscgen hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-mscgen /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-maxima hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-maxima /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-matlab hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-matlab /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-makefile hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-makefile /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-lob hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-lob /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-lisp hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-lisp /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-lilypond hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-lilypond /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ledger hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-ledger /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-latex hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-latex /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-keys hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-keys /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-js hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-js /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-java hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-java /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-io hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-io /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-haskell hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-haskell /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-gnuplot hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-gnuplot /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-fortran hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-fortran /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-exp hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-exp /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-eval hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-eval /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-emacs-lisp hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-emacs-lisp /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-dot hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-dot /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-ditaa hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-ditaa /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-css hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-css /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-core hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-core /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-comint hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-comint /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-clojure hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-clojure /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-calc hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-calc /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-awk hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-awk /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-asymptote hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-asymptote /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-R hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-R /home/oub/.emacs.d/elpa/org-plus-contrib-20160516/ob-C hides /opt/emacs25/share/emacs/25.1.50/lisp/org/ob-C /home/oub/.emacs.d/elpa/soap-client-3.1.1/soap-inspect hides /opt/emacs25/share/emacs/25.1.50/lisp/net/soap-inspect /home/oub/.emacs.d/elpa/soap-client-3.1.1/soap-client hides /opt/emacs25/share/emacs/25.1.50/lisp/net/soap-client /home/oub/emacs/site-lisp/versch/quickurl hides /opt/emacs25/share/emacs/25.1.50/lisp/net/quickurl /home/oub/emacs/site-lisp/versch/smtpmail hides /opt/emacs25/share/emacs/25.1.50/lisp/mail/smtpmail /home/oub/emacs/site-lisp/versch/hashcash hides /opt/emacs25/share/emacs/25.1.50/lisp/mail/hashcash /home/oub/emacs/site-lisp/versch/feedmail hides /opt/emacs25/share/emacs/25.1.50/lisp/mail/feedmail /home/oub/emacs/site-lisp/versch/hebrew hides /opt/emacs25/share/emacs/25.1.50/lisp/language/hebrew /home/oub/emacs/site-lisp/packages/personal-lisp/lisp hides /opt/emacs25/share/emacs/25.1.50/lisp/emacs-lisp/lisp /home/oub/emacs/site-lisp/versch/longlines hides /opt/emacs25/share/emacs/25.1.50/lisp/obsolete/longlines /home/oub/emacs/site-lisp/packages/iso-pkg/iso-swed hides /opt/emacs25/share/emacs/25.1.50/lisp/obsolete/iso-swed /home/oub/emacs/site-lisp/packages/iso-pkg/iso-insert hides /opt/emacs25/share/emacs/25.1.50/lisp/obsolete/iso-insert /home/oub/emacs/site-lisp/packages/iso-pkg/iso-acc hides /opt/emacs25/share/emacs/25.1.50/lisp/obsolete/iso-acc Features: (shadow gnus-cite hashcash footnote emacsbug gnus-topic cursor-sensor utf-7 nndraft nnmh bbdb-gnus bbdb-snarf mail-extr nnnil comment sort org-eldoc preview prv-emacs tex-fold reftex-dcr reftex-auc flyspell ispell cdlatex texmathp auto-capitalize tex-bar toolbar-x tex-buf font-latex latex tex-style tex dbus crm latexenc ibuf-ext ibuffer ibuffer-loaddefs map character-fold misearch multi-isearch dired-aux vc-mtn vc-git vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs diff-hl-dired vc-hg org-rmail org-mhe org-irc org-info org-gnus org-docview doc-view jka-compr image-mode org-bbdb org-w3m ob-R ffap browse-kill-ring+ browse-kill-ring uimage url-ftp url-file url-dired image-file iimage autobookmarks bookmark pp elpakit server shadchen latex-unicode-math-mode robin code-library gist gh-gist gh-oauth gh-api logito gh-cache pcache gh-auth gh-common gh-url gh-profile rx eieio-base writegood-mode writeroom-mode visual-fill-column xemacs-compat dired-zip calfw-ical calfw-cal calfw weekly-view cal-desk-calendar lunar solar cal-dst holidays hol-loaddefs bm diff-hl vc-dir goto-chg ahg vc-annotate grep ewoc log-edit pcvs-util add-log diff-mode align reftex-sel bibretrieve time monky bindat iso-acc iso-cvt ref-master my-hg-commit auto-insert-tkld date addl dired-tar all iso-cleanupmath hgignore-mode vc-change-login backup-each-save vc-ensure-checkin my-vc-addons vc vc-dispatcher latex-wcount dob-words latexdiff daily-journal gnus-dired matlab-addons matlab-publish matlab_init cus-edit cus-start cus-load company-matlab-shell matlab tempo company matlab-boxquote my-refill-msg my-sc-addons gnus-encrypt org_init org-protocol ox-pandoc ox-md ox-mediawiki ob-rec rec-mode org-tracktable orgtbl-join orglink org-auctex-keys ox-odt rng-loc rng-uri rng-parse rng-match rng-dt rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util my-org-ref org-ref org-ref-helm-bibtex helm-bibtex bibtex-completion helm-plugin helm-utils helm-net helm-help biblio biblio-dissemin biblio-hal biblio-dblp biblio-crossref biblio-arxiv biblio-doi biblio-core edebug-x edebug which-func imenu tar-mode let-alist url-queue ido hl-line autoload lisp-mnt mm-archive network-stream nsm starttls org-ref-helm helm helm-source eieio-compat helm-multi-match helm-lib helm-config helm-easymenu edmacro kmacro async-bytecomp async reftex-cite parsebib org-ref-glossary org-ref-utils org-ref-bibtex org-ref-citeproc key-chord hydra lv doi-utils org-bibtex bibtex f s ucs-normalize dash ob-octave ob-org org-mime org-readme http-post-simple url-http url-auth url-gw puny seq yaoddmuse thingatpt skeleton sgml-mode better-registers list-register query-replace-region ediff-addons re-builder extview pandoc anti-niqqud sigadapt-simple sigadapt sendmail bbdbadapt-sc texify-article TeX-escape-region supercite regi bbdbciteadpt filladapt-pat filladapt next-longline recentf tree-widget my-addons org-addons ox-latex ox-icalendar ox-html ox-ascii ox-publish ox org-element avl-tree org-table org-install sp-eng-ger-fr-minor folding-isearch folding gnus-init nnmairix nnml gnus-html url-cache url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf bbdbadapt-top-posting gnus-diary nndiary nnrss xml mm-url gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime dig nntp gnus-cache nnir gnus-sum gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source tls gnutls utf7 netrc parse-time gnus-spec gnus-int gnus-range message dired dired-loaddefs rfc822 mml mml-sec gmm-utils mailheader gnus-win gnus gnus-ems wid-edit nnoo nnheader mail-utils pgp-mime-attach-key my-smiley gnus-move-display-attachment epa-file epa derived epg boxquote rect icalendar diary-lib diary-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailcap shr dom subr-x browse-url bbdb-init bbdb-autoloads bbdbadapt-ispell bbdbadapt-gcc bbdbadapt-encrypt my-bbdb-addons remember-bbdb bbdb-com warnings mailabbrev bbdb timezone cl org-remember org-datetree org-location-google-maps org-agenda google-maps google-maps-static url-util google-maps-geocode google-maps-base json org org-macro org-footnote org-pcomplete org-list org-faces org-entities noutline outline easy-mmode org-version ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint ob-core ob-eval org-compat org-macs org-loaddefs format-spec find-func cal-menu calendar cal-loaddefs remember extra emacs_keys my-hebrew-init toggle my-mark quail my-latex-env my-auctex-init reftex reftex-loaddefs reftex-vars latex-keys my-fill-latex-sentence advice preview-latex tex-site auto-loads flyspell-abbrev-multilang iv-sp-am-br-ger-fr-minor my-change-prettify-list tex-mode compile shell pcomplete comint ansi-color ring finder-inf info package epg-config url-handlers url-parse auth-source cl-seq eieio byte-opt bytecomp byte-compile cl-extra cconv eieio-core cl-macs gv eieio-loaddefs gnus-util mm-util help-fns help-mode easymenu cl-loaddefs pcase cl-lib mail-prsvr password-cache url-vars time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote dbusbind inotify dynamic-setting font-render-setting move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 8 986039 16894) (symbols 24 111306 0) (miscs 20 2301 397) (strings 16 278734 35195) (string-bytes 1 7537116) (vectors 8 105015) (vector-slots 4 2305029 103954) (floats 8 1491 557) (intervals 28 10492 0) (buffers 520 51) (heap 1024 56404 1925)) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename=diff-rcs-result.txt Content-Transfer-Encoding: base64 5Imp5rmh54m54oGm5qWs5pWz4oG05pWz55Ct5o2o5qWu4rWq5oWw4rm05pW44KSy44Cx45iv44C1 4ryy44Sg44C546iz46C644i14KSx4riy4oGh5rmk4oG05pWz55Ct5o2o5qWu4rWq5oWw4rm05pW4 4KSy44Cx45iv44C14ryy44Sg44C546i045i644C04oGk5qWm5pml54iK54mj542k5qWm5pi64oG0 5pWz55Ct5o2o5qWu4rWq5oWw4rm05pW446ig5pGp5pmm4oGm5oWp5rGl5pCKDQo= --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=rcs-bug.txt Debugger entered--Lisp error: (error "Running rcsdiff -q -U0 test-chin-jap.tex...FAILED (status 2)") signal(error ("Running rcsdiff -q -U0 test-chin-jap.tex...FAILED (status 2)")) error("Running %s...FAILED (%s)" "rcsdiff -q -U0 test-chin-jap.tex" "status 2") vc-do-command(" *diff-hl* " 1 "rcsdiff" ("/home/oub/ALLES/Chin-Jap/rc/test-chin-jap.tex") "-q" nil nil "-U0") apply(vc-do-command " *diff-hl* " 1 "rcsdiff" ("/home/oub/ALLES/Chin-Jap/rc/test-chin-jap.tex") ("-q" nil nil "-U0")) vc-rcs-diff(("/home/oub/ALLES/Chin-Jap/rc/test-chin-jap.tex") nil nil " *diff-hl* ") apply(vc-rcs-diff (("/home/oub/ALLES/Chin-Jap/rc/test-chin-jap.tex") nil nil " *diff-hl* ")) vc-call-backend(RCS diff ("/home/oub/ALLES/Chin-Jap/rc/test-chin-jap.tex") nil nil " *diff-hl* ") diff-hl-changes-buffer("/home/oub/ALLES/Chin-Jap/rc/test-chin-jap.tex" RCS) diff-hl-changes() diff-hl-update() run-hooks(after-save-hook) basic-save-buffer(t) save-buffer(1) funcall-interactively(save-buffer 1) call-interactively(save-buffer nil nil) command-execute(save-buffer) --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=diff-result.txt diff --git a/test-chin-jap.tex b/test-chin-jap.tex index 7f0cb2d..fa9824d 100644 Binary files a/test-chin-jap.tex and b/test-chin-jap.tex differ --=-=-= Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename=diff-result-hg.txt Content-Transfer-Encoding: base64 5pGp5pmm4oCt54ig46C146Sy5oyx44Gj44Cy46Cx4oG05pWz55Ct5o2o5qWu4rWq5oWw4rm05pW4 4KmC5qWu5oWy56Sg5pmp5rGl4oG05pWz55Ct5o2o5qWu4rWq5oWw4rm05pW44oGo5oWz4oGj5qGh 5rmn5pWkDQo= --=-=-= Content-Type: application/octet-stream; charset=utf-16 Content-Disposition: attachment; filename=test-chin-jap.tex Content-Transfer-Encoding: base64 /v8ACgBcAGQAbwBjAHUAbQBlAG4AdABjAGwAYQBzAHMAewBhAHIAdABpAGMAbABlAH0ACgBcAHUA cwBlAHAAYQBjAGsAYQBnAGUAWwBiAG8AbABkAGYAbwBuAHQALABzAGwAYQBuAHQAZgBvAG4AdABd AHsAeABlAEMASgBLAH0ACgBcAHUAcwBlAHAAYQBjAGsAYQBnAGUAewB4AGMAbwBsAG8AcgB9AAoA XABzAGUAdABDAEoASwBtAGEAaQBuAGYAbwBuAHQAewBBAFIAIABQAEwAIABTAHUAbgBnAHQAaQBM ACAARwBCAH0ACgAKAFwAYgBlAGcAaQBuAHsAZABvAGMAdQBtAGUAbgB0AH0ACgAKAAoAVABoAGkA cwAgAGkAcwAgAGEAbgAgAGUAeABhAG0AcABsAGUAIABvAGYAIABhACAAbABhAHQAZQB4ACAAZABv AGMAdQBtAGUAbgB0ACwAIAB3AGgAaQBjAGgAIABjAG8AbgB0AGEAaQBuAHMAIABDAGgAaQBuAGUA cwBlACAAbwByAAoASgBhAHAAYQBuAGEAcwBlACAAYwBoAGEAcgBzAC4AIABJAHQAIABjAGEAbgAg AGIAZQAgAGMAbwBtAHAAaQBsAGUAZAAgAHcAaQB0AGgAIAB4AGUAbABhAHQAZQB4AC4AIABIAG8A dwBlAHYAZQByACAACgBkAGkAZgBmAC0AaABsAC0AZABpAGYAZgAtAGcAbwB0AG8ALQBoAHUAbgBr ACAAZgBhAGkAbABzAC4ACgAKAFwAYgBlAGcAaQBuAHsAdABhAGIAdQBsAGEAcgB9AHsAbABsAH0A Co/ZZi8AOgAmACAAOQA3ADUAIABrAGcAXABcAAqV9AA6ACAAIAAmACAAIAAxADMAIAAglpQAXABc AAoAXABlAG4AZAB7AHQAYQBiAHUAbABhAHIAfQAKAAoAXABlAG4AZAB7AGQAbwBjAHUAbQBlAG4A dAB9 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 07:52:13 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 11:52:13 +0000 Received: from localhost ([127.0.0.1]:33525 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4oP6-0003Sq-Oe for submit@debbugs.gnu.org; Mon, 23 May 2016 07:52:12 -0400 Received: from forward10h.cmail.yandex.net ([87.250.230.221]:49316) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4oP5-0003Sd-2I for 23595@debbugs.gnu.org; Mon, 23 May 2016 07:52:11 -0400 Received: from mxback7h.mail.yandex.net (mxback7h.mail.yandex.net [84.201.186.16]) by forward10h.cmail.yandex.net (Yandex) with ESMTP id 7012321AD2; Mon, 23 May 2016 14:52:04 +0300 (MSK) Received: from web25h.yandex.ru (web25h.yandex.ru [84.201.187.159]) by mxback7h.mail.yandex.net (nwsmtp/Yandex) with ESMTP id NxtSUekbEJ-q3vCOGDM; Mon, 23 May 2016 14:52:03 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1464004323; bh=1+qrsW+OntYpyXBrIlcLoXftQAebrxhMsolhCQkRkps=; h=X-Yandex-Sender-Uid:From:To:In-Reply-To:References:Subject: MIME-Version:Message-Id:X-Mailer:Date:Content-Transfer-Encoding: Content-Type; b=d+3bFQXW4Sw3p7QoRWXO/NZj/IOqjOEkytU9l80WYRg/nhUinWI3cIRYh/BPUC9h+ /tNDjTCgFhnpna7RXCtoYjjtAwDvxueBQxP+vmCbrRsh5gtsJTQIJIAAOw3/zLeHxT j76F5b+gdwjjWWFux6muW9j+j3YFI6de58YxP+lc= Authentication-Results: mxback7h.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-ForeignMX: US X-Yandex-Suid-Status: 1 0,1 0,1 32657580 X-Yandex-Sender-Uid: 14017277 Received: by web25h.yandex.ru with HTTP; Mon, 23 May 2016 14:52:03 +0300 From: Dmitry Gutov To: Uwe Brauer , "23595@debbugs.gnu.org" <23595@debbugs.gnu.org> In-Reply-To: <87bn3z4l9i.fsf@mat.ucm.es> References: <87bn3z4l9i.fsf@mat.ucm.es> Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) MIME-Version: 1.0 Message-Id: <1444321464004323@web25h.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Mon, 23 May 2016 14:52:03 +0300 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 23595 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 05/21/2016 04:02 PM, Uwe Brauer wrote: > Register the file (either with GIT, HG, RCS) modify commit and then run > vc-diff > > The resulting diff contains either rubbish or fails to run. > Files attached. It seems, to an extent, be caused by our setting coding-system-for-read inside vc-diff-internal (to utf-16be-with-signature-unix, which is also the value of buffer-file-coding-system). Without that, the result of vc-diff (at least with Git) is "Binary files a/test-chin-jap.tex and b/test-chin-jap.tex differ". Emacs 24.5 does the same. Which is weird, considering both vc-diff-internal and vc-coding-system-for-diff have both been virtually untouched for the last couple of years. But even if we figure out why happens, you (Uwe) probably want Git, Hg, etc, to treat this file as text, and not binary. Only then you'll be able to get meaningful diffs. I don't have a specific advice on that. From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 08:41:59 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 12:41:59 +0000 Received: from localhost ([127.0.0.1]:33548 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4pBH-0004bL-0Z for submit@debbugs.gnu.org; Mon, 23 May 2016 08:41:59 -0400 Received: from mail-wm0-f43.google.com ([74.125.82.43]:36074) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4pBF-0004b8-0U for 23595@debbugs.gnu.org; Mon, 23 May 2016 08:41:57 -0400 Received: by mail-wm0-f43.google.com with SMTP id n129so76564008wmn.1 for <23595@debbugs.gnu.org>; Mon, 23 May 2016 05:41:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mat.ucm.es; s=google; h=from:to:cc:subject:references:reply-to:date:message-id:user-agent :mime-version:content-transfer-encoding; bh=u07YrPV3zENTlxGpQTz3LEXPZPmZIpiromIvYGKsueM=; b=MzA+FhvDuZOEV0GiWWYgnLv2GaEi4kKP4ksMjIODP5EoPEy6yA85gyvKZqhfAf+2i1 vGY49fuVOuZOcQS65kvVkoDhPb1pDX9uxOVcHC9U6fp6Goej9yjNOJkfsihZYK9ubSNr UwjBY3u9HmmA0VUQoWjFRWFsS5b/t+W3OPUfg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:references:reply-to:date :message-id:user-agent:mime-version:content-transfer-encoding; bh=u07YrPV3zENTlxGpQTz3LEXPZPmZIpiromIvYGKsueM=; b=MOXxWAVVPnCsvNVXof8ydgsORD8h17ICAO9JjtuMx7Nw+8RX/tuEA5OsDfJ2gPEAC8 2iJk8Aos87q17S+EtF/jATyqhf1OiMwPWD22/s/Bx84o4xGe7ux6k9BFRtmoqXJvb3Y4 GcfI4l5SoABL/unnMTE3XAPqFoKYqheA/cR2cFkJRsi+9XEbrbRLY1fRzj7qFds6JAEr vURR9i1g0SH9inE03S1CbaRIrYlHtEcWEciJmSiKc4sbHQN9NeHfAM80Hfi4bdqYWSGF HYcQQhKpwYl2Z/akSGULDY0GZAm38r1hCg6VaRV+/wgl8/k8SELZwNHnxJRe9gV95lT/ qnZg== X-Gm-Message-State: AOPr4FVLqVVmSnramIoM8ZNlrQj52oQNpNtjq17G4tqzrCkQnfbEae1moMtL5iZV8JRCRZ+K X-Received: by 10.28.56.4 with SMTP id f4mr16537718wma.70.1464007311137; Mon, 23 May 2016 05:41:51 -0700 (PDT) Received: from Gilgamesch (gilgamesch.quim.ucm.es. [147.96.12.99]) by smtp.gmail.com with ESMTPSA id i190sm13548238wmf.10.2016.05.23.05.41.49 (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 23 May 2016 05:41:49 -0700 (PDT) X-Mailer: emacs 25.1.50.1 (via feedmail 11-beta-1 I) From: Uwe Brauer To: Dmitry Gutov Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> X-Hashcash: 1:20:160523:dgutov@yandex.ru::QvLb6cQ5pkPKpLyF:00qx0 X-Hashcash: 1:20:160523:23595@debbugs.gnu.org::z06oI0FHR8nGCMCh:00000000000000000000000000000000000000005K47 X-Hashcash: 1:20:160523:oub@mat.ucm.es::HA4onjV8zK/NH2e2:0004kqN Date: Mon, 23 May 2016 12:41:47 +0000 Message-ID: <877felj69g.fsf@mat.ucm.es> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 23595 Cc: Uwe Brauer , "23595@debbugs.gnu.org" <23595@debbugs.gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Uwe Brauer Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) > On 05/21/2016 04:02 PM, Uwe Brauer wrote: > But even if we figure out why happens, you (Uwe) probably want Git, > Hg, etc, to treat this file as text, and not binary. Only then you'll > be able to get meaningful diffs. I don't have a specific advice on > that. Right, but it is more dramatic. I don't care so much about the difference for the (short) Asian text, but I do care about the fact that now the file seems to be =C2=ABdoomed=C2=BB in the sense that the diff of e= ach and every new commit ends up in Chinese, that is the file is considered as binary from now on and that is really bad. Strange that nobody noticed that before. I am not sure what to do with this file. Uwe=20 From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 09:17:32 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 13:17:32 +0000 Received: from localhost ([127.0.0.1]:33564 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4pjg-0005QE-0i for submit@debbugs.gnu.org; Mon, 23 May 2016 09:17:32 -0400 Received: from mail-wm0-f41.google.com ([74.125.82.41]:34228) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4pje-0005Q0-SL for 23595@debbugs.gnu.org; Mon, 23 May 2016 09:17:31 -0400 Received: by mail-wm0-f41.google.com with SMTP id z87so7266227wmh.1 for <23595@debbugs.gnu.org>; Mon, 23 May 2016 06:17:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=DSPRmxtTONMB6WZnxeLXn1isi2oSXzcy65CHuZNXRZk=; b=nCX2Bh4o1eHpX8XfxvUZLKhSZn1kY3rwsaoYeb9WF40Bh7p4iuL1YVyZvkez7EtaeO QT3nXq7q+DLtJ60eEp2gxh3Gvx1oVTjKLb40XSgVQN1/rLZ1RDzbpv1jin9tADuZoIK8 x3LPL5o6jJApefpTuKMVJrEjarYkFVsU5lIY4Lm5lFNeqw+F/KhgYmEwfQJGw3GSWejE GpM2vzBMekStKS/HsFk29ZgDQF+Gf2aFem+Ss1xxoZxTdP0nMzLTcUW6S0ujuPQDcENX yFSwaAgFXafTu0tq+2MWxoRvOCF9an9dxuAkow+Kb25dDupj6H2a9A6RvQ2mQh9zjIpc wLUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=DSPRmxtTONMB6WZnxeLXn1isi2oSXzcy65CHuZNXRZk=; b=W7Dm0xYQRF+K/iL4zbAN+UH6UsuUVn2PffwnnxNYBI0d5b1S1x25whfMCXeb0zYHgv 7cwQjpfM86gKDN61z3hDDdoRcyaoxLby0IoUsB9jAsO+joKaVeK1x4s69HxSkyIpeQAy cNnurCWCk9HjRuA6JSJ9X+PWbXb7QuBEU1+yhXW/rpLPse4wwzVI7+DOwMUT8sL4KtVP fDHgFBZVNMWPcmsvvKFY13chrslP55iJamrKQnOrM1og2YKa1fTDiEDpO8+cIbCQynr6 WcNhDtgdiHIZHllvF+cSv1n/8Wy2fOLGWC3B0S6EzO5ITGdnupY2/jAjpZNig+/w81We 2Yzg== X-Gm-Message-State: AOPr4FWNe2Mr4EjDDFwrrP7EFa8h27Y6YnMGeE/8Codb1DJLmrxyuMQLhpgsgbJu4GNcNQ== X-Received: by 10.28.153.80 with SMTP id b77mr17458318wme.71.1464009445251; Mon, 23 May 2016 06:17:25 -0700 (PDT) Received: from [192.168.1.3] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id g3sm35321373wjb.47.2016.05.23.06.17.23 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 23 May 2016 06:17:24 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Uwe Brauer References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <877felj69g.fsf@mat.ucm.es> From: Dmitry Gutov Message-ID: <4e811c98-23c1-0977-4b54-d18d520dc61b@yandex.ru> Date: Mon, 23 May 2016 16:17:22 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1 MIME-Version: 1.0 In-Reply-To: <877felj69g.fsf@mat.ucm.es> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23595 Cc: "23595@debbugs.gnu.org" <23595@debbugs.gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 05/23/2016 03:41 PM, Uwe Brauer wrote: > ... that is the file is considered as > binary from now on and that is really bad. Yes, but it's Git that considers the file to be "binary". Emacs's VC probably can't do anything about that now. Maybe the problem was created by Emacs when you were editing this file. Someone should investigate that, especially if you have a scenario how to produce a problematic file like that from scratch. > Strange that nobody noticed that before. This file might be unique. From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 12:49:06 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 16:49:06 +0000 Received: from localhost ([127.0.0.1]:35015 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4t2P-0002Cb-Tf for submit@debbugs.gnu.org; Mon, 23 May 2016 12:49:06 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42830) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4t2O-0002C7-1x for 23595@debbugs.gnu.org; Mon, 23 May 2016 12:49:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4t2F-0005d1-0W for 23595@debbugs.gnu.org; Mon, 23 May 2016 12:48:58 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:45420) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4t2E-0005cx-Tq; Mon, 23 May 2016 12:48:54 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3405 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b4t2D-0007Vv-GO; Mon, 23 May 2016 12:48:54 -0400 Date: Mon, 23 May 2016 19:48:50 +0300 Message-Id: <83h9do67pp.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <1444321464004323@web25h.yandex.ru> (message from Dmitry Gutov on Mon, 23 May 2016 14:52:03 +0300) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > From: Dmitry Gutov > Date: Mon, 23 May 2016 14:52:03 +0300 > > > The resulting diff contains either rubbish or fails to run. > > Files attached. I don't see any rubbish in the Git output. With RCS, the command signals an error, so more digging is needed to find out what's wrong (although it could be that rcsdiff exits with non-zero status when it sees what looks like binary files). > It seems, to an extent, be caused by our setting coding-system-for-read inside vc-diff-internal (to utf-16be-with-signature-unix, which is also the value of buffer-file-coding-system). > > Without that, the result of vc-diff (at least with Git) is "Binary files a/test-chin-jap.tex and b/test-chin-jap.tex differ". Emacs 24.5 does the same. Setting coding-system-for-read is correct, because the important use case is when the diffs are actually output. The problem is that UTF-16 is not ASCII-compatible, and so text output by Git itself will be mishandled. Another problem is that Git doesn't show the diffs at all. > Which is weird, considering both vc-diff-internal and vc-coding-system-for-diff have both been virtually untouched for the last couple of years. Not sure what do you see as weird. > But even if we figure out why happens, you (Uwe) probably want Git, Hg, etc, to treat this file as text, and not binary. Only then you'll be able to get meaningful diffs. I don't have a specific advice on that. Why can't we invoke "git diff --text"? That should fix the second problem, I think. As for the first problem, we should probably refrain from binding coding-system-for-read to a CODING-SYSTEM for which (coding-system-get CODING-SYSTEM :ascii-compatible-p) returns nil. We should instead bind it to no-conversion and decode the file data parts by hand, skipping the parts that Git itself outputs (yes, this is messy). Patches to that effect are welcome. Bottom line: users who put UTF-16 encoded files into VCS are playing with fire, and are best advised not to do that! From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 12:51:25 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 16:51:25 +0000 Received: from localhost ([127.0.0.1]:35030 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4t4f-0002G1-BP for submit@debbugs.gnu.org; Mon, 23 May 2016 12:51:25 -0400 Received: from eggs.gnu.org ([208.118.235.92]:43322) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4t4d-0002Fn-Md for 23595@debbugs.gnu.org; Mon, 23 May 2016 12:51:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4t4U-0006Gs-Og for 23595@debbugs.gnu.org; Mon, 23 May 2016 12:51:18 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:45441) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4t4U-0006Gg-LS; Mon, 23 May 2016 12:51:14 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3406 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b4t4S-0007nD-CA; Mon, 23 May 2016 12:51:13 -0400 Date: Mon, 23 May 2016 19:51:10 +0300 Message-Id: <83fut867lt.fsf@gnu.org> From: Eli Zaretskii To: Uwe Brauer In-reply-to: <877felj69g.fsf@mat.ucm.es> (message from Uwe Brauer on Mon, 23 May 2016 12:41:47 +0000) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <877felj69g.fsf@mat.ucm.es> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: 23595@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > From: Uwe Brauer > Date: Mon, 23 May 2016 12:41:47 +0000 > Cc: Uwe Brauer , > "23595@debbugs.gnu.org" <23595@debbugs.gnu.org> > > I don't care so much about the difference for the (short) Asian > text, but I do care about the fact that now the file seems to be > «doomed» in the sense that the diff of each and every new commit > ends up in Chinese, that is the file is considered as binary from > now on and that is really bad. I'm guessing that Git uses the same strategy as GNU Diff for detecting binary files, which AFAIR is by looking for binary nulls (Paul can correct me if I'm wrong). And UTF-16 encoded files have lots of binary nulls. However, using the --text switch to "git diff" ought to fix that. Does it? From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 12:52:23 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 16:52:23 +0000 Received: from localhost ([127.0.0.1]:35039 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4t5b-0002Hg-Ln for submit@debbugs.gnu.org; Mon, 23 May 2016 12:52:23 -0400 Received: from eggs.gnu.org ([208.118.235.92]:43639) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4t5Z-0002HQ-NL for 23595@debbugs.gnu.org; Mon, 23 May 2016 12:52:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4t5T-0006Tt-Uo for 23595@debbugs.gnu.org; Mon, 23 May 2016 12:52:16 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:45453) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4t5T-0006Ta-RF; Mon, 23 May 2016 12:52:15 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3407 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b4t5R-0007sE-Oz; Mon, 23 May 2016 12:52:14 -0400 Date: Mon, 23 May 2016 19:52:12 +0300 Message-Id: <83eg8s67k3.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <4e811c98-23c1-0977-4b54-d18d520dc61b@yandex.ru> (message from Dmitry Gutov on Mon, 23 May 2016 16:17:22 +0300) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <877felj69g.fsf@mat.ucm.es> <4e811c98-23c1-0977-4b54-d18d520dc61b@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > From: Dmitry Gutov > Date: Mon, 23 May 2016 16:17:22 +0300 > Cc: "23595@debbugs.gnu.org" <23595@debbugs.gnu.org> > > > Strange that nobody noticed that before. > > This file might be unique. Yes, working with UTF-16 and UCS-4 encoded text files is rather rare and not recommended. From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 13:01:06 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 17:01:06 +0000 Received: from localhost ([127.0.0.1]:35062 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4tE1-0003OY-Ry for submit@debbugs.gnu.org; Mon, 23 May 2016 13:01:06 -0400 Received: from mail-wm0-f42.google.com ([74.125.82.42]:36469) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4tDz-0003HM-Vx for 23595@debbugs.gnu.org; Mon, 23 May 2016 13:01:04 -0400 Received: by mail-wm0-f42.google.com with SMTP id n129so89448174wmn.1 for <23595@debbugs.gnu.org>; Mon, 23 May 2016 10:01:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mat.ucm.es; s=google; h=from:to:cc:subject:references:reply-to:date:message-id:user-agent :mime-version; bh=pIKgdo4vnEzHQqEh419E2CZLnPWuxGaLEa4ckk3wZQE=; b=KrjhHdbUMghv7ScioTtiF8JMJtGK3V03990PIJw0Cqtpcc5ms6cIU+AeZJWL6uaBkK p09lJx8w2ncu5AjqJn2HPFnH05ApZyBw2Bre158dofmDl2AZ0tnmtgGMSCv42pWSr0z1 bcvJrXru5cNlTT920lgZBnCXKIWpGy6xTWsJo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:references:reply-to:date :message-id:user-agent:mime-version; bh=pIKgdo4vnEzHQqEh419E2CZLnPWuxGaLEa4ckk3wZQE=; b=HPpHL6UpoWU6/gaLuh3nhyjYYu450K0vVBapBbbtWOIaYoRku6TFd1OkC/J87QQF+f JeyHIH8Pdxx5M1e9RM4AuMgIQvyTx0RKt+TFIfF3N+lAoUS44Hhav0Lg7ROhIgD1mtaY DSSFtlXcjDRY4mUzkrv3pVXx2B8d0XQ9Dp7tNN2uk9nfoEhqZXHHbem27LbfjBcwQSjy /P2s7yWQFDVfcobbKSBmQCuJAo9dEctzv9uNANPkxY5DVu4WqzuJXpaz9L65mGy5xLnP C2g93lzveu/wf5lstZBv+zMSnBZJwws+JAtO/4UR2tj2tem4lYg9SfQFU8eOEtSvqcz+ P7cA== X-Gm-Message-State: ALyK8tLI1sMH4TrSN/40UoTKbc9nHl3C/Jpr8daRu2sup+4AIM1piRKvM5DCEbfRqITCgSiD X-Received: by 10.194.243.101 with SMTP id wx5mr13156219wjc.132.1464022858053; Mon, 23 May 2016 10:00:58 -0700 (PDT) Received: from Gilgamesch (gilgamesch.quim.ucm.es. [147.96.12.99]) by smtp.gmail.com with ESMTPSA id q76sm14671252wmd.6.2016.05.23.10.00.55 (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 23 May 2016 10:00:56 -0700 (PDT) X-Mailer: emacs 25.1.50.1 (via feedmail 11-beta-1 I) From: Uwe Brauer To: Eli Zaretskii Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> X-Hashcash: 1:20:160523:oub@mat.ucm.es::EBuIHAIMXzxyaktK:0001VGh X-Hashcash: 1:20:160523:dgutov@yandex.ru::OThocIdDf4O0D8OY:03ag7 X-Hashcash: 1:20:160523:eliz@gnu.org::u9LKQw50mVpc0uur:000003pQx X-Hashcash: 1:20:160523:23595@debbugs.gnu.org::rDWGQHhrLBAMJl61:0000000000000000000000000000000000000000A4/e Date: Mon, 23 May 2016 17:00:53 +0000 Message-ID: <8737p8k8u2.fsf@mat.ucm.es> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org, Dmitry Gutov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Uwe Brauer Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) >>> "Eli" == Eli Zaretskii writes: >> From: Dmitry Gutov >> Date: Mon, 23 May 2016 14:52:03 +0300 >> >> > The resulting diff contains either rubbish or fails to run. >> > Files attached. > I don't see any rubbish in the Git output. With RCS, the command > signals an error, so more digging is needed to find out what's wrong > (although it could be that rcsdiff exits with non-zero status when it > sees what looks like binary files). >> It seems, to an extent, be caused by our setting >> coding-system-for-read inside vc-diff-internal (to >> utf-16be-with-signature-unix, which is also the value of >> buffer-file-coding-system). >> >> Without that, the result of vc-diff (at least with Git) is "Binary >> files a/test-chin-jap.tex and b/test-chin-jap.tex differ". Emacs >> 24.5 does the same. > Setting coding-system-for-read is correct, because the important use > case is when the diffs are actually output. The problem is that > UTF-16 is not ASCII-compatible, and so text output by Git itself will > be mishandled. Another problem is that Git doesn't show the diffs at > all. >> Which is weird, considering both vc-diff-internal and >> vc-coding-system-for-diff have both been virtually untouched for the >> last couple of years. > Not sure what do you see as weird. >> But even if we figure out why happens, you (Uwe) probably want Git, >> Hg, etc, to treat this file as text, and not binary. Only then >> you'll be able to get meaningful diffs. I don't have a specific >> advice on that. > Why can't we invoke "git diff --text"? That should fix the second > problem, I think. I thought the problem was caused by the fact that I did not entered that chars, but rather copied it from some tex.stackexchange site, but I see that was not the reason. What is about mercurial?[1] > As for the first problem, we should probably refrain from binding > coding-system-for-read to a CODING-SYSTEM for which > (coding-system-get CODING-SYSTEM :ascii-compatible-p) > returns nil. We should instead bind it to no-conversion and decode > the file data parts by hand, skipping the parts that Git itself > outputs (yes, this is messy). Patches to that effect are welcome. > Bottom line: users who put UTF-16 encoded files into VCS are playing > with fire, and are best advised not to do that! Right, I see, that was just 2 chars in a document which contained latin-1 or UTF8. So Chinese and Japanese programmers are in a disadvantage, no? Footnotes: [1] I don't care so much about RCS in that context. From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 13:31:44 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 17:31:44 +0000 Received: from localhost ([127.0.0.1]:35082 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4thg-0004xr-30 for submit@debbugs.gnu.org; Mon, 23 May 2016 13:31:44 -0400 Received: from eggs.gnu.org ([208.118.235.92]:54384) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4thd-0004xe-Kv for 23595@debbugs.gnu.org; Mon, 23 May 2016 13:31:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4thU-0008E5-Cs for 23595@debbugs.gnu.org; Mon, 23 May 2016 13:31:36 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:45945) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4thU-0008E0-9O; Mon, 23 May 2016 13:31:32 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3434 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b4thS-0002O1-1L; Mon, 23 May 2016 13:31:30 -0400 Date: Mon, 23 May 2016 20:31:27 +0300 Message-Id: <838tz065qo.fsf@gnu.org> From: Eli Zaretskii To: Uwe Brauer In-reply-to: <8737p8k8u2.fsf@mat.ucm.es> (message from Uwe Brauer on Mon, 23 May 2016 17:00:53 +0000) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <8737p8k8u2.fsf@mat.ucm.es> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: 23595@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > From: Uwe Brauer > Cc: Dmitry Gutov , oub@mat.ucm.es, > 23595@debbugs.gnu.org > Date: Mon, 23 May 2016 17:00:53 +0000 > > I thought the problem was caused by the fact that I did not entered that > chars, but rather copied it from some tex.stackexchange site, but I see > that was not the reason. > > What is about mercurial?[1] No clue, sorry. I don't use it and don't know anything about it. The man page says "hg diff --text" might do what you want. > > Bottom line: users who put UTF-16 encoded files into VCS are playing > > with fire, and are best advised not to do that! > > Right, I see, that was just 2 chars in a document which contained > latin-1 or UTF8. So Chinese and Japanese programmers are in a > disadvantage, no? Why? UTF-8 supports Chinese just fine. From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 13:40:48 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 17:40:48 +0000 Received: from localhost ([127.0.0.1]:35097 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4tqS-0005B9-GY for submit@debbugs.gnu.org; Mon, 23 May 2016 13:40:48 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:55168) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4tqR-0005Ax-22 for 23595@debbugs.gnu.org; Mon, 23 May 2016 13:40:47 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 1ABCC161275; Mon, 23 May 2016 10:40:41 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id zhUS4KW3vgIX; Mon, 23 May 2016 10:40:40 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 595A216128C; Mon, 23 May 2016 10:40:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id jbNeQi2Zm5UB; Mon, 23 May 2016 10:40:40 -0700 (PDT) Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 3E72C161275; Mon, 23 May 2016 10:40:40 -0700 (PDT) To: 23595@debbugs.gnu.org From: Paul Eggert Subject: Re: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) Organization: UCLA Computer Science Department Message-ID: Date: Mon, 23 May 2016 10:40:40 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 23595 Cc: Uwe Brauer , Eli Zaretskii , Dmitry Gutov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.4 (-) On 05/23/2016 09:52 AM, Eli Zaretskii wrote: > Does "git diff --text" fix this? I tried something like the following, which as I understand it is the procedure recommended for putting UTF-16 files under Git control on GNU/Linux hosts (the basic idea is that the repository contains UTF-8 and the working files contain UTF-16): * Add the line 'test-chin-jap.tex filter=utf16' to .gitattributes. * git config filter.utf16.clean 'iconv -f utf-16 -t utf-8' * git config filter.utf16.smudge 'iconv -f utf-8 -t utf-16' * Commit the file all over again (as this stores the UTF-8 version in the repository, not the UTF-16 version). * Make a trivial edit to the file in the non-ASCII region. When I did all all this, Emacs 24.5 works and draft Emacs 25 shows mojibake, so we indeed have a regression. The shell command 'git diff' works fine, and outputs the difference in UTF-8, but I guess draft Emacs 25 treats the git diff output as UTF-16. From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 14:16:35 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 18:16:36 +0000 Received: from localhost ([127.0.0.1]:35119 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4uP5-00061E-Mj for submit@debbugs.gnu.org; Mon, 23 May 2016 14:16:35 -0400 Received: from eggs.gnu.org ([208.118.235.92]:37266) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4uP4-000611-52 for 23595@debbugs.gnu.org; Mon, 23 May 2016 14:16:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4uOy-0002tQ-3q for 23595@debbugs.gnu.org; Mon, 23 May 2016 14:16:28 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:46522) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4uOl-0002qM-N9; Mon, 23 May 2016 14:16:15 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3461 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b4uOh-0003Ho-Na; Mon, 23 May 2016 14:16:14 -0400 Date: Mon, 23 May 2016 21:15:57 +0300 Message-Id: <8337p863oi.fsf@gnu.org> From: Eli Zaretskii To: Paul Eggert In-reply-to: (message from Paul Eggert on Mon, 23 May 2016 10:40:40 -0700) Subject: Re: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, dgutov@yandex.ru, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > Cc: Uwe Brauer , Dmitry Gutov , > Eli Zaretskii > From: Paul Eggert > Date: Mon, 23 May 2016 10:40:40 -0700 > > I tried something like the following, which as I understand it is the > procedure recommended for putting UTF-16 files under Git control on > GNU/Linux hosts (the basic idea is that the repository contains UTF-8 > and the working files contain UTF-16): > > * Add the line 'test-chin-jap.tex filter=utf16' to .gitattributes. > > * git config filter.utf16.clean 'iconv -f utf-16 -t utf-8' > > * git config filter.utf16.smudge 'iconv -f utf-8 -t utf-16' > > * Commit the file all over again (as this stores the UTF-8 version in > the repository, not the UTF-16 version). > > * Make a trivial edit to the file in the non-ASCII region. > > When I did all all this, Emacs 24.5 works and draft Emacs 25 shows > mojibake, so we indeed have a regression. The shell command 'git diff' > works fine, and outputs the difference in UTF-8, but I guess draft Emacs > 25 treats the git diff output as UTF-16. If the above is the recommended procedure for putting such files under Git, then vc-git should bind coding-system-for-read to utf-8 whenever (coding-system-get buffer-file-coding-system :ascii-compatible-p) returns nil. Otherwise, the current binding is TRT. From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 16:40:23 2016 Received: (at submit) by debbugs.gnu.org; 23 May 2016 20:40:23 +0000 Received: from localhost ([127.0.0.1]:35162 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4weF-0000xW-Ev for submit@debbugs.gnu.org; Mon, 23 May 2016 16:40:23 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47162) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4weD-0000xF-IL for submit@debbugs.gnu.org; Mon, 23 May 2016 16:40:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4we2-0005Nu-VI for submit@debbugs.gnu.org; Mon, 23 May 2016 16:40:16 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:54184) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4we2-0005NR-SF for submit@debbugs.gnu.org; Mon, 23 May 2016 16:40:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38126) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4we0-00067R-PG for bug-gnu-emacs@gnu.org; Mon, 23 May 2016 16:40:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4wdx-0005Jm-KF for bug-gnu-emacs@gnu.org; Mon, 23 May 2016 16:40:08 -0400 Received: from plane.gmane.org ([80.91.229.3]:34255) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4wdx-0005Jh-DH for bug-gnu-emacs@gnu.org; Mon, 23 May 2016 16:40:05 -0400 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1b4wdv-0001EF-9v for bug-gnu-emacs@gnu.org; Mon, 23 May 2016 22:40:03 +0200 Received: from 84.red-79-153-137.dynamicip.rima-tde.net ([79.153.137.84]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 23 May 2016 22:40:03 +0200 Received: from oub by 84.red-79-153-137.dynamicip.rima-tde.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 23 May 2016 22:40:03 +0200 X-Injected-Via-Gmane: http://gmane.org/ Mail-Followup-To: bug-gnu-emacs@gnu.org To: bug-gnu-emacs@gnu.org From: Uwe Brauer Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) Date: Mon, 23 May 2016 20:37:29 +0000 Lines: 15 Message-ID: <87oa7wik8m.fsf@mat.ucm.es> References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <8737p8k8u2.fsf@mat.ucm.es> <838tz065qo.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 84.red-79-153-137.dynamicip.rima-tde.net Mail-Copies-To: never User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) Cancel-Lock: sha1:AWq63s++iEkfJK9EWlZiC3q05hw= X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.1 (----) >>> "Eli" == Eli Zaretskii writes: >> Right, I see, that was just 2 chars in a document which contained >> latin-1 or UTF8. So Chinese and Japanese programmers are in a >> disadvantage, no? > Why? UTF-8 supports Chinese just fine. Now I am confused. In my poor understanding I thought UTF-16 is needed for Chinese and Japanese. That seems not to be the case?! So the problem I reported was caused by the fact that I used UTF-16 instead of UTF-8? From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 17:01:57 2016 Received: (at submit) by debbugs.gnu.org; 23 May 2016 21:01:57 +0000 Received: from localhost ([127.0.0.1]:35193 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4wz6-0001Sc-OA for submit@debbugs.gnu.org; Mon, 23 May 2016 17:01:56 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51754) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4wz4-0001SN-Nf for submit@debbugs.gnu.org; Mon, 23 May 2016 17:01:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4wyy-0001lq-Ql for submit@debbugs.gnu.org; Mon, 23 May 2016 17:01:49 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:52780) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4wyy-0001lM-Nf for submit@debbugs.gnu.org; Mon, 23 May 2016 17:01:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42703) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4wyw-0003dw-Ag for bug-gnu-emacs@gnu.org; Mon, 23 May 2016 17:01:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4wys-0001kX-7L for bug-gnu-emacs@gnu.org; Mon, 23 May 2016 17:01:45 -0400 Received: from bouvier.getmail.no ([84.210.184.8]:56769) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4wys-0001jr-0d for bug-gnu-emacs@gnu.org; Mon, 23 May 2016 17:01:42 -0400 Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id 887FE492BA for ; Mon, 23 May 2016 23:01:32 +0200 (CEST) Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id K7wkExVofMta for ; Mon, 23 May 2016 23:01:31 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id B270049D9C for ; Mon, 23 May 2016 23:01:31 +0200 (CEST) X-Virus-Scanned: amavisd-new at bouvier.get.c.bitbit.net Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id iE07EC5M_YCH for ; Mon, 23 May 2016 23:01:31 +0200 (CEST) Received: from mouse (cm-84.215.1.64.getinternet.no [84.215.1.64]) by bouvier.getmail.no (Postfix) with ESMTPS id 8895B492BA for ; Mon, 23 May 2016 23:01:31 +0200 (CEST) From: Lars Ingebrigtsen To: bug-gnu-emacs@gnu.org Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <8737p8k8u2.fsf@mat.ucm.es> <838tz065qo.fsf@gnu.org> <87oa7wik8m.fsf@mat.ucm.es> Date: Mon, 23 May 2016 23:01:31 +0200 In-Reply-To: <87oa7wik8m.fsf@mat.ucm.es> (Uwe Brauer's message of "Mon, 23 May 2016 20:37:29 +0000") Message-ID: <87lh30qyj8.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Uwe Brauer writes: > > Why? UTF-8 supports Chinese just fine. > > Now I am confused. In my poor understanding I thought UTF-16 is needed > for Chinese and Japanese. That seems not to be the case?! No, UTF-8 and UTF-16 are just two different encodings of the same charset, which is Unicode. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 17:02:46 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 21:02:46 +0000 Received: from localhost ([127.0.0.1]:35198 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4wzu-0001UF-2r for submit@debbugs.gnu.org; Mon, 23 May 2016 17:02:46 -0400 Received: from mail-wm0-f44.google.com ([74.125.82.44]:35803) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4wzs-0001U2-NE for 23595@debbugs.gnu.org; Mon, 23 May 2016 17:02:45 -0400 Received: by mail-wm0-f44.google.com with SMTP id a136so42452292wme.0 for <23595@debbugs.gnu.org>; Mon, 23 May 2016 14:02:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=J3dhQKs/nrsDgrcbyEr8ZFg/5hB9b2IMzIoCI9Nk0+g=; b=Yo52xZd7i9xY/+Zmj1vIhj8D781lvQHkt2vWtnHSpvkaaAOCVoeui3gi85ppeUyC3e wLjoPb9tITgcnrsL8qORESq+YPlQCKKtPjnOjwAlMl9TcH2gSQghDV0VshQ48W989f+8 dUgO/R14CLQp5kSdqQWq4e2B1xoEf/D0mIWLHGr5Iuz5yYPLT/n+QVqpSm5dKqShjDn/ HkxMsBJl+QYJ/D1/OI6/GLKRMjdX+dQNEA2G0WTFY548DXCq7qzco8OwbyGvRWynhk19 fR80HM7AmgL0q0gLkVtx4ijpN8PwFkXdiwf0mhImZLwvSGZB6fJSknu3Jp3Bz9nwT4ca TJog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=J3dhQKs/nrsDgrcbyEr8ZFg/5hB9b2IMzIoCI9Nk0+g=; b=ZEO0Bh4J6ZmxF6IJC5SyGnWn5RFEltRiZaOGWO2hwn5lB0vG4D4TrO5D9mEaoX6/u6 zL+gVKroXp010ZWww9PCKyfkCQYZnECGX4i7KpHUhYWzDLoHbfUcNiK8nCH4arOiB3Ho xiioXj6YREiIrVQMuk44HNGKvYzHTcI1PeNONPB9L2rXkv2N4pdTdmhKTDuRZoswerXV Df1wG28wjkJcB16Xm4efBjiVlNqAUPaZ7i4Z91ISJfqsOy7QBGH21YLsPYBORFTtVagC 1SQVlunPGQdbhoM2O/0uQwAxJMYMAeMZEsQ+fw3gwinalGtKm3RNtywHfYhQzoh6w/uk P/vg== X-Gm-Message-State: ALyK8tL8Ax4FznIPOOuA6u8+T7UBInUmkTPFX8rGc4VL+GD2YpGMdO6ZFGYfl7vKz2L0lg== X-Received: by 10.194.62.99 with SMTP id x3mr737770wjr.128.1464037358972; Mon, 23 May 2016 14:02:38 -0700 (PDT) Received: from [192.168.1.3] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id kd7sm24606600wjc.33.2016.05.23.14.02.37 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 23 May 2016 14:02:37 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> From: Dmitry Gutov Message-ID: <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> Date: Tue, 24 May 2016 00:02:36 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1 MIME-Version: 1.0 In-Reply-To: <83h9do67pp.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, Paul Eggert , 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 05/23/2016 07:48 PM, Eli Zaretskii wrote: >>> The resulting diff contains either rubbish or fails to run. >>> Files attached. > > I don't see any rubbish in the Git output. Might that have to do something with your OS? I see the mojibake like others. > Setting coding-system-for-read is correct, because the important use > case is when the diffs are actually output. The problem is that > UTF-16 is not ASCII-compatible, and so text output by Git itself will > be mishandled. Another problem is that Git doesn't show the diffs at > all. Apparently so. >> Which is weird, considering both vc-diff-internal and vc-coding-system-for-diff have both been virtually untouched for the last couple of years. > > Not sure what do you see as weird. That we have a regression while the relevant functions didn't change. Something probably changed on the lower level, and we might be wise to figure out what (unless somebody already knows, and just didn't point that out because it's not a bug). >> But even if we figure out why happens, you (Uwe) probably want Git, Hg, etc, to treat this file as text, and not binary. Only then you'll be able to get meaningful diffs. I don't have a specific advice on that. > > Why can't we invoke "git diff --text"? That should fix the second > problem, I think. It does not. It forces Git to diff the file as text, but neither the current code, nor the patch at the end make the displayed file contents to be correctly decoded. I haven't tried Paul's solution for this myself, but it seems to be the way to go. > As for the first problem, we should probably refrain from binding > coding-system-for-read to a CODING-SYSTEM for which > > (coding-system-get CODING-SYSTEM :ascii-compatible-p) > > returns nil. We should instead bind it to no-conversion and decode > the file data parts by hand, skipping the parts that Git itself > outputs (yes, this is messy). Patches to that effect are welcome. Not sure what's the best place to do it, but the patch below gives me 24.5's behavior (correctly decoding the short "Binary files ... differ" output). Could someone try it together with Paul's solution? diff --git a/lisp/vc/vc.el b/lisp/vc/vc.el index 25b41e3..b62b68d 100644 --- a/lisp/vc/vc.el +++ b/lisp/vc/vc.el @@ -1696,6 +1696,8 @@ vc-diff-internal (setq coding-system-for-read (coding-system-change-eol-conversion coding-system-for-read 'dos))) + (unless (coding-system-get coding-system-for-read :ascii-compatible-p) + (setq coding-system-for-read nil)) (vc-setup-buffer buffer) (message "%s" (car messages)) ;; Many backends don't handle well the case of a file that has been From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 18:17:05 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 22:17:05 +0000 Received: from localhost ([127.0.0.1]:35286 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4y9p-0003Au-7X for submit@debbugs.gnu.org; Mon, 23 May 2016 18:17:05 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:51972) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4y9m-0003AK-TK for 23595@debbugs.gnu.org; Mon, 23 May 2016 18:17:03 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 37E9C1612EE; Mon, 23 May 2016 15:16:57 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id jx8CqTStmWUU; Mon, 23 May 2016 15:16:56 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 73F421612F5; Mon, 23 May 2016 15:16:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id gUv6LyiH9MHd; Mon, 23 May 2016 15:16:56 -0700 (PDT) Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 582A21612EE; Mon, 23 May 2016 15:16:56 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Dmitry Gutov , Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> Date: Mon, 23 May 2016 15:16:56 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.4 (-) On 05/23/2016 02:02 PM, Dmitry Gutov wrote: > Not sure what's the best place to do it, but the patch below gives me > 24.5's behavior (correctly decoding the short "Binary files ... > differ" output). Could someone try it together with Paul's solution? > It worked for me in the Bug#23595 test case, with Git configured with utf16<->utf8 filters as I described. However, it reintroduces a bug when the version-controlled uses ISO-2022-JP. If I make a trivial change to etc/HELLO, for example, the patch can cause vc-diff to display mojibake, as the output of "git diff" uses ISO0-2022-JP but vc-diff decodes it as UTF-8. Although this is the same mojibake that Emacs 24.5 generates so the behavior is not a regression from 24.5, it is a regression from current emacs-25. We are on thin ice here no matter what. One idea to improve on the current emacs-25 behavior is to test whether a simple ASCII message like "Binary files differ" encodes as itself using the file's coding system, and to use the file's coding system if it does and locale-coding-system if it doesn't. > diff --git a/lisp/vc/vc.el b/lisp/vc/vc.el > index 25b41e3..b62b68d 100644 > --- a/lisp/vc/vc.el > +++ b/lisp/vc/vc.el > @@ -1696,6 +1696,8 @@ vc-diff-internal > (setq coding-system-for-read > (coding-system-change-eol-conversion coding-system-for-read > 'dos))) > + (unless (coding-system-get coding-system-for-read > :ascii-compatible-p) > + (setq coding-system-for-read nil)) From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 18:29:08 2016 Received: (at 23595) by debbugs.gnu.org; 23 May 2016 22:29:08 +0000 Received: from localhost ([127.0.0.1]:35294 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4yLU-0003Uz-CH for submit@debbugs.gnu.org; Mon, 23 May 2016 18:29:08 -0400 Received: from mail-wm0-f46.google.com ([74.125.82.46]:36472) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4yLS-0003UV-HY for 23595@debbugs.gnu.org; Mon, 23 May 2016 18:29:06 -0400 Received: by mail-wm0-f46.google.com with SMTP id n129so101201547wmn.1 for <23595@debbugs.gnu.org>; Mon, 23 May 2016 15:29:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=NZ+V85LLNfUyR5FjvLv9od+942Yubjl5wBZbzv9DUeg=; b=eOhkyirbqDohfzEjdfstMy5gQb/10JTy7OZJc+xFQb/MuXq8ra+BGdj4h7w6COoNY2 z+JRw637LJhku1qQwqSd9PtMP6ZtTxzpYcFck5ZUcZ4H8EnzCVkUon45zFTsZkiLMVI/ pbhO2ddHgRtbGqWY8jriaCu1s39Yz2BfVURRf2qmcfJkjR7AOASWLG1tiaVLCc4FHi4Z 6OBtrYf7mv6Ha00WIUfec7IVrKjAQpho1yKdigOFoDSiE7Q1i7hCQ3sLuIoWPqFA6NhU q8mMV2cmL4yxAWOI8TSIEzJMfc+hHu8IH6hznq3F2Z1eu9m+T/D8ySn5Dklzu4CB0gRg nPeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=NZ+V85LLNfUyR5FjvLv9od+942Yubjl5wBZbzv9DUeg=; b=Okr88KodrKuq64sqWZVixyfqTgs+K1h/DEyflsjAPOzh2b/MBkPwS2L/oYjfdm1yBZ LAzl7hWwYHysGE7Awc3PlkZuX5tMM70rEeBe8+fTM06Y9Og3mcfRc0S+cs9JBmAEhqZz oR0LPsqdf8gCUHwzipDTN6SItyWvhR4iJQ+XdwIlxevNr9qn57exwuIryl5mELsYIDYu DvzlitewtpJrp2qeCjoeGDa3MErCuHNBVmh/ArXv/vQy7CX/j5JvUUJgkXJssWvdprIm pFY2pWnOnQLlLlB2s+cC6++Jf9x/bBM9Womju6teQwnOGNkmGTNAz5FB3c6CHx/+sYr8 GY3Q== X-Gm-Message-State: ALyK8tLpDxqjNnb4Qum41DO2T228R2Iq3ZVKg9b48KH4XCuMCGDqSuXDx2M0/lOfQmYeUQ== X-Received: by 10.28.4.79 with SMTP id 76mr8596524wme.41.1464042540919; Mon, 23 May 2016 15:29:00 -0700 (PDT) Received: from [192.168.1.3] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id 131sm309260wmu.17.2016.05.23.15.28.58 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 23 May 2016 15:28:59 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Paul Eggert , Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> From: Dmitry Gutov Message-ID: <86c6d05c-a37f-e223-d0d2-af63d09ed0cc@yandex.ru> Date: Tue, 24 May 2016 01:28:58 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1 MIME-Version: 1.0 In-Reply-To: <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 05/24/2016 01:16 AM, Paul Eggert wrote: > It worked for me in the Bug#23595 test case, with Git configured with > utf16<->utf8 filters as I described. However, it reintroduces a bug when > the version-controlled uses ISO-2022-JP. Does it have a bug report? Can we have a test case? > If I make a trivial change to > etc/HELLO, for example, the patch can cause vc-diff to display mojibake, > as the output of "git diff" uses ISO0-2022-JP but vc-diff decodes it as > UTF-8. Although this is the same mojibake that Emacs 24.5 generates so > the behavior is not a regression from 24.5, it is a regression from > current emacs-25. That's too bad. > We are on thin ice here no matter what. One idea to improve on the > current emacs-25 behavior is to test whether a simple ASCII message like > "Binary files differ" encodes as itself using the file's coding system, > and to use the file's coding system if it does and locale-coding-system > if it doesn't. How would we do that? We're currently picking conding-system-for-read well before the first byte of the output is generated. From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 20:07:41 2016 Received: (at 23595) by debbugs.gnu.org; 24 May 2016 00:07:41 +0000 Received: from localhost ([127.0.0.1]:35324 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4zsr-0005kC-89 for submit@debbugs.gnu.org; Mon, 23 May 2016 20:07:41 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:59331) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b4zso-0005jz-GX for 23595@debbugs.gnu.org; Mon, 23 May 2016 20:07:39 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 9E0F81612F9; Mon, 23 May 2016 17:07:32 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Tx0tnorFXgxW; Mon, 23 May 2016 17:07:31 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id E2FEB1612FA; Mon, 23 May 2016 17:07:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ZTWaOV-6quUO; Mon, 23 May 2016 17:07:31 -0700 (PDT) Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id C42A51612F9; Mon, 23 May 2016 17:07:31 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Dmitry Gutov , Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> <86c6d05c-a37f-e223-d0d2-af63d09ed0cc@yandex.ru> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: Date: Mon, 23 May 2016 17:07:31 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <86c6d05c-a37f-e223-d0d2-af63d09ed0cc@yandex.ru> Content-Type: multipart/mixed; boundary="------------0D89DDBF8B43066FB8BCAA78" X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.4 (-) This is a multi-part message in MIME format. --------------0D89DDBF8B43066FB8BCAA78 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 05/23/2016 03:28 PM, Dmitry Gutov wrote: > Does it have a bug report? Can we have a test case? Not as far as I know. I was hoping we wouldn't have to write a bug report now, as emacs-25 does not have that bug now. I suppose someone with more free time could write a test case.... > >> One idea to improve on the >> current emacs-25 behavior is to test whether a simple ASCII message like >> "Binary files differ" encodes as itself using the file's coding system, >> and to use the file's coding system if it does and locale-coding-system >> if it doesn't. > > How would we do that? We're currently picking conding-system-for-read > well before the first byte of the output is generated. Emacs can decide the coding system before git diff generates any output, by applying decode-coding-string to a canary string sample. The attached patch should work; please give it a try. --------------0D89DDBF8B43066FB8BCAA78 Content-Type: application/x-patch; name="0001-Fix-vc-diff-problems-with-UTF-16.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="0001-Fix-vc-diff-problems-with-UTF-16.patch" RnJvbSBkMmZkOGE3M2U5NzgwMDA1MGQ2ZmNlMjJjYTM4MTViMTU5Y2MwZWZlIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBNb24sIDIzIE1heSAyMDE2IDE2OjUzOjM0IC0wNzAwClN1YmplY3Q6IFtQQVRD SF0gRml4IHZjLWRpZmYgcHJvYmxlbXMgd2l0aCBVVEYtMTYKClByb2JsZW0gd2l0aCBVVEYt MTYgcmVwb3J0ZWQgYnkgVXdlIEJyYXVlciAoQnVnIzIzNTk1KS4KVGhlcmUgYXJlIHNpbWls YXIgcHJvYmxlbXMgd2l0aCBFQkNESUMgb3Igd2l0aCBvdGhlciBjb2RpbmcKc3lzdGVtcyBk aWZmZXJpbmcgZ3JlYXRseSBmcm9tIEFTQ0lJLiAgQWxzbywgdGhlIGhldXJpc3RpYyBzaG91 bGQKYmUgbW9yZSBjb25zZXJ2YXRpdmUgd2hlbiBkZWFsaW5nIHdpdGggbXVsdGlwbGUgZmls ZXMgdGhhdCBoYXZlCmluY29tcGF0aWJsZSBjb2Rpbmcgc3lzdGVtcy4KKiBsaXNwL3ZjL3Zj LmVsICh2Yy1kaWZmLWludGVybmFsKTogUmVxdWlyZSBhbGwgdGhlIGZpbGVzIHRvIGhhdmUK dGhlIHNhbWUgY29kaW5nIHN5c3RlbSwgYW5kIGZvciB0aGF0IGNvZGluZyBzeXN0ZW0gdG8g YmUKY29tcGF0aWJsZS1lbm91Z2ggd2l0aCBBU0NJSSBzbyB0aGF0IG1lc3NhZ2VzIGxpa2Ug IkJpbmFyeSBmaWxlcwpkaWZmZXIiIGFyZSBub3QgbWlzZGVjb2RlZC4KLS0tCiBsaXNwL3Zj L3ZjLmVsIHwgMjUgKysrKysrKysrKysrKysrKysrKysrLS0tLQogMSBmaWxlIGNoYW5nZWQs IDIxIGluc2VydGlvbnMoKyksIDQgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvbGlzcC92 Yy92Yy5lbCBiL2xpc3AvdmMvdmMuZWwKaW5kZXggMjViNDFlMy4uYzY5ZmYyMyAxMDA2NDQK LS0tIGEvbGlzcC92Yy92Yy5lbAorKysgYi9saXNwL3ZjL3ZjLmVsCkBAIC0xNjg0LDExICsx Njg0LDI4IEBAIHZjLWRpZmYtaW50ZXJuYWwKICAgICAgICAgICAgICAgICAgICAgICAgICAo Zm9ybWF0ICJObyBjaGFuZ2VzIGJldHdlZW4gJXMgYW5kICVzIgogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgKG9yIHJldjEgIndvcmtpbmcgcmV2aXNpb24iKQogICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgKG9yIHJldjIgIndvcmtmaWxlIikpKSkKLQkg OzsgU2V0IGNvZGluZyBzeXN0ZW0gYmFzZWQgb24gdGhlIGZpcnN0IGZpbGUuICBJdCdzIGEg a2x1Z2UsCi0JIDs7IGJ1dCB0aGUgb25seSB3YXkgdG8gc2V0IGl0IGZvciBlYWNoIGZpbGUg aW5jbHVkZWQgd291bGQKLQkgOzsgYmUgdG8gY2FsbCB0aGUgYmFjayBlbmQgc2VwYXJhdGVs eSBmb3IgZWFjaCBmaWxlLgogCSAoY29kaW5nLXN5c3RlbS1mb3ItcmVhZAotCSAgKGlmIGZp bGVzICh2Yy1jb2Rpbmctc3lzdGVtLWZvci1kaWZmIChjYXIgZmlsZXMpKSAndW5kZWNpZGVk KSkpCisJICAob3IgY29kaW5nLXN5c3RlbS1mb3ItcmVhZAorCSAgICAgICh3aGVuIGZpbGVz CisJCTs7IElmIGFsbCBmaWxlcyB1c2UgdGhlIHNhbWUgY29kaW5nIHN5c3RlbSwgZGVmYXVs dCB0byB0aGF0LgorCQk7OyBJdCdzIGEga2x1ZGdlLCBidXQgdGhlIG9ubHkgd2F5IHRvIHNl dCBpdCBmb3IgZWFjaCBmaWxlIHdvdWxkCisJCTs7IGJlIHRvIGNhbGwgdGhlIGJhY2sgZW5k IHNlcGFyYXRlbHkgZm9yIGVhY2ggZmlsZS4KKwkJKGxldCAoKGNvZGluZyAodmMtY29kaW5n LXN5c3RlbS1mb3ItZGlmZiAoY2FyIGZpbGVzKSkpKQorCQkgIChsZXQgKChmcyAoY2RyIGZp bGVzKSkpCisJCSAgICAod2hpbGUgKGFuZCBmcyBjb2RpbmcpCisJCSAgICAgICh1bmxlc3Mg KGVxIGNvZGluZyAodmMtY29kaW5nLXN5c3RlbS1mb3ItZGlmZiAoY2FyIGZzKSkpCisJCQko c2V0cSBjb2RpbmcgbmlsKSkKKwkJICAgICAgKHNldHEgZnMgKGNkciBmcykpKSkKKwkJICA7 OyBJZiB0aGUgZmlsZXMnIGNvZGluZyBzeXN0ZW0gaXMgVVRGLTE2LCBkaWZmIGxpa2VseQor CQkgIDs7IG91dHB1dHMgc29tZXRoaW5nIGxpa2UgIkJpbmFyeSBmaWxlcyBkaWZmZXIiIGlu IEFTQ0lJLAorCQkgIDs7IHdoaWNoIGlzIG1pc2RlY29kZWQgYXMgVVRGLTE2LiAgU28gdXNl IHRoZSBmaWxlcycgY29kaW5nCisJCSAgOzsgc3lzdGVtIG9ubHkgaWYgaXQgaXMgY29tcGF0 aWJsZS1lbm91Z2ggd2l0aCBBU0NJSS4KKwkJICAod2hlbiAoYW5kIGNvZGluZworCQkJICAg ICAobGV0ICgoc2FtcCAiQmluYXJ5IGZpbGVzIGRpZmZlciIpKQorCQkJICAgICAgIChzdHJp bmctZXF1YWwgc2FtcCAoZGVjb2RlLWNvZGluZy1zdHJpbmcKKwkJCQkJCSAgIHNhbXAgY29k aW5nIHQpKSkpCisJCSAgICBsYXN0LWNvZGluZy1zeXN0ZW0tdXNlZCkpKSkpKQorCiAgICAg OzsgT24gTVMtV2luZG93cyBhbmQgTVMtRE9TLCBEaWZmIGlzIGxpa2VseSB0byBwcm9kdWNl IERPUy1zdHlsZQogICAgIDs7IEVPTHMsIHdoaWNoIHdpbGwgbG9vayB1Z2x5IGlmIChjYXIg ZmlsZXMpIGhhcHBlbnMgdG8gaGF2ZSBVbml4CiAgICAgOzsgRU9Mcy4KLS0gCjIuNS41Cgo= --------------0D89DDBF8B43066FB8BCAA78-- From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 22:33:20 2016 Received: (at submit) by debbugs.gnu.org; 24 May 2016 02:33:20 +0000 Received: from localhost ([127.0.0.1]:35372 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b529o-0002MF-LT for submit@debbugs.gnu.org; Mon, 23 May 2016 22:33:20 -0400 Received: from eggs.gnu.org ([208.118.235.92]:54907) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b529m-0002M2-Kv for submit@debbugs.gnu.org; Mon, 23 May 2016 22:33:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b529g-0002Ot-MX for submit@debbugs.gnu.org; Mon, 23 May 2016 22:33:13 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:59615) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b529g-0002OQ-Jq for submit@debbugs.gnu.org; Mon, 23 May 2016 22:33:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45858) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b529e-0006EW-Gs for bug-gnu-emacs@gnu.org; Mon, 23 May 2016 22:33:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b529a-0002NT-93 for bug-gnu-emacs@gnu.org; Mon, 23 May 2016 22:33:09 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52447) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b529a-0002NH-61; Mon, 23 May 2016 22:33:06 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3802 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b529Y-00043E-Ai; Mon, 23 May 2016 22:33:04 -0400 Date: Tue, 24 May 2016 05:33:05 +0300 Message-Id: <83vb24423i.fsf@gnu.org> From: Eli Zaretskii To: Uwe Brauer In-reply-to: <87oa7wik8m.fsf@mat.ucm.es> (message from Uwe Brauer on Mon, 23 May 2016 20:37:29 +0000) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <8737p8k8u2.fsf@mat.ucm.es> <838tz065qo.fsf@gnu.org> <87oa7wik8m.fsf@mat.ucm.es> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: submit Cc: bug-gnu-emacs@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > From: Uwe Brauer > Date: Mon, 23 May 2016 20:37:29 +0000 > > >>> "Eli" == Eli Zaretskii writes: > >> Right, I see, that was just 2 chars in a document which contained > >> latin-1 or UTF8. So Chinese and Japanese programmers are in a > >> disadvantage, no? > > > Why? UTF-8 supports Chinese just fine. > > Now I am confused. In my poor understanding I thought UTF-16 is needed > for Chinese and Japanese. That seems not to be the case?! No, it's not the case. UTF-8 and UTF-16 both support the same space of Unicode codepoints. > So the problem I reported was caused by the fact that I used UTF-16 > instead of UTF-8? Yes! From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 22:37:02 2016 Received: (at 23595) by debbugs.gnu.org; 24 May 2016 02:37:02 +0000 Received: from localhost ([127.0.0.1]:35376 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b52DL-0002RE-6P for submit@debbugs.gnu.org; Mon, 23 May 2016 22:37:02 -0400 Received: from eggs.gnu.org ([208.118.235.92]:55637) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b52DK-0002R0-AT for 23595@debbugs.gnu.org; Mon, 23 May 2016 22:36:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b52DD-0003F8-VU for 23595@debbugs.gnu.org; Mon, 23 May 2016 22:36:52 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52480) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b52D9-0003EM-2e; Mon, 23 May 2016 22:36:47 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3806 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b52D8-0000xB-3y; Mon, 23 May 2016 22:36:46 -0400 Date: Tue, 24 May 2016 05:36:46 +0300 Message-Id: <83twho41xd.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> (message from Dmitry Gutov on Tue, 24 May 2016 00:02:36 +0300) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org, Paul Eggert > From: Dmitry Gutov > Date: Tue, 24 May 2016 00:02:36 +0300 > > On 05/23/2016 07:48 PM, Eli Zaretskii wrote: > > >>> The resulting diff contains either rubbish or fails to run. > >>> Files attached. > > > > I don't see any rubbish in the Git output. > > Might that have to do something with your OS? I see the mojibake like > others. I was talking about the attachment Uwe provided, so this has nothing to do with my OS. > > As for the first problem, we should probably refrain from binding > > coding-system-for-read to a CODING-SYSTEM for which > > > > (coding-system-get CODING-SYSTEM :ascii-compatible-p) > > > > returns nil. We should instead bind it to no-conversion and decode > > the file data parts by hand, skipping the parts that Git itself > > outputs (yes, this is messy). Patches to that effect are welcome. > > Not sure what's the best place to do it, but the patch below gives me > 24.5's behavior (correctly decoding the short "Binary files ... differ" > output). Could someone try it together with Paul's solution? Paul's solution is outside of Emacs's realm. What Emacs should do is bind coding-system-for-read to utf-8 in this case (not leave it unbound as in your patch), under the assumption that the user used the procedure outlined by Paul. From debbugs-submit-bounces@debbugs.gnu.org Mon May 23 22:40:56 2016 Received: (at 23595) by debbugs.gnu.org; 24 May 2016 02:40:56 +0000 Received: from localhost ([127.0.0.1]:35380 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b52HA-0002Wf-3x for submit@debbugs.gnu.org; Mon, 23 May 2016 22:40:56 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56393) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b52H7-0002WT-Vr for 23595@debbugs.gnu.org; Mon, 23 May 2016 22:40:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b52Gz-00041q-R6 for 23595@debbugs.gnu.org; Mon, 23 May 2016 22:40:48 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52598) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b52Gz-00041g-Nm; Mon, 23 May 2016 22:40:45 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3807 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b52Gx-0001Ku-OB; Mon, 23 May 2016 22:40:44 -0400 Date: Tue, 24 May 2016 05:40:44 +0300 Message-Id: <83shx841qr.fsf@gnu.org> From: Eli Zaretskii To: Paul Eggert In-reply-to: <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> (message from Paul Eggert on Mon, 23 May 2016 15:16:56 -0700) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org > From: Paul Eggert > Date: Mon, 23 May 2016 15:16:56 -0700 > > On 05/23/2016 02:02 PM, Dmitry Gutov wrote: > > Not sure what's the best place to do it, but the patch below gives me > > 24.5's behavior (correctly decoding the short "Binary files ... > > differ" output). Could someone try it together with Paul's solution? > > > > It worked for me in the Bug#23595 test case, with Git configured with > utf16<->utf8 filters as I described. However, it reintroduces a bug when > the version-controlled uses ISO-2022-JP. If I make a trivial change to > etc/HELLO, for example, the patch can cause vc-diff to display mojibake, > as the output of "git diff" uses ISO0-2022-JP but vc-diff decodes it as > UTF-8. Although this is the same mojibake that Emacs 24.5 generates so > the behavior is not a regression from 24.5, it is a regression from > current emacs-25. For some reason I don't quite understand, iso-2022-jp fails the ascii-compatible-p test. We could make an exception for the iso-202 family in this case. Then the bug would not creep back in. > We are on thin ice here no matter what. One idea to improve on the > current emacs-25 behavior is to test whether a simple ASCII message like > "Binary files differ" encodes as itself using the file's coding system, > and to use the file's coding system if it does and locale-coding-system > if it doesn't. Yes, but we know in advance which coding-systems will be unable to do that, so testing this at run time sounds like waste of cycles. From debbugs-submit-bounces@debbugs.gnu.org Tue May 24 05:35:50 2016 Received: (at 23595) by debbugs.gnu.org; 24 May 2016 09:35:50 +0000 Received: from localhost ([127.0.0.1]:35485 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b58kg-0003zV-39 for submit@debbugs.gnu.org; Tue, 24 May 2016 05:35:50 -0400 Received: from mail-wm0-f53.google.com ([74.125.82.53]:38709) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b58ke-0003zI-LJ for 23595@debbugs.gnu.org; Tue, 24 May 2016 05:35:48 -0400 Received: by mail-wm0-f53.google.com with SMTP id n129so16214041wmn.1 for <23595@debbugs.gnu.org>; Tue, 24 May 2016 02:35:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=8Civl5kQCVWHblIbpzUlxmp7+3JBbo/KHvP4vtevOn0=; b=YYo0cfCDXXxBjnE8Mt31Akhj2s7CwuZwn80OaPMLGdtljs6jmP+iINzWFkH/DX/mAO datQZanYFF6OT2Gim5mCnUmbNgTwH6rrrwGcqNs9HRMTAN+1H/6cAjLWjavK7XdtHQ2/ JQlf+y4lQO9fCtCwDH7zaGEIAF0KJUhW7S/xz43jOYS8D2TbzVnB1GOBstK1hT7Roe/O /yadzfmWvT7ROzdesh5pLiU0bDAOq1Ehl3x5pPzpqpUv4fBXzxRODB+t9Me30OuEIyjT 38mhRzRYSK5r2a9R8eFtbSP52ySidkePDpXZ978gixY/rQxf5Nz0uP0Nyxi0dlXdmI5W BQ3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=8Civl5kQCVWHblIbpzUlxmp7+3JBbo/KHvP4vtevOn0=; b=koAdxecx3oSykQnDJXVRqDDAt+Bcx24/jVKPZXNw+Awue37gCLyH4oMcKA2jpfafjl C6OtcxxFvFebHlFOUXTILUHuKxkJivamnplW8DfxYC0MZw8eTutQPIdBDYXdUWuENys2 sjRVbZ5TPYI6Q10+EnDBiZtBJubo+Ol2epZW+8zHk3ohQh9DeYp4JKgeGP2rbEdcn6lH FwCsiimwEWPh3JIKO+Pd2XO3wajyNDe6inKLZ/lqnBeAqHVizoui9jZUhPFjg3ZOIWHt VC8eilvLsP0xD2z5dmyZK/8ZwgZ9DwOCUI/3fci+d3YcoLo1JRMXRlZChO+c2KKCf5Pg omHw== X-Gm-Message-State: ALyK8tK05lC0Vrz9cwEegvdXFfnztWbbyVocUe5alwfTEdwxjshma0xqwOMhjxpRS7QXgA== X-Received: by 10.194.242.65 with SMTP id wo1mr3192391wjc.54.1464082542832; Tue, 24 May 2016 02:35:42 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id jq1sm2164912wjc.28.2016.05.24.02.35.41 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 24 May 2016 02:35:42 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> From: Dmitry Gutov Message-ID: <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> Date: Tue, 24 May 2016 12:35:40 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1 MIME-Version: 1.0 In-Reply-To: <83twho41xd.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 05/24/2016 05:36 AM, Eli Zaretskii wrote: >> Might that have to do something with your OS? I see the mojibake like >> others. > > I was talking about the attachment Uwe provided, so this has nothing > to do with my OS. Hm, yes, that's odd. I do see the same problem with Git as Use reported with Hg and RCS, on my machine. >What Emacs should do is > bind coding-system-for-read to utf-8 in this case (not leave it > unbound as in your patch), under the assumption that the user used the > procedure outlined by Paul. Should `utf-8' altogether replace `undecided' in vc-coding-system-for-diff? Then the use of buffer-file-coding-system could be predicated on its being compatible with ascii. From debbugs-submit-bounces@debbugs.gnu.org Tue May 24 05:47:29 2016 Received: (at 23595) by debbugs.gnu.org; 24 May 2016 09:47:29 +0000 Received: from localhost ([127.0.0.1]:35490 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b58vx-0004F7-5x for submit@debbugs.gnu.org; Tue, 24 May 2016 05:47:29 -0400 Received: from mail-wm0-f50.google.com ([74.125.82.50]:35191) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b58vv-0004Ev-6y for 23595@debbugs.gnu.org; Tue, 24 May 2016 05:47:27 -0400 Received: by mail-wm0-f50.google.com with SMTP id a136so63927498wme.0 for <23595@debbugs.gnu.org>; Tue, 24 May 2016 02:47:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=ZQTLYUEOHOpXSDmX5ZgrUGr+JRK4ubyzeQ8JpUkbJz8=; b=u7x8rtRtYQHYHqfuzM/Ah7v3fuUDGtIYCH9Qd0bL9dExFtiN8nvBFBynD1LIBt1jdu xqay7vcwS6DlVq13yQwRjl+fnkR9LnO6j0WCkkclVzmtR8EBTHFlHlKkfUthvbSG9GAK 1uKCQztmD9jkPIpV/6jUQcozTOeD0Yi978+qoniXe1uFGA3fuN4g9Bok4JjlZv3G5XKw eLgakD0RWJ90uZjNMtZZ/q9G69okar0YJu8a38YIaLEi7k0Q8AwWAWI+VMyBr+oBsAjl Zc1y/mPDUXMwJCwFizpNk6QBsawaJHc/6R5bd7GMRIvEQ4nWswdDQxU+qP1JkKTCE4qI cx2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=ZQTLYUEOHOpXSDmX5ZgrUGr+JRK4ubyzeQ8JpUkbJz8=; b=BlnOgKVvY6GPtmHdln5Y9DSfTcPQnNWs+dc0KuUEUSTJ5QnQgTqZD/4q8WyYM7kuqU PQCo6/7Pht6T4TUt9X3SrPP5I3D2Sb/lhWub9vqAC6ZX5+0lnFcbZRa8l2zgRKsiNtL7 Ube7NqCxnxLoTT05oOmP0aEqvGPbo0wDeLd6HZXMgqppB9Dhv/xs7aDz1DQDJEUpB2K+ CS3orj6vnVKXuvMtg5wLTXnAq9+N3FGNa+QhAYDo7aq90gOZ6d+mhj1P+8Nj9vejs8o8 ijo2ABC0EsW8Eyt2OYpE7Yf2Qy7mJmmICbZguZgF2nCuD+zg+ga4DRkIdfy9sU6n5XzB 6Ykw== X-Gm-Message-State: ALyK8tJ0JHW/w2NK84/iFDrRinEoYIIPGVFpthYWqfGCh2DgNWjYOWs96rWRmTHRlwBVug== X-Received: by 10.194.141.144 with SMTP id ro16mr3606231wjb.40.1464083241542; Tue, 24 May 2016 02:47:21 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id a75sm2710287wme.18.2016.05.24.02.47.20 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 24 May 2016 02:47:20 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Paul Eggert , Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> <86c6d05c-a37f-e223-d0d2-af63d09ed0cc@yandex.ru> From: Dmitry Gutov Message-ID: Date: Tue, 24 May 2016 12:47:18 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 05/24/2016 03:07 AM, Paul Eggert wrote: > Not as far as I know. I was hoping we wouldn't have to write a bug > report now, as emacs-25 does not have that bug now. I suppose someone > with more free time could write a test case.... OK, maybe it's not too important. > Emacs can decide the coding system before git diff generates any output, > by applying decode-coding-string to a canary string sample. The attached > patch should work; please give it a try. It works at least as well as my patch, or that's what I could test. But: - Shouldn't that change be in vc-coding-system-for-diff? - It seems to try to fix a separate issue (whether all files use the same coding system). - Like Eli pointed out, (coding-system-get coding-system-for-read :ascii-compatible-p) should work about as well. Why doesn't it? As an aside, how did you manage to create a patch that's using tabs for indentation, with indent-tabs-mode bound to nil in .dir-locals.el? That's troubling. From debbugs-submit-bounces@debbugs.gnu.org Tue May 24 11:37:13 2016 Received: (at 23595) by debbugs.gnu.org; 24 May 2016 15:37:13 +0000 Received: from localhost ([127.0.0.1]:36577 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5EOP-0005ci-7c for submit@debbugs.gnu.org; Tue, 24 May 2016 11:37:13 -0400 Received: from eggs.gnu.org ([208.118.235.92]:52601) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5EON-0005cU-8H for 23595@debbugs.gnu.org; Tue, 24 May 2016 11:37:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5EOF-0007mN-3L for 23595@debbugs.gnu.org; Tue, 24 May 2016 11:37:06 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60679) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5EOF-0007lp-04; Tue, 24 May 2016 11:37:03 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4380 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b5EOC-0006PS-Fm; Tue, 24 May 2016 11:37:01 -0400 Date: Tue, 24 May 2016 18:36:59 +0300 Message-Id: <83k2ij4gdg.fsf@gnu.org> From: Eli Zaretskii To: eggert@cs.ucla.edu In-reply-to: <83shx841qr.fsf@gnu.org> (message from Eli Zaretskii on Tue, 24 May 2016 05:40:44 +0300) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> <83shx841qr.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > Date: Tue, 24 May 2016 05:40:44 +0300 > From: Eli Zaretskii > Cc: oub@mat.ucm.es, dgutov@yandex.ru, 23595@debbugs.gnu.org > > > It worked for me in the Bug#23595 test case, with Git configured with > > utf16<->utf8 filters as I described. However, it reintroduces a bug when > > the version-controlled uses ISO-2022-JP. If I make a trivial change to > > etc/HELLO, for example, the patch can cause vc-diff to display mojibake, > > as the output of "git diff" uses ISO0-2022-JP but vc-diff decodes it as > > UTF-8. Although this is the same mojibake that Emacs 24.5 generates so > > the behavior is not a regression from 24.5, it is a regression from > > current emacs-25. > > For some reason I don't quite understand, iso-2022-jp fails the > ascii-compatible-p test. OK, I understand that now. ascii-compatible-p is not the right test, the right one is mime-text-unsuitable-p; and the test should be reversed, i.e. this: (coding-system-get CODING-SYSTEM :mime-text-unsuitable-p) should return nil for CODING-SYSTEM to be usable. From debbugs-submit-bounces@debbugs.gnu.org Tue May 24 11:40:59 2016 Received: (at 23595) by debbugs.gnu.org; 24 May 2016 15:40:59 +0000 Received: from localhost ([127.0.0.1]:36581 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5ES2-0005hn-Q0 for submit@debbugs.gnu.org; Tue, 24 May 2016 11:40:59 -0400 Received: from eggs.gnu.org ([208.118.235.92]:54059) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5ERx-0005hV-HW for 23595@debbugs.gnu.org; Tue, 24 May 2016 11:40:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5ERo-0000TT-7T for 23595@debbugs.gnu.org; Tue, 24 May 2016 11:40:48 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60747) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5ERo-0000Sr-40; Tue, 24 May 2016 11:40:44 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4384 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b5ERk-0008AK-Sf; Tue, 24 May 2016 11:40:42 -0400 Date: Tue, 24 May 2016 18:40:30 +0300 Message-Id: <83iny34g7l.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> (message from Dmitry Gutov on Tue, 24 May 2016 12:35:40 +0300) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org > From: Dmitry Gutov > Date: Tue, 24 May 2016 12:35:40 +0300 > > >What Emacs should do is > > bind coding-system-for-read to utf-8 in this case (not leave it > > unbound as in your patch), under the assumption that the user used the > > procedure outlined by Paul. > > Should `utf-8' altogether replace `undecided' in > vc-coding-system-for-diff? Then the use of buffer-file-coding-system > could be predicated on its being compatible with ascii. Not sure it's a good idea: the solution we found is only known to work with Git, whereas vc-coding-system-for-diff is for any VCS. Mercurial seems to have a similar encode/decode filter feature, but I'm not sure using it means the diff results will be in UTF-8. I think we should have a git-specific function that implements the above idea, and then we should use it in vc-coding-system-for-diff. (I prefer a separate function because my gut feeling is that we will need something like that in other Git operations, when UTF-16 files are involved.) WDYT? From debbugs-submit-bounces@debbugs.gnu.org Tue May 24 20:09:36 2016 Received: (at 23595) by debbugs.gnu.org; 25 May 2016 00:09:36 +0000 Received: from localhost ([127.0.0.1]:36859 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5MOG-0000jX-CJ for submit@debbugs.gnu.org; Tue, 24 May 2016 20:09:36 -0400 Received: from mail-wm0-f48.google.com ([74.125.82.48]:37104) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5MOF-0000jK-Hq for 23595@debbugs.gnu.org; Tue, 24 May 2016 20:09:35 -0400 Received: by mail-wm0-f48.google.com with SMTP id z87so42702681wmh.0 for <23595@debbugs.gnu.org>; Tue, 24 May 2016 17:09:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=h1SJpSQacK3tmlU96437eBRHHDVgcn9W1QNJI0kGw1o=; b=L89pm6FNO7BxoH5/MgODBzTBTQB5g0tuXTjgn5Zm34dG08S0Uhov1/pRKQ7IDw5D1T t/N6hn36hU2zMV4lTHq5RR9mg99S63XPk0YWKfU7WICKaPm8XVboSS3ZDOsmfcEfxk5P Lfs8F/reC0ytn5FrZ7/ktyQcLDyoaQSIII3ZDYUKU+oAg5082JfBe3k4wNCH2ojEmIVL utuY2EMn67TmwHEDGHE4eQtXYeBHiM5YNRUDDtS2SPbSKe59mNyc758rKydu2yQpFbj+ eJ44OMKkFB17Btc+d/kk+mNxPhVNlzFWINwsVpNK7hlAcd4nYZ9omPgPInkFXfHvYpYG hBcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=h1SJpSQacK3tmlU96437eBRHHDVgcn9W1QNJI0kGw1o=; b=lAP8W64xgMlKP+zhJRPBPi6treL4GnhDnJgjQxV0kjsCjpZtwLtNpsTpOKN0Fc2r0A PPgOzXLyCpmtEkB5Cmot0ZA31etZltiNv6NnccsvP+lhvH59bdUq9KiKPr1Ce7/hQtpb SBjoGmRCuN+shSvzEDZ9canTlfE/FV/6RFNXrICz7+oBE8i0Z0tV9fOqKMKyK1nTY0Ng Z7zP/LSwTfUo3Z/BmpD62AL7E8ah1wAl6YhwdqT0gSFi+6/AhqDhf0tdanv1M4n8W5pI lvCCQBx6t+SZ4tucoPl8fhox/A/quiDqB8VA4lWcCKzl/8T+v+4V2nUhUe8nnUD89V27 Gzkw== X-Gm-Message-State: ALyK8tL87vnP58EwF+cWlFREApsYTi+zQs8SEAlnZY7Ixv0iS7m2lZn3QtuHI10jZ4gSew== X-Received: by 10.194.95.40 with SMTP id dh8mr785522wjb.146.1464134969892; Tue, 24 May 2016 17:09:29 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id a128sm12787724wme.16.2016.05.24.17.09.28 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 24 May 2016 17:09:29 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> <83iny34g7l.fsf@gnu.org> From: Dmitry Gutov Message-ID: <66e22098-3368-775f-73f0-20a01f424acb@yandex.ru> Date: Wed, 25 May 2016 03:09:27 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1 MIME-Version: 1.0 In-Reply-To: <83iny34g7l.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 05/24/2016 06:40 PM, Eli Zaretskii wrote: > Not sure it's a good idea: the solution we found is only known to work > with Git, whereas vc-coding-system-for-diff is for any VCS. Mercurial > seems to have a similar encode/decode filter feature, but I'm not sure > using it means the diff results will be in UTF-8. Do we actually know that we'll need this behavior to be VCS-specific? So far, we've seem some pretty similar results with vc-diff using Git, Hg and RCS. > I think we should have a git-specific function that implements the > above idea, and then we should use it in vc-coding-system-for-diff. Git-specific or backend-specific? I suppose we could add some new encoding-handling logic at the beginning of vc-git-diff instead. > (I prefer a separate function because my gut feeling is that we will > need something like that in other Git operations, when UTF-16 files > are involved.) We can always extract a new function when it's needed, though. From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 02:19:12 2016 Received: (at 23595) by debbugs.gnu.org; 25 May 2016 06:19:13 +0000 Received: from localhost ([127.0.0.1]:36934 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5S9w-0001KB-Lw for submit@debbugs.gnu.org; Wed, 25 May 2016 02:19:12 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:57282) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5S9u-0001Jw-87 for 23595@debbugs.gnu.org; Wed, 25 May 2016 02:19:10 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 7A1CB161376; Tue, 24 May 2016 23:19:02 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 6itODWecZ5XO; Tue, 24 May 2016 23:19:01 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id C255716137E; Tue, 24 May 2016 23:19:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id E1dKJmdWig8e; Tue, 24 May 2016 23:19:01 -0700 (PDT) Received: from [192.168.1.9] (unknown [100.32.155.148]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 9F756161376; Tue, 24 May 2016 23:19:01 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Eli Zaretskii , Dmitry Gutov References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <574543D5.1030201@cs.ucla.edu> Date: Tue, 24 May 2016 23:19:01 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: <83twho41xd.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.4 (-) Eli Zaretskii wrote: > What Emacs should do is > bind coding-system-for-read to utf-8 in this case (not leave it > unbound as in your patch), under the assumption that the user used the > procedure outlined by Paul. I don't see how this would work for files like etc/HELLO, which use iso-2= 022-jp.=20 But perhaps the above comment is obsolete now. > ascii-compatible-p is not the right test, > the right one is mime-text-unsuitable-p; and the test should be > reversed, i.e. this: > > (coding-system-get CODING-SYSTEM :mime-text-unsuitable-p) > > should return nil for CODING-SYSTEM to be usable. Better, but this wouldn't work for coding systems like ebcdic-us, which a= re so=20 incompatible with ASCII that messages like "Binary files differ" would tu= rn into=20 gibberish. > testing this at run time sounds like waste of cycles. Not so many cycles that anyone will really care, I expect. We could establish a new coding system property for "close enough to ASCI= I that=20 most people won't mind". That would be a more-intrusive change, though. F= or=20 emacs-25 I thought it'd be better to have something that is more self-con= tained. From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 02:19:14 2016 Received: (at 23595) by debbugs.gnu.org; 25 May 2016 06:19:14 +0000 Received: from localhost ([127.0.0.1]:36936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5S9y-0001KP-0K for submit@debbugs.gnu.org; Wed, 25 May 2016 02:19:14 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:57292) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5S9w-0001K0-8O for 23595@debbugs.gnu.org; Wed, 25 May 2016 02:19:12 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 68DC916137E; Tue, 24 May 2016 23:19:06 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 5v6xD4fd4S3j; Tue, 24 May 2016 23:19:05 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id A2EC616137F; Tue, 24 May 2016 23:19:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id xQ0EC46wwrwL; Tue, 24 May 2016 23:19:05 -0700 (PDT) Received: from [192.168.1.9] (unknown [100.32.155.148]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 803F816137E; Tue, 24 May 2016 23:19:05 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Dmitry Gutov , Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <574543D9.6080901@cs.ucla.edu> Date: Tue, 24 May 2016 23:19:05 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.4 (-) Dmitry Gutov wrote: > Should `utf-8' altogether replace `undecided' in vc-coding-system-for-d= iff? Then > the use of buffer-file-coding-system could be predicated on its being c= ompatible > with ascii. That might be going too far. We want buffer-file-coding-system to be compatible-enough with ASCII for = the=20 case where diff output might contain ASCII metadata or non-ASCII file con= tents=20 or both. In this case, if buffer-file-coding-system is greatly incompatib= le with=20 ASCII, then ASCII will often be wrong (because the file data in the diff = output=20 will be mostly UTF-16, say), and buffer-file-coding-system will often be = wrong=20 too (because the non-file data will be mostly ASCII). So when=20 buffer-file-coding-system is greatly incompatible with ASCII, we can't us= e=20 either buffer-file-coding-system or UTF-8; they're both wrong too often. The way it's *supposed* to work in a POSIX system, is that diff is suppos= ed to=20 be applied to a file that is valid text according to the current locale's= =20 encoding, and diff is supposed to generate both metadata and data that us= es the=20 current locale's encoding. I expect that we should fall back on this appr= oach=20 when buffer-file-coding-system is greatly incompatible with ASCII. This w= ill=20 better handle unusual cases such as a system operating in an EBCDIC local= e=20 (which can happen on IBM mainframes, though admittedly Emacs is not likel= y to=20 work well on such platforms). And this argues for sticking with 'undecide= d'=20 instead of 'utf-8' here. (In theory it's possible for a GNU/Linux system to establish a locale wit= h=20 UTF-16 encoding, so that diff's metadata and data are consistently UTF-16= for=20 this example. However, I've never heard of such a thing, and couldn't fin= d any=20 evidence of one just now when I searched for it. So I don't think we need= to=20 worry about this now.) From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 02:51:43 2016 Received: (at 23595) by debbugs.gnu.org; 25 May 2016 06:51:43 +0000 Received: from localhost ([127.0.0.1]:36948 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5SfO-00024w-T0 for submit@debbugs.gnu.org; Wed, 25 May 2016 02:51:43 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:58477) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5SfN-00024j-FO for 23595@debbugs.gnu.org; Wed, 25 May 2016 02:51:41 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 0E647161371; Tue, 24 May 2016 23:51:35 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id mWtS7tCiw71B; Tue, 24 May 2016 23:51:34 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 1843D161380; Tue, 24 May 2016 23:51:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id BiTDQYg2RdUR; Tue, 24 May 2016 23:51:34 -0700 (PDT) Received: from [192.168.1.9] (unknown [100.32.155.148]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id E5AB4161371; Tue, 24 May 2016 23:51:33 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Dmitry Gutov , Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> <86c6d05c-a37f-e223-d0d2-af63d09ed0cc@yandex.ru> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <57454B75.6070506@cs.ucla.edu> Date: Tue, 24 May 2016 23:51:33 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------050309080908060600070203" X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.4 (-) This is a multi-part message in MIME format. --------------050309080908060600070203 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Dmitry Gutov wrote: > - Shouldn't that change be in vc-coding-system-for-diff? > - It seems to try to fix a separate issue (whether all files use the sa= me coding > system). Yes. For emacs-25 that's probably too much, as you suggest. So we can fix= the=20 problem in vc-coding-system-for-diff. Revised (more-conservative) patch a= ttached. > - Like Eli pointed out, (coding-system-get coding-system-for-read > :ascii-compatible-p) should work about as well. Why doesn't it? It doesn't work for EBCDIC. > As an aside, how did you manage to create a patch that's using tabs for > indentation, with indent-tabs-mode bound to nil in .dir-locals.el? That= 's > troubling. I override that setting, as I find it annoying in too many cases. It's ju= st a=20 minor annoyance, but there it is. --------------050309080908060600070203 Content-Type: text/x-diff; name="0001-Fix-vc-diff-problems-with-UTF-16.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0001-Fix-vc-diff-problems-with-UTF-16.patch" =46rom 4b608d04b5c71a580a962b014c0399d0c917d9ab Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Tue, 24 May 2016 22:28:44 -0700 Subject: [PATCH] Fix vc-diff problems with UTF-16 Problem with UTF-16 reported by Uwe Brauer (Bug#23595). There are similar problems with EBCDIC or with other coding systems differing greatly from ASCII. * lisp/vc/vc.el (vc-coding-system-for-diff): Require the file's coding system to be compatible-enough with ASCII so that messages like "Binary files differ" are not misdecoded. --- lisp/vc/vc.el | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/lisp/vc/vc.el b/lisp/vc/vc.el index af875e8..0dbdcb0 100644 --- a/lisp/vc/vc.el +++ b/lisp/vc/vc.el @@ -1601,18 +1601,31 @@ vc-coding-system-inherit-eol (defun vc-coding-system-for-diff (file) "Return the coding system for reading diff output for FILE." (or coding-system-for-read - ;; if we already have this file open, - ;; use the buffer's coding system - (let ((buf (find-buffer-visiting file))) - (when buf (with-current-buffer buf + (let ((coding + (or + ;; If we already have this file open, + ;; try the buffer's coding system. + (let ((buf (find-buffer-visiting file))) + (when buf + (with-current-buffer buf (if vc-coding-system-inherit-eol buffer-file-coding-system ;; Don't inherit the EOL part of the coding-system, ;; because some Diff tools may choose to use ;; a different one. bug#4451. (coding-system-base buffer-file-coding-system))))) - ;; otherwise, try to find one based on the file name - (car (find-operation-coding-system 'insert-file-contents file)) + ;; Otherwise, try to find one based on the file name. + (car (find-operation-coding-system 'insert-file-contents + file))))) + ;; Use the files' coding system only if it is compatible + ;; enough with ASCII. If the files' coding system is UTF-16, + ;; diff likely outputs something like "Binary files differ" in + ;; ASCII, which would be misdecoded by UTF-16. + (when (and coding + (let ((samp "Binary files differ")) + (string-equal samp (decode-coding-string + samp coding t)))) + last-coding-system-used)) ;; and a final fallback 'undecided)) =20 --=20 2.5.5 --------------050309080908060600070203-- From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 08:44:18 2016 Received: (at 23595) by debbugs.gnu.org; 25 May 2016 12:44:18 +0000 Received: from localhost ([127.0.0.1]:37062 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5YAc-0003fB-CS for submit@debbugs.gnu.org; Wed, 25 May 2016 08:44:18 -0400 Received: from mail-wm0-f44.google.com ([74.125.82.44]:35878) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5YAa-0003ez-NC for 23595@debbugs.gnu.org; Wed, 25 May 2016 08:44:17 -0400 Received: by mail-wm0-f44.google.com with SMTP id n129so179941173wmn.1 for <23595@debbugs.gnu.org>; Wed, 25 May 2016 05:44:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=DboRBdNfsinu6Ug5Fiq1OFs4DM4cnQmvs98YFYhaCOk=; b=yOo1TD7wSipkiwj6xLWQN7Bma+EaWmausaZpmf3TfVGS0X4vOy3mi6lHhJRU1Dn0uq DoUzhGj6H9vU4yVsZngx55yHK9ej8zs+zd6eqkpaw650TG2lMlfnDKJSNt9V3k2nNdBZ X4iZk2cR/KCPCMM2sJFOFU246SCRzNZUmCFqJIlMtD74wkxMk83nGl5vbKRGQ5MbroDi qTvZjxLF9BYUDamRZBcREZkY6hLTt+eg+LVL1ZWzwuw0xGOv0gAL9M/Lsdqj5te1yunx /Zzjrl76YmiXibVtFM2dxF8ext4ub6/KvQCtYldy+1U0zxuDa2PuMguxWiK/ZsggOAms uGZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=DboRBdNfsinu6Ug5Fiq1OFs4DM4cnQmvs98YFYhaCOk=; b=GcAhRVxgf7pWElO1rJfQuWuFEkd6kx3vVZAvjSdKpE0ww23eL/Jwf3NeuT8e0mNFqX 3YSxMhfQS8wPiT0qo5PKeGi2eQWjsOcLzrAep9FPWfG1xNrJ6bhY7nKhBnMkDCJAipBR ruJgj+0mjgHe2mxyckSwsZi+lGfcEyDG+P3JtB9VL/5z6dl3fXuKqjR5zbdFP9/5ca2H qOO9boCFGTtktMz+RW/i8pNHnCK8lUJ+e4D7Ign11SthPNtG1rPTFdxrQMG6hox6VXW7 e/IOUTADtG+fIu+1iMv5rHkTIr2KNtPl2rt7kxhK32OIlahrkKHwwavJudNmIlppepzs IGPw== X-Gm-Message-State: ALyK8tKrYwbxgRvWEVvSOrMDg36V4xxIGT/1GVE/lfmkmmKOsZcJn68tS41axZbzYe6gZg== X-Received: by 10.194.206.39 with SMTP id ll7mr3573792wjc.179.1464180250731; Wed, 25 May 2016 05:44:10 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id on2sm8618654wjc.32.2016.05.25.05.44.09 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 25 May 2016 05:44:09 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Paul Eggert , Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <32b48032-8b30-d1d4-259c-8715aad3e7b8@cs.ucla.edu> <86c6d05c-a37f-e223-d0d2-af63d09ed0cc@yandex.ru> <57454B75.6070506@cs.ucla.edu> From: Dmitry Gutov Message-ID: <5fb39ec5-a6fc-1e9d-8a44-a73fdfd5c14c@yandex.ru> Date: Wed, 25 May 2016 15:44:08 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1 MIME-Version: 1.0 In-Reply-To: <57454B75.6070506@cs.ucla.edu> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 05/25/2016 09:51 AM, Paul Eggert wrote: > Yes. For emacs-25 that's probably too much, as you suggest. So we can > fix the problem in vc-coding-system-for-diff. Revised > (more-conservative) patch attached. Looks good, thanks. We should also test it with different backends, to see if it helps with this problem in each of them (at least to some extent). Like Eli said, it might also cause new problems in some of them. From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 12:22:25 2016 Received: (at 23595) by debbugs.gnu.org; 25 May 2016 16:22:25 +0000 Received: from localhost ([127.0.0.1]:38469 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5bZh-0002Qo-DO for submit@debbugs.gnu.org; Wed, 25 May 2016 12:22:25 -0400 Received: from eggs.gnu.org ([208.118.235.92]:35977) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5bZf-0002QZ-Kr for 23595@debbugs.gnu.org; Wed, 25 May 2016 12:22:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5bZU-0007st-OH for 23595@debbugs.gnu.org; Wed, 25 May 2016 12:22:18 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52248) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5bZU-0007so-Kp; Wed, 25 May 2016 12:22:12 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1399 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b5bZS-0001LI-PF; Wed, 25 May 2016 12:22:11 -0400 Date: Wed, 25 May 2016 19:22:14 +0300 Message-Id: <8360u23y6h.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <66e22098-3368-775f-73f0-20a01f424acb@yandex.ru> (message from Dmitry Gutov on Wed, 25 May 2016 03:09:27 +0300) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> <83iny34g7l.fsf@gnu.org> <66e22098-3368-775f-73f0-20a01f424acb@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org > From: Dmitry Gutov > Date: Wed, 25 May 2016 03:09:27 +0300 > > Not sure it's a good idea: the solution we found is only known to work > with Git, whereas vc-coding-system-for-diff is for any VCS. Mercurial > seems to have a similar encode/decode filter feature, but I'm not sure > using it means the diff results will be in UTF-8. > > Do we actually know that we'll need this behavior to be VCS-specific? I think we can make an educated guess, see below. My conclusion is that this does need to be VCS-specific. > So far, we've seem some pretty similar results with vc-diff using Git, Hg and RCS. Not quite. They all fail, but in different ways. More importantly, the solutions are most probably going to be different. > I think we should have a git-specific function that implements the > above idea, and then we should use it in vc-coding-system-for-diff. > > Git-specific or backend-specific? Backend-specific, most probably. Except that currently we only have a good idea about the Git backend, for which it is explicitly documented that the output will be in UTF-8 when content filters are used. Mercurial and Bazaar both support similar filters, but I cannot find any documentation on what encoding will be used for the output. For Bazaar, there's a general statement somewhere that it defaults to the locale's encoding (there's a config variable to change that). SVN doesn't seem to support filters at all, so with it, the user will have to manually set the mime-type property of the UTF-16 files as text, and install a replacement Diff command that can produce diffs from UTF-16 files (I believe GNU Diff cannot currently do that). Since no canonical way exists, I don't see how we can know for sure the encoding of the Diff output; my best guess is that it will also be in UTF-16. (Similar problems exist in SVN with other operations on such files.) For RCS and CVS, I don't see any solution at all, since AFAIK these don't support any such features or anything similar. These will always treat UTF-16 files as binary, so no meaningful diffs can be produced for them. > I suppose we could add some new encoding-handling logic at the beginning of vc-git-diff instead. > > (I prefer a separate function because my gut feeling is that we will > need something like that in other Git operations, when UTF-16 files > are involved.) > > We can always extract a new function when it's needed, though. True, but I think if we want to support UTF-16 files, the need is already here. vc-diff and its derivatives are just the tip of the iceberg, we will need similar stuff for every command that includes both text from the versioned file(s) and some text output by the VCS program itself. From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 12:26:14 2016 Received: (at 23595) by debbugs.gnu.org; 25 May 2016 16:26:14 +0000 Received: from localhost ([127.0.0.1]:38478 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5bdO-0002We-74 for submit@debbugs.gnu.org; Wed, 25 May 2016 12:26:14 -0400 Received: from eggs.gnu.org ([208.118.235.92]:37055) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5bdN-0002WS-1M for 23595@debbugs.gnu.org; Wed, 25 May 2016 12:26:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5bdD-0000i3-Kx for 23595@debbugs.gnu.org; Wed, 25 May 2016 12:26:07 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52296) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5bdD-0000hp-HM; Wed, 25 May 2016 12:26:03 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1402 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b5bdB-0001ta-Gx; Wed, 25 May 2016 12:26:01 -0400 Date: Wed, 25 May 2016 19:26:06 +0300 Message-Id: <831t4q3y01.fsf@gnu.org> From: Eli Zaretskii To: Paul Eggert In-reply-to: <574543D5.1030201@cs.ucla.edu> (message from Paul Eggert on Tue, 24 May 2016 23:19:01 -0700) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <574543D5.1030201@cs.ucla.edu> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org > From: Paul Eggert > Date: Tue, 24 May 2016 23:19:01 -0700 > > ascii-compatible-p is not the right test, > the right one is mime-text-unsuitable-p; and the test should be > reversed, i.e. this: > > (coding-system-get CODING-SYSTEM :mime-text-unsuitable-p) > > should return nil for CODING-SYSTEM to be usable. > > Better, but this wouldn't work for coding systems like ebcdic-us, which are so incompatible with ASCII that messages like "Binary files differ" would turn into gibberish. It's easy enough to exempt EBCDIC (and any other similar encodings). There are only 3 of them, AFAICS. > We could establish a new coding system property for "close enough to ASCII that most people won't mind". That would be a more-intrusive change, though. For emacs-25 I thought it'd be better to have something that is more self-contained. A :mime-text-unsuitable-p test augmented by a list of additional coding-systems we find unsuitable is simple, self-contained, and safe for emacs-25, IMO. For master, we could add a cleaner, but more intrusive fix. From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 12:27:06 2016 Received: (at 23595) by debbugs.gnu.org; 25 May 2016 16:27:06 +0000 Received: from localhost ([127.0.0.1]:38482 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5beE-0002YB-IB for submit@debbugs.gnu.org; Wed, 25 May 2016 12:27:06 -0400 Received: from eggs.gnu.org ([208.118.235.92]:37244) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5beD-0002XZ-Jt for 23595@debbugs.gnu.org; Wed, 25 May 2016 12:27:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5be4-0000sn-Ew for 23595@debbugs.gnu.org; Wed, 25 May 2016 12:27:00 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52306) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5be4-0000sj-BI; Wed, 25 May 2016 12:26:56 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1403 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b5be3-00033i-D6; Wed, 25 May 2016 12:26:55 -0400 Date: Wed, 25 May 2016 19:27:00 +0300 Message-Id: <83zire2je3.fsf@gnu.org> From: Eli Zaretskii To: Paul Eggert In-reply-to: <574543D9.6080901@cs.ucla.edu> (message from Paul Eggert on Tue, 24 May 2016 23:19:05 -0700) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> <574543D9.6080901@cs.ucla.edu> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org, dgutov@yandex.ru X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > Cc: oub@mat.ucm.es, 23595@debbugs.gnu.org > From: Paul Eggert > Date: Tue, 24 May 2016 23:19:05 -0700 > > - Like Eli pointed out, (coding-system-get coding-system-for-read > :ascii-compatible-p) should work about as well. Why doesn't it? > > It doesn't work for EBCDIC. It's easy enough to exempt EBCDIC (and any other similar encodings). > - Shouldn't that change be in vc-coding-system-for-diff? > - It seems to try to fix a separate issue (whether all files use the same coding > system). > > Yes. For emacs-25 that's probably too much, as you suggest. So we can fix the problem in vc-coding-system-for-diff. Revised (more-conservative) patch attached. That patch relies on subtleties of comparing unibyte and multibyte strings, so I don't like it, even if we ignore the issue of computing something whose result is known in advance. I still think we can come up with a simple (and safe for emacs-25) method of identifying the problematic encodings, and leave the general issue of having a new attribute for master. Can we have a patch along those lines, please? From debbugs-submit-bounces@debbugs.gnu.org Wed May 25 19:28:25 2016 Received: (at 23595) by debbugs.gnu.org; 25 May 2016 23:28:25 +0000 Received: from localhost ([127.0.0.1]:39124 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5iDw-00062w-Ox for submit@debbugs.gnu.org; Wed, 25 May 2016 19:28:24 -0400 Received: from mail-lb0-f171.google.com ([209.85.217.171]:35821) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5iDv-00062k-4t for 23595@debbugs.gnu.org; Wed, 25 May 2016 19:28:23 -0400 Received: by mail-lb0-f171.google.com with SMTP id ww9so20274522lbc.2 for <23595@debbugs.gnu.org>; Wed, 25 May 2016 16:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=lQx9ZW0Wo5r3KOphIQcvpxmkPCHVrnseJBuQX3EStx4=; b=KfXpFMw1HceJF/5efplxbPLQW3T6HFnHbAvLhYBDLi+xyRWnA1/qxOF1oNau3o2FqT yRyulcam/hzvWxsCBSDbINqEeyDBez8Ewl/N1tyovz3NkBGWnHK5A6qkq0JloBD0ds1w a5n94jDe/Onn4/6NtLJotBFiq7KD/RVZ8mSvJrRQEhlzc7Ce3IDCU0mF//uzd4423sQ1 svDQhi3DylrYXFiF69V3KWh0nywDHcsI+Ht0H6z95mt6DXFzF3JZoKiWgTPq2ChtDy2V SH1A6aIIhVZfXScjYqixEq5MAvq/QxVWDnj6dQvIMSEzzpUgV63l6rL0vMbla5VHmfIb /3iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to; bh=lQx9ZW0Wo5r3KOphIQcvpxmkPCHVrnseJBuQX3EStx4=; b=CGnbmP29NoYFbvJQ6CC78uA8mRJ8rJdTQ78jnAYod7m/gbbRMNfntcEAdxdz71nhrH VXBPmqp2SyDZGP1W/2Wgmf2fGesea2cAkd6LFWfB899/9mGWogOdnDQl5zBQKUhp2Lxo Gqm7AjGnnKKuG9Pi72tMf1b9URR+6LmxKMMng/qtDjlBQGV2FrsZO+3xC/aHkfiYIyl7 GxiwoQSi7w/9O65shsB7A/jsRBEzfM+fPYrLCZH7hgmY4/96v1l+we9iAS4VMGyp4ENn BM1CyrKaN7Svxp6z3AxoAfm/38uoiriQhzx3/lw1y55EmY4OPEXIJRhEobKUkhZvkPhQ W0fw== X-Gm-Message-State: ALyK8tJM5XBJETgwPCHv1TsFhgiQw4pKP27V+Oy7zkxTcqA62DtpFAq/VP5w/kDI2Bla6g== X-Received: by 10.194.69.106 with SMTP id d10mr6224929wju.165.1464218503746; Wed, 25 May 2016 16:21:43 -0700 (PDT) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id x124sm400121wmg.24.2016.05.25.16.21.41 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 25 May 2016 16:21:42 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> <83iny34g7l.fsf@gnu.org> <66e22098-3368-775f-73f0-20a01f424acb@yandex.ru> <8360u23y6h.fsf@gnu.org> From: Dmitry Gutov Message-ID: <18c9d07f-a6d6-c326-5d5e-5c4517126147@yandex.ru> Date: Thu, 26 May 2016 02:21:40 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1 MIME-Version: 1.0 In-Reply-To: <8360u23y6h.fsf@gnu.org> Content-Type: multipart/mixed; boundary="------------9C43E1EFC5FA432F4FE5024A" X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) This is a multi-part message in MIME format. --------------9C43E1EFC5FA432F4FE5024A Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 05/25/2016 07:22 PM, Eli Zaretskii wrote: > I think we can make an educated guess, see below. My conclusion is > that this does need to be VCS-specific. > >> So far, we've seem some pretty similar results with vc-diff using Git, Hg and RCS. > > Not quite. They all fail, but in different ways. More importantly, > the solutions are most probably going to be different. OK, that makes sense. > Backend-specific, most probably. Except that currently we only have a > good idea about the Git backend, for which it is explicitly documented > that the output will be in UTF-8 when content filters are used. > > ... Taking a step back, I'm not sure it's really necessary to fix this. Because Uwe, the only person to complain about diffing UTF-16 files so far, can solve it much easier by switching the file to UTF-8. And properly fixing it, across the board, is either impossible, or at least pretty hard. >> We can always extract a new function when it's needed, though. > > True, but I think if we want to support UTF-16 files, the need is > already here. vc-diff and its derivatives are just the tip of the > iceberg, we will need similar stuff for every command that includes > both text from the versioned file(s) and some text output by the VCS > program itself. Here's a patch everybody is welcome to try. --------------9C43E1EFC5FA432F4FE5024A Content-Type: text/x-patch; name="git-asciify.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="git-asciify.diff" diff --git a/lisp/vc/vc-git.el b/lisp/vc/vc-git.el index f35c84d..f7bd867 100644 --- a/lisp/vc/vc-git.el +++ b/lisp/vc/vc-git.el @@ -1083,12 +1083,19 @@ vc-git-region-history-mode (cdr font-lock-defaults)))) +(defun git--asciify-coding-system () + (unless (let ((samp "Binary files differ")) + (string-equal samp (decode-coding-string + samp coding-system-for-read t))) + (setq coding-system-for-read 'undecided))) + (autoload 'vc-switches "vc") (defun vc-git-diff (files &optional rev1 rev2 buffer _async) "Get a difference report using Git between two revisions of FILES." (let (process-file-side-effects (command "diff-tree")) + (git--asciify-coding-system) (if rev2 ;; Diffing against the empty tree. (unless rev1 (setq rev1 "4b825dc642cb6eb9a060e54bf8d69288fbee4904")) @@ -1127,6 +1134,7 @@ vc-git-revision-completion-table table)) (defun vc-git-annotate-command (file buf &optional rev) + (git--asciify-coding-system) (let ((name (file-relative-name file))) (apply #'vc-git-command buf 'async nil "blame" "--date=short" (append (vc-switches 'git 'annotate) --------------9C43E1EFC5FA432F4FE5024A-- From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 06:44:20 2016 Received: (at 23595) by debbugs.gnu.org; 26 May 2016 10:44:20 +0000 Received: from localhost ([127.0.0.1]:39308 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5sm4-0005DM-4W for submit@debbugs.gnu.org; Thu, 26 May 2016 06:44:20 -0400 Received: from mail-wm0-f44.google.com ([74.125.82.44]:35973) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5sm2-0005D9-4e for 23595@debbugs.gnu.org; Thu, 26 May 2016 06:44:18 -0400 Received: by mail-wm0-f44.google.com with SMTP id n129so220487216wmn.1 for <23595@debbugs.gnu.org>; Thu, 26 May 2016 03:44:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mat.ucm.es; s=google; h=from:to:cc:subject:references:reply-to:date:message-id:user-agent :mime-version; bh=OuJfT72u/gLlsDxVlw0c4nKSTOH58BPZu4tuyfo1A28=; b=R4vHoEprS7oMVYbWNynmCp8FVmHhGbxSIOUbipEkz8UKebsUBGspdScDlfZZlIzEN5 s+DBUZ5sN03jsRIxQeLRVMwOv79SPsJcNBhkd+EDktzLKYyh6b+JRN000C2tSjLvtMn3 bNQssKgR8CZtRybuCp7Yt+LYu45oh2rI6zyxE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:references:reply-to:date :message-id:user-agent:mime-version; bh=OuJfT72u/gLlsDxVlw0c4nKSTOH58BPZu4tuyfo1A28=; b=Uow2eTHf1A+chpO93Fy7ed7RoCQEmGO2PYYZTs6cCf8DrJX6xBBjS+5FCGZNuRaLeI XpukOz5vcQiXZOJiQTt6V1watB/Si3JGTDx8Q1u2I7qVy/JnvudO9aLh42GaQLpA4MJm qemZqcyvsT/jF0veDh0vvox3brKEGjkPVutksMYYZkCMOyXmOAwthEyx4tvc+o1yEw3q hv8589wupEWtT/6PSNhdUtxLaIGLO/QsTxmA9D0x0GsQSQ0zTDegmwJzu2xuaGSAq47g iX8ZYg/uvMGrwER9wLEjl0XPPSGw2HSeojAY1m+7yi5zUSvCw8R7hiVz0nKzIvEkfBc+ G0BA== X-Gm-Message-State: ALyK8tKWsVcOJd8GVlJPxMsHOXePFC5+IFd7E2hBI0gDGinHTc6s8mEqMkRGROSKsB9Xbd9E X-Received: by 10.28.232.136 with SMTP id f8mr3198630wmi.52.1464259452183; Thu, 26 May 2016 03:44:12 -0700 (PDT) Received: from Gilgamesch (gilgamesch.quim.ucm.es. [147.96.12.99]) by smtp.gmail.com with ESMTPSA id q194sm2857571wme.18.2016.05.26.03.44.10 (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 26 May 2016 03:44:10 -0700 (PDT) X-Mailer: emacs 25.1.50.1 (via feedmail 11-beta-1 I) From: Uwe Brauer To: Dmitry Gutov Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> <83iny34g7l.fsf@gnu.org> <66e22098-3368-775f-73f0-20a01f424acb@yandex.ru> <8360u23y6h.fsf@gnu.org> <18c9d07f-a6d6-c326-5d5e-5c4517126147@yandex.ru> X-Hashcash: 1:20:160526:dgutov@yandex.ru::41JCUGeO04Qezeze:00SeF X-Hashcash: 1:20:160526:eggert@cs.ucla.edu::v/CKf3nFzljRL5FX:00000000000000000000000000000000000000000002Y1D X-Hashcash: 1:20:160526:oub@mat.ucm.es::KKqhP4Trfm8JIxRI:0005lCV X-Hashcash: 1:20:160526:eliz@gnu.org::ALkA5K7JYxk/ztkp:0000048IC X-Hashcash: 1:20:160526:23595@debbugs.gnu.org::3PBO3bzj5IunKuTB:0000000000000000000000000000000000000000CE0k Date: Thu, 26 May 2016 10:44:09 +0000 Message-ID: <87shx59k06.fsf@mat.ucm.es> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, Eli Zaretskii , eggert@cs.ucla.edu, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Uwe Brauer Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) > On 05/25/2016 07:22 PM, Eli Zaretskii wrote: > OK, that makes sense. > Taking a step back, I'm not sure it's really necessary to fix this. > Because Uwe, the only person to complain about diffing UTF-16 files so > far, can solve it much easier by switching the file to UTF-8. Correct and I already did. :-D so for me the situation is fine. I would however suggest to add some short information, maybe in the docstring of vc-diff, to mention the problem with UTF-16. From debbugs-submit-bounces@debbugs.gnu.org Thu May 26 11:35:13 2016 Received: (at 23595) by debbugs.gnu.org; 26 May 2016 15:35:13 +0000 Received: from localhost ([127.0.0.1]:40677 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5xJW-0005Vb-9O for submit@debbugs.gnu.org; Thu, 26 May 2016 11:35:13 -0400 Received: from eggs.gnu.org ([208.118.235.92]:45067) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b5xJV-0005VI-8Z for 23595@debbugs.gnu.org; Thu, 26 May 2016 11:35:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5xJM-0007QY-4x for 23595@debbugs.gnu.org; Thu, 26 May 2016 11:35:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:39659) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5xJM-0007Po-2n; Thu, 26 May 2016 11:35:00 -0400 Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2344 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1b5xJL-0005Rz-3m; Thu, 26 May 2016 11:34:59 -0400 Date: Thu, 26 May 2016 18:35:06 +0300 Message-Id: <83bn3s3k9h.fsf@gnu.org> From: Eli Zaretskii To: Dmitry Gutov In-reply-to: <18c9d07f-a6d6-c326-5d5e-5c4517126147@yandex.ru> (message from Dmitry Gutov on Thu, 26 May 2016 02:21:40 +0300) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> <83iny34g7l.fsf@gnu.org> <66e22098-3368-775f-73f0-20a01f424acb@yandex.ru> <8360u23y6h.fsf@gnu.org> <18c9d07f-a6d6-c326-5d5e-5c4517126147@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -6.4 (------) X-Debbugs-Envelope-To: 23595 Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.4 (------) > Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595@debbugs.gnu.org > From: Dmitry Gutov > Date: Thu, 26 May 2016 02:21:40 +0300 > > > Backend-specific, most probably. Except that currently we only have a > > good idea about the Git backend, for which it is explicitly documented > > that the output will be in UTF-8 when content filters are used. > > > > Taking a step back, I'm not sure it's really necessary to fix this. > Because Uwe, the only person to complain about diffing UTF-16 files so > far, can solve it much easier by switching the file to UTF-8. > > And properly fixing it, across the board, is either impossible, or at > least pretty hard. Well, it's not too hard for Git, so I think it would be good to fix it, at least on master. > >> We can always extract a new function when it's needed, though. > > > > True, but I think if we want to support UTF-16 files, the need is > > already here. vc-diff and its derivatives are just the tip of the > > iceberg, we will need similar stuff for every command that includes > > both text from the versioned file(s) and some text output by the VCS > > program itself. > > Here's a patch everybody is welcome to try. Thanks. Like I said, I don't like using string-equal to compare unencoded and encoded strings, but that aspect could be fixed by a follow-up change. If no one objects in a week, I suggest to push to master. From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 19 15:09:28 2016 Received: (at 23595-done) by debbugs.gnu.org; 19 Jun 2016 19:09:28 +0000 Received: from localhost ([127.0.0.1]:46661 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bEi64-00017p-0j for submit@debbugs.gnu.org; Sun, 19 Jun 2016 15:09:28 -0400 Received: from mail-wm0-f43.google.com ([74.125.82.43]:37815) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bEi62-00017d-Lj for 23595-done@debbugs.gnu.org; Sun, 19 Jun 2016 15:09:27 -0400 Received: by mail-wm0-f43.google.com with SMTP id a66so51548428wme.0 for <23595-done@debbugs.gnu.org>; Sun, 19 Jun 2016 12:09:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=HhlFSKwkygQx/sPm/GDx4PdX60jhD78eymBLONQYPR8=; b=tQUfZNMpzjvBNz5kkcmPlzXZpALf5tLFl5mQSzGNtd0DRwgWDZXDG78c6EOKrCM094 zgKapkI0TKBrYS0FuoWW1r2nGtYxm4oC+0G1BxlJvgdHelkN9mmD2gX67dLSGVdAGlds ncOnyrPvar+7qLr1QaTXO2HMsBUO8LjTpHiUUletNNT7BwVyCufNEEzl6SiwFh9Hifz9 ceu1xvFyqREoTc39SxyHBCRd2APqXL8ZDYDXyFFiFlLISyD92MXrSHnb2/O8hQtAbJyU N6GS82Vb+X5+Vp0JqrWk9olK4s//t1gwKBBR9wrghr7fKxUDcCNCo+CddKm4PvqBq75A 21HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=HhlFSKwkygQx/sPm/GDx4PdX60jhD78eymBLONQYPR8=; b=ExtOs05jtEyKG9iYFNk49pHx7wZnz7bH6bYa/SiX/BFdltZaN9QQ3OYQ0wggzGUZQC ftxBNfUH1Bt23qVZAFL42QRpqptAoe13A2tA3TigaYWo533MihJNV6WX45m3zNLw7G+P TqzbaupYH5TvqOA01Kh5jg3HmjYKk/L/YI0SZ2NIoo7QlVAkZ10Z+Fm3DzFZfa0GD3gt o2N1Q+KLWp5JQ4qIuDfmCSA+2zWv5CvUgWgJ9fOVsu8xNGeVagDK/HiWXSE9BTm5njgy UqmNaEc0nR97Oo5uc+oydBHYuB0Ro974urJ9tSg67AraIIECrwZ5Sqel4jejs87BiWoT eKJQ== X-Gm-Message-State: ALyK8tI+MBQpcrTFlsHK6KrRoqlPUFbizgxloPoujfNoLBRn5FycimnOMHn2OgA3rgBvTg== X-Received: by 10.28.5.147 with SMTP id 141mr8199422wmf.48.1466363361020; Sun, 19 Jun 2016 12:09:21 -0700 (PDT) Received: from [192.168.1.2] ([185.105.173.135]) by smtp.googlemail.com with ESMTPSA id w76sm9739297wmd.11.2016.06.19.12.09.19 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 19 Jun 2016 12:09:20 -0700 (PDT) Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS) To: Eli Zaretskii References: <87bn3z4l9i.fsf@mat.ucm.es> <1444321464004323@web25h.yandex.ru> <83h9do67pp.fsf@gnu.org> <21f6198c-a2fc-365f-caf7-79fad5027f1c@yandex.ru> <83twho41xd.fsf@gnu.org> <1f8cf525-c138-03f6-7f17-65015dc5cdfa@yandex.ru> <83iny34g7l.fsf@gnu.org> <66e22098-3368-775f-73f0-20a01f424acb@yandex.ru> <8360u23y6h.fsf@gnu.org> <18c9d07f-a6d6-c326-5d5e-5c4517126147@yandex.ru> <83bn3s3k9h.fsf@gnu.org> From: Dmitry Gutov Message-ID: <9122abea-f771-d3ec-ce45-0df001b47617@yandex.ru> Date: Sun, 19 Jun 2016 22:09:18 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2 MIME-Version: 1.0 In-Reply-To: <83bn3s3k9h.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 23595-done Cc: oub@mat.ucm.es, eggert@cs.ucla.edu, 23595-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 05/26/2016 06:35 PM, Eli Zaretskii wrote: >> Here's a patch everybody is welcome to try. > > Thanks. Like I said, I don't like using string-equal to compare > unencoded and encoded strings, but that aspect could be fixed by a > follow-up change. > > If no one objects in a week, I suggest to push to master. Pushed, and closing. From unknown Sat Aug 09 04:56:40 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 18 Jul 2016 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator