From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: One character can be lost if colors are enabled Resent-From: Jaroslav Skarvada Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Mon, 23 Sep 2013 13:35:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: 15444@debbugs.gnu.org X-Debbugs-Original-To: bug-grep@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.137994328114712 (code B ref -1); Mon, 23 Sep 2013 13:35:02 +0000 Received: (at submit) by debbugs.gnu.org; 23 Sep 2013 13:34:41 +0000 Received: from localhost ([127.0.0.1]:57946 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VO6Hg-0003pD-Co for submit@debbugs.gnu.org; Mon, 23 Sep 2013 09:34:41 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56642) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VO6Hb-0003ot-Hk for submit@debbugs.gnu.org; Mon, 23 Sep 2013 09:34:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VO6HN-0005jp-QG for submit@debbugs.gnu.org; Mon, 23 Sep 2013 09:34:30 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:57297) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VO6HN-0005jl-Nt for submit@debbugs.gnu.org; Mon, 23 Sep 2013 09:34:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58345) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VO6HH-0001yL-Jy for bug-grep@gnu.org; Mon, 23 Sep 2013 09:34:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VO6HB-0005gs-Kt for bug-grep@gnu.org; Mon, 23 Sep 2013 09:34:15 -0400 Received: from mx4-phx2.redhat.com ([209.132.183.25]:35247) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VO6HB-0005gh-Cj for bug-grep@gnu.org; Mon, 23 Sep 2013 09:34:09 -0400 Received: from zmail14.collab.prod.int.phx2.redhat.com (zmail14.collab.prod.int.phx2.redhat.com [10.5.83.16]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id r8NDY8Ft020262 for ; Mon, 23 Sep 2013 09:34:08 -0400 Date: Mon, 23 Sep 2013 09:34:05 -0400 (EDT) From: Jaroslav Skarvada Message-ID: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> In-Reply-To: <649516636.557185.1379942182710.JavaMail.root@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.3_GA_5664 (ZimbraWebClient - FF23 (Linux)/8.0.3_GA_5664) Thread-Topic: One character can be lost if colors are enabled Thread-Index: AYWhRYLQ5bDuN0lWiV+Oj3poa8DTKA== X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Reproducer: $ printf 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1234xxxxxxxxx\n' | grep 1234 --color=always xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx123xxxxxxxxx This can be reproduced at least on xterm and linux console, but it works on xfce4-terminal and konsole (at least). The EL command (\E[K) is sent to clear the line. The VT100 terminal autowraps when the 81th character is received and the EL command is not counted as a character. So if 80 characters are received, it waits on the 80th character and the following EL command erases characters from the current one to the end of line (i.e. the last character on the line). Workaround is to use GREP_COLORS=ne. The original Red Hat bugzilla that contains discussion with the xterm upstream: http://bugzilla.redhat.com/show_bug.cgi?id=1006310 From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: One character can be lost if colors are enabled Resent-From: Jim Meyering Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Mon, 23 Sep 2013 17:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: Jaroslav Skarvada Cc: 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.13799577999646 (code B ref 15444); Mon, 23 Sep 2013 17:37:02 +0000 Received: (at 15444) by debbugs.gnu.org; 23 Sep 2013 17:36:39 +0000 Received: from localhost ([127.0.0.1]:58577 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VOA3r-0002VV-26 for submit@debbugs.gnu.org; Mon, 23 Sep 2013 13:36:39 -0400 Received: from mail-pa0-f49.google.com ([209.85.220.49]:51203) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VOA3o-0002VG-VT for 15444@debbugs.gnu.org; Mon, 23 Sep 2013 13:36:37 -0400 Received: by mail-pa0-f49.google.com with SMTP id ld10so3849897pab.36 for <15444@debbugs.gnu.org>; Mon, 23 Sep 2013 10:36:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=3EfGdZ+celOo9xxf8D58FTBVB926WEuUCOAm1MMAWoc=; b=malbalVjZ3TkHJuXdKL0ieMhhKTDW5KuVA2LjRWqMbym3LLudRS2+EyzVM8s6rLs64 92259NdcJgfok0MLpIY6GkCOnCdbXAXta5QTM8oerb4PYga5KYuWBi5ybNs9Bem04AKA Vj0pvnF2hT8wlRlombL8SZ/PItBSv703LQtJNyF9Jc5vWUKQWhXXOPRsLufe+55mXGAH h1yR4GWN+2x5t4LW6+SRYraT6VZK+y5qio3IEGmcoCiruXsZMe0UzpF9uVUt4da42Mx4 Ynb/FaAZCaqE6Z7pHVynI5W5H6n0rypeo35Mnb648LhDUAjSJ3adUsj9j9964HlKjbIb BvFA== X-Received: by 10.68.180.34 with SMTP id dl2mr14731983pbc.6.1379957790825; Mon, 23 Sep 2013 10:36:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.68.6.66 with HTTP; Mon, 23 Sep 2013 10:35:58 -0700 (PDT) In-Reply-To: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> References: <649516636.557185.1379942182710.JavaMail.root@redhat.com> <1237403814.574122.1379943245507.JavaMail.root@redhat.com> From: Jim Meyering Date: Mon, 23 Sep 2013 10:35:58 -0700 X-Google-Sender-Auth: 7Ao6tLIdOvOUBLeQqF9-6iCN4FI Message-ID: Content-Type: text/plain; charset=ISO-8859-1 X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Mon, Sep 23, 2013 at 6:34 AM, Jaroslav Skarvada wrote: > printf 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1234xxxxxxxxx\n' | grep 1234 --color=always Thank you for the report. I confirm that setting GREP_COLORS=ne is a work-around. Does that have unwelcome side effects on any other type of terminal that you've tried? From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: One character can be lost if colors are enabled Resent-From: Jaroslav Skarvada Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Tue, 24 Sep 2013 07:42:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: Jim Meyering Cc: 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.138000846923861 (code B ref 15444); Tue, 24 Sep 2013 07:42:01 +0000 Received: (at 15444) by debbugs.gnu.org; 24 Sep 2013 07:41:09 +0000 Received: from localhost ([127.0.0.1]:59864 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VONF5-0006Cm-8Q for submit@debbugs.gnu.org; Tue, 24 Sep 2013 03:41:08 -0400 Received: from mx4-phx2.redhat.com ([209.132.183.25]:51610) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VONF2-0006Cc-2v for 15444@debbugs.gnu.org; Tue, 24 Sep 2013 03:41:05 -0400 Received: from zmail14.collab.prod.int.phx2.redhat.com (zmail14.collab.prod.int.phx2.redhat.com [10.5.83.16]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id r8O7f2sH001349; Tue, 24 Sep 2013 03:41:02 -0400 Date: Tue, 24 Sep 2013 03:41:01 -0400 (EDT) From: Jaroslav Skarvada Message-ID: <554364705.1075194.1380008461304.JavaMail.root@redhat.com> In-Reply-To: References: <649516636.557185.1379942182710.JavaMail.root@redhat.com> <1237403814.574122.1379943245507.JavaMail.root@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.3_GA_5664 (ZimbraWebClient - FF23 (Linux)/8.0.3_GA_5664) Thread-Topic: bug#15444: One character can be lost if colors are enabled Thread-Index: gWGfAblcNhmNI0yqMm/GhLdAnIJ/+Q== X-Spam-Score: -7.3 (-------) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -7.3 (-------) ----- Original Message ----- > On Mon, Sep 23, 2013 at 6:34 AM, Jaroslav Skarvada > wrote: > > printf > > 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1234xxxxxxxxx\n' > > | grep 1234 --color=always > > Thank you for the report. > I confirm that setting GREP_COLORS=ne is a work-around. Does that > have unwelcome side effects on any other type of terminal that you've > tried? > It's only workaround, I think it cannot be used by default, because it can have undesired effect if custom background is used, e.g.: $ printf 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx12345x\n' | GREP_COLORS='sl=01;41:ne' grep --color=always 1234 (on some terms you need to repeat this more times until the term vertically scrolls to see the difference w/wo 'ne'). >From the discussion in the original report it seems, the reported xterm autowrap behaviour is DEC VT100 feature (for me historical bug that became feature) and could be deduced from the xenl terminfo capability. But as e.g. libvte interprets this different way, there is probably no simple fix (even in case grep would read terminfo). Maybe the xterm upstream will come with some idea Jaroslav From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: [debian-reportbug@plan9.de: Bug#734147: grep: colorisation corrupts character at end of line] References: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> In-Reply-To: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> Resent-From: Santiago Ruano =?UTF-8?Q?Rinc=C3=B3n?= Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Fri, 05 Dec 2014 08:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: 15444@debbugs.gnu.org Cc: 734147-submitter@bugs.debian.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.14177679707500 (code B ref 15444); Fri, 05 Dec 2014 08:27:01 +0000 Received: (at 15444) by debbugs.gnu.org; 5 Dec 2014 08:26:10 +0000 Received: from localhost ([127.0.0.1]:54246 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XwoDK-0001wt-2L for submit@debbugs.gnu.org; Fri, 05 Dec 2014 03:26:10 -0500 Received: from mx1.riseup.net ([198.252.153.129]:33263) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XwoDI-0001wk-3n for 15444@debbugs.gnu.org; Fri, 05 Dec 2014 03:26:09 -0500 Received: from berryeater.riseup.net (berryeater-pn.riseup.net [10.0.1.120]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.riseup.net (Postfix) with ESMTPS id AE95A4136B; Fri, 5 Dec 2014 08:26:05 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: santiagorr) with ESMTPSA id DDCB240DF3 Received: by nomada (sSMTP sendmail emulation); Fri, 05 Dec 2014 09:25:51 +0100 Date: Fri, 5 Dec 2014 09:25:51 +0100 From: Santiago Ruano =?UTF-8?Q?Rinc=C3=B3n?= Message-ID: <20141205082551.GA4657@nomada> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Virus-Scanned: clamav-milter 0.98.4 at mx1 X-Virus-Status: Clean X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Hi, Forwarding some user's thoughts about this issue. Hope this helps to solve this bug. Regards, Santiago ----- Forwarded message from Marc Lehmann ----- Date: Fri, 05 Dec 2014 06:01:33 +0100 From: Marc Lehmann , To: Debian Bug Tracking System <734147@bugs.debian.org>, Subject: Bug#734147: grep: colorisation corrupts character at end of line X-Mailer: reportbug 6.4.4 Package: grep Version: 2.21-1 Followup-For: Bug #734147 Dear Maintainer, seeing that the bug is still in grep, here are some thoughts: Foremost, this is a bug in grep, but it is also a bug in xterm (and rxvt), which claim to emulate vt102 behaviour, but both vt100 and vt102 behave like urxvt, namely space at the end of the line. At least, thats what the ROM image of either emulator does when run in a hardware simulator and fed with the example. As for grep, you can currently choose between a) data corruption and b) annoying background colour bars IFF the user configures it. The choice of "a) data corruption" over the alternatives is puzzling, as the default settings of grep do not change the background colour, so the "fix" (that corrupts the output) is entirely unnecessary when the goal is just to change the text colour. That is, setting "GREP_COLORS=ne" by default should always be safe unless the user configured a backgorund colour change, or I am missing something about the defaults. Even if background colours were the default, there are a multitude of workarounds that are all better than a) or b) above, for example, one could output every character twice, once with the default bg + backspace + character with coloured background. I don't know on which terminals this might break, but unless your terminal is one line high, this should work in vt10x emulators. Another alternative would be to force a scroll before changing attributes (move down/move up), which should give results visually indistinguishable from the correct solution under all normal usage patterns. I didn't think long about this problem, there might be even better solutions. So, in short, grep should simply change to sane defaults ("GREP_COLORS=ne", leaving the problem to the user who configures his/her own GREP_COLORS), or implement a suitable workaround (such as forcing a scroll with default attributes). Both of these would result in no data corruption. The first will result in (presumably) ugly background colour changes if the user configures that, the second should always work. Some experimentation might be in order, but surely grep can do better than just corrupt the search results. -- System Information: Debian Release: 7.5 APT prefers stable APT policy: (990, 'stable'), (500, 'unstable'), (500, 'testing'), (500, 'oldstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.16-3-amd64 (SMP w/12 CPU cores) Locale: LANG=C, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/bash Versions of packages grep depends on: ii dpkg 1.16.15 ii install-info 4.13a.dfsg.1-10 ii libc6 2.19-1 ii libpcre3 1:8.31-5 grep recommends no packages. grep suggests no packages. -- no debconf information ----- End forwarded message ----- From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: [debian-reportbug@plan9.de: Bug#734147: grep: colorisation corrupts character at end of line] Resent-From: Jim Meyering Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Fri, 05 Dec 2014 20:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: Santiago Ruano =?UTF-8?Q?Rinc=C3=B3n?= Cc: 15444@debbugs.gnu.org, 734147-submitter@bugs.debian.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.141781114326759 (code B ref 15444); Fri, 05 Dec 2014 20:26:01 +0000 Received: (at 15444) by debbugs.gnu.org; 5 Dec 2014 20:25:43 +0000 Received: from localhost ([127.0.0.1]:54997 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XwzRf-0006xW-BG for submit@debbugs.gnu.org; Fri, 05 Dec 2014 15:25:43 -0500 Received: from mail-yk0-f170.google.com ([209.85.160.170]:50945) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XwzRc-0006xO-Ex for 15444@debbugs.gnu.org; Fri, 05 Dec 2014 15:25:41 -0500 Received: by mail-yk0-f170.google.com with SMTP id q200so648912ykb.15 for <15444@debbugs.gnu.org>; Fri, 05 Dec 2014 12:25:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=4fI7ukys6cOdaKuqt/qdoN9gJVrbz7JG3So+bafYK+I=; b=p48OHYg+gO/5yg6r/2uR7pm4xN36IGlvZBRYHO49+j6gMySStXlFQNJUsNrk5/ZpUI gk7wtG/KISE2Dd+VDWE2e+YBVrTABtiOqty9/MUdIgOYa6mewsDbKpMEd1U9x6cS4zQP i/qkfOZzCn8kAvaqKcwYAYIo65BKunSfe8MfyD6ltdiOQrnid/0d4VXFh4uW/L7wfL8w 4ycLzcSjISa1dMrPvLMfheUGHyFc6FPil+4sjZQQpEfQoYcnO4i+icSXShr3FnFtPkTb WMkbevfRuUyBVmplnS2GVspWlPQ3oMDp2xAQJOgCjLP8Z6FRf377FosEUSUj6BbqqRdz HtVA== X-Received: by 10.170.160.137 with SMTP id b131mr17247528ykd.92.1417811139881; Fri, 05 Dec 2014 12:25:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.170.139.67 with HTTP; Fri, 5 Dec 2014 12:25:19 -0800 (PST) In-Reply-To: <20141205082551.GA4657@nomada> References: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> <20141205082551.GA4657@nomada> From: Jim Meyering Date: Fri, 5 Dec 2014 12:25:19 -0800 X-Google-Sender-Auth: 6msCIy2lJyYTPkeNKAynK38om-U Message-ID: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Fri, Dec 5, 2014 at 12:25 AM, Santiago Ruano Rinc=F3n wrote: > Hi, > > Forwarding some user's thoughts about this issue. Hope this helps to > solve this bug. > > Regards, > > Santiago > > ----- Forwarded message from Marc Lehmann ---= -- > > Date: Fri, 05 Dec 2014 06:01:33 +0100 > From: Marc Lehmann , > To: Debian Bug Tracking System <734147@bugs.debian.org>, > Subject: Bug#734147: grep: colorisation corrupts character at end of line > X-Mailer: reportbug 6.4.4 > > Package: grep > Version: 2.21-1 > Followup-For: Bug #734147 > > Dear Maintainer, > > seeing that the bug is still in grep, here are some thoughts: > > Foremost, this is a bug in grep, but it is also a bug in xterm (and rxvt)= , > which claim to emulate vt102 behaviour, but both vt100 and vt102 behave > like urxvt, namely space at the end of the line. At least, thats what the > ROM image of either emulator does when run in a hardware simulator and fe= d > with the example. > > As for grep, you can currently choose between a) data corruption and b) > annoying background colour bars IFF the user configures it. Thank you for forwarding that. However, note that there is another downside to making the proposed change, not just "annoying background color bars...". Currently, when grep matches a line containing a mix of \r and \t bytes, it does what most expect/desire (with the HT replacing each preexisting glyph with a space): $ printf 'asdfqwerzxcv\rASDF\tZXCV\n'|grep --color A ASDF ZXCV If we were to change grep so that it would no longer print those offending \e[K codes, that same command would no longer overwrite: $ printf 'asdfqwerzxcv\rASDF\tZXCV\n'|grep --color A ASDFqwerZXCV I think the underlying issue is that we want grep's --color option to be "safe", i.e., we want grep to stop conspiring with common terminal emulators to mangle certain output lines. Currently, it handles the \r...\t case at the expense of mangling any match that begins in the rightmost column of your terminal. The \r...\t issue is just one example of what can happen when a tool prints arbitrary control sequences to a terminal, so it feels like a red herring. The mangling of real matches can happen even with carefully sanitized input. That is more problematic. As such, I am leaning towards making the default be the same as what setting "GREP_COLORS=3Dne" does now. Opinions to the contrary? From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: (no subject) References: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> In-Reply-To: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> Resent-From: Leon Meier Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Wed, 13 Jul 2016 22:01:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: 15444@debbugs.gnu.org, 456943@bugs.debian.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.14684472352626 (code B ref 15444); Wed, 13 Jul 2016 22:01:02 +0000 Received: (at 15444) by debbugs.gnu.org; 13 Jul 2016 22:00:35 +0000 Received: from localhost ([127.0.0.1]:50251 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNSCo-0000gG-5q for submit@debbugs.gnu.org; Wed, 13 Jul 2016 18:00:34 -0400 Received: from forward19p.cmail.yandex.net ([77.88.31.22]:52600) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNRcv-0008E6-CZ for 15444@debbugs.gnu.org; Wed, 13 Jul 2016 17:23:30 -0400 Received: from smtp14.mail.yandex.net (smtp14.mail.yandex.net [IPv6:2a02:6b8:0:801:1::13]) by forward19p.cmail.yandex.net (Yandex) with ESMTP id AEA7421538; Thu, 14 Jul 2016 00:23:22 +0300 (MSK) Received: from smtp14.mail.yandex.net (localhost [127.0.0.1]) by smtp14.mail.yandex.net (Yandex) with ESMTP id AADE81B60365; Thu, 14 Jul 2016 00:23:22 +0300 (MSK) Received: by smtp14.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id 8tOdQLj8tv-NLd0tM0Z; Thu, 14 Jul 2016 00:23:22 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1468445002; bh=nXg8zBuH5qE61x+Wxuq6Ur+N4lgk/PyB5BROv9q6KHU=; h=To:From:Message-ID:Date; b=K1JZYkiXKLUz4FtKeMbz1fgaDJ+/+9Gj/CAhizCGi8SChsebJSs6GHkEXMK1Xwj/S cGxsrPLARaSaWJqtkf3dNR/GDB4f0I0oGPLNDwz9e/BOosla1Chp1XagnYQ+zmVOu6 EjC+fsgYag84tKdjHlt3zidaLiNTztMxt/RMjvM4= Authentication-Results: smtp14.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-Suid-Status: 1 0,1 0 From: Leon Meier Message-ID: Date: Wed, 13 Jul 2016 23:23:20 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: As of today, the test case echo " 1234" | grep --color=auto '[1-9]' (80 spaces and 1234) still fails in (u)xterm. Any resolution in sight? [...] Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [77.88.31.22 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (leon.meier[at]yandex.ru) 1.8 MISSING_SUBJECT Missing Subject: header 0.2 NO_SUBJECT Extra score for no subject 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Mailman-Approved-At: Wed, 13 Jul 2016 18:00:32 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: As of today, the test case echo " 1234" | grep --color=auto '[1-9]' (80 spaces and 1234) still fails in (u)xterm. Any resolution in sight? [...] Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [77.88.31.22 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (leon.meier[at]yandex.ru) 1.8 MISSING_SUBJECT Missing Subject: header 0.2 NO_SUBJECT Extra score for no subject 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid As of today, the test case echo " 1234" | grep --color=auto '[1-9]' (80 spaces and 1234) still fails in (u)xterm. Any resolution in sight? (Sorry for not providing any opinion of _how_ to resolve it technically at the moment; I'm not good enough for that. It is just the bug has been kind of disturbing me for a few years already...) From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: (no subject) Resent-From: Paul Jackson Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Wed, 13 Jul 2016 22:38:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: 15444@debbugs.gnu.org X-Debbugs-Original-To: bug-grep@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.14684494585994 (code B ref -1); Wed, 13 Jul 2016 22:38:01 +0000 Received: (at submit) by debbugs.gnu.org; 13 Jul 2016 22:37:38 +0000 Received: from localhost ([127.0.0.1]:50276 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNSmg-0001Yc-MD for submit@debbugs.gnu.org; Wed, 13 Jul 2016 18:37:38 -0400 Received: from eggs.gnu.org ([208.118.235.92]:59048) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNSme-0001YO-W1 for submit@debbugs.gnu.org; Wed, 13 Jul 2016 18:37:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bNSmZ-00037l-2Z for submit@debbugs.gnu.org; Wed, 13 Jul 2016 18:37:31 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:38705) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bNSmY-00037g-Sm for submit@debbugs.gnu.org; Wed, 13 Jul 2016 18:37:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50005) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bNSmW-0005mP-KF for bug-grep@gnu.org; Wed, 13 Jul 2016 18:37:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bNSmS-00036y-Dh for bug-grep@gnu.org; Wed, 13 Jul 2016 18:37:27 -0400 Received: from out5-smtp.messagingengine.com ([66.111.4.29]:39568) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bNSmR-00036i-27 for bug-grep@gnu.org; Wed, 13 Jul 2016 18:37:24 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 1BF5A20850 for ; Wed, 13 Jul 2016 18:37:19 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute3.internal (MEProxy); Wed, 13 Jul 2016 18:37:19 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=fyc9qa0Ylsz4rmu w9NegzCznF8U=; b=KLQ0U1meB5VJGZkOH85+xLNCEOcCG7b6JsH+1nXChNw+GPC Bru49GMGxBRt1SUx7dxpp6ITYensROwYkFZg6TR1GOexur3+eKyfXxMjbvxUy8Ri aBjHjtg3BmDmVc8VH+nV3rA0bhsEdQ9XT7ob46fG7Xvr+bozc6UToKoQtujw= Received: by mailuser.nyi.internal (Postfix, from userid 99) id E458016711; Wed, 13 Jul 2016 18:37:18 -0400 (EDT) Message-Id: <1468449438.3221809.665600449.4C8FDC5A@webmail.messagingengine.com> X-Sasl-Enc: ZXjEz73/ES0/GY3xrLDKeaT93A6BYuBQWZomd4on861q 1468449438 From: Paul Jackson MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-bf4e2c8f Date: Wed, 13 Jul 2016 17:37:18 -0500 In-Reply-To: References: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -2.4 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Leon wrote: >> As of today, the test case >> >> echo " >> 1234" | grep --color=auto '[1-9]' >> (80 spaces and 1234) >> >> still fails in (u)xterm. Any resolution in sight? Does the following also fail for you? It uses python instead of (difficult to see in email) explicit spaces to get the 80 spaces. python2 -c 'print " " * 80, 1234' | grep --color=auto '[1-9]' However they both appear to fail in uxterm, IF I have the terminal set to 80 columns wide. If the terminal is wider, then I can see the red 1234 out in "right field". For example, if the terminal is 84 chars wide, I can see the red "12", but not the "34", unless the terminal is at least 86 chars wide. If I invoke uxterm with the -cm option, to disable ANSI color sequences, then the extra 2 bytes aren't needed, and a terminal that's 84 chars wide will show all, uncolored, four "1234" numbers. In xterm, I need a terminal 82 chars wide (not 84 like uxterm) to see the red "12", and I need a terminal of 84 chars wide (not 86 like uxterm) to see all four red numbers "1234". I did not double check the perfection of the above details, so might be off by one or two, here or there. But I'd guess that this might be a "bug" (unexpected feature) in the terminal emulator's handling of ANSI color escape sequences, rather than a bug in grep. -- Paul Jackson pj@usa.net From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: Bug#456943: (no subject) Resent-From: "Walter Doekes" Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Thu, 14 Jul 2016 15:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: 456943@bugs.debian.org Cc: 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.146851067615859 (code B ref 15444); Thu, 14 Jul 2016 15:38:02 +0000 Received: (at 15444) by debbugs.gnu.org; 14 Jul 2016 15:37:56 +0000 Received: from localhost ([127.0.0.1]:51610 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNii3-00047i-Vj for submit@debbugs.gnu.org; Thu, 14 Jul 2016 11:37:56 -0400 Received: from wjd.osso.nl ([217.21.198.142]:44815 helo=wjdsys.wjd.nu) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNafc-0003lO-PU for 15444@debbugs.gnu.org; Thu, 14 Jul 2016 03:02:54 -0400 Received: by wjdsys.wjd.nu (Postfix, from userid 33) id 8B6A29611C; Thu, 14 Jul 2016 09:02:48 +0200 (CEST) Received: from 91.194.225.4 (SquirrelMail authenticated user walter@wjd.nu) by mail.wjd.nu with HTTP; Thu, 14 Jul 2016 09:02:48 +0200 Message-ID: In-Reply-To: References: Date: Thu, 14 Jul 2016 09:02:48 +0200 From: "Walter Doekes" User-Agent: SquirrelMail/1.4.23 [SVN] MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Leon Meier wrote: > As of today, the test case [...] still fails in (u)xterm. > Any resolution in sight? I tried to reproduce, and indeed, it fails on xterm (without the 'ne' grep option), but not in gnome-terminal. [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 SLIGHTLY_BAD_SUBJECT Subject contains something slightly spammy -0.0 SPF_PASS SPF: sender matches SPF record X-Mailman-Approved-At: Thu, 14 Jul 2016 11:37:55 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Leon Meier wrote: > As of today, the test case [...] still fails in (u)xterm. > Any resolution in sight? I tried to reproduce, and indeed, it fails on xterm (without the 'ne' grep option), but not in gnome-terminal. [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 SLIGHTLY_BAD_SUBJECT Subject contains something slightly spammy -0.0 SPF_PASS SPF: sender matches SPF record Leon Meier wrote: > As of today, the test case [...] still fails in (u)xterm. > Any resolution in sight? I tried to reproduce, and indeed, it fails on xterm (without the 'ne' grep option), but not in gnome-terminal. Does that mean that this is an xterm bug again and not a grep bug? See this: http://wjd.nu/files/2016/07/debian-bug-456943.png?view This works both in gnome-terminal and xterm: python -c 'print(" "*'$COLUMNS'+"1234")' | GREP_COLORS=ne grep --color=auto '[1-9]' This works in gnome-terminal and *fails* in xterm: python -c 'print(" "*'$COLUMNS'+"1234")' | grep --color=auto '[1-9]' Versions (Ubuntu, Trusty, latest): gnome-terminal 3.6.2-0ubuntu1 grep 2.16-1 xterm 297-1ubuntu1 Greetings, Walter Doekes From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: (no subject) References: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> In-Reply-To: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> Resent-From: Leon Meier Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Thu, 14 Jul 2016 15:40:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.146851074516038 (code B ref 15444); Thu, 14 Jul 2016 15:40:02 +0000 Received: (at 15444) by debbugs.gnu.org; 14 Jul 2016 15:39:05 +0000 Received: from localhost ([127.0.0.1]:51619 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNijB-0004Ab-8s for submit@debbugs.gnu.org; Thu, 14 Jul 2016 11:39:05 -0400 Received: from forward11h.cmail.yandex.net ([87.250.230.153]:41462) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNctF-0000eb-U7 for 15444@debbugs.gnu.org; Thu, 14 Jul 2016 05:25:06 -0400 Received: from smtp13.mail.yandex.net (smtp13.mail.yandex.net [95.108.130.68]) by forward11h.cmail.yandex.net (Yandex) with ESMTP id B03EA2160D for <15444@debbugs.gnu.org>; Thu, 14 Jul 2016 12:24:59 +0300 (MSK) Received: from smtp13.mail.yandex.net (localhost [127.0.0.1]) by smtp13.mail.yandex.net (Yandex) with ESMTP id ADC9CE40439 for <15444@debbugs.gnu.org>; Thu, 14 Jul 2016 12:24:59 +0300 (MSK) Received: by smtp13.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id D79Z8d0elh-Ow2eH6Mw; Thu, 14 Jul 2016 12:24:59 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1468488299; bh=sZz5ggjaee3FnKyPEF+ykfX4KlCIX4GY8qZBMAR3iew=; h=From:To:Message-ID:Date; b=qib7g848RCabUoGby2PE5ayR7OylJ4ZjCtEu0eYo1zkypJZHn7JwnSIr1KZfIoZ7f L15HJ/z1Uj3R91rIOEiDloLZFof5A1DjVztyMnGX6QL3sjQQB8Kz3/1K6V4bnKBPOx NXKYhZRD7VWKy09DKx9GC7o5JkGP8cZErSa00BbE= Authentication-Results: smtp13.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-Suid-Status: 1 0 From: Leon Meier Message-ID: <7f63c635-f0d7-a842-e8c7-ad317bb1a22c@yandex.ru> Date: Thu, 14 Jul 2016 11:24:58 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: > Does the following also fail for you? It uses python instead of > (difficult to see in email) explicit spaces to get the 80 spaces. Sorry; in my previous e-mail there was a wraparound induced by the e-mail composer. This one is hopefully better: $ echo " "| grep --color=always '[1-9]' [...] Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [87.250.230.153 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [87.250.230.153 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (leon.meier[at]yandex.ru) 1.8 MISSING_SUBJECT Missing Subject: header 0.2 NO_SUBJECT Extra score for no subject -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Mailman-Approved-At: Thu, 14 Jul 2016 11:39:04 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: > Does the following also fail for you? It uses python instead of > (difficult to see in email) explicit spaces to get the 80 spaces. Sorry; in my previous e-mail there was a wraparound induced by the e-mail composer. This one is hopefully better: $ echo " "| grep --color=always '[1-9]' [...] Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [87.250.230.153 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [87.250.230.153 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (leon.meier[at]yandex.ru) 1.8 MISSING_SUBJECT Missing Subject: header 0.2 NO_SUBJECT Extra score for no subject -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid > Does the following also fail for you? It uses python instead of > (difficult to see in email) explicit spaces to get the 80 spaces. Sorry; in my previous e-mail there was a wraparound induced by the e-mail composer. This one is hopefully better: $ echo " "| grep --color=always '[1-9]' > python2 -c 'print " " * 80, 1234' | grep --color=auto '[1-9]' This one does work for a standard 80-char-wide (u)xterm. From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: Bug#456943: (no subject) Resent-From: Vincent Lefevre Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Thu, 14 Jul 2016 21:23:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: Walter Doekes , 456943@bugs.debian.org Cc: 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.146853134427508 (code B ref 15444); Thu, 14 Jul 2016 21:23:01 +0000 Received: (at 15444) by debbugs.gnu.org; 14 Jul 2016 21:22:24 +0000 Received: from localhost ([127.0.0.1]:51821 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNo5P-00079c-Q3 for submit@debbugs.gnu.org; Thu, 14 Jul 2016 17:22:23 -0400 Received: from ioooi.vinc17.net ([92.243.22.117]:46477) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNo5M-00079R-6f for 15444@debbugs.gnu.org; Thu, 14 Jul 2016 17:22:21 -0400 Received: from smtp-zira.vinc17.net (unknown [216.9.110.15]) by ioooi.vinc17.net (Postfix) with ESMTPSA id EFD64603; Thu, 14 Jul 2016 23:22:17 +0200 (CEST) Received: by zira.vinc17.org (Postfix, from userid 1000) id BF0A9C245BD; Thu, 14 Jul 2016 14:22:15 -0700 (PDT) Date: Thu, 14 Jul 2016 14:22:15 -0700 From: Vincent Lefevre Message-ID: <20160714212215.GB5305@zira.vinc17.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Mailer-Info: https://www.vinc17.net/mutt/ User-Agent: Mutt/1.6.2-6722-vl-r89564 (2016-07-11) X-Spam-Score: 0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.7 (/) On 2016-07-14 09:02:48 +0200, Walter Doekes wrote: > Leon Meier wrote: > > As of today, the test case [...] still fails in (u)xterm. > > Any resolution in sight? > > I tried to reproduce, and indeed, it fails on xterm (without the 'ne' grep > option), but not in gnome-terminal. > > Does that mean that this is an xterm bug again and not a grep bug? It is GNOME Terminal that is buggy, so that the grep bug is not visible. -- Vincent Lefèvre - Web: 100% accessible validated (X)HTML - Blog: Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: (no subject) Resent-From: Paul Jackson Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Fri, 15 Jul 2016 05:19:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: 15444@debbugs.gnu.org X-Debbugs-Original-To: bug-grep@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.146855989428195 (code B ref -1); Fri, 15 Jul 2016 05:19:01 +0000 Received: (at submit) by debbugs.gnu.org; 15 Jul 2016 05:18:14 +0000 Received: from localhost ([127.0.0.1]:51940 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNvVt-0007Kh-OU for submit@debbugs.gnu.org; Fri, 15 Jul 2016 01:18:13 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56964) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bNvVr-0007KR-9I for submit@debbugs.gnu.org; Fri, 15 Jul 2016 01:18:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bNvVl-0005rQ-6i for submit@debbugs.gnu.org; Fri, 15 Jul 2016 01:18:05 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:54685) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bNvVl-0005rK-0f for submit@debbugs.gnu.org; Fri, 15 Jul 2016 01:18:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47924) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bNvVi-0000fW-VE for bug-grep@gnu.org; Fri, 15 Jul 2016 01:18:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bNvVe-0005qo-O1 for bug-grep@gnu.org; Fri, 15 Jul 2016 01:18:01 -0400 Received: from out5-smtp.messagingengine.com ([66.111.4.29]:48061) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bNvVd-0005qb-Hx for bug-grep@gnu.org; Fri, 15 Jul 2016 01:17:58 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id E6AE620471 for ; Fri, 15 Jul 2016 01:17:52 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute6.internal (MEProxy); Fri, 15 Jul 2016 01:17:52 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=7GeROkdY0jj5Ip3 4wug2PxkXTRo=; b=icycKESmWMLhf/YKFUa7UgaEdjUdFzp+ETd5Bn7Go2lOnt6 7ROKKgyf2sSIXtx0N4ZzeBpQw8+raZXJmZqLP0FJQ1WvEtHlQbd8c/BVh+g6NPtT UWOaZqAO11qUhyIWgcj4KSx1bom+2HYdM9gdq/8dVC3f0C/PQko2meVCiyeM= Received: by mailuser.nyi.internal (Postfix, from userid 99) id B8383161A9; Fri, 15 Jul 2016 01:17:52 -0400 (EDT) Message-Id: <1468559872.2959316.666910489.33F75EBF@webmail.messagingengine.com> X-Sasl-Enc: nkMq/KVkbkHuGYVdFfs9iiuTcwT9J8+NNWg1zLSGFHq3 1468559872 From: Paul Jackson MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-bf4e2c8f Date: Fri, 15 Jul 2016 00:17:52 -0500 In-Reply-To: <7f63c635-f0d7-a842-e8c7-ad317bb1a22c@yandex.ru> References: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> <7f63c635-f0d7-a842-e8c7-ad317bb1a22c@yandex.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Leon wrote: >> Sorry; in my previous e-mail there was a wraparound induced by the e-mail composer. This one is hopefully better: >> $ echo " "| grep --color=always '[1-9]' >> >> python2 -c 'print " " * 80, 1234' | grep --color=auto '[1-9]' >> >> This one does work for a standard 80-char-wide (u)xterm. The above two commands are not the same. That echo command seems to emit 79 spaces, followed by one newline (but no numbers at all ??) That (my) python2 command emits 81 spaces, the four numbers, and a newline. Try this one (fixing my off-by-one space count, as the ',' comma in print args emits a space of its own): python2 -c 'print " " * 79, 1234' | grep --color=always '[1-9]' And try this one (fixing what I guess is your accidental omission of one space and the four numbers): echo " 1234"| grep --color=always '[1-9]' I get the same results from either of the last two test cases above, with the red "1234" numbers NOT appearing in an 80 wide xterm or uxterm, but appearing ok if I widen the terminal. -- Paul Jackson pj@usa.net From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: Bug#456943: (no subject) Resent-From: Vincent Lefevre Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Mon, 25 Mar 2024 15:33:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: Walter Doekes , 456943@bugs.debian.org Cc: 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.1711380738831 (code B ref 15444); Mon, 25 Mar 2024 15:33:01 +0000 Received: (at 15444) by debbugs.gnu.org; 25 Mar 2024 15:32:18 +0000 Received: from localhost ([127.0.0.1]:35711 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1romJ7-0000DL-K7 for submit@debbugs.gnu.org; Mon, 25 Mar 2024 11:32:17 -0400 Received: from cventin.lip.ens-lyon.fr ([140.77.13.17]:47756) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1romJ4-0000D5-WB for 15444@debbugs.gnu.org; Mon, 25 Mar 2024 11:32:15 -0400 Received: from vlefevre by cventin.lip.ens-lyon.fr with local (Exim 4.97) (envelope-from ) id 1role4-00000000462-2BNO; Mon, 25 Mar 2024 15:49:52 +0100 Date: Mon, 25 Mar 2024 15:49:52 +0100 From: Vincent Lefevre Message-ID: <20240325144952.GA2199@cventin.lip.ens-lyon.fr> References: <20160714212215.GB5305@zira.vinc17.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20160714212215.GB5305@zira.vinc17.org> X-Mailer-Info: https://www.vinc17.net/mutt/ User-Agent: Mutt/2.2.12+69 (354c5b11) vl-149028 (2023-12-10) X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On 2016-07-14 14:22:15 -0700, Vincent Lefevre wrote: > On 2016-07-14 09:02:48 +0200, Walter Doekes wrote: > > Leon Meier wrote: > > > As of today, the test case [...] still fails in (u)xterm. > > > An [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 SLIGHTLY_BAD_SUBJECT Subject contains something slightly spammy 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) On 2016-07-14 14:22:15 -0700, Vincent Lefevre wrote: > On 2016-07-14 09:02:48 +0200, Walter Doekes wrote: > > Leon Meier wrote: > > > As of today, the test case [...] still fails in (u)xterm. > > > Any resolution in sight? > > > > I tried to reproduce, and indeed, it fails on xterm (without the 'ne' grep > > option), but not in gnome-terminal. > > > > Does that mean that this is an xterm bug again and not a grep bug? > > It is GNOME Terminal that is buggy, so that the grep bug is not > visible. A solution for "grep" would be to add a space+backspace before the escape sequence. Testcase: for i in `seq 5` ; do printf "%0$(($(tput cols)+i-5))dab \bc\n" ; done | \ GREP_COLORS="mt=41;97:ne" grep --color c This works fine in Xterm, giving on a 80-column terminal: 0000000000000000000000000000000000000000000000000000000000000000000000000000abc 00000000000000000000000000000000000000000000000000000000000000000000000000000abc 000000000000000000000000000000000000000000000000000000000000000000000000000000abc 0000000000000000000000000000000000000000000000000000000000000000000000000000000abc 00000000000000000000000000000000000000000000000000000000000000000000000000000000abc where only the "c" has the red background. However, this triggers the bug in GNOME Terminal (and other libvte-based terminals): 0000000000000000000000000000000000000000000000000000000000000000000000000000abc 00000000000000000000000000000000000000000000000000000000000000000000000000000ac 000000000000000000000000000000000000000000000000000000000000000000000000000000abc 0000000000000000000000000000000000000000000000000000000000000000000000000000000abc 00000000000000000000000000000000000000000000000000000000000000000000000000000000abc -- Vincent Lefèvre - Web: 100% accessible validated (X)HTML - Blog: Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: Bug#456943: (no subject) Resent-From: Vincent Lefevre Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Mon, 25 Mar 2024 17:03:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: 456943@bugs.debian.org Cc: 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.171138617721945 (code B ref 15444); Mon, 25 Mar 2024 17:03:02 +0000 Received: (at 15444) by debbugs.gnu.org; 25 Mar 2024 17:02:57 +0000 Received: from localhost ([127.0.0.1]:36069 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ronir-0005ht-G5 for submit@debbugs.gnu.org; Mon, 25 Mar 2024 13:02:57 -0400 Received: from cventin.lip.ens-lyon.fr ([140.77.13.17]:57224) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ronip-0005hk-CG for 15444@debbugs.gnu.org; Mon, 25 Mar 2024 13:02:56 -0400 Received: from vlefevre by cventin.lip.ens-lyon.fr with local (Exim 4.97) (envelope-from ) id 1ronio-000000008xs-1R83; Mon, 25 Mar 2024 18:02:54 +0100 Date: Mon, 25 Mar 2024 18:02:54 +0100 From: Vincent Lefevre Message-ID: <20240325170254.GA34301@cventin.lip.ens-lyon.fr> References: <20160714212215.GB5305@zira.vinc17.org> <20240325144952.GA2199@cventin.lip.ens-lyon.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20240325144952.GA2199@cventin.lip.ens-lyon.fr> X-Mailer-Info: https://www.vinc17.net/mutt/ User-Agent: Mutt/2.2.12+69 (354c5b11) vl-149028 (2023-12-10) X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On 2024-03-25 15:49:52 +0100, Vincent Lefevre wrote: > A solution for "grep" would be to add a space+backspace before the > escape sequence. An additional note: One of the following is needed: * Detect the end of line (this may be tricky) and split the coloring into 2 parts (each one with space + backspace + escape sequence) at this point. If this detection is incorrect, then the background [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 SLIGHTLY_BAD_SUBJECT Subject contains something slightly spammy 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) On 2024-03-25 15:49:52 +0100, Vincent Lefevre wrote: > A solution for "grep" would be to add a space+backspace before the > escape sequence. An additional note: One of the following is needed: * Detect the end of line (this may be tricky) and split the coloring into 2 parts (each one with space + backspace + escape sequence) at this point. If this detection is incorrect, then the background problem would still be visible, but with no major drawbacks. * Split the coloring so that each matching character gets a space + backspace + escape sequence. I think that this is acceptable since this trick should be used only when the output is a tty (i.e. it is not redirected to a file / not piped to a pager or something else). -- Vincent Lefèvre - Web: 100% accessible validated (X)HTML - Blog: Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: One character can be lost if colors are enabled Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Tue, 26 Mar 2024 17:48:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: Vincent Lefevre , Walter Doekes , 456943@bugs.debian.org Cc: 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.17114752589171 (code B ref 15444); Tue, 26 Mar 2024 17:48:02 +0000 Received: (at 15444) by debbugs.gnu.org; 26 Mar 2024 17:47:38 +0000 Received: from localhost ([127.0.0.1]:35002 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpAte-0002No-3z for submit@debbugs.gnu.org; Tue, 26 Mar 2024 13:47:38 -0400 Received: from mail.cs.ucla.edu ([131.179.128.66]:43226) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpAta-0002N5-A4 for 15444@debbugs.gnu.org; Tue, 26 Mar 2024 13:47:37 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 7CDC73C00E402; Tue, 26 Mar 2024 10:47:28 -0700 (PDT) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavis, port 10032) with ESMTP id 5_wphn-l84xB; Tue, 26 Mar 2024 10:47:28 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 34C203C00D408; Tue, 26 Mar 2024 10:47:28 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu 34C203C00D408 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1711475248; bh=GZQYivC8lgKHRkVIDaoCv9EhpVZKIE/LTkbr602W4eI=; h=Message-ID:Date:MIME-Version:To:From; b=fVgoAqkM5nRsfmqxexGmrNJR+RJmTImp+rJcj8Z9Ea+I80k2yodHuuin77/clJQCq o/TZeWWepvfnwp0wXM/20KnrWfllAADgYHPFkxDnTZDJQPlPkUTDDtIonsgOqIc/pz ZGB6gq0/WlTbNeqHh1Lwj2hy0nrTT3/YtuXEn044qmmTAKoeUuYO1ELfmJxu8H8kJg Kc/6d3XzrVScvOcyiJkx4TKvAF8nWqYunQy/OBogldWcssMJvUad5IriDkuURztaMu CuHuCc42YtU2gfJFBizzgfMJLaU/5c40i4iqEhkYDj7hbZB+w0ueR/cvmu2sD9BKDY UaUAw91Kb6lAg== X-Virus-Scanned: amavis at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavis, port 10026) with ESMTP id J6TNWrvf4dft; Tue, 26 Mar 2024 10:47:28 -0700 (PDT) Received: from [10.10.33.175] (unknown [96.69.135.29]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id C7D6C3C00E402; Tue, 26 Mar 2024 10:47:27 -0700 (PDT) Message-ID: Date: Tue, 26 Mar 2024 11:47:26 -0600 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird References: <20160714212215.GB5305@zira.vinc17.org> <20240325144952.GA2199@cventin.lip.ens-lyon.fr> Content-Language: en-US From: Paul Eggert In-Reply-To: <20240325144952.GA2199@cventin.lip.ens-lyon.fr> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 3/25/24 08:49, Vincent Lefevre wrote: > This works fine in Xterm, giving on a 80-column terminal: > > ... > > However, this triggers the bug in GNOME Terminal (and other > libvte-based terminals): That's not good. Is there some escape sequence that will work on both xterm and libvte? I assume the space+backspace trick you suggested later assumes xterm behavior. From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: One character can be lost if colors are enabled Resent-From: Vincent Lefevre Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Tue, 26 Mar 2024 23:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: Paul Eggert Cc: 15444@debbugs.gnu.org, 456943@bugs.debian.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.17114970122807 (code B ref 15444); Tue, 26 Mar 2024 23:51:02 +0000 Received: (at 15444) by debbugs.gnu.org; 26 Mar 2024 23:50:12 +0000 Received: from localhost ([127.0.0.1]:35435 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpGYV-0000j0-Bc for submit@debbugs.gnu.org; Tue, 26 Mar 2024 19:50:12 -0400 Received: from joooj.vinc17.net ([2001:4b99:1:3:216:3eff:fe20:ac98]:47882) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpGYS-0000ib-O5 for 15444@debbugs.gnu.org; Tue, 26 Mar 2024 19:50:10 -0400 Received: from smtp-qaa.vinc17.net (135.197.67.86.rev.sfr.net [86.67.197.135]) by joooj.vinc17.net (Postfix) with ESMTPSA id D7B10888; Wed, 27 Mar 2024 00:50:05 +0100 (CET) Received: by qaa.vinc17.org (Postfix, from userid 1000) id 9A897CA00B3; Wed, 27 Mar 2024 00:50:05 +0100 (CET) Date: Wed, 27 Mar 2024 00:50:05 +0100 From: Vincent Lefevre Message-ID: <20240326235005.GG2636@qaa.vinc17.org> References: <20160714212215.GB5305@zira.vinc17.org> <20240325144952.GA2199@cventin.lip.ens-lyon.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Mailer-Info: https://www.vinc17.net/mutt/ User-Agent: Mutt/2.2.12+69 (354c5b11) vl-149028 (2023-12-10) X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 2024-03-26 11:47:26 -0600, Paul Eggert wrote: > On 3/25/24 08:49, Vincent Lefevre wrote: > > This works fine in Xterm, giving on a 80-column terminal: > > > > ... > > However, this triggers the bug in GNOME Terminal (and other > > libvte-based terminals): > > That's not good. Is there some escape sequence that will work on > both xterm and libvte? I assume the space+backspace trick you > suggested later assumes xterm behavior. I've eventually found that this works in xterm only because I have the reverseWrap option set (a setting I've had since at least June 1996... so that I forgot about it). In the past, I think that I set it in order to be able to go backward to the previous line in cooked mode. But this is not the default behavior in xterm (at least nowadays). It could still be nice to have this trick in grep as an *option* for GREP_COLORS (just like one already has "ne"). But perhaps the best solution would be to make terminals implement new escape sequences to enable/disable the use of the default background color (instead of the current background color, which is the current behavior and the cause of this bug) for the new line when scrolling, or something similar. -- Vincent Lefèvre - Web: 100% accessible validated (X)HTML - Blog: Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: One character can be lost if colors are enabled References: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> In-Reply-To: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> Resent-From: Egmont Koblinger Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Wed, 27 Mar 2024 08:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.17115273459531 (code B ref 15444); Wed, 27 Mar 2024 08:16:02 +0000 Received: (at 15444) by debbugs.gnu.org; 27 Mar 2024 08:15:45 +0000 Received: from localhost ([127.0.0.1]:35699 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpORl-0002Te-62 for submit@debbugs.gnu.org; Wed, 27 Mar 2024 04:15:45 -0400 Received: from mail-yb1-xb2e.google.com ([2607:f8b0:4864:20::b2e]:58742) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpORi-0002Sp-Po for 15444@debbugs.gnu.org; Wed, 27 Mar 2024 04:15:43 -0400 Received: by mail-yb1-xb2e.google.com with SMTP id 3f1490d57ef6-dcbef31a9dbso4660175276.1 for <15444@debbugs.gnu.org>; Wed, 27 Mar 2024 01:15:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711527336; x=1712132136; darn=debbugs.gnu.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=t3L5X4p+6+tEP8LR6SXFlZVPkfIauaWS3l2b2nuviNY=; b=E/XGfkrNm3594lMj7C/VgI2exmSh9ioN5XdVz5nNqG8tfG9XLSeujNO52Yu4zu6c1X wPar7/bE5e8zg7J3zVWgatiN018jl7GDY08OVC4VLQtpqCj7Lfvvlm6GcXJI8S6xglq6 Td+rLrp1NfJws4y2DRT1i4gCeeQ6QFG5+++rLFXPnExeyRODj8GfN7R7gv2wJjLnNQ1z C41eeknykgNkPqv2QWWEnyws3hh+ExMRlglhmLcJ6BWkTGdYvMygr79/r+8mgcm29RKc b/lBC4R3zLPk2NEi1Lvc85cX/j343nBpL2AlpaBzMrlieIg0D7iKec03kMQo6Ta5maUz i9Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711527336; x=1712132136; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=t3L5X4p+6+tEP8LR6SXFlZVPkfIauaWS3l2b2nuviNY=; b=PLdBxBtEQB3CuI9GUd3HXhu2ag0LqKd503HGOfLyusJsR/HbLHwnptAmeh6ohTiKDQ fprtOOnMlOuSlPoScXnI2WyaY7tFKv/n3Bxy0JURbtlfG6gU658XPgZpzIDUx5Tp+oyN DgZEdxSJLJdKgBtfDJlUJ2EAFGBytWZzW1QZFgCjLOkqxKCXcsvh8hfh4nizO/Lw+3lh XCAOwASMOzGeafsx0Txf9nK3qPm1ZF2j8uPB7btZWhN6lyAZGnusKc0qY4AwxB5q4y2h VxFUMzK8qJMngXAjeN0gimutTZeKLxz4UnfVY90tf1RIk+SWWClxbmGNea7gCu2EJ1uV bvyQ== X-Gm-Message-State: AOJu0Yw+JmEkXqDjyQXR+aMRST6fsm0Xpyk8d0UsOzI4u3ulXdw/T4Fn TcwX0M2F/Unw0pi9Varu8/CUoVt7q09L4btLrYw6ZUbWhG1SaH/QMlP+0HkzeH1MtM9Aa2njiS7 39SVAVs1eOclkKQqT7YCchPYPQLkK/4CSveY= X-Google-Smtp-Source: AGHT+IHmI3UWXnHKoMN5o1Ws3UA+03RbsxllBqZ8TQ3bZYI6dzVfrz1My2DdGGIgz3z9FL8WKykdFXUQa5YfhygtlJo= X-Received: by 2002:a25:b03:0:b0:dc6:9d35:f9aa with SMTP id 3-20020a250b03000000b00dc69d35f9aamr357666ybl.19.1711527336512; Wed, 27 Mar 2024 01:15:36 -0700 (PDT) MIME-Version: 1.0 From: Egmont Koblinger Date: Wed, 27 Mar 2024 09:15:00 +0100 Message-ID: Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, VTE co-developer here. (VTE, a.k.a. libvte is the terminal emulation widget behind GNOME Terminal and several other terminal emulator apps.) Note that this very bug is also filed at https://savannah.gnu.org/bugs/?36831 . --- The "background color erase" ("bce") feature of most terminals suffers from a fundamental design flaw. You cannot "just print" some piece of text containing non-default background color, and expect it to appear on the screen correctly. By "just printing" what I mean is that you don't go into the business of querying the terminal size, tracking the cursor position, knowing which characters will be double wide, computing where the line would wrap to the next one, etc. I'm only considering simple apps that print messages using printf() or similar, like grep does. The problem is that if you're at the bottom of the screen and autowrapping occurs, the entire new line is added with the currently active background color, which might not be the terminal's default background color. If that new line isn't filled with text entirely then the rest remains with that color. The workaround applied by "grep" by default is to switch back to the default background color and emit the sequence that clears to the end of the line. This, while fixes the background color issue (apart from a possible temporary flicker if the terminal doesn't process grep's output in a single step and updates its display in the middle), introduces a different bug: possibly chopping off one letter of output. For the sake of explaining it, assume 80 columns numbered from 1 to 80. The way terminals behave is: After printing 78 chars the cursor is in column 79. After printing one more char the cursor is in column 80. After printing the 80th char in the 80th column, the cursor remains in column 80 (over this newly printed character) (*), but a special flag is set, in Xterm's source I believe it's called "do_wrap", it denotes that the next character will wrap to the next line. ((*) VTE hides the cursor in this case, but it's irrelevant for this bug.) Some operations, like handling a backspace, or clearing to the end of the line unfortunately don't take this "do_wrap" flag into account. That is, after printing 80 characters and being in the "do_wrap" state, the "clear to end of line" escape sequence only checks that the cursor is in column 80, therefore clears everything in column 80 and its right (i.e. column 80 only). And it also clears the "do_wrap" flag. Perhaps about 8-10 years ago I talked about this conceptual problem with the developer of xterm and ncurses, and he wasn't interested in doing anything about the situation. I, however, still was. --- In VTE I made two modifications, i.e. at two places we intentionally deviate from the standard behavior to fix this nonsense and make grep work as expected. One of the workarounds is for when grep is asked to emit that clear-to-eol, one is for when it isn't. In https://bugzilla.gnome.org/show_bug.cgi?id=740789 (2014) I changed the behavior of the clear-to-eol escape sequence. If in that "do_wrap" state then it doesn't clear the last character, nor this "do_wrap" state, it becomes no-op. In https://bugzilla.gnome.org/show_bug.cgi?id=754596 (2015-2017) I modified the bce feature to fill the newly appearing line with the current background only if scrolling occurs due to an explicit newline or escape sequence, but not when it occurs to text autowrapping, in which case we use the terminal's default background color. Neither of these changes cause any problem in real life that we're aware of. The first iteration of the bce change in 2015 had some problem that I fixed in 2017. We haven't received any bugreport since then about either of these intentional deviations from the standard. The result is that grep's output renders correctly in VTE, no matter if you use its "ne" feature or not. However, the goal should be to fix it in all terminals out there. --- So, what should "grep" do? Its maintainers, or anyone interested in pushing this through (it's not going to be me, sorry) could talk to Xterm's and other terminals' authors to change the default, or find a solution. They could point out that these two workarounds in VTE have turned out to be fine, and suggest to change the default behavior to these. (However, I wouldn't expect everyone to agree. I'd expect that many of these folks would insist on strictly adhering to the standards, rather than acknowledging that the standard is conceptually broken and being okay with deviating from it.) As for clear-to-eol, I think urxvt kept the official behavior, but added the same sequence with numeric parameter 3, i.e. \e[3K, to perform the slightly modified (fixed) behavior. Could be a reasonable route to go down. As for the immediate future, i.e. assuming that grep maintainers don't want to or don't succeed in driving such a change across leading / reference terminals (probably most notably Xterm) and then wait for years for this to become widespread, the question is: Would you rather have a wrong background color, or a letter missing? I'm puzzled how upstream grep, for many-many years, has chosen to possibly chop off a letter as its default behavior, I firmly believe that it's by far the worse of the two. Or start thinking about brand new workarounds. Like Vincent's idea with a space + backspace. Or space + backspace + clear-to-eol. Or space + clear-to-eol. These _could_ be robust against issues at the right margin. Obviously these brainstorming ideas I'm randomly throwing in now have to be tested carefully across many popular terminals. Maybe with both possible values of Xterm's reverse wraparound, maybe with only the default because the other value causes ncurses corruptions in Xterm (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1067679#10) so you can't really count on that anyway. Also, I'm not sure if currently grep emits clear-to-eol wherever the background color changes within a line, or at the end of the line after reverting to the terminal's default. Workarounds involving space / backspace chars probably only work with the latter approach. Also, a minor issue, but could be important / annoying to some users: printing space / backspace can modify how trailing spaces get copy-pasted in some terminals. It would be nice to copy-paste grep's output exactly as it appeared in the file. But if that's the price to pay to fix the other two problems, probably it's still the least bad. e. From unknown Sun Jun 15 09:00:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#15444: One character can be lost if colors are enabled Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-grep@gnu.org Resent-Date: Wed, 27 Mar 2024 20:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15444 X-GNU-PR-Package: grep X-GNU-PR-Keywords: To: Egmont Koblinger , 15444@debbugs.gnu.org Received: via spool by 15444-submit@debbugs.gnu.org id=B15444.17115717496493 (code B ref 15444); Wed, 27 Mar 2024 20:36:02 +0000 Received: (at 15444) by debbugs.gnu.org; 27 Mar 2024 20:35:49 +0000 Received: from localhost ([127.0.0.1]:38428 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpZzx-0001gf-3i for submit@debbugs.gnu.org; Wed, 27 Mar 2024 16:35:49 -0400 Received: from mail.cs.ucla.edu ([131.179.128.66]:41248) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpZzt-0001fj-Db for 15444@debbugs.gnu.org; Wed, 27 Mar 2024 16:35:47 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 020C13C00E407; Wed, 27 Mar 2024 13:35:39 -0700 (PDT) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavis, port 10032) with ESMTP id I4r4O254jv8x; Wed, 27 Mar 2024 13:35:38 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 7BD193C00E408; Wed, 27 Mar 2024 13:35:38 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu 7BD193C00E408 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1711571738; bh=rAAUuzwCCkTLJ+mBaG53MWgZDkVzlrvkX6ety6SvuEU=; h=Message-ID:Date:MIME-Version:To:From; b=QxvlYp8HKBwpXj4a02YlLfvo6JRh7XIejZ9pNKsns2esL7no/i8pgbUJcUMnZVbhf P1nNUqhPJ6w9eLDz+5NzRLgo93fyCaSsgDHKFvW738e4A1LzA1WddKp6Pk3nhAwjsB WBUOlWB8FBl6EaclkAJDBOxII9Cy9smIVBT5vsE6qhyl9aD4UVqURj61yl26xuYJ17 mmGIj/OycVDoFLuoTrMGzKnkkGPCWpu0f1OpE5ATmQ1iJ2d89hvBJRPrOckdDtr5cJ SvSvRAXCNgum+Tt6Sc2u8mxmbQotYO4DInLjr5bwf++6h0SoGnneBLTOYNa6CULbcw tIt5Ks2WGwhpw== X-Virus-Scanned: amavis at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavis, port 10026) with ESMTP id ThX8nGUaZKfc; Wed, 27 Mar 2024 13:35:38 -0700 (PDT) Received: from [10.10.33.175] (unknown [96.69.135.29]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id 3DC903C00E407; Wed, 27 Mar 2024 13:35:38 -0700 (PDT) Message-ID: <9a79a08b-5091-4ffd-9e67-b1db45601010@cs.ucla.edu> Date: Wed, 27 Mar 2024 14:35:37 -0600 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird References: <1237403814.574122.1379943245507.JavaMail.root@redhat.com> Content-Language: en-US From: Paul Eggert In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 3/27/24 02:15, Egmont Koblinger wrote: > The problem is that if you're at the bottom of the screen and > autowrapping occurs, the entire new line is added with the currently > active background color, which might not be the terminal's default > background color. If that new line isn't filled with text entirely > then the rest remains with that color. Thanks for the detailed analysis. Surely the issue quoted above is the main problem. That is, grep (and other programs) should be able to output color changes without knowing or worrying about line length, and shouldn't have to do any hack like what grep does now, where it switches back to the default background color and clears to the end of the line (and here I'm talking about either the current grep behavior, or the \e[3K of urxvt). Shouldn't there be a better way to do things - perhaps a new escape sequence that grep etc. can use, which says "I don't know what the physical line length is and I don't care"? > I'm puzzled how > upstream grep, for many-many years, has chosen to possibly chop off a > letter as its default behavior A good chunk of this is plain caution, as you can see from the bug report. No matter what we do it'll be "wrong" for some users and we've lacked the time or desire to think things out as carefully as you have. > Like Vincent's idea > with a space + backspace. Or space + backspace + clear-to-eol I dunno, those hacks would likely just compound our misery. They sounded a bit like trying to clean the lipstick off a pig. We're better off not adding the lipstick in the first place, or even better yet not buying the pig.