From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 07 12:46:32 2011 Received: (at submit) by debbugs.gnu.org; 7 Dec 2011 17:46:32 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYLZf-00008n-8V for submit@debbugs.gnu.org; Wed, 07 Dec 2011 12:46:32 -0500 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYEQM-0003uJ-Lo for submit@debbugs.gnu.org; Wed, 07 Dec 2011 05:08:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RYEPc-00067U-Tz for submit@debbugs.gnu.org; Wed, 07 Dec 2011 05:07:43 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:50512) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RYEPc-00067L-DO for submit@debbugs.gnu.org; Wed, 07 Dec 2011 05:07:40 -0500 Received: from eggs.gnu.org ([140.186.70.92]:57687) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RYEPb-0006Qc-1F for bug-coreutils@gnu.org; Wed, 07 Dec 2011 05:07:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RYEPY-00066Q-4P for bug-coreutils@gnu.org; Wed, 07 Dec 2011 05:07:38 -0500 Received: from mail-ww0-f49.google.com ([74.125.82.49]:43848) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RYEPX-000665-SX for bug-coreutils@gnu.org; Wed, 07 Dec 2011 05:07:36 -0500 Received: by wgbdt11 with SMTP id dt11so649324wgb.30 for ; Wed, 07 Dec 2011 02:07:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=maven.pl; s=maven; h=from:to:subject:date:user-agent:mime-version:content-type :content-transfer-encoding:message-id; bh=Vwch8WkAl27QVFH5svBl0ZYiIJ+sKQzCRIVY8QJHi0s=; b=bvYMdQFz8M9DC1lveW7LneRjliNffJjtEscR1gcPMI+ZjLpmmjch64+P20kSMjF/E9 IWZYnZ8aBGmcvI71AiASDrUT25QexTxp2Tw0AZvt2RuseLZIb1Sd+c8HbqspEbqzHRNp qJKoQo3gfuxQpOlR1c0Tk032XNd+rWyhrbqJs= Received: by 10.216.161.80 with SMTP id v58mr470566wek.21.1323252454354; Wed, 07 Dec 2011 02:07:34 -0800 (PST) Received: from t400.localnet ([83.238.65.58]) by mx.google.com with ESMTPS id hq5sm2050616wib.7.2011.12.07.02.07.32 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 07 Dec 2011 02:07:33 -0800 (PST) From: Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?= To: bug-coreutils@gnu.org Subject: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) Date: Wed, 7 Dec 2011 11:07:30 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.4; KDE/4.7.4; x86_64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201112071107.30380.arekm@maven.pl> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -6.6 (------) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Wed, 07 Dec 2011 12:46:30 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.6 (------) Hello, When doing "ls --color=3Dtty" or "ls --color=3Dauto" on directory then ls i= gnores=20 (?) ctrl+c or ctrl+z signals. Basically I'm unable to interrupt ls in such= =20 case. Easily reproducible with bigger directories. # ls --color=3Dtty ^C^C^C^C^C^C^C^C^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z (no reaction) There is no such problem if ls is being straced (strace -f -F -s 200 ls) at= =20 that time or if pipe is used ("ls --color=3Dtty | less") - in such cases ct= rl+c=20 works immediately. If --color is not used then there is no problem, too. =46irst I thought that this is filesystem issue but it is not. One of xfs=20 filesystem developers was also able to reproduce the problem and confirm th= at=20 most likely ls is doing something incorrect. # LC_ALL=3DC ls --version ls (GNU coreutils) 8.14 Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later . This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Richard M. Stallman and David MacKenzie. Linux 3.0.9, glibc 2.14 here. =2D-=20 Arkadiusz Mi=C5=9Bkiewicz PLD/Linux Team arekm / maven.pl http://ftp.pld-linux.org/ From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 07 12:56:53 2011 Received: (at 10243) by debbugs.gnu.org; 7 Dec 2011 17:56:53 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYLjg-0000aX-PG for submit@debbugs.gnu.org; Wed, 07 Dec 2011 12:56:53 -0500 Received: from mx.meyering.net ([88.168.87.75]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYLje-0000aN-VX for 10243@debbugs.gnu.org; Wed, 07 Dec 2011 12:56:52 -0500 Received: from rho.meyering.net (localhost.localdomain [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id 3A4796001B; Wed, 7 Dec 2011 18:56:05 +0100 (CET) From: Jim Meyering To: Arkadiusz =?utf-8?Q?Mi=C5=9Bkiewicz?= Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) In-Reply-To: <201112071107.30380.arekm@maven.pl> ("Arkadiusz \=\?utf-8\?Q\?Mi\?\= \=\?utf-8\?Q\?\=C5\=9Bkiewicz\=22's\?\= message of "Wed, 7 Dec 2011 11:07:30 +0100") References: <201112071107.30380.arekm@maven.pl> Date: Wed, 07 Dec 2011 18:56:05 +0100 Message-ID: <87obvknu4q.fsf@rho.meyering.net> Lines: 54 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 10243 Cc: 10243@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) Arkadiusz Mi=C5=9Bkiewicz wrote: > When doing "ls --color=3Dtty" or "ls --color=3Dauto" on directory then ls= ignores > (?) ctrl+c or ctrl+z signals. Basically I'm unable to interrupt ls in such > case. Easily reproducible with bigger directories. > > # ls --color=3Dtty > ^C^C^C^C^C^C^C^C^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z > (no reaction) > > There is no such problem if ls is being straced (strace -f -F -s 200 ls) = at > that time or if pipe is used ("ls --color=3Dtty | less") - in such cases = ctrl+c > works immediately. If --color is not used then there is no problem, too. > > First I thought that this is filesystem issue but it is not. One of xfs > filesystem developers was also able to reproduce the problem and confirm = that > most likely ls is doing something incorrect. > > # LC_ALL=3DC ls --version > ls (GNU coreutils) 8.14 > Copyright (C) 2011 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later . > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. > > Written by Richard M. Stallman and David MacKenzie. > > Linux 3.0.9, glibc 2.14 here. Thanks for the report. I reproduced it starting in an empty directory like this: seq 100000|xargs touch env ls --color -1 and tried to interrupt that. Failed to interrupt every time. Here's one way to fix it: diff --git a/src/ls.c b/src/ls.c index 8be9b6a..58bb196 100644 --- a/src/ls.c +++ b/src/ls.c @@ -4060,9 +4060,9 @@ print_name_with_quoting (const struct fileinfo *f, if (stack) PUSH_CURRENT_DIRED_POS (stack); + process_signals (); if (used_color_this_time) { - process_signals (); prep_non_filename_text (); if (start_col / line_length !=3D (start_col + width - 1) / line_leng= th) put_indicator (&color_indicator[C_CLR_TO_EOL]); From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 07 15:12:48 2011 Received: (at 10243) by debbugs.gnu.org; 7 Dec 2011 20:12:48 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYNrD-00040V-NV for submit@debbugs.gnu.org; Wed, 07 Dec 2011 15:12:48 -0500 Received: from mail2.vodafone.ie ([213.233.128.44]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYNrB-00040N-LZ for 10243@debbugs.gnu.org; Wed, 07 Dec 2011 15:12:46 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqABAH3H305tTAz3/2dsb2JhbAAMN4UGoxmFLQEBAQMBIw8BRgULCw0LAgIFFgsCAgkDAgECAUUGDQEHAQGIA6UmkU6BNIJAhiqBFgSaVYw5 Received: from unknown (HELO [192.168.1.79]) ([109.76.12.247]) by mail2.vodafone.ie with ESMTP; 07 Dec 2011 20:11:58 +0000 Message-ID: <4EDFC88E.1040500@draigBrady.com> Date: Wed, 07 Dec 2011 20:11:58 +0000 From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20110816 Thunderbird/6.0 MIME-Version: 1.0 To: Jim Meyering Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) References: <201112071107.30380.arekm@maven.pl> <87obvknu4q.fsf@rho.meyering.net> In-Reply-To: <87obvknu4q.fsf@rho.meyering.net> X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.5 (--) X-Debbugs-Envelope-To: 10243 Cc: 10243@debbugs.gnu.org, =?UTF-8?B?QXJrYWRpdXN6IE1pxZtraWV3aWN6?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.5 (--) On 12/07/2011 05:56 PM, Jim Meyering wrote: > Arkadiusz Miśkiewicz wrote: >> When doing "ls --color=tty" or "ls --color=auto" on directory then ls ignores >> (?) ctrl+c or ctrl+z signals. Basically I'm unable to interrupt ls in such >> case. Easily reproducible with bigger directories. > Thanks for the report. > I reproduced it starting in an empty directory like this: > > seq 100000|xargs touch > env ls --color -1 > > and tried to interrupt that. > Failed to interrupt every time. > > Here's one way to fix it: > > diff --git a/src/ls.c b/src/ls.c > index 8be9b6a..58bb196 100644 > --- a/src/ls.c > +++ b/src/ls.c > @@ -4060,9 +4060,9 @@ print_name_with_quoting (const struct fileinfo *f, > if (stack) > PUSH_CURRENT_DIRED_POS (stack); > > + process_signals (); > if (used_color_this_time) > { > - process_signals (); > prep_non_filename_text (); > if (start_col / line_length != (start_col + width - 1) / line_length) > put_indicator (&color_indicator[C_CLR_TO_EOL]); Looks like a good fix. It works here and had negligible impact on performance. cheers, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 07 15:17:20 2011 Received: (at 10243) by debbugs.gnu.org; 7 Dec 2011 20:17:20 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYNvb-00048g-KO for submit@debbugs.gnu.org; Wed, 07 Dec 2011 15:17:20 -0500 Received: from mail-fx0-f44.google.com ([209.85.161.44]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYNvY-00048W-SP for 10243@debbugs.gnu.org; Wed, 07 Dec 2011 15:17:18 -0500 Received: by faas1 with SMTP id s1so326820faa.3 for <10243@debbugs.gnu.org>; Wed, 07 Dec 2011 12:16:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=maven.pl; s=maven; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:content-transfer-encoding:message-id; bh=cO5GAgqDONG+UR9cMY6fny81i5x9wK3ztc1Yi7qOLSw=; b=R3rWAQrfTfeOxpz2wX3/aXqvl+NOkjTfCMtqwLMrGGj23se12QIe2s7FjCQ0A5W48n +p+vy1qPhegnu/PkPt/dLUTfcHe0b/1/wYa4dqeSsBYj4Z/5cQ4lzhjFLLIRkzvIh0NR FTAw4aHhGxxuawwihUh+KuWEkvHeQMTkd+mfw= Received: by 10.180.92.169 with SMTP id cn9mr25058718wib.62.1323288990905; Wed, 07 Dec 2011 12:16:30 -0800 (PST) Received: from t400.localnet (89-69-21-174.dynamic.chello.pl. [89.69.21.174]) by mx.google.com with ESMTPS id z35sm4801964wbm.12.2011.12.07.12.16.29 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 07 Dec 2011 12:16:30 -0800 (PST) From: Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?= To: =?utf-8?q?P=C3=A1draig_Brady?= Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) Date: Wed, 7 Dec 2011 21:16:28 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.4; KDE/4.7.4; x86_64; ; ) References: <201112071107.30380.arekm@maven.pl> <87obvknu4q.fsf@rho.meyering.net> <4EDFC88E.1040500@draigBrady.com> In-Reply-To: <4EDFC88E.1040500@draigBrady.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201112072116.28827.arekm@maven.pl> X-Spam-Score: -3.6 (---) X-Debbugs-Envelope-To: 10243 Cc: 10243@debbugs.gnu.org, Jim Meyering X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.6 (---) On Wednesday 07 of December 2011, P=C3=A1draig Brady wrote: > On 12/07/2011 05:56 PM, Jim Meyering wrote: > > Arkadiusz Mi=C5=9Bkiewicz wrote: > >> When doing "ls --color=3Dtty" or "ls --color=3Dauto" on directory then= ls > >> ignores (?) ctrl+c or ctrl+z signals. Basically I'm unable to interrupt > >> ls in such case. Easily reproducible with bigger directories. > >=20 > > Thanks for the report. > >=20 > > I reproduced it starting in an empty directory like this: > > seq 100000|xargs touch > > env ls --color -1 > >=20 > > and tried to interrupt that. > > Failed to interrupt every time. > >=20 > > Here's one way to fix it: > >=20 > > diff --git a/src/ls.c b/src/ls.c > > index 8be9b6a..58bb196 100644 > > --- a/src/ls.c > > +++ b/src/ls.c > > @@ -4060,9 +4060,9 @@ print_name_with_quoting (const struct fileinfo *f, > >=20 > > if (stack) > > =20 > > PUSH_CURRENT_DIRED_POS (stack); > >=20 > > + process_signals (); > >=20 > > if (used_color_this_time) > > =20 > > { > >=20 > > - process_signals (); > >=20 > > prep_non_filename_text (); > > if (start_col / line_length !=3D (start_col + width - 1) / > > line_length) > > =20 > > put_indicator (&color_indicator[C_CLR_TO_EOL]); >=20 > Looks like a good fix. > It works here and had negligible impact on performance. That part works for me too. Unfortunately more changes is needed since befo= re=20 printing happens it it still not possible to interrupt ls (and for huge dir= s=20 it can take a while). Moving code that enables special signal handling just before actuall printi= ng=20 starts? >=20 > cheers, > P=C3=A1draig. =2D-=20 Arkadiusz Mi=C5=9Bkiewicz PLD/Linux Team arekm / maven.pl http://ftp.pld-linux.org/ From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 07 15:42:39 2011 Received: (at 10243) by debbugs.gnu.org; 7 Dec 2011 20:42:39 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYOK7-0005TI-0Q for submit@debbugs.gnu.org; Wed, 07 Dec 2011 15:42:39 -0500 Received: from mail2.vodafone.ie ([213.233.128.44]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYOK4-0005TA-Fb for 10243@debbugs.gnu.org; Wed, 07 Dec 2011 15:42:37 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApcBAIbO305tTAz3/2dsb2JhbAAMN4UGoxmFLgEBAQQjDwFGEAsNCwICBRYLAgIJAwIBAgFFBg0BBwEBrS2RT4E0gkCGKoEWBJpVjDk Received: from unknown (HELO [192.168.1.79]) ([109.76.12.247]) by mail2.vodafone.ie with ESMTP; 07 Dec 2011 20:41:49 +0000 Message-ID: <4EDFCF8D.7000706@draigBrady.com> Date: Wed, 07 Dec 2011 20:41:49 +0000 From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20110816 Thunderbird/6.0 MIME-Version: 1.0 To: =?UTF-8?B?QXJrYWRpdXN6IE1pxZtraWV3aWN6?= Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) References: <201112071107.30380.arekm@maven.pl> <87obvknu4q.fsf@rho.meyering.net> <4EDFC88E.1040500@draigBrady.com> <201112072116.28827.arekm@maven.pl> In-Reply-To: <201112072116.28827.arekm@maven.pl> X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.5 (--) X-Debbugs-Envelope-To: 10243 Cc: 10243@debbugs.gnu.org, Jim Meyering X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.5 (--) On 12/07/2011 08:16 PM, Arkadiusz Miśkiewicz wrote: > On Wednesday 07 of December 2011, Pádraig Brady wrote: >> On 12/07/2011 05:56 PM, Jim Meyering wrote: >>> Arkadiusz Miśkiewicz wrote: >>>> When doing "ls --color=tty" or "ls --color=auto" on directory then ls >>>> ignores (?) ctrl+c or ctrl+z signals. Basically I'm unable to interrupt >>>> ls in such case. Easily reproducible with bigger directories. >>> >>> Thanks for the report. >>> >>> I reproduced it starting in an empty directory like this: >>> seq 100000|xargs touch >>> env ls --color -1 >>> >>> and tried to interrupt that. >>> Failed to interrupt every time. >>> >>> Here's one way to fix it: >>> >>> diff --git a/src/ls.c b/src/ls.c >>> index 8be9b6a..58bb196 100644 >>> --- a/src/ls.c >>> +++ b/src/ls.c >>> @@ -4060,9 +4060,9 @@ print_name_with_quoting (const struct fileinfo *f, >>> >>> if (stack) >>> >>> PUSH_CURRENT_DIRED_POS (stack); >>> >>> + process_signals (); >>> >>> if (used_color_this_time) >>> >>> { >>> >>> - process_signals (); >>> >>> prep_non_filename_text (); >>> if (start_col / line_length != (start_col + width - 1) / >>> line_length) >>> >>> put_indicator (&color_indicator[C_CLR_TO_EOL]); >> >> Looks like a good fix. >> It works here and had negligible impact on performance. > > That part works for me too. Unfortunately more changes is needed since before > printing happens it it still not possible to interrupt ls (and for huge dirs > it can take a while). > > Moving code that enables special signal handling just before actuall printing > starts? Probably, as long as there are no long blocking calls when processing large dirs, after we've starting printing. Do you get the delays with -U too? I guess we should test over NFS too. cheers, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 07 15:55:41 2011 Received: (at 10243) by debbugs.gnu.org; 7 Dec 2011 20:55:41 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYOWj-0005ml-GB for submit@debbugs.gnu.org; Wed, 07 Dec 2011 15:55:41 -0500 Received: from mail-bw0-f44.google.com ([209.85.214.44]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYOWg-0005mb-Oy for 10243@debbugs.gnu.org; Wed, 07 Dec 2011 15:55:40 -0500 Received: by bkbzs8 with SMTP id zs8so961746bkb.3 for <10243@debbugs.gnu.org>; Wed, 07 Dec 2011 12:54:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=maven.pl; s=maven; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:content-transfer-encoding:message-id; bh=3boC6oeFmj2etpAThvwomCVXFJ3GTK1BHow6jWtyUwM=; b=JhZBkKRP2QBfU6twDhXq+iBqZC6AjvP+m1DbvEeJzKnnnHdoNFr2Y7ulo/d20ojpT8 xDoqFXZQiEinpGfePOaL1FuoeXLCa1vf3NXOxLFrbEi8HOzLtNYpJnTtAAZ74z7taT8l MKDiYUzUj/Pm48BjeGROTeY4CfEQ7anC+7iUg= Received: by 10.180.92.169 with SMTP id cn9mr7733wib.62.1323291292339; Wed, 07 Dec 2011 12:54:52 -0800 (PST) Received: from t400.localnet (89-69-21-174.dynamic.chello.pl. [89.69.21.174]) by mx.google.com with ESMTPS id hn15sm4781116wib.22.2011.12.07.12.54.51 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 07 Dec 2011 12:54:51 -0800 (PST) From: Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?= To: =?utf-8?q?P=C3=A1draig_Brady?= Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) Date: Wed, 7 Dec 2011 21:54:50 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.4; KDE/4.7.4; x86_64; ; ) References: <201112071107.30380.arekm@maven.pl> <201112072116.28827.arekm@maven.pl> <4EDFCF8D.7000706@draigBrady.com> In-Reply-To: <4EDFCF8D.7000706@draigBrady.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201112072154.50459.arekm@maven.pl> X-Spam-Score: -3.6 (---) X-Debbugs-Envelope-To: 10243 Cc: 10243@debbugs.gnu.org, Jim Meyering X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.6 (---) On Wednesday 07 of December 2011, P=C3=A1draig Brady wrote: > On 12/07/2011 08:16 PM, Arkadiusz Mi=C5=9Bkiewicz wrote: > > On Wednesday 07 of December 2011, P=C3=A1draig Brady wrote: > >> On 12/07/2011 05:56 PM, Jim Meyering wrote: > >>> Arkadiusz Mi=C5=9Bkiewicz wrote: > >>>> When doing "ls --color=3Dtty" or "ls --color=3Dauto" on directory th= en ls > >>>> ignores (?) ctrl+c or ctrl+z signals. Basically I'm unable to > >>>> interrupt ls in such case. Easily reproducible with bigger > >>>> directories. > >>>=20 > >>> Thanks for the report. > >>>=20 > >>> I reproduced it starting in an empty directory like this: > >>> seq 100000|xargs touch > >>> env ls --color -1 > >>>=20 > >>> and tried to interrupt that. > >>> Failed to interrupt every time. > >>>=20 > >>> Here's one way to fix it: > >>>=20 > >>> diff --git a/src/ls.c b/src/ls.c > >>> index 8be9b6a..58bb196 100644 > >>> --- a/src/ls.c > >>> +++ b/src/ls.c > >>> @@ -4060,9 +4060,9 @@ print_name_with_quoting (const struct fileinfo > >>> *f, > >>>=20 > >>> if (stack) > >>> =20 > >>> PUSH_CURRENT_DIRED_POS (stack); > >>>=20 > >>> + process_signals (); > >>>=20 > >>> if (used_color_this_time) > >>> =20 > >>> { > >>>=20 > >>> - process_signals (); > >>>=20 > >>> prep_non_filename_text (); > >>> if (start_col / line_length !=3D (start_col + width - 1) / > >>> line_length) > >>> =20 > >>> put_indicator (&color_indicator[C_CLR_TO_EOL]); > >>=20 > >> Looks like a good fix. > >> It works here and had negligible impact on performance. > >=20 > > That part works for me too. Unfortunately more changes is needed since > > before printing happens it it still not possible to interrupt ls (and > > for huge dirs it can take a while). > >=20 > > Moving code that enables special signal handling just before actuall > > printing starts? >=20 > Probably, as long as there are no long blocking calls when > processing large dirs, after we've starting printing. Don't know why that should matter. IMO before printing happens there should be no difference in signal handlin= g=20 behaviour between ls --color and ls (since the whole signal handling is jus= t=20 to "protect" printing from what I understand). I was thinking about something like: =2D do ususal things =2D setup special signal handling =2D print + process_signals () at print_name_with_quoting =2D back to original signal handling =2D do the rest of things so most of the time there won't be any special signal handling (=3D=3D will= be the=20 same as ls without --color). > Do you get the delays with -U too? Yes, too. ls doesn't call process_signals() at all before printing starts in --color= =20 mode thus making ls uninterruptible in that period. > I guess we should test over NFS too. > cheers, > P=C3=A1draig. =2D-=20 Arkadiusz Mi=C5=9Bkiewicz PLD/Linux Team arekm / maven.pl http://ftp.pld-linux.org/ From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 07 17:05:31 2011 Received: (at 10243) by debbugs.gnu.org; 7 Dec 2011 22:05:31 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYPcI-0007RL-Np for submit@debbugs.gnu.org; Wed, 07 Dec 2011 17:05:31 -0500 Received: from mx.meyering.net ([88.168.87.75]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYPcE-0007RB-Pi for 10243@debbugs.gnu.org; Wed, 07 Dec 2011 17:05:29 -0500 Received: from rho.meyering.net (localhost.localdomain [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id 3C3BD60023; Wed, 7 Dec 2011 23:04:40 +0100 (CET) From: Jim Meyering To: Arkadiusz =?utf-8?Q?Mi=C5=9Bkiewicz?= Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) In-Reply-To: <201112072154.50459.arekm@maven.pl> ("Arkadiusz \=\?utf-8\?Q\?Mi\?\= \=\?utf-8\?Q\?\=C5\=9Bkiewicz\=22's\?\= message of "Wed, 7 Dec 2011 21:54:50 +0100") References: <201112071107.30380.arekm@maven.pl> <201112072116.28827.arekm@maven.pl> <4EDFCF8D.7000706@draigBrady.com> <201112072154.50459.arekm@maven.pl> Date: Wed, 07 Dec 2011 23:04:40 +0100 Message-ID: <87r50gm41z.fsf@rho.meyering.net> Lines: 370 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 10243 Cc: 10243@debbugs.gnu.org, =?iso-8859-1?Q?P=E1draig?= Brady X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) Arkadiusz Mi=C5=9Bkiewicz wrote: > On Wednesday 07 of December 2011, P=C3=A1draig Brady wrote: >> On 12/07/2011 08:16 PM, Arkadiusz Mi=C5=9Bkiewicz wrote: >> > On Wednesday 07 of December 2011, P=C3=A1draig Brady wrote: >> >> On 12/07/2011 05:56 PM, Jim Meyering wrote: >> >>> Arkadiusz Mi=C5=9Bkiewicz wrote: >> >>>> When doing "ls --color=3Dtty" or "ls --color=3Dauto" on directory t= hen ls >> >>>> ignores (?) ctrl+c or ctrl+z signals. Basically I'm unable to >> >>>> interrupt ls in such case. Easily reproducible with bigger >> >>>> directories. >> >>> >> >>> Thanks for the report. >> >>> >> >>> I reproduced it starting in an empty directory like this: >> >>> seq 100000|xargs touch >> >>> env ls --color -1 >> >>> >> >>> and tried to interrupt that. >> >>> Failed to interrupt every time. >> >>> >> >>> Here's one way to fix it: >> >>> >> >>> diff --git a/src/ls.c b/src/ls.c >> >>> index 8be9b6a..58bb196 100644 >> >>> --- a/src/ls.c >> >>> +++ b/src/ls.c >> >>> @@ -4060,9 +4060,9 @@ print_name_with_quoting (const struct fileinfo >> >>> *f, >> >>> >> >>> if (stack) >> >>> >> >>> PUSH_CURRENT_DIRED_POS (stack); >> >>> >> >>> + process_signals (); >> >>> >> >>> if (used_color_this_time) >> >>> >> >>> { >> >>> >> >>> - process_signals (); >> >>> >> >>> prep_non_filename_text (); >> >>> if (start_col / line_length !=3D (start_col + width - 1) / >> >>> line_length) >> >>> >> >>> put_indicator (&color_indicator[C_CLR_TO_EOL]); >> >> >> >> Looks like a good fix. >> >> It works here and had negligible impact on performance. >> > >> > That part works for me too. Unfortunately more changes is needed since >> > before printing happens it it still not possible to interrupt ls (and >> > for huge dirs it can take a while). >> > >> > Moving code that enables special signal handling just before actuall >> > printing starts? >> >> Probably, as long as there are no long blocking calls when >> processing large dirs, after we've starting printing. > Don't know why that should matter. > > IMO before printing happens there should be no difference in signal handl= ing > behaviour between ls --color and ls (since the whole signal handling is j= ust > to "protect" printing from what I understand). > > I was thinking about something like: > - do ususal things > - setup special signal handling > - print + process_signals () at print_name_with_quoting > - back to original signal handling > - do the rest of things > > so most of the time there won't be any special signal handling (=3D=3D wi= ll be the > same as ls without --color). > >> Do you get the delays with -U too? > > Yes, too. > > ls doesn't call process_signals() at all before printing starts in --color > mode thus making ls uninterruptible in that period. Thanks for the feedback. Here's a more complete patch, but I haven't re-reviewed it yet, so caveat emptor. It may write a test, too... but it will have to be racy, so maybe not. Oh, and I've just realized this needs a NEWS entry, too. >From 54d7dafb0b10daa54d9beb8e1020c2e1bfbe0370 Mon Sep 17 00:00:00 2001 From: Jim Meyering Date: Wed, 7 Dec 2011 23:00:42 +0100 Subject: [PATCH] ls: do not inhibit interrupts when listing large directori= es MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit Starting with commit adc30a83, ls inhibited interrupts to avoid corrupting the state of the terminal, but for very large directories, that inhibition renders ls uninterruptible for too long, including a potentially long period even before any output is generated. Act on interrupts as much as possible. * src/ls.c (Sig, nsigs): New globals, moved here from "main". Global Sig renamed from sig, to avoid shadowing "int sig". Now used in install_signal_handlers and in main. (install_signal_handlers): New function, extracted from... (main): ...here. The top-level function for printing file names is print_current_files. There are two execution paths by which to actually print a possibly- colorized name from this function: via print_file_name_and_frills or via the "case long_format". We cannot simply install signal handlers at the top, because both print_many_per_line and print_horizontal end up calling calculate_columns, which would cause a time-consuming and uninterruptible prelude to printing (when there are many file names and many columns). Thus, we opt to install the signal handlers in two places: that "case long_format", and upon the first call to print_file_name_and_frills. Reported by Arkadiusz Mi=C5=9Bkiewicz in http://bugs.gnu.org/10243 --- THANKS.in | 1 + src/ls.c | 165 +++++++++++++++++++++++++++++++++------------------------= --- 2 files changed, 92 insertions(+), 74 deletions(-) diff --git a/THANKS.in b/THANKS.in index 5ecc29e..afed5d4 100644 --- a/THANKS.in +++ b/THANKS.in @@ -59,6 +59,7 @@ Anthony Thyssen anthony@griffith.edu.= au Antonio Rendas ajrendas@yahoo.com Ariel Faigon ariel@cthulhu.engr.sgi.com Arjan Opmeer arjan.opmeer@gmail.com +Arkadiusz Mi=C5=9Bkiewicz arekm@maven.pl Arne Henrik Juul arnej@imf.unit.no Arnold Robbins arnold@skeeve.com Arthur Pool pool@commerce.uq.edu.au diff --git a/src/ls.c b/src/ls.c index 8be9b6a..69e40a9 100644 --- a/src/ls.c +++ b/src/ls.c @@ -155,6 +155,36 @@ # define st_author st_uid #endif +/* The signals that are trapped, and the number of such signals. */ +static int const Sig[] =3D + { + /* This one is handled specially. */ + SIGTSTP, + + /* The usual suspects. */ + SIGALRM, SIGHUP, SIGINT, SIGPIPE, SIGQUIT, SIGTERM, +#ifdef SIGPOLL + SIGPOLL, +#endif +#ifdef SIGPROF + SIGPROF, +#endif +#ifdef SIGVTALRM + SIGVTALRM, +#endif +#ifdef SIGXCPU + SIGXCPU, +#endif +#ifdef SIGXFSZ + SIGXFSZ, +#endif + }; +enum { nsigs =3D ARRAY_CARDINALITY (Sig) }; + +#if ! SA_NOCLDSTOP +static bool caught_sig[nsigs]; +#endif + enum filetype { unknown, @@ -1229,6 +1259,53 @@ process_signals (void) } } +static void +install_signal_handlers (void) +{ + if (print_with_color) + { + /* If the standard output is a controlling terminal, watch out + for signals, so that the colors can be restored to the + default state if "ls" is suspended or interrupted. */ + + if (0 <=3D tcgetpgrp (STDOUT_FILENO)) + { + int j; +#if SA_NOCLDSTOP + struct sigaction act; + + sigemptyset (&caught_signals); + for (j =3D 0; j < nsigs; j++) + { + sigaction (Sig[j], NULL, &act); + if (act.sa_handler !=3D SIG_IGN) + sigaddset (&caught_signals, Sig[j]); + } + + act.sa_mask =3D caught_signals; + act.sa_flags =3D SA_RESTART; + + for (j =3D 0; j < nsigs; j++) + if (sigismember (&caught_signals, Sig[j])) + { + act.sa_handler =3D Sig[j] =3D=3D SIGTSTP ? stophandler : s= ighandler; + sigaction (Sig[j], &act, NULL); + } +#else + for (j =3D 0; j < nsigs; j++) + { + caught_sig[j] =3D (signal (Sig[j], SIG_IGN) !=3D SIG_IGN); + if (caught_sig[j]) + { + signal (Sig[j], Sig[j] =3D=3D SIGTSTP ? stophandler : si= ghandler); + siginterrupt (Sig[j], 0); + } + } +#endif + } + } +} + int main (int argc, char **argv) { @@ -1236,36 +1313,6 @@ main (int argc, char **argv) struct pending *thispend; int n_files; - /* The signals that are trapped, and the number of such signals. */ - static int const sig[] =3D - { - /* This one is handled specially. */ - SIGTSTP, - - /* The usual suspects. */ - SIGALRM, SIGHUP, SIGINT, SIGPIPE, SIGQUIT, SIGTERM, -#ifdef SIGPOLL - SIGPOLL, -#endif -#ifdef SIGPROF - SIGPROF, -#endif -#ifdef SIGVTALRM - SIGVTALRM, -#endif -#ifdef SIGXCPU - SIGXCPU, -#endif -#ifdef SIGXFSZ - SIGXFSZ, -#endif - }; - enum { nsigs =3D ARRAY_CARDINALITY (sig) }; - -#if ! SA_NOCLDSTOP - bool caught_sig[nsigs]; -#endif - initialize_main (&argc, &argv); set_program_name (argv[0]); setlocale (LC_ALL, ""); @@ -1299,46 +1346,6 @@ main (int argc, char **argv) || (is_colored (C_EXEC) && color_symlink_as_referent) || (is_colored (C_MISSING) && format =3D=3D long_format)) check_symlink_color =3D true; - - /* If the standard output is a controlling terminal, watch out - for signals, so that the colors can be restored to the - default state if "ls" is suspended or interrupted. */ - - if (0 <=3D tcgetpgrp (STDOUT_FILENO)) - { - int j; -#if SA_NOCLDSTOP - struct sigaction act; - - sigemptyset (&caught_signals); - for (j =3D 0; j < nsigs; j++) - { - sigaction (sig[j], NULL, &act); - if (act.sa_handler !=3D SIG_IGN) - sigaddset (&caught_signals, sig[j]); - } - - act.sa_mask =3D caught_signals; - act.sa_flags =3D SA_RESTART; - - for (j =3D 0; j < nsigs; j++) - if (sigismember (&caught_signals, sig[j])) - { - act.sa_handler =3D sig[j] =3D=3D SIGTSTP ? stophandler : s= ighandler; - sigaction (sig[j], &act, NULL); - } -#else - for (j =3D 0; j < nsigs; j++) - { - caught_sig[j] =3D (signal (sig[j], SIG_IGN) !=3D SIG_IGN); - if (caught_sig[j]) - { - signal (sig[j], sig[j] =3D=3D SIGTSTP ? stophandler : si= ghandler); - siginterrupt (sig[j], 0); - } - } -#endif - } } if (dereference =3D=3D DEREF_UNDEFINED) @@ -1468,12 +1475,12 @@ main (int argc, char **argv) /* Restore the default signal handling. */ #if SA_NOCLDSTOP for (j =3D 0; j < nsigs; j++) - if (sigismember (&caught_signals, sig[j])) - signal (sig[j], SIG_DFL); + if (sigismember (&caught_signals, Sig[j])) + signal (Sig[j], SIG_DFL); #else for (j =3D 0; j < nsigs; j++) if (caught_sig[j]) - signal (sig[j], SIG_DFL); + signal (Sig[j], SIG_DFL); #endif /* Act on any signals that arrived before the default was restored. @@ -3483,6 +3490,7 @@ print_current_files (void) break; case long_format: + install_signal_handlers (); for (i =3D 0; i < cwd_n_used; i++) { set_normal_color (); @@ -4060,9 +4068,9 @@ print_name_with_quoting (const struct fileinfo *f, if (stack) PUSH_CURRENT_DIRED_POS (stack); + process_signals (); if (used_color_this_time) { - process_signals (); prep_non_filename_text (); if (start_col / line_length !=3D (start_col + width - 1) / line_leng= th) put_indicator (&color_indicator[C_CLR_TO_EOL]); @@ -4092,6 +4100,15 @@ static size_t print_file_name_and_frills (const struct fileinfo *f, size_t start_col) { char buf[MAX (LONGEST_HUMAN_READABLE + 1, INT_BUFSIZE_BOUND (uintmax_t))= ]; + static bool first =3D true; + if (first) + { + /* handle interrupts so that ls cannot be made to output an + incomplete multi-byte sequence or a color-change escape + sequence that could leave the terminal messed up. */ + install_signal_handlers (); + first =3D false; + } set_normal_color (); -- 1.7.8 From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 07 17:54:35 2011 Received: (at 10243) by debbugs.gnu.org; 7 Dec 2011 22:54:35 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYQNm-0000xD-9i for submit@debbugs.gnu.org; Wed, 07 Dec 2011 17:54:34 -0500 Received: from mail2.vodafone.ie ([213.233.128.44]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYQNj-0000ws-Lt for 10243@debbugs.gnu.org; Wed, 07 Dec 2011 17:54:32 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBACnu305tTAz3/2dsb2JhbAAMN4UGqEcBAQEEIw8BRhALDQ0CBRYLAgIJAwIBAgFFBg0BBwEBrS2RT4E0iGqBFgSaVYw5 Received: from unknown (HELO [192.168.1.79]) ([109.76.12.247]) by mail2.vodafone.ie with ESMTP; 07 Dec 2011 22:53:44 +0000 Message-ID: <4EDFEE77.6080105@draigBrady.com> Date: Wed, 07 Dec 2011 22:53:43 +0000 From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20110816 Thunderbird/6.0 MIME-Version: 1.0 To: Jim Meyering Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) References: <201112071107.30380.arekm@maven.pl> <201112072116.28827.arekm@maven.pl> <4EDFCF8D.7000706@draigBrady.com> <201112072154.50459.arekm@maven.pl> <87r50gm41z.fsf@rho.meyering.net> In-Reply-To: <87r50gm41z.fsf@rho.meyering.net> X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.5 (--) X-Debbugs-Envelope-To: 10243 Cc: 10243@debbugs.gnu.org, =?UTF-8?B?QXJrYWRpdXN6IE1pxZtraWV3aWN6?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.5 (--) >From a cursory glance the change looks sensible. Note a separate issue is there is a small chance that the ^C representation output to the terminal, will mess things up: http://debbugs.gnu.org/9620#14 I.E. even when signals are blocked the ^C etc. is output, which can mess up ANSI codes and multibyte characters (I have seen such corruption). I'm just noting it here, as it's probably overkill/problematic to disable echoing to the terminal, and explicitly output ^C when we process_signals() like readline does. cheers, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 07 18:17:41 2011 Received: (at 10243) by debbugs.gnu.org; 7 Dec 2011 23:17:41 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYQk8-0002EY-W9 for submit@debbugs.gnu.org; Wed, 07 Dec 2011 18:17:41 -0500 Received: from mx1.redhat.com ([209.132.183.28]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYQk5-0002EO-Re for 10243@debbugs.gnu.org; Wed, 07 Dec 2011 18:17:39 -0500 Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id pB7NGoX2010451 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 7 Dec 2011 18:16:51 -0500 Received: from [10.3.113.35] (ovpn-113-35.phx2.redhat.com [10.3.113.35]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id pB7NGoSf014008; Wed, 7 Dec 2011 18:16:50 -0500 Message-ID: <4EDFF3E1.8070809@redhat.com> Date: Wed, 07 Dec 2011 16:16:49 -0700 From: Eric Blake Organization: Red Hat User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20111115 Thunderbird/8.0 MIME-Version: 1.0 To: Jim Meyering Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) References: <201112071107.30380.arekm@maven.pl> <201112072116.28827.arekm@maven.pl> <4EDFCF8D.7000706@draigBrady.com> <201112072154.50459.arekm@maven.pl> <87r50gm41z.fsf@rho.meyering.net> In-Reply-To: <87r50gm41z.fsf@rho.meyering.net> X-Enigmail-Version: 1.3.3 OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enig8E03C076F24A09CC1C2FC749" X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Spam-Score: -10.3 (----------) X-Debbugs-Envelope-To: 10243 Cc: 10243@debbugs.gnu.org, =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= , =?UTF-8?B?QXJrYWRpdXN6IE1pxZtraWV3aWN6?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -10.3 (----------) This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig8E03C076F24A09CC1C2FC749 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 12/07/2011 03:04 PM, Jim Meyering wrote: >> I was thinking about something like: >> - do ususal things >> - setup special signal handling >> - print + process_signals () at print_name_with_quoting >> - back to original signal handling >> - do the rest of things >> >> so most of the time there won't be any special signal handling (=3D=3D= will be the >> same as ls without --color). >> >>> Do you get the delays with -U too? >> >> Yes, too. >> >> ls doesn't call process_signals() at all before printing starts in --c= olor >> mode thus making ls uninterruptible in that period. >=20 > Thanks for the feedback. > Here's a more complete patch, but I haven't re-reviewed it yet, > so caveat emptor. Question - is ls processing entirely two-phase (collect all names, then format them), or is this a repetitive loop? Seeing as how long listings of two directories on the command line produces differing column widths between the two directories, I have to think it is the latter (besides, that makes better sense from a memory management perspective - it's cheaper to sort one directory at a time than to store the sorting of all directories at once). In which case, consider: ls --color=3Dalways largedir1 largedir2 If we enable our signal handle just before outputting the sorted, columnized, and colred largedir1 listings, only to then start our readdir() of largedir2, we haven't solved the problem - the second readdir() and/or calculate_columns() between the two directories will be lengthy enough to be noticeably interrupt-starved. > @@ -4092,6 +4100,15 @@ static size_t > print_file_name_and_frills (const struct fileinfo *f, size_t start_col= ) > { > char buf[MAX (LONGEST_HUMAN_READABLE + 1, INT_BUFSIZE_BOUND (uintmax= _t))]; > + static bool first =3D true; > + if (first) > + { > + /* handle interrupts so that ls cannot be made to output an > + incomplete multi-byte sequence or a color-change escape > + sequence that could leave the terminal messed up. */ That is, I think a one-shot static is wrong, and that you will have to repeatedly install and uninstall the signal handlers, according to which phase of the processing loop you are in. --=20 Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --------------enig8E03C076F24A09CC1C2FC749 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBCAAGBQJO3/PhAAoJEKeha0olJ0NqtawIAKYFHBjS9h3uKR84lBRsB4u9 pAog8q/TRIsuVnVIiTcRzO2BjDk8bAhKNahgcUasOpBz2CyytKoh2nUc0ftXcGzy GyzpEDqEIlCR1S/mBqH8apixefo28+FQGj8u/ZaazRRhkob30gDPtMe+cDxROtEh 465Z9ZY3efJQjOqbRI3i0rvCcyzh+JdXqMCCnfleg7VLwlx5E4Lg+vdIvevD6jAv 51qluf34w8qHXapwTZW8BXVjln7Y1kD4A5ByEYgLC7nEiaM8h3/0SK+4cekCNHp5 e13+rU5/DH9ruCgAEoWWB0d5RG/U8KF7W6XmXIu8AWrb3xIbj8zutPLgCCVETPQ= =flC2 -----END PGP SIGNATURE----- --------------enig8E03C076F24A09CC1C2FC749-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 08 05:06:47 2011 Received: (at 10243-done) by debbugs.gnu.org; 8 Dec 2011 10:06:47 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYasI-0000yS-W6 for submit@debbugs.gnu.org; Thu, 08 Dec 2011 05:06:47 -0500 Received: from mx.meyering.net ([88.168.87.75]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYasF-0000yI-2R for 10243-done@debbugs.gnu.org; Thu, 08 Dec 2011 05:06:45 -0500 Received: from rho.meyering.net (localhost.localdomain [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id 9F06260398; Thu, 8 Dec 2011 11:05:53 +0100 (CET) From: Jim Meyering To: Eric Blake Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) In-Reply-To: <4EDFF3E1.8070809@redhat.com> (Eric Blake's message of "Wed, 07 Dec 2011 16:16:49 -0700") References: <201112071107.30380.arekm@maven.pl> <201112072116.28827.arekm@maven.pl> <4EDFCF8D.7000706@draigBrady.com> <201112072154.50459.arekm@maven.pl> <87r50gm41z.fsf@rho.meyering.net> <4EDFF3E1.8070809@redhat.com> Date: Thu, 08 Dec 2011 11:05:52 +0100 Message-ID: <87d3bztm2n.fsf@rho.meyering.net> Lines: 118 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 10243-done Cc: =?iso-8859-1?Q?P=E1draig?= Brady , Arkadiusz =?utf-8?Q?Mi=C5=9Bkiewicz?= , 10243-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) Eric Blake wrote: > On 12/07/2011 03:04 PM, Jim Meyering wrote: ... >> Thanks for the feedback. >> Here's a more complete patch, but I haven't re-reviewed it yet, >> so caveat emptor. > > Question - is ls processing entirely two-phase (collect all names, then > format them), or is this a repetitive loop? Seeing as how long listings > of two directories on the command line produces differing column widths > between the two directories, I have to think it is the latter (besides, > that makes better sense from a memory management perspective - it's > cheaper to sort one directory at a time than to store the sorting of all > directories at once). In which case, consider: > > ls --color=3Dalways largedir1 largedir2 Thanks, you're right. This is my first use of the new Co-authored-by: syntax, so I confirmed that our updated gitlog-to-changelog does produce the expected ChangeLog in the distribution tarball. Here's a simpler patch. I've tested it by running ls -R --color big something-distinctive big2 where big has 500K entries and big2 has 2M (on a tmpfs). Before (using my initial patch), I could interrupt any time before the ls' readdir loop for big2, but not during that loop, which would run for a few seconds after ls printed its "/t/big2:" Now, it's interruptible, as one would expect. >From aaf5b61e9999d0ece522faa4508a6565d51a8446 Mon Sep 17 00:00:00 2001 From: Jim Meyering Date: Thu, 8 Dec 2011 10:49:03 +0100 Subject: [PATCH] ls: be responsive to interrupts when color-listing large directories MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit Starting with commit adc30a83, when using --color, ls inhibited interrupts to avoid corrupting the state of an output terminal. However, for very large directories, that inhibition rendered ls uninterruptible for too long, including a potentially long period even before any output is generated. * src/ls.c: There are two phases of processing that are time- consuming enough that they can cause this sort of trouble: the readdir loop and the printing loop. The printing side of things was nominally covered by a call to process_signals in (print_name_with_quoting): ... but that call was mistakenly guarded by a condition that might be false for many or even all files being processed. Call process_signals unconditionally. (print_dir): Also call process_signals in the readdir loop. * NEWS (Bug fixes): Mention it. Reported by Arkadiusz Mi=C5=9Bkiewicz in http://bugs.gnu.org/10243 Co-authored-by: Eric Blake --- NEWS | 3 +++ THANKS.in | 1 + src/ls.c | 7 ++++++- 3 files changed, 10 insertions(+), 1 deletions(-) diff --git a/NEWS b/NEWS index de3888d..0d4c83b 100644 --- a/NEWS +++ b/NEWS @@ -4,6 +4,9 @@ GNU coreutils NEWS -*- o= utline -*- ** Bug fixes + ls --color many-entry-directory was uninterruptible for too long + [bug introduced in coreutils-5.2.1] + ls's -k option no longer affects how ls -l outputs file sizes. It now affects only the per-directory block counts written by -l, and the sizes written by -s. This is for compatibility with BSD diff --git a/THANKS.in b/THANKS.in index 5ecc29e..afed5d4 100644 --- a/THANKS.in +++ b/THANKS.in @@ -59,6 +59,7 @@ Anthony Thyssen anthony@griffith.edu.= au Antonio Rendas ajrendas@yahoo.com Ariel Faigon ariel@cthulhu.engr.sgi.com Arjan Opmeer arjan.opmeer@gmail.com +Arkadiusz Mi=C5=9Bkiewicz arekm@maven.pl Arne Henrik Juul arnej@imf.unit.no Arnold Robbins arnold@skeeve.com Arthur Pool pool@commerce.uq.edu.au diff --git a/src/ls.c b/src/ls.c index 8be9b6a..0d64bab 100644 --- a/src/ls.c +++ b/src/ls.c @@ -2595,6 +2595,11 @@ print_dir (char const *name, char const *realname, b= ool command_line_arg) } else break; + + /* When processing a very large directory, and since we've inhibited + interrupts, this loop would take so long that ls would be annoyin= gly + uninterruptible. This ensures that it handles signals promptly. = */ + process_signals (); } if (closedir (dirp) !=3D 0) @@ -4060,9 +4065,9 @@ print_name_with_quoting (const struct fileinfo *f, if (stack) PUSH_CURRENT_DIRED_POS (stack); + process_signals (); if (used_color_this_time) { - process_signals (); prep_non_filename_text (); if (start_col / line_length !=3D (start_col + width - 1) / line_leng= th) put_indicator (&color_indicator[C_CLR_TO_EOL]); -- 1.7.8.110.g4cb5d1 From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 08 08:03:35 2011 Received: (at 10243-done) by debbugs.gnu.org; 8 Dec 2011 13:03:35 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYddN-0005kl-SI for submit@debbugs.gnu.org; Thu, 08 Dec 2011 08:03:34 -0500 Received: from mx1.redhat.com ([209.132.183.28]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYddK-0005kc-CE for 10243-done@debbugs.gnu.org; Thu, 08 Dec 2011 08:03:32 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id pB8D2eeb021589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 8 Dec 2011 08:02:40 -0500 Received: from [10.3.113.35] (ovpn-113-35.phx2.redhat.com [10.3.113.35]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id pB8D2dDl019891; Thu, 8 Dec 2011 08:02:39 -0500 Message-ID: <4EE0B56E.6010609@redhat.com> Date: Thu, 08 Dec 2011 06:02:38 -0700 From: Eric Blake Organization: Red Hat User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20111115 Thunderbird/8.0 MIME-Version: 1.0 To: Jim Meyering Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) References: <201112071107.30380.arekm@maven.pl> <201112072116.28827.arekm@maven.pl> <4EDFCF8D.7000706@draigBrady.com> <201112072154.50459.arekm@maven.pl> <87r50gm41z.fsf@rho.meyering.net> <4EDFF3E1.8070809@redhat.com> <87d3bztm2n.fsf@rho.meyering.net> In-Reply-To: <87d3bztm2n.fsf@rho.meyering.net> X-Enigmail-Version: 1.3.3 OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enig2DB3EF2CC255D6C7BD9E4618" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Spam-Score: -10.3 (----------) X-Debbugs-Envelope-To: 10243-done Cc: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= , =?UTF-8?B?QXJrYWRpdXN6IE1pxZtraWV3aWN6?= , 10243-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -10.3 (----------) This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig2DB3EF2CC255D6C7BD9E4618 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 12/08/2011 03:05 AM, Jim Meyering wrote: > Starting with commit adc30a83, when using --color, ls inhibited > interrupts to avoid corrupting the state of an output terminal. > However, for very large directories, that inhibition rendered ls > uninterruptible for too long, including a potentially long period > even before any output is generated. > * src/ls.c: There are two phases of processing that are time- > consuming enough that they can cause this sort of trouble: > the readdir loop and the printing loop. The printing side of things > was nominally covered by a call to process_signals in > (print_name_with_quoting): ... but that call was mistakenly guarded > by a condition that might be false for many or even all files being > processed. Call process_signals unconditionally. > (print_dir): Also call process_signals in the readdir loop. Aren't there actually three phases of long processing? You covered the readdir loop, and printing, but what about time spent in sorting? And if sorting is indeed a long enough processing hog, should we be calling process_signals in the qsort() callback as a reliable frequently reached point, or is that too dangerous? --=20 Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --------------enig2DB3EF2CC255D6C7BD9E4618 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBCAAGBQJO4LVvAAoJEKeha0olJ0NqfLEH/22E/a7dFV5FU/KF/81lIW3V g6qkge/o37qhIxVEbEVxjenrBSkKQ6Nyd0tflgUJm/KGTHfiquDmpKQqZO/5bZoO wO64Yp0ZAUsZV1IFRsdewsXAv7uM9p/O+UPYVKeOSvxk6MkcVHu5WZBQrIfIh4zF Dn+oGd4qBSGROvxlAPsWvxF+ygCl2mieA/fn35sBM4qFXHPzH+kU6Ga8ygR4uC8n J+9j8xsOC7WergpjmeEl7Csw43rGXnDiHV97l9N0RlwIMRyfRL77TEDj2d6fkKb1 RHfW4ig6aw928nE9Zw6iczzQi3Q8jIn+EOJ8+0rB1oIoNIPB3c99AH43E92U9Lo= =anP1 -----END PGP SIGNATURE----- --------------enig2DB3EF2CC255D6C7BD9E4618-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 08 08:10:27 2011 Received: (at 10243-done) by debbugs.gnu.org; 8 Dec 2011 13:10:27 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYdk2-0005u0-TJ for submit@debbugs.gnu.org; Thu, 08 Dec 2011 08:10:27 -0500 Received: from mx.meyering.net ([88.168.87.75]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RYdjy-0005tr-Sr for 10243-done@debbugs.gnu.org; Thu, 08 Dec 2011 08:10:25 -0500 Received: from rho.meyering.net (localhost.localdomain [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id A591C60267; Thu, 8 Dec 2011 14:09:32 +0100 (CET) From: Jim Meyering To: Eric Blake Subject: Re: bug#10243: 8.14: ls --color is uninterruptible with ctrl+c (and no network fs in use) In-Reply-To: <4EE0B56E.6010609@redhat.com> (Eric Blake's message of "Thu, 08 Dec 2011 06:02:38 -0700") References: <201112071107.30380.arekm@maven.pl> <201112072116.28827.arekm@maven.pl> <4EDFCF8D.7000706@draigBrady.com> <201112072154.50459.arekm@maven.pl> <87r50gm41z.fsf@rho.meyering.net> <4EDFF3E1.8070809@redhat.com> <87d3bztm2n.fsf@rho.meyering.net> <4EE0B56E.6010609@redhat.com> Date: Thu, 08 Dec 2011 14:09:32 +0100 Message-ID: <87ehwfb46r.fsf@rho.meyering.net> Lines: 34 MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 10243-done Cc: =?iso-8859-1?Q?P=E1draig?= Brady , Arkadiusz =?utf-8?Q?Mi=C5=9Bkiewicz?= , 10243-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) Eric Blake wrote: > On 12/08/2011 03:05 AM, Jim Meyering wrote: >> Starting with commit adc30a83, when using --color, ls inhibited >> interrupts to avoid corrupting the state of an output terminal. >> However, for very large directories, that inhibition rendered ls >> uninterruptible for too long, including a potentially long period >> even before any output is generated. >> * src/ls.c: There are two phases of processing that are time- >> consuming enough that they can cause this sort of trouble: >> the readdir loop and the printing loop. The printing side of things >> was nominally covered by a call to process_signals in >> (print_name_with_quoting): ... but that call was mistakenly guarded >> by a condition that might be false for many or even all files being >> processed. Call process_signals unconditionally. >> (print_dir): Also call process_signals in the readdir loop. > > Aren't there actually three phases of long processing? You covered the Not for today's systems. > readdir loop, and printing, but what about time spent in sorting? And > if sorting is indeed a long enough processing hog, should we be calling > process_signals in the qsort() callback as a reliable frequently reached > point, or is that too dangerous? It's not "long", as in noticeable with a 2-million-entry directory. Sorting that many entries is quick enough that no change is needed. Of course you can construct a situation in which sorting will take too long, i.e., so little RAM that swapping is required, or so many dir entries that sorting really does take too long, but on systems where you don't run out of i-nodes first, a directory with that many entries will cause you far more trouble than merely rendering ls uninterruptible for a few seconds. From unknown Sun Aug 17 22:07:33 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 06 Jan 2012 12:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator