From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 02:38:52 2024 Received: (at submit) by debbugs.gnu.org; 6 Apr 2024 06:38:52 +0000 Received: from localhost ([127.0.0.1]:38021 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rszhS-000783-Hs for submit@debbugs.gnu.org; Sat, 06 Apr 2024 02:38:52 -0400 Received: from lists.gnu.org ([2001:470:142::17]:43480) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rswFh-0003Qb-0g for submit@debbugs.gnu.org; Fri, 05 Apr 2024 22:57:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rswFT-00021p-5u for bug-coreutils@gnu.org; Fri, 05 Apr 2024 22:57:43 -0400 Received: from mail-os0jpn01on20700.outbound.protection.outlook.com ([2a01:111:f403:201a::700] helo=JPN01-OS0-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rswFB-0000nn-O3 for bug-coreutils@gnu.org; Fri, 05 Apr 2024 22:57:27 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JjDjX6kZb3vwJpM3ZT8FffXfJhL087gVeV2NK27TAqbtLVQ/N7SCrY1/vvDstwbpKo7W+t8gS3htGTuE30p8jXDHfUcBM1oajccUtzHw5LnwOUCHbD7ZD66xCSK1WD+5KQ3no696d9ZgOX9sMez3WGPeogNXB6jmA0D3DwHvV8xGUCZCP7/xf+0YmKwsedkeDEkV0ew1+OKM/8QbsMUl/9RPd32Ny3cxZEKooiz14d8XS0SWcu8dQHdpmpidQ9k/kq/r0sMJ2JwzvOpciNDAPwq9Wch3Ul+Pk4/W7z+qB49ONmI1mtcecLbCrZzViReLyFRH0OmewQQNfnShs115Aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hYJHjpGaaWvBWkSw9/5lMydMncB4577oG9vDJUlFTHk=; b=CRUC4kj1YOrzAxDNEXuvAN1bBiEY9lQJdwBzlo9UX5NcRyGoESyIJKZJxmCtCZYRV1vDRk5qe54XCk1mro/GJU3bGd13AlbAfTHQReNoNJZ7MDvo1innmGNeCk3GZVgHb15V7+YIA4TZ/YZuSvlAPub6GHzkYPljfBP/hE6YEDFZI3t7G+3q54SKo7CU9n6qeBmXOu7wAdIjA5WDMtnD9Gul5RslYaGfjUaT2CPD72x3RZzCsxmFb6Xci7FT5oB/aPyKoXSp/o1LcVJOp9u4SC3glr6PhZZ1XnwrRtT2x8a90l9ZkjDdezVwnpMWIyNDIH8nZ/XECwFpva9eZL8KqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=zlab.co.jp; dmarc=pass action=none header.from=zlab.co.jp; dkim=pass header.d=zlab.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yjgroup1.onmicrosoft.com; s=selector2-yjgroup1-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hYJHjpGaaWvBWkSw9/5lMydMncB4577oG9vDJUlFTHk=; b=kO+GSJB+gZCRykbEgIwZ8/lZ+GuUtGcVeTQaCph6Y+lANq8t3/Pfgt4FkAmTP4AEx8w0xv6KdCzFx1Awzkwo3GG7RoBAkGpUZn5fbjFwm4K5ejeIvbJWWdgACriQn8gSJuoTMu/AsLtFUmZMYokOvJC8PUp3mxjF51TzZRfLfk4= Received: from OSAPR01MB1619.jpnprd01.prod.outlook.com (2603:1096:603:2e::15) by TYWPR01MB9969.jpnprd01.prod.outlook.com (2603:1096:400:1e1::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7409.46; Sat, 6 Apr 2024 02:52:18 +0000 Received: from OSAPR01MB1619.jpnprd01.prod.outlook.com ([fe80::9555:3b3d:ba6:147e]) by OSAPR01MB1619.jpnprd01.prod.outlook.com ([fe80::9555:3b3d:ba6:147e%5]) with mapi id 15.20.7409.042; Sat, 6 Apr 2024 02:52:18 +0000 Content-Type: multipart/mixed; boundary="------------scVQ1eSWWXgHHmIO5FRVkQAM" Message-ID: <5d9a5fa2-e716-4d68-b4fe-8ef58e4e4e0c@zlab.co.jp> Date: Sat, 6 Apr 2024 11:52:17 +0900 User-Agent: Mozilla Thunderbird Content-Language: en-US From: Takashi Kusumi Subject: Performance issue on sort with zero-sized pseudo files To: bug-coreutils@gnu.org X-ClientProxiedBy: TY2PR01CA0006.jpnprd01.prod.outlook.com (2603:1096:404:a::18) To OSAPR01MB1619.jpnprd01.prod.outlook.com (2603:1096:603:2e::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: OSAPR01MB1619:EE_|TYWPR01MB9969:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iqgayc0hMGYCCVIPCH9QVPjrulHLFmpDw/VHNocfnf7Itefb2Nei593vN4eb3FRuDDH9etTUyEl2sYyYz5strEWMuZKke6ytio+0091NnfyIpjgQA19RhLbaQaygZFJxWhOx9zPpDLvz9tWOmn8vJAEeNkM+wlXfBurBCPBGJZgOetbhsTDPYByD/9apelCnwMRlIr7hTF+qi5fseBkMda8zPtVZ99tP98NqmpMn3MXzG91Sa6b7rQDpH6HVGcu/83ZkLcIG0m9fauymMVuG/l1rzKFU119rMhjZGjByP2rTJUdy0TzzRyrOqWUr7insOA+5ayVz+YGVmsl+4imz+uZf9ObLvHuVO8+7XnOFfU//pnT+PSPAXGc68GYiiuIEE5i9IS1vuqPZnhO7nFBrnKHpvttBstb1EMNJ8Hz3re9W7LA7/ymbkx77XwDAm1cT9qD1QvCWg5Z2QiXXn5MWvb/KOh4qO4jlddnB9trh4BaqN9un6y7krbUeOcyzS4h9Pf7pMmsqIBaP191jiK/D45SC3v3K6v420O5oGI8SzqoCE9G9gzbuRt2WnnnX0UHhF2JnBqfYcGio8WOeE867UqRMgiFPI7u4PYf5VT7TKt54GFnTqYA9ZHYZ5bubYYeDo1BTPuuZjtxIn+JEZjJUvOPE5SH12gbrq5Wm0WNta7o= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:OSAPR01MB1619.jpnprd01.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366007)(376005)(1800799015); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aUcvbi9FNkhoYzZNUVRVanJCNGVITS96VWRxOVVCN2Vac1pTRXErMTl5TWFI?= =?utf-8?B?dkZmakFWZFJpUjgrMnh4anIwODc1ZUVUcUJpM3Z3bTd5T3M2SmZJV2tlRVg2?= =?utf-8?B?dktjNlU5cGRDYWZmNVJwV2tkbXIxRkhGTmJyVSs0Z08yMmQ2WHBpbDlMai9j?= =?utf-8?B?MDJvR2Q4Mlk5WHdEVjdqMGprVVBrNnNCbndqY0l3SjNLT3h3aFNweVk0QkdN?= =?utf-8?B?aG9RSDVaV3E5cEJmdU81ZGI5NUdsUU5GQk9iU2xFTkFKd2FBWEVSV0pFZHdl?= =?utf-8?B?OHR1WUR1ZXI1RWpZZExNRHZqMGFoQ015OFJmN0R4WmhFZ2IxckV5c213VEpK?= =?utf-8?B?TDdZaUlIQ0tOSDVJRm9hOHRlNzdKbjY3REozMXhVRzR6SFFxaW5TTVhtb2NX?= =?utf-8?B?RTVONzdPSERsai9uMlA4ZTV2a2x3Mk1Wb0dJalRWM0NhTklrWktnN2oxdWFx?= =?utf-8?B?WmcrdWJPdlB4MWNtTXI3OEhZRytQSWJRbGdUUk1IRmVNQnk4MjE0TTEyZzhK?= =?utf-8?B?S3k5Z09tQlByaFlEaUZ1Vyt2dlNncWM4Rk9mNG82WmgrMTZXMXlVYkJ3Rk1q?= =?utf-8?B?MDJ0TUQwQUczMm01K3pVUEhPUmtueGhZaDFXeHRvbEl3eWpiTEZIbW5yQzY1?= =?utf-8?B?SVV2Y1JWdzhianZSZmo3aWNhVWxvWmM2VThaOE14Umtld2UwK1NFNEZjZm16?= =?utf-8?B?UGliRjh6SU03NW5IaEJLQ05MKzdLTFFuK25lcmNJMEYyWE9SUlEwWWFPY1ha?= =?utf-8?B?MDNiekFBZGJzNm5EL3RMdzI4cHd6czdnQ2xqM3FpLzF4NkJEUUdsNUliMG5C?= =?utf-8?B?aFhINzZRRGVwMzFWU0dUU0VzUDFOL2pNWVhJMXFRbmZKdDFrZkUwa25ZWC9m?= =?utf-8?B?WTFJVUYyNlNNalhJeHlIS3k5c3dhYWtqbThCaktVcDM3U24yOG1DNmdLblNp?= =?utf-8?B?RndMVDduZ1Zoa3E4WlUzK2Uwa1VBWm5Pb2Jxd1hpT2NsZ2NrWHRBeVJldHdR?= =?utf-8?B?ZmE2MElvUWp3M3Jwa3BzeU9HTE9lMzJQQkJtVzZNL3B0dFZkNURzOXdlTE9s?= =?utf-8?B?cVdDOGtSR2dtUWduZCtRZ1dXNkxCRjc3NE51RWdiYmxXSVJzZkNiYWllbDRt?= =?utf-8?B?Zm5UYmM3VGpLRXNDN1pmRnBvK0w5aFoySXlaOEYySnlFSkp5ZG5sejUyYklq?= =?utf-8?B?WVVaenAyVzNBY0VxZ1pOb0k0cTg5UFdPZmtCMGZIa1kvcWdBR2xsSExCeTQx?= =?utf-8?B?eHZGK2RDcy9zOEY3b0JycFI5VXUrcXdMd2JXenVmM25WZ0pua2YvTGt0S1FC?= =?utf-8?B?TWRpajhLRTNTa1Y4MXNKQU1wWGFsNU95Wmo4aVJNMkMwMmY3akd1YndDTUJn?= =?utf-8?B?NVNkUjZrYjJVV1hFUFR0RC9VYVhiQ24yT0xjTCtVT2NtOFJ3dnhBU1Q0L293?= =?utf-8?B?Mk5VdEhEUStNTndvTnJGbGtjdythekJ2WGthdjBYTnl6Z2s0SHNSRFdVM0Vt?= =?utf-8?B?SlpxS2RDWmlyQzM0NW5WeWkzV0ZWaHI3UG1rUExMeDhmWmNPaXF6a2NCcFg4?= =?utf-8?B?blpPOU44bkFmVWg0TFh0WnR5dWw4dVhLbVhpZGxRMm4yL01iZEc5Tjk4QTZx?= =?utf-8?B?bmt4MVBiWlk4SzNrTkFtSHlHc2hkQ2gzOTZhUm1na2ZWYTRvbWVhQmRtTHJv?= =?utf-8?B?YnRad2FubVJCVDJKdWh1OStjR1psUUhjRDRRY0NCWFdNRHZTQ1hDa0lCSm9J?= =?utf-8?B?UDdZR1Q0VUdNcERaQ1NlS0JaRFkxcm5LS3MxTmpPQjRaUm94eUk4Wk5GcHRC?= =?utf-8?B?c3FYcDdCVGtyTnVINjk2K21IS1RnVGc2OVBONHQxb1Z2MFV0MmpFTFByUkZR?= =?utf-8?B?aTRvNUZPVEdWdXZza01oMGwvSlhycU5MRHo3U09ka0JHMyt4U1Q0WFdpMUk4?= =?utf-8?B?aHdkL3FsTnhJQmZBcXd6Zm5ZZ2FNenpqcXR1MDd1NTRxcEpHNmhGK243SGhm?= =?utf-8?B?WW5iNzlxS1k5bHUxRXdwVmhsTFNqeDZMZTBRZmlKcHJ1TmNGN25QUDk2aVlp?= =?utf-8?B?Yk9VeWVhY1phZXFaY00zcTJQV1pCT3FOcms2UjFaWUs5ZWNQNnA1c1NaN2xx?= =?utf-8?Q?lQOolbrfH7C0TWGNeOuO2vJQ9?= X-OriginatorOrg: zlab.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 7051c020-3341-4154-1b89-08dc55e492e4 X-MS-Exchange-CrossTenant-AuthSource: OSAPR01MB1619.jpnprd01.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Apr 2024 02:52:18.6408 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 58f17630-b846-437d-ac06-7ecf8a587a66 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xOae3lwLkmb8c3RcU41A5XNxrezr3ACXgd6pLBuEsGtL/M9i+gbNbRbv7syY6ISCjF4xiSzz8wWkUaTPPFXiQg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYWPR01MB9969 Received-SPF: pass client-ip=2a01:111:f403:201a::700; envelope-from=tkusumi@zlab.co.jp; helo=JPN01-OS0-obe.outbound.protection.outlook.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Sat, 06 Apr 2024 02:38:42 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) --------------scVQ1eSWWXgHHmIO5FRVkQAM Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi, I have found a performance issue with the sort command when used on pseudo files with zero size. For instance, sorting `/proc/kallsyms`, as demonstrated below, takes significantly longer than executing with `cat`, generating numerous temporary files. I confirmed this issue on v8.32 as well as on commit 8f3989d in the master branch. $ time cat /proc/kallsyms | sort > /dev/null real 0m0.954s user 0m0.873s sys 0m0.096s $ time sort /proc/kallsyms > /dev/null real 0m8.555s user 0m3.367s sys 0m5.064s $ strace -e trace=openat sort /proc/kallsyms 2>&1 > /dev/null \ | grep /tmp/sort | head -100 ... openat(AT_FDCWD, "/tmp/sortM6Y6Y1", ... openat(AT_FDCWD, "/tmp/sortPrHKMG", ... $ strace -e trace=openat -c sort /proc/kallsyms > /dev/null % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 100.00 6.419777 19 333258 8 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 6.419777 19 333258 8 total It appears that the buffer size allocated for pseudo files with zero size is insufficient, likely because it is based on their file size, which is zero. As seen in the attached patch, I think using `INPUT_FILE_SIZE_GUESS` to calculate the buffer size when the file size is zero would resolve this issue. Best regards, Takashi Kusumi --------------scVQ1eSWWXgHHmIO5FRVkQAM Content-Type: text/plain; charset=UTF-8; name="0001-sort-fix-performance-issue-on-zero-sized-pseudo-file.patch" Content-Disposition: attachment; filename*0="0001-sort-fix-performance-issue-on-zero-sized-pseudo-file.pa"; filename*1="tch" Content-Transfer-Encoding: base64 RnJvbSA5Zjc1OWNjNzIwMTRhYjY2ZGEwZTE0MzE4ZDRhYTBjNzJlOTMxMWQ5IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBUYWthc2hpIEt1c3VtaSA8dGt1c3VtaUB6bGFiLmNvLmpwPgpE YXRlOiBGcmksIDUgQXByIDIwMjQgMTI6MDM6NDIgKzA5MDAKU3ViamVjdDogW1BBVENIXSBzb3J0 OiBmaXggcGVyZm9ybWFuY2UgaXNzdWUgb24gemVyby1zaXplZCBwc2V1ZG8gZmlsZXMKClByZXZp b3VzbHksIGFuIGluc3VmZmljaWVudCBidWZmZXIgc2l6ZSB3YXMgY2hvc2VuIGZvciB6ZXJvLXNp emVkIHBzZXVkbwpmaWxlcyAoZS5nLiwgL3Byb2Mva2FsbHN5bXMpLiBOb3csIHRoZSBidWZmZXIg c2l6ZSBpcyBjYWxjdWxhdGVkIHVzaW5nCklOUFVUX0ZJTEVfU0laRV9HVUVTUyB3aGVuIHRoZSBm aWxlIHNpemUgaXMgemVyby4KLS0tCiBzcmMvc29ydC5jIHwgMiArLQogMSBmaWxlIGNoYW5nZWQs IDEgaW5zZXJ0aW9uKCspLCAxIGRlbGV0aW9uKC0pCgpkaWZmIC0tZ2l0IGEvc3JjL3NvcnQuYyBi L3NyYy9zb3J0LmMKaW5kZXggMzI5ZWQ0NWRjLi44ZDc1N2RhNTUgMTAwNjQ0Ci0tLSBhL3NyYy9z b3J0LmMKKysrIGIvc3JjL3NvcnQuYwpAQCAtMTUzOCw3ICsxNTM4LDcgQEAgc29ydF9idWZmZXJf c2l6ZSAoRklMRSAqY29uc3QgKmZwcywgc2l6ZV90IG5mcHMsCiAgICAgICAgICAgIT0gMCkKICAg ICAgICAgc29ydF9kaWUgKF8oInN0YXQgZmFpbGVkIiksIGZpbGVzW2ldKTsKIAotICAgICAgaWYg KFNfSVNSRUcgKHN0LnN0X21vZGUpKQorICAgICAgaWYgKFNfSVNSRUcgKHN0LnN0X21vZGUpICYm IHN0LnN0X3NpemUgIT0gMCkKICAgICAgICAgZmlsZV9zaXplID0gc3Quc3Rfc2l6ZTsKICAgICAg IGVsc2UKICAgICAgICAgewotLSAKMi40MS4wCgo= --------------scVQ1eSWWXgHHmIO5FRVkQAM-- From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 06:09:24 2024 Received: (at 70231) by debbugs.gnu.org; 6 Apr 2024 10:09:24 +0000 Received: from localhost ([127.0.0.1]:38331 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rt2zC-0001hF-Gg for submit@debbugs.gnu.org; Sat, 06 Apr 2024 06:09:24 -0400 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:55388) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rt2z7-0001g0-Bm for 70231@debbugs.gnu.org; Sat, 06 Apr 2024 06:09:21 -0400 Received: by mail-wr1-x434.google.com with SMTP id ffacd0b85a97d-343eb6cc46bso857017f8f.2 for <70231@debbugs.gnu.org>; Sat, 06 Apr 2024 03:09:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712398145; x=1713002945; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=yQDIbPjqM4XzOW+AgjIPC7zqdqHrX1T38JPEKgPqfFg=; b=TMhU6o2tKkOS85056YzccSDiskk9Fbx/DAXKkpTQCwRPWxV/lAonjez1JEalZVpkk1 FZjU6IEX9kvXExQ3b2PvjfW9oLfaYF9rf+vkUarAnyqRFm6ktzR78pUTnrXKKbFl111H FjcjkfiJQIAWGYebompyDPKuqMBQpqii9eSQZ+1OM3+BL3ZxCIPulxyUpYdt75ef/XR5 8L9tFyu3rLO0zLzRRlFrGQVXEpOIk8sIhHF2HUeZLE8WqyNCkn5e0Txc6n0c7i2drK9o FPdQEnMxMdk5YBSuLegT4XbMpKvR6q8tSTk/FHWm+jW5loYpTfgdnCI58X2yjW14EIGV Zw0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712398145; x=1713002945; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yQDIbPjqM4XzOW+AgjIPC7zqdqHrX1T38JPEKgPqfFg=; b=NNerYlo+SvphObqxb6bUmJ1S53ctNfxvyz74R0wnXnSiB44lABDj3fvxPBUIM8/fXD YbDfnSQmvQKtw6UqfSwV3STPo4YS/3YlXlGF9f4OsXthlaS4r2Z0k3HeYvIGqETJBy7S JLh64UYcNsR+JbB0TC4S+q3VNqYsXZUTO8kbGfiX+8I0ARibn/qD3YTY/o9GDxQQCn5J T7fJQUB7+Iz3UeslwG8qWTp3yKXWGlbTDIU6wj6qMNaV4XayTeYjgbG5xLFPu6MqJJbR Dj/G2VmpGHx+CE+0/jRbSNk5HMoQgQ7pvzTHpkOmukVD3SYihTo8gvvyo6SnV3OBoyL6 u6Ng== X-Forwarded-Encrypted: i=1; AJvYcCUAX79xLLQwYDPxhycWkbSWfF9ta9Rry+CQ9+ima+ahkfXhNfsPF2yuXNqMwoCTSmZW91GF8ig97LCEJ2WoMT7k//n5GrI= X-Gm-Message-State: AOJu0YyIwZBflCIBNxj/0KK7JMNHHG4zB66EJ9yON6SGXdFnWmdDyqnr h2Hjrrz82lUrM/HfBdDc1jRSu2dtVTnFyXP+7k3dpgpLbkSAVbZR8zVsECp8 X-Google-Smtp-Source: AGHT+IHNNIsPz7aTA5X6UO6A0hFjZJdaQTLutu5d7F0G7OPXvDZvJnFy06A77OsgLyfWWGK/kTuakQ== X-Received: by 2002:adf:ef10:0:b0:343:8994:3145 with SMTP id e16-20020adfef10000000b0034389943145mr2756718wro.4.1712398144855; Sat, 06 Apr 2024 03:09:04 -0700 (PDT) Received: from [192.168.1.39] (86-44-211-146-dynamic.agg2.lod.rsl-rtd.eircom.net. [86.44.211.146]) by smtp.googlemail.com with ESMTPSA id d15-20020adfa34f000000b00341c88ab493sm4181144wrb.10.2024.04.06.03.09.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 06 Apr 2024 03:09:04 -0700 (PDT) Message-ID: <84d19933-f17b-f052-a3e0-2e67905b8fc9@draigBrady.com> Date: Sat, 6 Apr 2024 11:09:02 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#70231: Performance issue on sort with zero-sized pseudo files Content-Language: en-US To: Takashi Kusumi , 70231@debbugs.gnu.org References: <5d9a5fa2-e716-4d68-b4fe-8ef58e4e4e0c@zlab.co.jp> From: =?UTF-8?Q?P=C3=A1draig_Brady?= In-Reply-To: <5d9a5fa2-e716-4d68-b4fe-8ef58e4e4e0c@zlab.co.jp> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 70231 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 06/04/2024 03:52, Takashi Kusumi wrote: > Hi, > > I have found a performance issue with the sort command when used on > pseudo files with zero size. For instance, sorting `/proc/kallsyms`, as > demonstrated below, takes significantly longer than executing with > `cat`, generating numerous temporary files. I confirmed this issue on > v8.32 as well as on commit 8f3989d in the master branch. > > $ time cat /proc/kallsyms | sort > /dev/null > real 0m0.954s > user 0m0.873s > sys 0m0.096s > > $ time sort /proc/kallsyms > /dev/null > real 0m8.555s > user 0m3.367s > sys 0m5.064s > > $ strace -e trace=openat sort /proc/kallsyms 2>&1 > /dev/null \ > | grep /tmp/sort | head -100 > ... > openat(AT_FDCWD, "/tmp/sortM6Y6Y1", ... > openat(AT_FDCWD, "/tmp/sortPrHKMG", ... > > $ strace -e trace=openat -c sort /proc/kallsyms > /dev/null > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 100.00 6.419777 19 333258 8 openat > ------ ----------- ----------- --------- --------- ---------------- > 100.00 6.419777 19 333258 8 total > > It appears that the buffer size allocated for pseudo files with zero > size is insufficient, likely because it is based on their file size, > which is zero. As seen in the attached patch, I think using > `INPUT_FILE_SIZE_GUESS` to calculate the buffer size when the file size > is zero would resolve this issue. I'll apply this. BTW we should improve sort buffer handling in general. From my TODO... 0. Have sort --debug output memory buffer sizes and space avail at $TMPDIR(s) 1. auto increase buffer when reading from pipe or zero sized files. This will be more efficient and more importantly enable parallel operation. See http://superuser.com/questions/938558/sort-parallel-isnt-parallelizing/ At least your more appropriate default buffer sizes in this case. I.e. bigger mins and probably smaller maxs as half avail mem is too aggressive. 2. check() should not need full buffer size? only merge buffer size or something small at least. 3. Look at minimizing the amount of mem used by default. Hmm, sort auto adjusts down to avail mem in initbuf() (Test with ulimit -v) 4. Careful with too small buffers as that may initiate an extra merge step (see section above). If anyone wants to look at the above give me a heads up, or I'll get to it sometime in the next release cycle. thanks! Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 18:22:36 2024 Received: (at 70231) by debbugs.gnu.org; 6 Apr 2024 22:22:36 +0000 Received: from localhost ([127.0.0.1]:41215 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rtEQm-0006Ud-0W for submit@debbugs.gnu.org; Sat, 06 Apr 2024 18:22:36 -0400 Received: from mail.cs.ucla.edu ([131.179.128.66]:58810) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rtEQh-0006UL-2V for 70231@debbugs.gnu.org; Sat, 06 Apr 2024 18:22:34 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 52F0C3C011BDD; Sat, 6 Apr 2024 15:22:18 -0700 (PDT) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavis, port 10032) with ESMTP id BEcofzUQVLNG; Sat, 6 Apr 2024 15:22:17 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id A38943C011BDE; Sat, 6 Apr 2024 15:22:17 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu A38943C011BDE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1712442137; bh=flDyk8JAZcMA1JomWYyjpA8Wv2xnvjFp/fEGAjDGGns=; h=Message-ID:Date:MIME-Version:To:From; b=BI0jjfjvtc5OTEvqbF5rJLOJhrze0vb2sX+kXxLSCSi9R8EwBNnj8w1Sm32TSRaEz n5bVnthqb1IGkVkFV2kP8Y+zfRIgFTyZFTSSp4TaCjdJ/rIOCD6clq2erEytzgwZJ9 xba7IdPdDGwu49jr2cx7zOthqGk0iCn2K+b67EDO904XN+USHOD8hv80M/HFQex+Uq 6sfojRnGhS0mufrs/plHjN82Q6fuaE2c/z+85etV2mPG7I+VxEQAc0K3bpTwEoP8h3 TTz1ay3nnHx1qEtt1tiBsZx/S75VCNZlUbn9+lrTOSldpNzfZ5cX98Agvld1WJ3QBA S19DKML6RPNKA== X-Virus-Scanned: amavis at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavis, port 10026) with ESMTP id bUou80nZLzx9; Sat, 6 Apr 2024 15:22:17 -0700 (PDT) Received: from [192.168.254.12] (unknown [47.154.17.165]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id 7A0663C011BDD; Sat, 6 Apr 2024 15:22:17 -0700 (PDT) Content-Type: multipart/mixed; boundary="------------dY3qJdOk63prc4xW12tfOlZV" Message-ID: <7b7f1e6f-633b-4851-927d-f77ac6a68c4b@cs.ucla.edu> Date: Sat, 6 Apr 2024 15:22:17 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#70231: Performance issue on sort with zero-sized pseudo files To: =?UTF-8?Q?P=C3=A1draig_Brady?= , Takashi Kusumi , 70231@debbugs.gnu.org References: <5d9a5fa2-e716-4d68-b4fe-8ef58e4e4e0c@zlab.co.jp> <84d19933-f17b-f052-a3e0-2e67905b8fc9@draigBrady.com> Content-Language: en-US From: Paul Eggert Organization: UCLA Computer Science Department In-Reply-To: <84d19933-f17b-f052-a3e0-2e67905b8fc9@draigBrady.com> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 70231 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) This is a multi-part message in MIME format. --------------dY3qJdOk63prc4xW12tfOlZV Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable On 2024-04-06 03:09, P=C3=A1draig Brady wrote: > I'll apply this. Heh, I beat you to it by looking for similar errors elsewhere and=20 applying the attached patches to fix the issues I found. None of them=20 look like serious bugs. > BTW we should improve sort buffer handling in general Oh yes. PS. My current little task is to get i18n to work better with 'sort'.=20 Among other things I want Unicode-style full case folding. --------------dY3qJdOk63prc4xW12tfOlZV Content-Type: text/x-patch; charset=UTF-8; name="0001-cat-don-t-trust-st_size-on-proc-files.patch" Content-Disposition: attachment; filename="0001-cat-don-t-trust-st_size-on-proc-files.patch" Content-Transfer-Encoding: base64 RnJvbSAyMjVjYjhkNzQ3M2VhZGI0ODFhNDg4NGU5MjliZjIzNTg5ZDRiZDgyIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBTYXQsIDYgQXByIDIwMjQgMTU6MTM6MjMgLTA3MDAKU3ViamVjdDogW1BBVENI IDEvNF0gPT9VVEYtOD9xP2NhdDo9MjBkb249RTI9ODA9OTl0PTIwdHJ1c3Q9MjBzdD01RnNp emU9MjA/PQogPT9VVEYtOD9xP29uPTIwL3Byb2M9MjBmaWxlcz89Ck1JTUUtVmVyc2lvbjog MS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD1VVEYtOApDb250ZW50LVRy YW5zZmVyLUVuY29kaW5nOiA4Yml0CgoqIHNyYy9jYXQuYyAobWFpbik6CkltcHJvdmUgdGVz dCBmb3Igd2hlbiBjb3B5aW5nIHdpbGwgZXhoYXVzdCB0aGUgb3V0cHV0IGRldmljZS4KRG8g bm90IHJlbHkgb24gc3Rfc2l6ZSwgd2hpY2ggaXMgdW5yZWxpYWJsZSBpbiAvcHJvYy4KVXNl IGxzZWVrIGluc3RlYWQ7IHRoaXMgaXMgZ29vZCBlbm91Z2ggaGVyZS4KKiB0ZXN0cy9jYXQv Y2F0LXNlbGYuc2g6IFRlc3QgdGhlIHJlbGF4YXRpb24gb2YgdGhlIGhldXJpc3RpYwpmb3Ig c2VsZi1jb3B5aW5nLgotLS0KIHNyYy9jYXQuYyAgICAgICAgICAgICB8IDMxICsrKysrKysr KysrKysrKysrKysrKy0tLS0tLS0tLS0KIHRlc3RzL2NhdC9jYXQtc2VsZi5zaCB8IDIwICsr KysrKysrKysrKysrKysrKysrCiAyIGZpbGVzIGNoYW5nZWQsIDQxIGluc2VydGlvbnMoKyks IDEwIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3NyYy9jYXQuYyBiL3NyYy9jYXQuYwpp bmRleCA0ZWQ0MDQzNjMuLmIzM2ZhZWIzNSAxMDA2NDQKLS0tIGEvc3JjL2NhdC5jCisrKyBi L3NyYy9jYXQuYwpAQCAtNjQ1LDkgKzY0NSwxMCBAQCBtYWluIChpbnQgYXJnYywgY2hhciAq KmFyZ3YpCiAgIC8qIE9wdGltYWwgc2l6ZSBvZiBpL28gb3BlcmF0aW9ucyBvZiBvdXRwdXQu ICAqLwogICBpZHhfdCBvdXRzaXplID0gaW9fYmxrc2l6ZSAoJnN0YXRfYnVmKTsKIAotICAv KiBEZXZpY2UgYW5kIEktbm9kZSBudW1iZXIgb2YgdGhlIG91dHB1dC4gICovCisgIC8qIERl dmljZSwgSS1ub2RlIG51bWJlciBhbmQgbGF6aWx5LWFjcXVpcmVkIGZsYWdzIG9mIHRoZSBv dXRwdXQuICAqLwogICBkZXZfdCBvdXRfZGV2ID0gc3RhdF9idWYuc3RfZGV2OwogICBpbm9f dCBvdXRfaW5vID0gc3RhdF9idWYuc3RfaW5vOworICBpbnQgb3V0X2ZsYWdzID0gLTI7CiAK ICAgLyogVHJ1ZSBpZiB0aGUgb3V0cHV0IGlzIGEgcmVndWxhciBmaWxlLiAgKi8KICAgYm9v bCBvdXRfaXNyZWcgPSBTX0lTUkVHIChzdGF0X2J1Zi5zdF9tb2RlKSAhPSAwOwpAQCAtNzAx LDE3ICs3MDIsMjcgQEAgbWFpbiAoaW50IGFyZ2MsIGNoYXIgKiphcmd2KQogCiAgICAgICBm ZGFkdmlzZSAoaW5wdXRfZGVzYywgMCwgMCwgRkFEVklTRV9TRVFVRU5USUFMKTsKIAotICAg ICAgLyogRG9uJ3QgY29weSBhIG5vbmVtcHR5IHJlZ3VsYXIgZmlsZSB0byBpdHNlbGYsIGFz IHRoYXQgd291bGQKLSAgICAgICAgIG1lcmVseSBleGhhdXN0IHRoZSBvdXRwdXQgZGV2aWNl LiAgSXQncyBiZXR0ZXIgdG8gY2F0Y2ggdGhpcwotICAgICAgICAgZXJyb3IgZWFybGllciBy YXRoZXIgdGhhbiBsYXRlci4gICovCisgICAgICAvKiBEb24ndCBjb3B5IGEgZmlsZSB0byBp dHNlbGYgaWYgdGhhdCB3b3VsZCBtZXJlbHkgZXhoYXVzdCB0aGUKKyAgICAgICAgIG91dHB1 dCBkZXZpY2UuICBJdCdzIGJldHRlciB0byBjYXRjaCB0aGlzIGVycm9yIGVhcmxpZXIKKyAg ICAgICAgIHJhdGhlciB0aGFuIGxhdGVyLiAgKi8KIAotICAgICAgaWYgKG91dF9pc3JlZwot ICAgICAgICAgICYmIHN0YXRfYnVmLnN0X2RldiA9PSBvdXRfZGV2ICYmIHN0YXRfYnVmLnN0 X2lubyA9PSBvdXRfaW5vCi0gICAgICAgICAgJiYgbHNlZWsgKGlucHV0X2Rlc2MsIDAsIFNF RUtfQ1VSKSA8IHN0YXRfYnVmLnN0X3NpemUpCisgICAgICBpZiAoc3RhdF9idWYuc3RfZGV2 ID09IG91dF9kZXYgJiYgc3RhdF9idWYuc3RfaW5vID09IG91dF9pbm8pCiAgICAgICAgIHsK LSAgICAgICAgICBlcnJvciAoMCwgMCwgXygiJXM6IGlucHV0IGZpbGUgaXMgb3V0cHV0IGZp bGUiKSwgcXVvdGVmIChpbmZpbGUpKTsKLSAgICAgICAgICBvayA9IGZhbHNlOwotICAgICAg ICAgIGdvdG8gY29udGluOworICAgICAgICAgIGlmIChvdXRfZmxhZ3MgPCAtMSkKKyAgICAg ICAgICAgIG91dF9mbGFncyA9IGZjbnRsIChTVERPVVRfRklMRU5PLCBGX0dFVEZMKTsKKyAg ICAgICAgICBib29sIGV4aGF1c3RpbmcgPSAwIDw9IG91dF9mbGFncyAmJiBvdXRfZmxhZ3Mg JiBPX0FQUEVORDsKKyAgICAgICAgICBpZiAoIWV4aGF1c3RpbmcpCisgICAgICAgICAgICB7 CisgICAgICAgICAgICAgIG9mZl90IGluX3BvcyA9IGxzZWVrIChpbnB1dF9kZXNjLCAwLCBT RUVLX0NVUik7CisgICAgICAgICAgICAgIGlmICgwIDw9IGluX3BvcykKKyAgICAgICAgICAg ICAgICBleGhhdXN0aW5nID0gaW5fcG9zIDwgbHNlZWsgKFNURE9VVF9GSUxFTk8sIDAsIFNF RUtfQ1VSKTsKKyAgICAgICAgICAgIH0KKyAgICAgICAgICBpZiAoZXhoYXVzdGluZykKKyAg ICAgICAgICAgIHsKKyAgICAgICAgICAgICAgZXJyb3IgKDAsIDAsIF8oIiVzOiBpbnB1dCBm aWxlIGlzIG91dHB1dCBmaWxlIiksIHF1b3RlZiAoaW5maWxlKSk7CisgICAgICAgICAgICAg IG9rID0gZmFsc2U7CisgICAgICAgICAgICAgIGdvdG8gY29udGluOworICAgICAgICAgICAg fQogICAgICAgICB9CiAKICAgICAgIC8qIFBvaW50ZXIgdG8gdGhlIGlucHV0IGJ1ZmZlci4g ICovCmRpZmYgLS1naXQgYS90ZXN0cy9jYXQvY2F0LXNlbGYuc2ggYi90ZXN0cy9jYXQvY2F0 LXNlbGYuc2gKaW5kZXggZTBmNjQ1NWMwLi44NTQ4MjVkZWYgMTAwNzU1Ci0tLSBhL3Rlc3Rz L2NhdC9jYXQtc2VsZi5zaAorKysgYi90ZXN0cy9jYXQvY2F0LXNlbGYuc2gKQEAgLTMwLDQg KzMwLDI0IEBAIGVjaG8geSA+ZG9jLmVuZCB8fCBmcmFtZXdvcmtfZmFpbHVyZV8KIGNhdCBk b2MgZG9jLmVuZCA+ZG9jIHx8IGZhaWw9MQogY29tcGFyZSBkb2MgZG9jLmVuZCB8fCBmYWls PTEKIAorIyBUaGlzIHRlcm1pbmF0ZXMgZXZlbiB0aG91Z2ggaXQgY29waWVzIGEgZmlsZSB0 byBpdHNlbGYuCisjIENvcmV1dGlscyA5LjUgYW5kIGVhcmxpZXIgcmVqZWN0ZWQgdGhpcy4K K2VjaG8geCA+ZnggfHwgZnJhbWV3b3JrX2ZhaWx1cmVfCitlY2hvIHkgPmZ5IHx8IGZyYW1l d29ya19mYWlsdXJlXworY2F0IGZ4IGZ5ID5meHkgfHwgZmFpbD0xCitmb3IgaSBpbiAxIDI7 IGRvCisgIGNhdCBmeCA+Znh5JGkgfHwgZmFpbD0xCitkb25lCitmb3IgaSBpbiAzIDQgNSA2 OyBkbworICBjYXQgZnggPmZ4JGkgfHwgZmFpbD0xCitkb25lCitjYXQgLSBmeSA8Znh5MSAx PD5meHkxIHx8IGZhaWw9MQorY29tcGFyZSBmeHkgZnh5MSB8fCBmYWlsPTEKK2NhdCBmeHky IGZ5IDE8PmZ4eTIgfHwgZmFpbD0xCitjb21wYXJlIGZ4eSBmeHkyIHx8IGZhaWw9MQorcmV0 dXJuc18gMSBjYXQgZnggZngzIDE8PmZ4MyB8fCBmYWlsPTEKK3JldHVybnNfIDEgY2F0IC0g Zng0IDxmeCAxPD5meDQgfHwgZmFpbD0xCityZXR1cm5zXyAxIGNhdCBmeDUgPj5meDUgfHwg ZmFpbD0xCityZXR1cm5zXyAxIGNhdCA8Zng2ID4+Zng2IHx8IGZhaWw9MQorCiBFeGl0ICRm YWlsCi0tIAoyLjQwLjEKCg== --------------dY3qJdOk63prc4xW12tfOlZV Content-Type: text/x-patch; charset=UTF-8; name="0002-dd-don-t-trust-st_size-on-proc-files.patch" Content-Disposition: attachment; filename="0002-dd-don-t-trust-st_size-on-proc-files.patch" Content-Transfer-Encoding: base64 RnJvbSBhYzZiOGQ4MjI0ZGUxNDBmNWE2ZjJjYTY2ZTZjZTI3OTYwNGEzN2U2IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBTYXQsIDYgQXByIDIwMjQgMTU6MTU6MDQgLTA3MDAKU3ViamVjdDogW1BBVENI IDIvNF0gPT9VVEYtOD9xP2RkOj0yMGRvbj1FMj04MD05OXQ9MjB0cnVzdD0yMHN0PTVGc2l6 ZT0yMG8/PQogPT9VVEYtOD9xP249MjAvcHJvYy9maWxlcz89Ck1JTUUtVmVyc2lvbjogMS4w CkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD1VVEYtOApDb250ZW50LVRyYW5z ZmVyLUVuY29kaW5nOiA4Yml0CgoqIHNyYy9kZC5jIChza2lwKTogRG9u4oCZdCB0cnVzdCBz dF9zaXplID09IDAuCi0tLQogc3JjL2RkLmMgfCAyICstCiAxIGZpbGUgY2hhbmdlZCwgMSBp bnNlcnRpb24oKyksIDEgZGVsZXRpb24oLSkKCmRpZmYgLS1naXQgYS9zcmMvZGQuYyBiL3Ny Yy9kZC5jCmluZGV4IDM3MGE0MmM5Ny4uNmUwZDcyNWRhIDEwMDY0NAotLS0gYS9zcmMvZGQu YworKysgYi9zcmMvZGQuYwpAQCAtMTgwOSw3ICsxODA5LDcgQEAgc2tpcCAoaW50IGZkZXNj LCBjaGFyIGNvbnN0ICpmaWxlLCBpbnRtYXhfdCByZWNvcmRzLCBpZHhfdCBibG9ja3NpemUs CiAgICAgICAgICAgIHN0cnVjdCBzdGF0IHN0OwogICAgICAgICAgICBpZiAoaWZzdGF0IChT VERJTl9GSUxFTk8sICZzdCkgIT0gMCkKICAgICAgICAgICAgICBlcnJvciAoRVhJVF9GQUlM VVJFLCBlcnJubywgXygiY2Fubm90IGZzdGF0ICVzIiksIHF1b3RlYWYgKGZpbGUpKTsKLSAg ICAgICAgICAgaWYgKHVzYWJsZV9zdF9zaXplICgmc3QpICYmIDAgPD0gaW5wdXRfb2Zmc2V0 CisgICAgICAgICAgIGlmICh1c2FibGVfc3Rfc2l6ZSAoJnN0KSAmJiAwIDwgc3Quc3Rfc2l6 ZSAmJiAwIDw9IGlucHV0X29mZnNldAogICAgICAgICAgICAgICAgJiYgc3Quc3Rfc2l6ZSAt IGlucHV0X29mZnNldCA8IG9mZnNldCkKICAgICAgICAgICAgICB7CiAgICAgICAgICAgICAg ICAvKiBXaGVuIHNraXBwaW5nIHBhc3QgRU9GLCByZXR1cm4gdGhlIG51bWJlciBvZiBfZnVs bF8gYmxvY2tzCi0tIAoyLjQwLjEKCg== --------------dY3qJdOk63prc4xW12tfOlZV Content-Type: text/x-patch; charset=UTF-8; name="0003-sort-don-t-trust-st_size-on-proc-files.patch" Content-Disposition: attachment; filename="0003-sort-don-t-trust-st_size-on-proc-files.patch" Content-Transfer-Encoding: base64 RnJvbSA4ZmYzOTAzMjgxZTAzZDM2ZGQxYWEyYTIwMmE1NmYzOGFmNzI2ZDkxIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBTYXQsIDYgQXByIDIwMjQgMTU6MTc6MTQgLTA3MDAKU3ViamVjdDogW1BBVENI IDMvNF0gPT9VVEYtOD9xP3NvcnQ6PTIwZG9uPUUyPTgwPTk5dD0yMHRydXN0PTIwc3Q9NUZz aXplPz0KID0/VVRGLTg/cT89MjBvbj0yMC9wcm9jPTIwZmlsZXM/PQpNSU1FLVZlcnNpb246 IDEuMApDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9VVRGLTgKQ29udGVudC1U cmFuc2Zlci1FbmNvZGluZzogOGJpdAoKUHJvYmxlbSBhbmQgZml4IHJlcG9ydGVkIGJ5IFRh a2FzaGkgS3VzdW1pIGluOgpodHRwczovL2J1Z3MuZ251Lm9yZy83MDIzMQoqIHNyYy9zb3J0 LmMgKHNvcnRfYnVmZmVyX3NpemUpOiBEb27igJl0IHRydXN0IHN0X3NpemUgPT0gMC4KLS0t CiBzcmMvc29ydC5jIHwgMiArLQogMSBmaWxlIGNoYW5nZWQsIDEgaW5zZXJ0aW9uKCspLCAx IGRlbGV0aW9uKC0pCgpkaWZmIC0tZ2l0IGEvc3JjL3NvcnQuYyBiL3NyYy9zb3J0LmMKaW5k ZXggMmQ4MzI0Y2E0Li43ODk4M2ZmMjcgMTAwNjQ0Ci0tLSBhL3NyYy9zb3J0LmMKKysrIGIv c3JjL3NvcnQuYwpAQCAtMTUzOSw3ICsxNTM5LDcgQEAgc29ydF9idWZmZXJfc2l6ZSAoRklM RSAqY29uc3QgKmZwcywgc2l6ZV90IG5mcHMsCiAgICAgICAgICAgIT0gMCkKICAgICAgICAg c29ydF9kaWUgKF8oInN0YXQgZmFpbGVkIiksIGZpbGVzW2ldKTsKIAotICAgICAgaWYgKFNf SVNSRUcgKHN0LnN0X21vZGUpKQorICAgICAgaWYgKHVzYWJsZV9zdF9zaXplICgmc3QpICYm IDAgPCBzdC5zdF9zaXplKQogICAgICAgICBmaWxlX3NpemUgPSBzdC5zdF9zaXplOwogICAg ICAgZWxzZQogICAgICAgICB7Ci0tIAoyLjQwLjEKCg== --------------dY3qJdOk63prc4xW12tfOlZV Content-Type: text/x-patch; charset=UTF-8; name="0004-split-don-t-trust-st_size-on-proc-files.patch" Content-Disposition: attachment; filename="0004-split-don-t-trust-st_size-on-proc-files.patch" Content-Transfer-Encoding: base64 RnJvbSAxMTE2MzY3NTgxOGFiODc3ZjIwZDM3NDBhN2MzZTU5ZDU2NWI4ZTljIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBTYXQsIDYgQXByIDIwMjQgMTU6MTg6MDQgLTA3MDAKU3ViamVjdDogW1BBVENI IDQvNF0gPT9VVEYtOD9xP3NwbGl0Oj0yMGRvbj1FMj04MD05OXQ9MjB0cnVzdD0yMHN0PTVG c2l6ZT89CiA9P1VURi04P3E/PTIwb249MjAvcHJvYz0yMGZpbGVzPz0KTUlNRS1WZXJzaW9u OiAxLjAKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0PVVURi04CkNvbnRlbnQt VHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQKCiogc3JjL3NwbGl0LmMgKGNyZWF0ZSk6IERvbuKA mXQgdHJ1c3Qgc3Rfc2l6ZSA9PSAwLgotLS0KIHNyYy9zcGxpdC5jIHwgNiArKy0tLS0KIDEg ZmlsZSBjaGFuZ2VkLCAyIGluc2VydGlvbnMoKyksIDQgZGVsZXRpb25zKC0pCgpkaWZmIC0t Z2l0IGEvc3JjL3NwbGl0LmMgYi9zcmMvc3BsaXQuYwppbmRleCAwMzc5NjBhNTkuLmY4MmE3 Zjc0YiAxMDA2NDQKLS0tIGEvc3JjL3NwbGl0LmMKKysrIGIvc3JjL3NwbGl0LmMKQEAgLTQ4 OSwxMCArNDg5LDggQEAgY3JlYXRlIChjaGFyIGNvbnN0ICpuYW1lKQogICAgICAgaWYgKHBz YW1lX2lub2RlICgmaW5fc3RhdF9idWYsICZvdXRfc3RhdF9idWYpKQogICAgICAgICBlcnJv ciAoRVhJVF9GQUlMVVJFLCAwLCBfKCIlcyB3b3VsZCBvdmVyd3JpdGUgaW5wdXQ7IGFib3J0 aW5nIiksCiAgICAgICAgICAgICAgICBxdW90ZWFmIChuYW1lKSk7Ci0gICAgICBib29sIHJl Z3VsYXJpc2gKLSAgICAgICAgPSBTX0lTUkVHIChvdXRfc3RhdF9idWYuc3RfbW9kZSkgfHwg U19UWVBFSVNTSE0gKCZvdXRfc3RhdF9idWYpOwotICAgICAgaWYgKCEgKHJlZ3VsYXJpc2gg JiYgb3V0X3N0YXRfYnVmLnN0X3NpemUgPT0gMCkKLSAgICAgICAgICAmJiBmdHJ1bmNhdGUg KGZkLCAwKSA8IDAgJiYgcmVndWxhcmlzaCkKKyAgICAgIGlmIChmdHJ1bmNhdGUgKGZkLCAw KSA8IDAKKyAgICAgICAgICAmJiAoU19JU1JFRyAob3V0X3N0YXRfYnVmLnN0X21vZGUpIHx8 IFNfVFlQRUlTU0hNICgmb3V0X3N0YXRfYnVmKSkpCiAgICAgICAgIGVycm9yIChFWElUX0ZB SUxVUkUsIGVycm5vLCBfKCIlczogZXJyb3IgdHJ1bmNhdGluZyIpLCBxdW90ZWYgKG5hbWUp KTsKIAogICAgICAgcmV0dXJuIGZkOwotLSAKMi40MC4xCgo= --------------dY3qJdOk63prc4xW12tfOlZV-- From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 07 08:56:00 2024 Received: (at 70231-done) by debbugs.gnu.org; 7 Apr 2024 12:56:00 +0000 Received: from localhost ([127.0.0.1]:41813 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rtS3z-0007KX-Eu for submit@debbugs.gnu.org; Sun, 07 Apr 2024 08:55:59 -0400 Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]:52375) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rtS3v-0007J5-Kz for 70231-done@debbugs.gnu.org; Sun, 07 Apr 2024 08:55:57 -0400 Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-4162b8cb3e9so31112265e9.0 for <70231-done@debbugs.gnu.org>; Sun, 07 Apr 2024 05:55:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712494543; x=1713099343; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=SVlZlD1cNak3D/Mz29dtOBWKiyDmzcjLSjAQfArO3Rk=; b=IZljnBtdSkR15tVwZ7vFKFCPLrLzjZC3qTYcK8XINkJbnziu7qjGP3kKh2IDqcXoVg qDnroileND6zmr0//Uzeh2kjeMahk9SsB8SNd4ZVueqE7qfAI7PGE7jnnnEtV2/Ahdgv fad1kSSknzjoqXmotwu89U42HR+t64FMAVU+0OAVDaSjvHluoK0X5qgJqSSHaGAInnfz l7wmYjAxw+gPR42v8e6TLdaKeMjgBKp/ydM0DsX+HqdkbfCDkLuOp+onDJHGfUETHoLx SA5qhSRj4yc2GJpdZJ3lxvudSUbnl9anllYAlYTqOLSs3twJrkpUcyg5ObOfSIZ2wdeh 746g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712494543; x=1713099343; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SVlZlD1cNak3D/Mz29dtOBWKiyDmzcjLSjAQfArO3Rk=; b=HOBmDIAJHhL4niEXvOfgu688iv4oHZMs1vkq7yCLg60SGUg5HJwaaIjc8ZzcRkYwvl mUCL6rj48wZpM8I01iMqERnXDVQMCKA2scIbw88XYGDEeCHk+5nD9iuZe6YPKE6zwxd5 ercxDpbzkciMX3i4sXVHUMSKfA3pJRcJe8+aqnnYEg3CF5dcVwgrARZO4rub5cuLBIPH txhTk5U+UbMdo0ytvb/0HgPIaG3nE4NK9AMAz8CM7P0H3eHpxw+o0cfC+kMkrdn1fBBt r4OqfRsJSiw57iKehv4isAcg2HrEGd79x9EzQKRWbp9vS6dFlBXGYR698V/7mKLRat6C Fsgw== X-Forwarded-Encrypted: i=1; AJvYcCXjSz9yRFMoc9+4+PTHVk+QR1VBOjtB3WazTUhtIhBsF3eLf1swTcQLas+wxTgcFfESxbEis3waU+e1Qbv0iwaWeQA8n9qFYTyrtA== X-Gm-Message-State: AOJu0YzlFEdtfBvC8eRqQgNlg3hTql5vdU/Boe8wAAhNLhtgyEM4EuIz B3LtvdTVYbu8Sgt02ojYo3bPtbzZGd4/CPcH+2IvXRQw8DrZn0EN X-Google-Smtp-Source: AGHT+IG/zLtvP1jhzJ/M3tuj+l4eTqPz+aM9/mv6hYBGCaUQIr/K1iiodT4Qoidopv8YwlCnsUXzDA== X-Received: by 2002:a7b:c44e:0:b0:415:6860:eafb with SMTP id l14-20020a7bc44e000000b004156860eafbmr6147819wmi.15.1712494542636; Sun, 07 Apr 2024 05:55:42 -0700 (PDT) Received: from [192.168.1.39] (86-44-211-146-dynamic.agg2.lod.rsl-rtd.eircom.net. [86.44.211.146]) by smtp.googlemail.com with ESMTPSA id iv9-20020a05600c548900b004162ce49719sm10770745wmb.6.2024.04.07.05.55.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 07 Apr 2024 05:55:42 -0700 (PDT) Message-ID: <4f0a25da-ad79-33bf-85f3-e88da627e0da@draigBrady.com> Date: Sun, 7 Apr 2024 13:55:40 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#70231: Performance issue on sort with zero-sized pseudo files Content-Language: en-US To: Paul Eggert , Takashi Kusumi , 70231-done@debbugs.gnu.org References: <5d9a5fa2-e716-4d68-b4fe-8ef58e4e4e0c@zlab.co.jp> <84d19933-f17b-f052-a3e0-2e67905b8fc9@draigBrady.com> <7b7f1e6f-633b-4851-927d-f77ac6a68c4b@cs.ucla.edu> From: =?UTF-8?Q?P=C3=A1draig_Brady?= In-Reply-To: <7b7f1e6f-633b-4851-927d-f77ac6a68c4b@cs.ucla.edu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 70231-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 06/04/2024 23:22, Paul Eggert wrote: > On 2024-04-06 03:09, Pádraig Brady wrote: >> I'll apply this. > > Heh, I beat you to it by looking for similar errors elsewhere and > applying the attached patches to fix the issues I found. None of them > look like serious bugs. Cool. I thought the sort(1) change worthy of a NEWS entry so pushed one. Marking this as done. >> BTW we should improve sort buffer handling in general > > Oh yes. > > PS. My current little task is to get i18n to work better with 'sort'. > Among other things I want Unicode-style full case folding. Excellent, that will help keep the related uniq(1) and join(1) commands more aligned in their ordering. cheers, Pádraig From unknown Sat Jun 14 19:13:48 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 06 May 2024 11:24:08 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator