From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 02 15:34:28 2021 Received: (at submit) by debbugs.gnu.org; 2 Mar 2021 20:34:28 +0000 Received: from localhost ([127.0.0.1]:54332 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHBiu-0004Yz-I9 for submit@debbugs.gnu.org; Tue, 02 Mar 2021 15:34:28 -0500 Received: from lists.gnu.org ([209.51.188.17]:54260) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHBiq-0004Yn-Kt for submit@debbugs.gnu.org; Tue, 02 Mar 2021 15:34:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42560) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lHBiq-0006PU-Du for bug-gnu-emacs@gnu.org; Tue, 02 Mar 2021 15:34:24 -0500 Received: from mail-ot1-x32f.google.com ([2607:f8b0:4864:20::32f]:37839) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lHBil-0005Q0-Ss for bug-gnu-emacs@gnu.org; Tue, 02 Mar 2021 15:34:24 -0500 Received: by mail-ot1-x32f.google.com with SMTP id g8so17697918otk.4 for ; Tue, 02 Mar 2021 12:34:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=5b9IwmcF3bL0NiSEoxEpASmYQ/yzhyOmJN0qSxpQJWw=; b=BJfWl/Ra5v1psfnx4Oys7EkG28sHuSAeXhkjotuARlPH/u8DdiCBDefCVyCKH6C14q /1r0sPKDWZA7SO7qHmhqeLhwSAC/EdTwFqNU79W1C6K9d7fntEuN8280SvFJroTq+Ufh mKGj2fXAbRz1dqbFvHd4gsPQOtk/5sB6XBDtiqEP7FBW2KeaZEZ5X1jJ951NsMjv3+tg 5LJEAK92qK6cgotj8G36e8X6ifD0pskYVJPMu3isMOFMK4srPigyveO9k+ro1D5h6rzx jDKGq0pX84AGVf5DZIuVzbQ7mndEgLilSZuw1pVd7KTOP1g37sgp/tTrZDVGr5S+x3mv i89Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=5b9IwmcF3bL0NiSEoxEpASmYQ/yzhyOmJN0qSxpQJWw=; b=N1rpogs53ZQPaSMwnYCxtSARWfMh+01gVHsaVNmf8qGAohoyk6Ze0gnGpz3Ob7rJuo tUDHpnkO1TduGPFqSfT1Od1JhEbX1mQX2VzFxiejAiUyXYyQbMdGQKAuHSXEwYz0aFMz QqEegOYYABFy56EmKZTDqEiP1pDSf2tagU5iGmuLmH0jmX+lJUvBnmD/Xy9EaJJQqbdy 2UfpYeL5ZrP0QgdmGmebvQZLl6BLsHRu277eL+QICsBdf2pJ4Ea0Nm1fWwsJ22kX3xhp 2IQcBLzmAuOB9YtqOijSD8ssCi4iCGtf10nLiGrXg8BR3AcgO+3V91mNQ1Zj076Sai6P aW0Q== X-Gm-Message-State: AOAM531+fU3Al4eXJLePou897P2Vw3ulkOaHNNcp7I89VjpID7qDNfCV 2qWq69Kl2o9v8BxJM7PjXwAs5xQciaZauBXxhlVpiRZAvxw= X-Google-Smtp-Source: ABdhPJx0ANq9SgORjfvFY9p6A0UFIA+L4KtWywysakl68tnr180nyHLuZN6ottAsi2/tK8xtrFAgfiPUKjnAcH0vB40= X-Received: by 2002:a9d:131:: with SMTP id 46mr19269769otu.287.1614717258619; Tue, 02 Mar 2021 12:34:18 -0800 (PST) MIME-Version: 1.0 From: Pip Cet Date: Tue, 2 Mar 2021 20:33:42 +0000 Message-ID: Subject: 28.0.50; pdumper dumping causes way too many syscalls To: bug-gnu-emacs@gnu.org Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::32f; envelope-from=pipcet@gmail.com; helo=mail-ot1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Playing around with the WebAssembly port, I noticed that pdumper, in creating the dump file, makes way too many syscalls: it uses emacs_write(), not fwrite(), so these calls translate to actual syscalls and context switches. On immature systems (or in special circumstances like a device mounted synchronously), they might actually cause a hardware write for each syscall, which would wear out flash quickly and be generally wasteful. I've looked into the problem, and it seems easy to solve and worth it in terms of debuggability and performance. Patch will be attached once this has a bug number. From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 02 15:45:48 2021 Received: (at 46881) by debbugs.gnu.org; 2 Mar 2021 20:45:48 +0000 Received: from localhost ([127.0.0.1]:54341 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHBtr-0004p9-VY for submit@debbugs.gnu.org; Tue, 02 Mar 2021 15:45:48 -0500 Received: from mail-oi1-f174.google.com ([209.85.167.174]:40507) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHBtq-0004ow-Bk for 46881@debbugs.gnu.org; Tue, 02 Mar 2021 15:45:46 -0500 Received: by mail-oi1-f174.google.com with SMTP id 21so13828146oiq.7 for <46881@debbugs.gnu.org>; Tue, 02 Mar 2021 12:45:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=WCycNmA77HpRI9xt4Gr+dg728tCYASb74Wtzho97ICc=; b=pY2GEpx8PsbbHj3XYOjuSr9K6B9Ds6q+sL6N5aJECRVlNu2QxH0d3JZl8YNnqFMVp3 8JLQu7UL74Xu4DJf+NkvURNCcOpM2xfd+y0GL8PFgHdiyL3oKfjXSd1sgVKlePlQvpJq YYdWCoeK9iFehPo+/Q4oiqsdM/OiQ3FKmcgpXKtuwPKphMRxQyGpesymBeao/f4omWXx 6AigW0a6U1qC7RvR/sP41ghmOIcq1ctwEpM1muFuaUQjyD/jNqp9TxMC9wg0biEn+YiR Nc/frP/Pv/dMJJEB75hcY+U1BMQU97vRjDY07OlDi0DaIadYcPlwsmp1sGtL3duaWTo/ knaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=WCycNmA77HpRI9xt4Gr+dg728tCYASb74Wtzho97ICc=; b=LmaDAh1DRZk+bjHXa3gXNz0MZFj1d4CKeG6IF5aU91jNpIm8KC0cwu83FVXSSAl9as 1tXwDW//hpzT0Y6t8TWMMLg4yUsYdgfBl8YHxjd0wok6siErFe+9yzjix0BilEc2mIq4 46UoF0ChVkgaDoAZ1MwuytJjVna+7teIQF1ScCD9Qb6qdY5pQnfeqw7J7XF8R3ncN3u5 xGw31Rj5MqnXJ7zyekJFJtgaMitQ3PTnCIVcbURxJIz5tjVKi98jYkb/ptifkrPFTScX jILr5IK8KJLTmrh/qtWVrvVW0K40eoQemfJV2OgBO1k7XSyGJmqcIJgUTLOZ8To81SpU q91w== X-Gm-Message-State: AOAM532kk53TTkeAOBcHlkbwxycyV9/SmchxhZCXBE3vczRaYDV3yRLW EEut2ORCJXqQy+JJtBJQKz1L57zTTaL53U34T3K/xj9wruN5Lw== X-Google-Smtp-Source: ABdhPJydEwKrvPLuo+xN+t+YFTvZU+QoELiX6iK00P1J+r5SwwAc0CDo08g4BfMZ3hz2TvenDR+jasyaUynu6uMXQUA= X-Received: by 2002:aca:d905:: with SMTP id q5mr1111434oig.30.1614717940575; Tue, 02 Mar 2021 12:45:40 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Pip Cet Date: Tue, 2 Mar 2021 20:45:04 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: 46881@debbugs.gnu.org Content-Type: multipart/mixed; boundary="00000000000042cc0805bc93d141" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --00000000000042cc0805bc93d141 Content-Type: text/plain; charset="UTF-8" On Tue, Mar 2, 2021 at 8:35 PM Pip Cet wrote: > I've looked into the problem, and it seems easy to solve and worth it > in terms of debuggability and performance. Very rough benchmarks, but this seems to be clearly worth it: Performance: With patch: real 0m3.861s user 0m3.776s sys 0m0.085s Without patch: real 0m7.001s user 0m4.476s sys 0m2.511s Number of syscalls: With patch: 415442 Without patch: 2028307 > Patch will be attached once this has a bug number. And here's the patch. Testing would be very appreciated. I'm unsure about the precise usage of dump_off vs ptrdiff_t here; I don't think it matters, but suggestions, nitpicks, and comments, on this or any other aspect, would be very appreciated. Pip --00000000000042cc0805bc93d141 Content-Type: text/x-patch; charset="US-ASCII"; name="0001-Prepare-pdumper-dump-file-in-memory-write-it-in-one-.patch" Content-Disposition: attachment; filename="0001-Prepare-pdumper-dump-file-in-memory-write-it-in-one-.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_klshagd70 RnJvbSA5MmVlMTM4ODUyYjM0ZWRlMmY0M2RkN2Y5M2YzMTBmYzc0NmJiM2JmIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBQaXAgQ2V0IDxwaXBjZXRAZ21haWwuY29tPgpEYXRlOiBUdWUs IDIgTWFyIDIwMjEgMjA6Mzg6MjMgKzAwMDAKU3ViamVjdDogW1BBVENIXSBQcmVwYXJlIHBkdW1w ZXIgZHVtcCBmaWxlIGluIG1lbW9yeSwgd3JpdGUgaXQgaW4gb25lIGdvCiAoQnVnIzQ2ODgxKQoK KiBzcmMvcGR1bXBlci5jIChzdHJ1Y3QgZHVtcF9jb250ZXh0KTogQWRkIGJ1ZiwgYnVmX3NpemUs IG1heF9vZmZzZXQgZmllbGRzLgooZ3Jvd19idWZmZXIpOiBOZXcgZnVuY3Rpb24uCihkdW1wX3dy aXRlKTogVXNlIG1lbWNweSwgbm90IGFuIGFjdHVhbCBlbWFjc193cml0ZS4KKGR1bXBfc2Vlayk6 IEtlZXAgdHJhY2sgb2YgbWF4aW11bSBzZWVuIG9mZnNldC4KKEZkdW1wX2VtYWNzX3BvcnRhYmxl KTogV3JpdGUgb3V0IHRoZSBmaWxlIGNvbnRlbnRzIHdoZW4gZG9uZS4KLS0tCiBzcmMvcGR1bXBl ci5jIHwgMjAgKysrKysrKysrKysrKysrKysrLS0KIDEgZmlsZSBjaGFuZ2VkLCAxOCBpbnNlcnRp b25zKCspLCAyIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3NyYy9wZHVtcGVyLmMgYi9zcmMv cGR1bXBlci5jCmluZGV4IDMzNzc0MmZkYTRhZGUuLjYyZGRhZDhlZTVlMzQgMTAwNjQ0Ci0tLSBh L3NyYy9wZHVtcGVyLmMKKysrIGIvc3JjL3BkdW1wZXIuYwpAQCAtNDczLDYgKzQ3MywxMCBAQCBk dW1wX2ZpbmdlcnByaW50IChjaGFyIGNvbnN0ICpsYWJlbCwKIHsKICAgLyogSGVhZGVyIHdlJ2xs IHdyaXRlIHRvIHRoZSBkdW1wIGZpbGUgd2hlbiBkb25lLiAgKi8KICAgc3RydWN0IGR1bXBfaGVh ZGVyIGhlYWRlcjsKKyAgLyogRGF0YSB0aGF0IHdpbGwgYmUgd3JpdHRlbiB0byB0aGUgZHVtcCBm aWxlLiAgKi8KKyAgdm9pZCAqYnVmOworICBwdHJkaWZmX3QgYnVmX3NpemU7CisgIHB0cmRpZmZf dCBtYXhfb2Zmc2V0OwogCiAgIExpc3BfT2JqZWN0IG9sZF9wdXJpZnlfZmxhZzsKICAgTGlzcF9P YmplY3Qgb2xkX3Bvc3RfZ2NfaG9vazsKQEAgLTU4MSw2ICs1ODUsMTMgQEAgZHVtcF9maW5nZXJw cmludCAoY2hhciBjb25zdCAqbGFiZWwsCiAMCiAvKiBEdW1wIGZpbGUgY3JlYXRpb24gKi8KIAor c3RhdGljIHZvaWQgZHVtcF9ncm93X2J1ZmZlciAoc3RydWN0IGR1bXBfY29udGV4dCAqY3R4KQor eworICBjdHgtPmJ1ZiA9IHhyZWFsbG9jIChjdHgtPmJ1ZiwgY3R4LT5idWZfc2l6ZSA9IChjdHgt PmJ1Zl9zaXplID8KKwkJCQkJCSAgKGN0eC0+YnVmX3NpemUgKiAyKQorCQkJCQkJICA6IDEwMjQg KiAxMDI0KSk7Cit9CisKIHN0YXRpYyBkdW1wX29mZiBkdW1wX29iamVjdCAoc3RydWN0IGR1bXBf Y29udGV4dCAqY3R4LCBMaXNwX09iamVjdCBvYmplY3QpOwogc3RhdGljIGR1bXBfb2ZmIGR1bXBf b2JqZWN0X2Zvcl9vZmZzZXQgKHN0cnVjdCBkdW1wX2NvbnRleHQgKmN0eCwKIAkJCQkJTGlzcF9P YmplY3Qgb2JqZWN0KTsKQEAgLTc0Nyw4ICs3NTgsOSBAQCBkdW1wX3dyaXRlIChzdHJ1Y3QgZHVt cF9jb250ZXh0ICpjdHgsIGNvbnN0IHZvaWQgKmJ1ZiwgZHVtcF9vZmYgbmJ5dGUpCiAgIGVhc3Nl cnQgKG5ieXRlID09IDAgfHwgYnVmICE9IE5VTEwpOwogICBlYXNzZXJ0IChjdHgtPm9ial9vZmZz ZXQgPT0gMCk7CiAgIGVhc3NlcnQgKGN0eC0+ZmxhZ3MuZHVtcF9vYmplY3RfY29udGVudHMpOwot ICBpZiAoZW1hY3Nfd3JpdGUgKGN0eC0+ZmQsIGJ1ZiwgbmJ5dGUpIDwgbmJ5dGUpCi0gICAgcmVw b3J0X2ZpbGVfZXJyb3IgKCJDb3VsZCBub3Qgd3JpdGUgdG8gZHVtcCBmaWxlIiwgY3R4LT5kdW1w X2ZpbGVuYW1lKTsKKyAgd2hpbGUgKGN0eC0+b2Zmc2V0ICsgbmJ5dGUgPiBjdHgtPmJ1Zl9zaXpl KQorICAgIGR1bXBfZ3Jvd19idWZmZXIgKGN0eCk7CisgIG1lbWNweSAoKGNoYXIgKiljdHgtPmJ1 ZiArIGN0eC0+b2Zmc2V0LCBidWYsIG5ieXRlKTsKICAgY3R4LT5vZmZzZXQgKz0gbmJ5dGU7CiB9 CiAKQEAgLTgyOCw2ICs4NDAsOCBAQCBkdW1wX3RhaWxxX3BvcCAoc3RydWN0IGR1bXBfdGFpbHEg KnRhaWxxKQogc3RhdGljIHZvaWQKIGR1bXBfc2VlayAoc3RydWN0IGR1bXBfY29udGV4dCAqY3R4 LCBkdW1wX29mZiBvZmZzZXQpCiB7CisgIGlmIChjdHgtPm1heF9vZmZzZXQgPCBjdHgtPm9mZnNl dCkKKyAgICBjdHgtPm1heF9vZmZzZXQgPSBjdHgtPm9mZnNldDsKICAgZWFzc2VydCAoY3R4LT5v Ympfb2Zmc2V0ID09IDApOwogICBpZiAobHNlZWsgKGN0eC0+ZmQsIG9mZnNldCwgU0VFS19TRVQp IDwgMCkKICAgICByZXBvcnRfZmlsZV9lcnJvciAoIlNldHRpbmcgZmlsZSBwb3NpdGlvbiIsCkBA IC00MTU5LDYgKzQxNzMsOCBAQCBERUZVTiAoImR1bXAtZW1hY3MtcG9ydGFibGUiLAogICBjdHgt PmhlYWRlci5tYWdpY1swXSA9IGR1bXBfbWFnaWNbMF07CiAgIGR1bXBfc2VlayAoY3R4LCAwKTsK ICAgZHVtcF93cml0ZSAoY3R4LCAmY3R4LT5oZWFkZXIsIHNpemVvZiAoY3R4LT5oZWFkZXIpKTsK KyAgaWYgKGVtYWNzX3dyaXRlIChjdHgtPmZkLCBjdHgtPmJ1ZiwgY3R4LT5tYXhfb2Zmc2V0KSA8 IGN0eC0+bWF4X29mZnNldCkKKyAgICByZXBvcnRfZmlsZV9lcnJvciAoIkNvdWxkIG5vdCB3cml0 ZSB0byBkdW1wIGZpbGUiLCBjdHgtPmR1bXBfZmlsZW5hbWUpOwogCiAgIGR1bXBfb2ZmCiAgICAg aGVhZGVyX2J5dGVzID0gaGVhZGVyX2VuZCAtIGhlYWRlcl9zdGFydCwKLS0gCjIuMzAuMQoK --00000000000042cc0805bc93d141-- From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 02 16:08:08 2021 Received: (at 46881) by debbugs.gnu.org; 2 Mar 2021 21:08:08 +0000 Received: from localhost ([127.0.0.1]:54354 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHCFT-0005MA-6T for submit@debbugs.gnu.org; Tue, 02 Mar 2021 16:08:08 -0500 Received: from outbound.soverin.net ([116.202.65.218]:36931) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHCFN-0005LY-Sc for 46881@debbugs.gnu.org; Tue, 02 Mar 2021 16:08:05 -0500 Received: from smtp.soverin.net (unknown [10.10.3.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by outbound.soverin.net (Postfix) with ESMTPS id AC4E4600F5 for <46881@debbugs.gnu.org>; Tue, 2 Mar 2021 21:07:55 +0000 (UTC) Received: from smtp.soverin.net (smtp.soverin.net [159.69.232.138]) by soverin.net DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=idiocy.org; s=soverin; t=1614719275; bh=Gn/fFdJQmHej7f8G4cjroZ7nBBxCt7DRt2VL9NyGNpw=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=RB+7EVHCU9wyz/duzQr5DatpVzj+Wg34btNdMPbXYvAaZeEDoFrBQth3cbNnyU+eO DgkbitZB9XGkrk7aRol1PumAk8sOPSisVfjGveXS5Za4A6c/bnQRzrYQDQ3NZEox3E 5nmTaaoKNG/dYYTwtggXYn7PLfy8ELkwNtqDRIXao6j2VnuHrAefqNCRQeQNdwH9q3 QjssipDNBV6c6C+NHVZyMVhIVO6HuOxc3yTjMBW6WGeSCB74iwR5CbzQsRaCopcWNr JUTbfhFuFeNmPqunflg09m3EdwZ2oF55oRe5bLdBGjvEcdxq/OFvOg2/n3e4IzWpXO tZm5VbYky6a4A== Received: by breton.holly.idiocy.org (Postfix, from userid 501) id D1914202AC44B9; Tue, 2 Mar 2021 21:07:52 +0000 (GMT) Date: Tue, 2 Mar 2021 21:07:52 +0000 From: Alan Third To: Pip Cet Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls Message-ID: Mail-Followup-To: Alan Third , Pip Cet , 46881@debbugs.gnu.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Tue, Mar 02, 2021 at 08:45:04PM +0000, Pip Cet wrote: > On Tue, Mar 2, 2021 at 8:35 PM Pip Cet wrote: > > I've looked into the problem, and it seems easy to solve and worth it > > in terms of debuggability and performance. > > Very rough benchmarks, but this seems to be clearly worth it: > > Performance: > With patch: > real 0m3.861s > user 0m3.776s > sys 0m0.085s > > Without patch: > real 0m7.001s > user 0m4.476s > sys 0m2.511s > > Number of syscalls: > With patch: 415442 > Without patch: 2028307 My quick test on macOS by doing: rm src/*.pdmp time make sees it going from ~26s without patch to ~10s with patch, so a considerable improvement. > > Patch will be attached once this has a bug number. > > And here's the patch. Testing would be very appreciated. It appears to work fine here, but I don't know if there's anything specific to test other than just running Emacs. -- Alan Third From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 03 00:51:23 2021 Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 05:51:23 +0000 Received: from localhost ([127.0.0.1]:54974 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHKPq-0003A2-Ul for submit@debbugs.gnu.org; Wed, 03 Mar 2021 00:51:23 -0500 Received: from eggs.gnu.org ([209.51.188.92]:41330) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHKPo-00039p-46 for 46881@debbugs.gnu.org; Wed, 03 Mar 2021 00:51:21 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59393) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lHKPh-0000S3-Ue; Wed, 03 Mar 2021 00:51:13 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2473 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lHKPf-0003vJ-MQ; Wed, 03 Mar 2021 00:51:13 -0500 Date: Wed, 03 Mar 2021 07:51:05 +0200 Message-Id: <83r1kw6b06.fsf@gnu.org> From: Eli Zaretskii To: Pip Cet , Daniel Colascione , Paul Eggert In-Reply-To: (message from Pip Cet on Tue, 2 Mar 2021 20:45:04 +0000) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Pip Cet > Date: Tue, 2 Mar 2021 20:45:04 +0000 > > On Tue, Mar 2, 2021 at 8:35 PM Pip Cet wrote: > > I've looked into the problem, and it seems easy to solve and worth it > > in terms of debuggability and performance. > > Very rough benchmarks, but this seems to be clearly worth it: > > Performance: > With patch: > real 0m3.861s > user 0m3.776s > sys 0m0.085s > > Without patch: > real 0m7.001s > user 0m4.476s > sys 0m2.511s > > Number of syscalls: > With patch: 415442 > Without patch: 2028307 > > > Patch will be attached once this has a bug number. > > And here's the patch. Testing would be very appreciated. > > I'm unsure about the precise usage of dump_off vs ptrdiff_t here; I > don't think it matters, but suggestions, nitpicks, and comments, on > this or any other aspect, would be very appreciated. > From 92ee138852b34ede2f43dd7f93f310fc746bb3bf Mon Sep 17 00:00:00 2001 > From: Pip Cet > Date: Tue, 2 Mar 2021 20:38:23 +0000 > Subject: [PATCH] Prepare pdumper dump file in memory, write it in one go > (Bug#46881) > > * src/pdumper.c (struct dump_context): Add buf, buf_size, max_offset fields. > (grow_buffer): New function. > (dump_write): Use memcpy, not an actual emacs_write. > (dump_seek): Keep track of maximum seen offset. > (Fdump_emacs_portable): Write out the file contents when done. > --- > src/pdumper.c | 20 ++++++++++++++++++-- > 1 file changed, 18 insertions(+), 2 deletions(-) > > diff --git a/src/pdumper.c b/src/pdumper.c > index 337742fda4ade..62ddad8ee5e34 100644 > --- a/src/pdumper.c > +++ b/src/pdumper.c > @@ -473,6 +473,10 @@ dump_fingerprint (char const *label, > { > /* Header we'll write to the dump file when done. */ > struct dump_header header; > + /* Data that will be written to the dump file. */ > + void *buf; > + ptrdiff_t buf_size; > + ptrdiff_t max_offset; > > Lisp_Object old_purify_flag; > Lisp_Object old_post_gc_hook; > @@ -581,6 +585,13 @@ dump_fingerprint (char const *label, > > /* Dump file creation */ > > +static void dump_grow_buffer (struct dump_context *ctx) > +{ > + ctx->buf = xrealloc (ctx->buf, ctx->buf_size = (ctx->buf_size ? > + (ctx->buf_size * 2) > + : 1024 * 1024)); > +} > + > static dump_off dump_object (struct dump_context *ctx, Lisp_Object object); > static dump_off dump_object_for_offset (struct dump_context *ctx, > Lisp_Object object); > @@ -747,8 +758,9 @@ dump_write (struct dump_context *ctx, const void *buf, dump_off nbyte) > eassert (nbyte == 0 || buf != NULL); > eassert (ctx->obj_offset == 0); > eassert (ctx->flags.dump_object_contents); > - if (emacs_write (ctx->fd, buf, nbyte) < nbyte) > - report_file_error ("Could not write to dump file", ctx->dump_filename); > + while (ctx->offset + nbyte > ctx->buf_size) > + dump_grow_buffer (ctx); > + memcpy ((char *)ctx->buf + ctx->offset, buf, nbyte); > ctx->offset += nbyte; > } > > @@ -828,6 +840,8 @@ dump_tailq_pop (struct dump_tailq *tailq) > static void > dump_seek (struct dump_context *ctx, dump_off offset) > { > + if (ctx->max_offset < ctx->offset) > + ctx->max_offset = ctx->offset; > eassert (ctx->obj_offset == 0); > if (lseek (ctx->fd, offset, SEEK_SET) < 0) > report_file_error ("Setting file position", > @@ -4159,6 +4173,8 @@ DEFUN ("dump-emacs-portable", > ctx->header.magic[0] = dump_magic[0]; > dump_seek (ctx, 0); > dump_write (ctx, &ctx->header, sizeof (ctx->header)); > + if (emacs_write (ctx->fd, ctx->buf, ctx->max_offset) < ctx->max_offset) > + report_file_error ("Could not write to dump file", ctx->dump_filename); > > dump_off > header_bytes = header_end - header_start, > -- > 2.30.1 Thanks. Daniel, Paul: any comments? In particular, is it safe to allocate large amounts of memory off the heap while dumping? A couple of places in pdumper.c says some parts of code should call malloc. From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 03 02:11:13 2021 Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 07:11:13 +0000 Received: from localhost ([127.0.0.1]:55030 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHLf7-0005Dc-5n for submit@debbugs.gnu.org; Wed, 03 Mar 2021 02:11:13 -0500 Received: from mail-oo1-f46.google.com ([209.85.161.46]:38681) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHLf3-0005DN-TD for 46881@debbugs.gnu.org; Wed, 03 Mar 2021 02:11:11 -0500 Received: by mail-oo1-f46.google.com with SMTP id f26so5451516oog.5 for <46881@debbugs.gnu.org>; Tue, 02 Mar 2021 23:11:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ohuOvlUVhjnXo88gJbDdHE0o+bd1U4Y1hNfdQ5HU7+A=; b=TdVVj+jXpk9iCc0QbhdwAqheQ4uL/NVsohBHY65b4OQPjgs9Fw9XWvpMsADl9sXv6R PIZCdYmbwVVjQYLeNKA0IiH8xGx1OzhUSPAuwumJi+/PEdGRDJoyuBTvYuzJFO8TYtwA J0W4Iu3SqjJrSGRHZaQtuc8trDGoPP+0bz0QjclLvNYkFxtLsVn6B5KolajG/sfyHvGy qPFIp9+L1hsnsWCmH7pP8xFZ4VgmBmCrU/OHcVDpN9QlCtxUh8NWVTghY+yNlCCRalyr OIqCcCdb35s5A+A21zcKW1Fuq0t32zW1KMzUClh91AyEaCEZQzKXOPPXTgJhzlnrQNOX hA3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ohuOvlUVhjnXo88gJbDdHE0o+bd1U4Y1hNfdQ5HU7+A=; b=rm/GI8yhU2RQ/KfPcXDNQiqmM0PHvYpEaf37OJD1P486hrhXqwI+HyhCRBL/yMZRrt AyFrWjLc+ns2sqvYsB+65QVgTpsEx43FH6zV+OhHJHT3G2GAMqXuHr5yB54RS1hmK7dH o3aWc0sZXrZ6uwt2PIKYI9dUy9X27tEncc6C8jxnmhx7F/sLGDtmZ2VEcQk/TkVuYTCx kyEm3O3rrymwpSTuB/jiaBvFvwvSjVw2i7CklmZSGS+neww+wPvWp7tCFR1x9xRZ2Iml f6wxaz4cQwreDrOPA/2Z0+vtc6W3jSl8/ZvrTOKyQPIDsnc34r1c2ostsvmUdMSzAW12 rOZA== X-Gm-Message-State: AOAM530HNHCincOZ4uj7LqEU8tjF3pDsER/vLWSllq3a02yoLpnIgcSM j1L7S4VGa2AksrLwdvpmrP2oAYzI3qA0YxMoUqc= X-Google-Smtp-Source: ABdhPJwDiWksJBkLfj1cBZo0thE98VJ2g8+ALlu2EPNzM/ePAwZHKRJ9nvGlZDFcCUsGYcdm4vl6mJ/+OD2GwgjJWuo= X-Received: by 2002:a4a:2511:: with SMTP id g17mr20213317ooa.22.1614755464381; Tue, 02 Mar 2021 23:11:04 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Pip Cet Date: Wed, 3 Mar 2021 07:10:28 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Alan Third , Pip Cet Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > My quick test on macOS by doing: > > rm src/*.pdmp > time make > > sees it going from ~26s without patch to ~10s with patch, so a > considerable improvement. Thanks for testing! > > > Patch will be attached once this has a bug number. > > > > And here's the patch. Testing would be very appreciated. > > It appears to work fine here, but I don't know if there's anything > specific to test other than just running Emacs. I suspect there may be problems on systems with very little memory. Do you have an easy way to determine maximum resident size (on Debian GNU/Linux, /bin/time works)? It would be interesting to see if that's actually different. Thanks again! Pip From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 03 02:36:32 2021 Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 07:36:32 +0000 Received: from localhost ([127.0.0.1]:55060 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHM3b-0005oy-MQ for submit@debbugs.gnu.org; Wed, 03 Mar 2021 02:36:32 -0500 Received: from mail-oi1-f176.google.com ([209.85.167.176]:32780) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHM3X-0005og-FX for 46881@debbugs.gnu.org; Wed, 03 Mar 2021 02:36:30 -0500 Received: by mail-oi1-f176.google.com with SMTP id a13so24990995oid.0 for <46881@debbugs.gnu.org>; Tue, 02 Mar 2021 23:36:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+EECznTjamyCy/0zBbo7zaWPtTrz+NZQaFEIg71G6L8=; b=OyxWogHD1aqtW7C98jy5/rqH272gJ7yTlXMVLA5H5zBqqxUmhqJ8orR4jOSBgzWsYF KFs887UQD8Kj+kTNIwRFacM0dsp/oIi+iXvhvPyIemILHJtYJ6v/iOXaWzMEMRMy86Ty poszsU9oNAW2Tth8ozljY37IvWEjzQwDpb7U5wUmVKTBpaZRgN8HJ73K1WYQXS51Vv6V LDcKNTF47OjXIGOxJx/B2xVuF5LInqkb9902GXC+bNHXAHHQ2iZixLe+y06dsZtauMaY WZZpJB7Zb6v68ETh/V4Pn2t48Px9hqBxoi1D4HhbU4btNGdd9OKC5PVEvFDuJCO8MfoI OfQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+EECznTjamyCy/0zBbo7zaWPtTrz+NZQaFEIg71G6L8=; b=j5Pb+j1rI593i89onfQZUZ9l3N/cFdylRyg6kItjQlAtKlXC1xltSyAJSFmu9yVVGb BochGszsIlfZ7EN0QZjSsAY5fp7uVWy2lV/8AjZJeqkJL+rnxZ/U31o7yf6l1SekndBc RwUDUdfyGUWN8I2RKII0NK7EWD18RdPoQhOM+q9QVBZZv/pk+Ugb0YhP8LQWjeM3gJPH mNgU1e6L514IlaMNuSX5bxkjyt4UqwZ4sDMzJjeHUHwx+IcMDR1MXi1bIoGDEuE5Ui8l G8rxwfSzwQ9u1JMuAS48NbrBNj+jnVa9lINJmZP1n3ghRiALRiXULkWCJMnMv4sYh/Th brew== X-Gm-Message-State: AOAM531vKJA1RWJ9VSwH1gHeGFiIjf+lk5MkkUHMahvvTPDTcRGpM9CI W1VIozKCj/ieXhpsEDbFu0A+5C+yOTMJoM5ZBDs= X-Google-Smtp-Source: ABdhPJwhcJOmI9ybAmsRtall1VyrbFUVFaWB4MoQ2UZ2NfJUJJxRA/Zc/xHYjJG1pH4Ar+zcOjifczLTCo/KEcoadvw= X-Received: by 2002:aca:d905:: with SMTP id q5mr2758632oig.30.1614756981917; Tue, 02 Mar 2021 23:36:21 -0800 (PST) MIME-Version: 1.0 References: <83r1kw6b06.fsf@gnu.org> In-Reply-To: <83r1kw6b06.fsf@gnu.org> From: Pip Cet Date: Wed, 3 Mar 2021 07:35:45 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Eli Zaretskii Content-Type: multipart/mixed; boundary="0000000000004e66b405bc9ce8e0" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Daniel Colascione , Paul Eggert X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --0000000000004e66b405bc9ce8e0 Content-Type: text/plain; charset="UTF-8" On Wed, Mar 3, 2021 at 5:51 AM Eli Zaretskii wrote: > > From: Pip Cet > > Date: Tue, 2 Mar 2021 20:45:04 +0000 > > > > On Tue, Mar 2, 2021 at 8:35 PM Pip Cet wrote: > > > I've looked into the problem, and it seems easy to solve and worth it > > > in terms of debuggability and performance. Since debuggability is such a concern, we probably shouldn't leak the buffer memory. Revised patch attached. (This patch also removes the lseek() syscalls; while not quite as numerous as the read() ones, those did clutter up straces here). > In particular, is it safe to allocate > large amounts of memory off the heap while dumping? Even if it isn't, we'd still be faster re-running the dump after growing the dumper image than the current approach is. >A couple of > places in pdumper.c says some parts of code should call malloc. IIUC, the prohibition on calling malloc, if it is still a concern, applies only when loading the dump, not while writing it. My main concern is the possibility of a partly-written dump file, since we no longer turn "!UMPEDGNUEMACS" into "DUMPEDGNUEMACS" after the dump. Maybe it would make sense to restore that feature? Pip --0000000000004e66b405bc9ce8e0 Content-Type: text/x-patch; charset="US-ASCII"; name="0001-Prepare-pdumper-dump-file-in-memory-write-it-in-one-.patch" Content-Disposition: attachment; filename="0001-Prepare-pdumper-dump-file-in-memory-write-it-in-one-.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_klt4kmx30 RnJvbSBmYWU2N2MwMjk1NWE1YmJlYTE2ZDU1NGI4ZTczNWRjOGJlZjZhOWUyIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBQaXAgQ2V0IDxwaXBjZXRAZ21haWwuY29tPgpEYXRlOiBUdWUs IDIgTWFyIDIwMjEgMjA6Mzg6MjMgKzAwMDAKU3ViamVjdDogW1BBVENIXSBQcmVwYXJlIHBkdW1w ZXIgZHVtcCBmaWxlIGluIG1lbW9yeSwgd3JpdGUgaXQgaW4gb25lIGdvCiAoQnVnIzQ2ODgxKQoK KiBzcmMvcGR1bXBlci5jIChzdHJ1Y3QgZHVtcF9jb250ZXh0KTogQWRkIGJ1ZiwgYnVmX3NpemUs IG1heF9vZmZzZXQgZmllbGRzLgooZHVtcF9ncm93X2J1ZmZlcik6IE5ldyBmdW5jdGlvbi4KKGR1 bXBfd3JpdGUpOiBVc2UgbWVtY3B5LCBub3QgYW4gYWN0dWFsIGVtYWNzX3dyaXRlLgooZHVtcF9z ZWVrKTogS2VlcCB0cmFjayBvZiBtYXhpbXVtIHNlZW4gb2Zmc2V0LiBEb24ndCBhY3R1YWxseSBz ZWVrLgooRmR1bXBfZW1hY3NfcG9ydGFibGUpOiBXcml0ZSBvdXQgdGhlIGZpbGUgY29udGVudHMg d2hlbiBkb25lLgotLS0KIHNyYy9wZHVtcGVyLmMgfCAyNyArKysrKysrKysrKysrKysrKysrKysr LS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCAyMiBpbnNlcnRpb25zKCspLCA1IGRlbGV0aW9ucygtKQoK ZGlmZiAtLWdpdCBhL3NyYy9wZHVtcGVyLmMgYi9zcmMvcGR1bXBlci5jCmluZGV4IDMzNzc0MmZk YTRhZGUuLjI5ZDFjYjg2MmUwN2YgMTAwNjQ0Ci0tLSBhL3NyYy9wZHVtcGVyLmMKKysrIGIvc3Jj L3BkdW1wZXIuYwpAQCAtNDczLDYgKzQ3MywxMCBAQCBkdW1wX2ZpbmdlcnByaW50IChjaGFyIGNv bnN0ICpsYWJlbCwKIHsKICAgLyogSGVhZGVyIHdlJ2xsIHdyaXRlIHRvIHRoZSBkdW1wIGZpbGUg d2hlbiBkb25lLiAgKi8KICAgc3RydWN0IGR1bXBfaGVhZGVyIGhlYWRlcjsKKyAgLyogRGF0YSB0 aGF0IHdpbGwgYmUgd3JpdHRlbiB0byB0aGUgZHVtcCBmaWxlLiAgKi8KKyAgdm9pZCAqYnVmOwor ICBkdW1wX29mZiBidWZfc2l6ZTsKKyAgZHVtcF9vZmYgbWF4X29mZnNldDsKIAogICBMaXNwX09i amVjdCBvbGRfcHVyaWZ5X2ZsYWc7CiAgIExpc3BfT2JqZWN0IG9sZF9wb3N0X2djX2hvb2s7CkBA IC01ODEsNiArNTg1LDEzIEBAIGR1bXBfZmluZ2VycHJpbnQgKGNoYXIgY29uc3QgKmxhYmVsLAog DAogLyogRHVtcCBmaWxlIGNyZWF0aW9uICovCiAKK3N0YXRpYyB2b2lkIGR1bXBfZ3Jvd19idWZm ZXIgKHN0cnVjdCBkdW1wX2NvbnRleHQgKmN0eCkKK3sKKyAgY3R4LT5idWYgPSB4cmVhbGxvYyAo Y3R4LT5idWYsIGN0eC0+YnVmX3NpemUgPSAoY3R4LT5idWZfc2l6ZSA/CisJCQkJCQkgIChjdHgt PmJ1Zl9zaXplICogMikKKwkJCQkJCSAgOiA4ICogMTAyNCAqIDEwMjQpKTsKK30KKwogc3RhdGlj IGR1bXBfb2ZmIGR1bXBfb2JqZWN0IChzdHJ1Y3QgZHVtcF9jb250ZXh0ICpjdHgsIExpc3BfT2Jq ZWN0IG9iamVjdCk7CiBzdGF0aWMgZHVtcF9vZmYgZHVtcF9vYmplY3RfZm9yX29mZnNldCAoc3Ry dWN0IGR1bXBfY29udGV4dCAqY3R4LAogCQkJCQlMaXNwX09iamVjdCBvYmplY3QpOwpAQCAtNzQ3 LDggKzc1OCw5IEBAIGR1bXBfd3JpdGUgKHN0cnVjdCBkdW1wX2NvbnRleHQgKmN0eCwgY29uc3Qg dm9pZCAqYnVmLCBkdW1wX29mZiBuYnl0ZSkKICAgZWFzc2VydCAobmJ5dGUgPT0gMCB8fCBidWYg IT0gTlVMTCk7CiAgIGVhc3NlcnQgKGN0eC0+b2JqX29mZnNldCA9PSAwKTsKICAgZWFzc2VydCAo Y3R4LT5mbGFncy5kdW1wX29iamVjdF9jb250ZW50cyk7Ci0gIGlmIChlbWFjc193cml0ZSAoY3R4 LT5mZCwgYnVmLCBuYnl0ZSkgPCBuYnl0ZSkKLSAgICByZXBvcnRfZmlsZV9lcnJvciAoIkNvdWxk IG5vdCB3cml0ZSB0byBkdW1wIGZpbGUiLCBjdHgtPmR1bXBfZmlsZW5hbWUpOworICB3aGlsZSAo Y3R4LT5vZmZzZXQgKyBuYnl0ZSA+IGN0eC0+YnVmX3NpemUpCisgICAgZHVtcF9ncm93X2J1ZmZl ciAoY3R4KTsKKyAgbWVtY3B5ICgoY2hhciAqKWN0eC0+YnVmICsgY3R4LT5vZmZzZXQsIGJ1Ziwg bmJ5dGUpOwogICBjdHgtPm9mZnNldCArPSBuYnl0ZTsKIH0KIApAQCAtODI4LDEwICs4NDAsOSBA QCBkdW1wX3RhaWxxX3BvcCAoc3RydWN0IGR1bXBfdGFpbHEgKnRhaWxxKQogc3RhdGljIHZvaWQK IGR1bXBfc2VlayAoc3RydWN0IGR1bXBfY29udGV4dCAqY3R4LCBkdW1wX29mZiBvZmZzZXQpCiB7 CisgIGlmIChjdHgtPm1heF9vZmZzZXQgPCBjdHgtPm9mZnNldCkKKyAgICBjdHgtPm1heF9vZmZz ZXQgPSBjdHgtPm9mZnNldDsKICAgZWFzc2VydCAoY3R4LT5vYmpfb2Zmc2V0ID09IDApOwotICBp ZiAobHNlZWsgKGN0eC0+ZmQsIG9mZnNldCwgU0VFS19TRVQpIDwgMCkKLSAgICByZXBvcnRfZmls ZV9lcnJvciAoIlNldHRpbmcgZmlsZSBwb3NpdGlvbiIsCi0gICAgICAgICAgICAgICAgICAgICAg IGN0eC0+ZHVtcF9maWxlbmFtZSk7CiAgIGN0eC0+b2Zmc2V0ID0gb2Zmc2V0OwogfQogCkBAIC00 MTU5LDYgKzQxNzAsMTIgQEAgREVGVU4gKCJkdW1wLWVtYWNzLXBvcnRhYmxlIiwKICAgY3R4LT5o ZWFkZXIubWFnaWNbMF0gPSBkdW1wX21hZ2ljWzBdOwogICBkdW1wX3NlZWsgKGN0eCwgMCk7CiAg IGR1bXBfd3JpdGUgKGN0eCwgJmN0eC0+aGVhZGVyLCBzaXplb2YgKGN0eC0+aGVhZGVyKSk7Cisg IGlmIChlbWFjc193cml0ZSAoY3R4LT5mZCwgY3R4LT5idWYsIGN0eC0+bWF4X29mZnNldCkgPCBj dHgtPm1heF9vZmZzZXQpCisgICAgcmVwb3J0X2ZpbGVfZXJyb3IgKCJDb3VsZCBub3Qgd3JpdGUg dG8gZHVtcCBmaWxlIiwgY3R4LT5kdW1wX2ZpbGVuYW1lKTsKKyAgeGZyZWUgKGN0eC0+YnVmKTsK KyAgY3R4LT5idWYgPSBOVUxMOworICBjdHgtPmJ1Zl9zaXplID0gMDsKKyAgY3R4LT5tYXhfb2Zm c2V0ID0gMDsKIAogICBkdW1wX29mZgogICAgIGhlYWRlcl9ieXRlcyA9IGhlYWRlcl9lbmQgLSBo ZWFkZXJfc3RhcnQsCi0tIAoyLjMwLjEKCg== --0000000000004e66b405bc9ce8e0-- From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 03 10:09:34 2021 Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 15:09:35 +0000 Received: from localhost ([127.0.0.1]:56615 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHT82-0000Se-LT for submit@debbugs.gnu.org; Wed, 03 Mar 2021 10:09:34 -0500 Received: from quimby.gnus.org ([95.216.78.240]:40160) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHT80-0000SO-Ci for 46881@debbugs.gnu.org; Wed, 03 Mar 2021 10:09:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=dWcvz+PRXKBKPusURui7a7AcOm+ihfRCKPJ+xgNRmCE=; b=RrAyIQ/t0jwcKEzBRiEWb3w3fr DcyVjjH77m2bN5fSpyjJdovZgCIV8iuQKzr2KpmurclmlhSKjybnUYoU/nqM6klOpSi/yJjLWRf+8 fYTHrv/Q+fuJHVIL353fWN1+BybXxP/hWXsv3OPk2jOuVnWwa8aFKuy1j+jd8aa0xSwQ=; Received: from cm-84.212.220.105.getinternet.no ([84.212.220.105] helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lHT7k-0001G5-Jr; Wed, 03 Mar 2021 16:09:19 +0100 From: Lars Ingebrigtsen To: Pip Cet Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <83r1kw6b06.fsf@gnu.org> X-Now-Playing: Front 242's _Geography_: "He Runs Too Fast For Us" Date: Wed, 03 Mar 2021 16:09:15 +0100 In-Reply-To: (Pip Cet's message of "Wed, 3 Mar 2021 07:35:45 +0000") Message-ID: <87tupsw9yc.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Pip Cet writes: > Since debuggability is such a concern, we probably shouldn't leak the > buffer memory. Revised patch attached. (This patch also removes the > lseek() syscalls; while not quite as numerous as the rea [...] Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Eli Zaretskii , Daniel Colascione , Paul Eggert X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Pip Cet writes: > Since debuggability is such a concern, we probably shouldn't leak the > buffer memory. Revised patch attached. (This patch also removes the > lseek() syscalls; while not quite as numerous as the read() ones, > those did clutter up straces here). I've tried the patch on a couple of systems here, and the resulting Emacs works fine (as expected), and the pdumping is significantly faster here, too. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 03 14:35:40 2021 Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 19:35:40 +0000 Received: from localhost ([127.0.0.1]:57036 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHXHY-0003Bp-2l for submit@debbugs.gnu.org; Wed, 03 Mar 2021 14:35:40 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:57290) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHXHX-0003Bc-00 for 46881@debbugs.gnu.org; Wed, 03 Mar 2021 14:35:39 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id D2E591600F3; Wed, 3 Mar 2021 11:35:32 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id UMigWG2Oy0O3; Wed, 3 Mar 2021 11:35:32 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 2EF4F1600CC; Wed, 3 Mar 2021 11:35:32 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 931QkZ5XMszY; Wed, 3 Mar 2021 11:35:32 -0800 (PST) Received: from [192.168.1.9] (cpe-23-243-218-95.socal.res.rr.com [23.243.218.95]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 048BE1600BC; Wed, 3 Mar 2021 11:35:31 -0800 (PST) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Pip Cet , Eli Zaretskii References: <83r1kw6b06.fsf@gnu.org> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: Date: Wed, 3 Mar 2021 11:35:31 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Daniel Colascione X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On 3/2/21 11:35 PM, Pip Cet wrote: > IIUC, the prohibition on calling malloc, if it is still a concern, > applies only when loading the dump, not while writing it. That's my understanding as well. > My main concern is the possibility of a partly-written dump file, > since we no longer turn "!UMPEDGNUEMACS" into "DUMPEDGNUEMACS" after > the dump. Maybe it would make sense to restore that feature? Wouldn't hurt, though I'd make it low priority. From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 03 14:58:12 2021 Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 19:58:12 +0000 Received: from localhost ([127.0.0.1]:57064 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHXdM-0003jR-I7 for submit@debbugs.gnu.org; Wed, 03 Mar 2021 14:58:12 -0500 Received: from outbound.soverin.net ([116.202.65.218]:43347) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHXdK-0003jE-DZ for 46881@debbugs.gnu.org; Wed, 03 Mar 2021 14:58:11 -0500 Received: from smtp.soverin.net (unknown [10.10.3.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by outbound.soverin.net (Postfix) with ESMTPS id 8F72160293 for <46881@debbugs.gnu.org>; Wed, 3 Mar 2021 19:58:02 +0000 (UTC) Received: from smtp.soverin.net (smtp.soverin.net [159.69.232.138]) by soverin.net DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=idiocy.org; s=soverin; t=1614801482; bh=NX9Txa0W5RxAPS57duXK5gx3K+SNhSY779l6SVevE8E=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=OA3Tv6McME/Xc9gQLKlTxJhKAUwdL+hLgwlLrUocZX5dKjdWiVtkiXEBHAjP2ackY HctFnCYWjSl8/j7FinLxj3F+VrvI/20xWNiYpZiP/Yp5V+EU2QE56b+ab6YBrtFE8c RdBgZCl3G0pijOsLgggYmgmPZgaH/3+a3/m4/4GsVjlj8VTw2EQhZ3lk0CWEaU/aHp GJeLmOeqz4HcTPe9uYxeP9b2Z0OO4ph3QI3HcWI86zC7SPaoxWj6P775TzGiosofhs VJch/fK6TbuctnWF1Kdz+vEF6/36FVEHHSzMnAsCeh3Xc0QrJpUeZ5LyXN8V6XRpAB N4B2fbWMZZFsQ== Received: by breton.holly.idiocy.org (Postfix, from userid 501) id 6D72D202AC618E; Wed, 3 Mar 2021 19:57:59 +0000 (GMT) Date: Wed, 3 Mar 2021 19:57:59 +0000 From: Alan Third To: Pip Cet Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls Message-ID: Mail-Followup-To: Alan Third , Pip Cet , 46881@debbugs.gnu.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Wed, Mar 03, 2021 at 07:10:28AM +0000, Pip Cet wrote: > > My quick test on macOS by doing: > > > > rm src/*.pdmp > > time make > > > > sees it going from ~26s without patch to ~10s with patch, so a > > considerable improvement. > > Thanks for testing! > > > > > Patch will be attached once this has a bug number. > > > > > > And here's the patch. Testing would be very appreciated. > > > > It appears to work fine here, but I don't know if there's anything > > specific to test other than just running Emacs. > > I suspect there may be problems on systems with very little memory. Do > you have an easy way to determine maximum resident size (on Debian > GNU/Linux, /bin/time works)? It would be interesting to see if that's > actually different. I tried using time -l the same as above and both came out with roughly the same values, but I suspect that's probably just me getting the values for running make. Is there a better way of testing dumping? -- Alan Third From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 04 02:26:05 2021 Received: (at 46881) by debbugs.gnu.org; 4 Mar 2021 07:26:06 +0000 Received: from localhost ([127.0.0.1]:57554 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHiMu-0001YM-D1 for submit@debbugs.gnu.org; Thu, 04 Mar 2021 02:26:05 -0500 Received: from mail-ot1-f46.google.com ([209.85.210.46]:32946) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHiMs-0001Y9-Pw for 46881@debbugs.gnu.org; Thu, 04 Mar 2021 02:25:55 -0500 Received: by mail-ot1-f46.google.com with SMTP id j8so890800otc.0 for <46881@debbugs.gnu.org>; Wed, 03 Mar 2021 23:25:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=RSykA11zPfpk99eUA+3H9NyfJn/uSfzUhZ5+YCNxFng=; b=E78xJTA7uWmlzYQJE1IeHqQNFFY6+Wtp7Ar74ep4OvuHiHfL5q3LhLABY2j3w4b5VB Cst0ltm5mRlx2nKOKOqFfHa5pSHOoqdISjylBHceSa5rCrA510MdiII2TbedCrWVF0bR wCTngaw+hyyR4PbFAm+pI0VtFYCx9nm6Ldz48Lsycw49d54rlFVvdB1GvurOGAvmNFdS ZK34w7T4k6HI/vPGnpEd3BjMJV6rTg6M7e0lm9GV8ZdAvoFX+JLnP2gg66shQttpcyKU i3JQss61a9E6+m5M1F0k/nHB9b72kCi3QzJkXtGN5NadFRcDBvkBcGL6HNdP3mRmur9s uyEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=RSykA11zPfpk99eUA+3H9NyfJn/uSfzUhZ5+YCNxFng=; b=Es7j4L/xZW13zFc0l9ayd0JHYyhXh/gHcdktwEzong/fi49Cd9yvrFa4qull8CSx5x 0YiiOD6XTAdYyd8kyyCI+7QDVn90GYLqXsoV744ntBJAJ778aw8MKarxIXyS1a1ngflM 6LqMuGexN44E6G1Xxzei3GMcwp2LoZxDySxilpHX5t7vatWqA222hGrClorzmYICsU2S RmTxueiBvLbTjhbF3dHzEE0N45Dy64Gj0tsWpj/8eFouY+6XB6cJaR9h7cIIPjHD66Al m2Qp4s61IKz8y/Qd7qKMS2wdpK41lF7l+bPMGcjFGu17Bbe+l+HCs/Lw5j42rfN2ksEm D58A== X-Gm-Message-State: AOAM532o/AO7vFQYcqkzg4PbXK41/gfgme5KnXNVwo4X+UtbiZY3Y33V KSMY9s9Dhk3q3xcwwj/EoGj3RArmI6WHFs8NSUA= X-Google-Smtp-Source: ABdhPJzfXsuMZZGujGssQAPGbkQ0pGdkuJA0UB77t7oy2v3f+nRA7ValeOLHaxX5xlx0HSEJQiMSWPdwnW+fKfihdro= X-Received: by 2002:a05:6830:1011:: with SMTP id a17mr221042otp.154.1614842749289; Wed, 03 Mar 2021 23:25:49 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Pip Cet Date: Thu, 4 Mar 2021 07:25:13 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Alan Third , Pip Cet , 46881@debbugs.gnu.org Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, Mar 3, 2021 at 7:58 PM Alan Third wrote: > On Wed, Mar 03, 2021 at 07:10:28AM +0000, Pip Cet wrote: > > > My quick test on macOS by doing: > > > > > > rm src/*.pdmp > > > time make > > > > > > sees it going from ~26s without patch to ~10s with patch, so a > > > considerable improvement. > > > > Thanks for testing! > > > > > > > Patch will be attached once this has a bug number. > > > > > > > > And here's the patch. Testing would be very appreciated. > > > > > > It appears to work fine here, but I don't know if there's anything > > > specific to test other than just running Emacs. > > > > I suspect there may be problems on systems with very little memory. Do > > you have an easy way to determine maximum resident size (on Debian > > GNU/Linux, /bin/time works)? It would be interesting to see if that's > > actually different. > > I tried using time -l the same as above and both came out with > roughly the same values, but I suspect that's probably just me getting > the values for running make. Thanks. > Is there a better way of testing dumping? I guess you could run "time ./temacs --batch -l loadup --temacs=pbootstrap" directly (in src/)... I realize that's not an answer to your question, but IME, dumping bugs will often lead to immediate failures to bootstrap, whereas GC bugs are tricky and don't. Since this bug doesn't affect GC in any way, I think we've done enough initial testing. Pip From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 04 17:26:41 2021 Received: (at 46881) by debbugs.gnu.org; 4 Mar 2021 22:26:41 +0000 Received: from localhost ([127.0.0.1]:60555 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHwQa-0001WI-TA for submit@debbugs.gnu.org; Thu, 04 Mar 2021 17:26:41 -0500 Received: from dancol.org ([96.126.100.184]:35692) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHwQW-0001W8-3X for 46881@debbugs.gnu.org; Thu, 04 Mar 2021 17:26:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=AH3jYq7hr+YyCbABU+2+sUNtbROHnDTbfObLdyzhWms=; b=UB24kcXCIPvrgAOb/19/PtG/IL 7WwiRVbTX03Y5aL5u4LckDkYGagGLrp5vzbW2Pf5LtDMM8MhDPpCqikgYT+PfIplfynUEuoIdoI74 QAOw/Sxv214AUCXu0wSDXUutWFNqRvS2lPWAEqD0EQy6xkqsgeqiOEyu1EXzivEsAzcf5OHKfiv7o FEl8aJYOz5xM7ZseepNowy7USVozRyXS6qNzoJ18UB+OLfCFzgPIfNRhXcEVGgg5GaquO9McMZZBg 3NzMwaC+LLDBKbLTiI2gayi0gEBwn5/l09WUTBXEniNAFiXHyCa4LNZkWrJQ7uHCrrMn8QgS6s4zk YRlX37RA==; Received: from [97.104.73.87] (port=46596 helo=[192.168.1.148]) by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1lHwQU-0003cW-5G; Thu, 04 Mar 2021 14:26:34 -0800 Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Eli Zaretskii , Pip Cet , Paul Eggert References: <83r1kw6b06.fsf@gnu.org> From: Daniel Colascione Message-ID: <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> Date: Thu, 4 Mar 2021 17:26:32 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <83r1kw6b06.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 3/3/21 12:51 AM, Eli Zaretskii wrote: >> From: Pip Cet >> Date: Tue, 2 Mar 2021 20:45:04 +0000 >> >> On Tue, Mar 2, 2021 at 8:35 PM Pip Cet wrote: >>> I've looked into the problem, and it seems easy to solve and worth it >>> in terms of debuggability and performance. >> Very rough benchmarks, but this seems to be clearly worth it: >> >> Performance: >> With patch: >> real 0m3.861s >> user 0m3.776s >> sys 0m0.085s >> >> Without patch: >> real 0m7.001s >> user 0m4.476s >> sys 0m2.511s >> >> Number of syscalls: >> With patch: 415442 >> Without patch: 2028307 >> >>> Patch will be attached once this has a bug number. >> And here's the patch. Testing would be very appreciated. >> >> I'm unsure about the precise usage of dump_off vs ptrdiff_t here; I >> don't think it matters, but suggestions, nitpicks, and comments, on >> this or any other aspect, would be very appreciated. >> From 92ee138852b34ede2f43dd7f93f310fc746bb3bf Mon Sep 17 00:00:00 2001 >> From: Pip Cet >> Date: Tue, 2 Mar 2021 20:38:23 +0000 >> Subject: [PATCH] Prepare pdumper dump file in memory, write it in one go >> (Bug#46881) >> >> * src/pdumper.c (struct dump_context): Add buf, buf_size, max_offset fields. >> (grow_buffer): New function. >> (dump_write): Use memcpy, not an actual emacs_write. >> (dump_seek): Keep track of maximum seen offset. >> (Fdump_emacs_portable): Write out the file contents when done. >> --- >> src/pdumper.c | 20 ++++++++++++++++++-- >> 1 file changed, 18 insertions(+), 2 deletions(-) >> >> diff --git a/src/pdumper.c b/src/pdumper.c >> index 337742fda4ade..62ddad8ee5e34 100644 >> --- a/src/pdumper.c >> +++ b/src/pdumper.c >> @@ -473,6 +473,10 @@ dump_fingerprint (char const *label, >> { >> /* Header we'll write to the dump file when done. */ >> struct dump_header header; >> + /* Data that will be written to the dump file. */ >> + void *buf; >> + ptrdiff_t buf_size; >> + ptrdiff_t max_offset; >> >> Lisp_Object old_purify_flag; >> Lisp_Object old_post_gc_hook; >> @@ -581,6 +585,13 @@ dump_fingerprint (char const *label, >> >> /* Dump file creation */ >> >> +static void dump_grow_buffer (struct dump_context *ctx) >> +{ >> + ctx->buf = xrealloc (ctx->buf, ctx->buf_size = (ctx->buf_size ? >> + (ctx->buf_size * 2) >> + : 1024 * 1024)); >> +} >> + >> static dump_off dump_object (struct dump_context *ctx, Lisp_Object object); >> static dump_off dump_object_for_offset (struct dump_context *ctx, >> Lisp_Object object); >> @@ -747,8 +758,9 @@ dump_write (struct dump_context *ctx, const void *buf, dump_off nbyte) >> eassert (nbyte == 0 || buf != NULL); >> eassert (ctx->obj_offset == 0); >> eassert (ctx->flags.dump_object_contents); >> - if (emacs_write (ctx->fd, buf, nbyte) < nbyte) >> - report_file_error ("Could not write to dump file", ctx->dump_filename); >> + while (ctx->offset + nbyte > ctx->buf_size) >> + dump_grow_buffer (ctx); >> + memcpy ((char *)ctx->buf + ctx->offset, buf, nbyte); >> ctx->offset += nbyte; >> } >> >> @@ -828,6 +840,8 @@ dump_tailq_pop (struct dump_tailq *tailq) >> static void >> dump_seek (struct dump_context *ctx, dump_off offset) >> { >> + if (ctx->max_offset < ctx->offset) >> + ctx->max_offset = ctx->offset; >> eassert (ctx->obj_offset == 0); >> if (lseek (ctx->fd, offset, SEEK_SET) < 0) >> report_file_error ("Setting file position", >> @@ -4159,6 +4173,8 @@ DEFUN ("dump-emacs-portable", >> ctx->header.magic[0] = dump_magic[0]; >> dump_seek (ctx, 0); >> dump_write (ctx, &ctx->header, sizeof (ctx->header)); >> + if (emacs_write (ctx->fd, ctx->buf, ctx->max_offset) < ctx->max_offset) >> + report_file_error ("Could not write to dump file", ctx->dump_filename); >> >> dump_off >> header_bytes = header_end - header_start, >> -- >> 2.30.1 > Thanks. > > Daniel, Paul: any comments? In particular, is it safe to allocate > large amounts of memory off the heap while dumping? A couple of > places in pdumper.c says some parts of code should call malloc. It looks fine, but wouldn't dumping to a FILE* (with internal buffering) do the same basic thing in a simpler way? There aren't any particular constraints on the environment _during_ the dump: we even make new lisp objects. It's when loading the dump, early in initialization, that you have to be careful. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 04 21:30:57 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 02:30:57 +0000 Received: from localhost ([127.0.0.1]:60702 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI0Ez-0007Hi-8B for submit@debbugs.gnu.org; Thu, 04 Mar 2021 21:30:57 -0500 Received: from mail-ot1-f54.google.com ([209.85.210.54]:43412) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI0Ex-0007HV-DV for 46881@debbugs.gnu.org; Thu, 04 Mar 2021 21:30:56 -0500 Received: by mail-ot1-f54.google.com with SMTP id v12so363540ott.10 for <46881@debbugs.gnu.org>; Thu, 04 Mar 2021 18:30:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Yz+5IQwz1PHNAgvclmhCEIHgyfGRA67qmVd0f2CpqfY=; b=aQGhbbaYcIu6PXyoVeiOaBgqs0O9MaZcBo7eRMscwwfCxVMYj+Q0uZxFSu9sD5nRkR l+tfV3OqPlX+uLkn0ANbHbESCePX4Y0Gj9UStdPAnOx+lZvHquco3e4O5HfgDYvl4JHh 7eujLJn9FrJBBU9iYL9KOG++UB27bxaGKzMr/OMyu0PvY+tE8/lcFZr0NrUnO/lI0Y4T omd/5hs4jOZbQpDU5kFST5Plx/a2f3RFYeZes+MBT8Em7EAtkP3Wh+6JmWcVPvezOjMh kF/97XiZmpSi7ZGHMCYl0P8Na5oJtHYN4NZKmGs7MYPC7DusLL05lyFEKpugir9Hu14c 0XzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Yz+5IQwz1PHNAgvclmhCEIHgyfGRA67qmVd0f2CpqfY=; b=MGuVYQ/vxXa9gzx0pGl38dL0oZSoRFMOEIxgkVza26uDzAPiZwjQZA26IsB16FF51L IpqL3ht2+42PnSPMiOiIc8BsWtno8YqiyxhjSMWySkkAfF8/qTFaK/w/pRqh6XsVn1WV CsCcIJFBBw2kbryP1bi8wk3+mEpl+XqFwlxqjypHHjTM8kP+oD+4BCkSfdhRTvQo496C dfBVFNMNeq62csS8wFZcx0jZzAWcdMu2CWsVPDQnzMfand3hreyPkHuPmreYtn0/UY3x 6aVmF5nxrLs56OHTaqYDeKG3ZHONQtfalF27mUwhXk8ltj7W4jLqPTWm9o5Ax9b20BQn VyLQ== X-Gm-Message-State: AOAM533LU+s0syYr9oAO0FAkAk4MuoysLsVbMgQji7NVm7pMQhv1W91Q NYONz46rxDs1ybcC07k5bQfdBj8RmM9lxgyoBt4= X-Google-Smtp-Source: ABdhPJwIQH7cO+h2H+YYei8PWyPiS1L921F16Fu5I/aqTG9jzxcPhYtm2585s40h7ZZryU0sTAknPMQfYfgTorNTIw0= X-Received: by 2002:a05:6830:1e51:: with SMTP id e17mr6132947otj.292.1614911449852; Thu, 04 Mar 2021 18:30:49 -0800 (PST) MIME-Version: 1.0 References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> In-Reply-To: <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> From: Pip Cet Date: Fri, 5 Mar 2021 02:30:13 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Daniel Colascione Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Eli Zaretskii , Paul Eggert X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Thu, Mar 4, 2021 at 10:26 PM Daniel Colascione wrote: > On 3/3/21 12:51 AM, Eli Zaretskii wrote: > > Daniel, Paul: any comments? In particular, is it safe to allocate > > large amounts of memory off the heap while dumping? A couple of > > places in pdumper.c says some parts of code should call malloc. > > It looks fine, but wouldn't dumping to a FILE* (with internal buffering) > do the same basic thing in a simpler way? I initially set out to do that, but decided against it. We don't just write sequentially (when FILE I/O helps, a little), we also have the seek-and-fixup phase, and it didn't seem any simpler at that point.. > There aren't any particular > constraints on the environment _during_ the dump: we even make new lisp > objects. It's when loading the dump, early in initialization, that you > have to be careful. Thanks! Pip From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 02:19:26 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 07:19:26 +0000 Received: from localhost ([127.0.0.1]:60870 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI4kA-00068n-2A for submit@debbugs.gnu.org; Fri, 05 Mar 2021 02:19:26 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45688) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI4k7-00068X-SO for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 02:19:25 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52967) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lI4k2-0003sk-6I; Fri, 05 Mar 2021 02:19:18 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3748 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lI4k0-0007wX-1s; Fri, 05 Mar 2021 02:19:18 -0500 Date: Fri, 05 Mar 2021 09:19:00 +0200 Message-Id: <83a6riysnv.fsf@gnu.org> From: Eli Zaretskii To: Pip Cet In-Reply-To: (message from Pip Cet on Fri, 5 Mar 2021 02:30:13 +0000) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, dancol@dancol.org, eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Pip Cet > Date: Fri, 5 Mar 2021 02:30:13 +0000 > Cc: Eli Zaretskii , Paul Eggert , 46881@debbugs.gnu.org > > > It looks fine, but wouldn't dumping to a FILE* (with internal buffering) > > do the same basic thing in a simpler way? > > I initially set out to do that, but decided against it. We don't just > write sequentially (when FILE I/O helps, a little), we also have the > seek-and-fixup phase, and it didn't seem any simpler at that point.. I'm not sure I understand: what's wrong with fseek? From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 02:39:12 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 07:39:12 +0000 Received: from localhost ([127.0.0.1]:60900 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI53H-0006fF-L9 for submit@debbugs.gnu.org; Fri, 05 Mar 2021 02:39:12 -0500 Received: from mail-oi1-f180.google.com ([209.85.167.180]:36144) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI53F-0006f2-Fq for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 02:39:10 -0500 Received: by mail-oi1-f180.google.com with SMTP id j1so1546666oiw.3 for <46881@debbugs.gnu.org>; Thu, 04 Mar 2021 23:39:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=VZ0z9pEvfnZzbQ0sPXcm6gvsEnaOshuFZVeRShqCOBs=; b=psOPkvKQ24fOwb2ymV0kqfJQqZ33IJ1QiJgCfFYlaPEh5j/5nhAcR/ZLf6qHZcAZDs tiNTOBv6yub40h+HsironL7uzCbpgRSjmQyrvrmJW8iL6J4baVPDKZfmHuBlFXlxgPRW WvpVj2SZ108TxyPYV3A9WAinFu5xf3gvLn53MV8ylmEwnUy4Baim1oj35KTsKGlsB9hb ThpWb7XoZE5VoaySaorsisE7dLFfuYTEvScZCfYA4rH+nyZY2HVJb4OdArAuYrklC0i3 zM6+trT4AqX3MoGvGH8rBYuP5qRWTs2TYT2ODv2/i/OPR+IFySndmLKXc03uMjFSy4Dk r6yQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=VZ0z9pEvfnZzbQ0sPXcm6gvsEnaOshuFZVeRShqCOBs=; b=by9gFRev5bfXVYv/1rjjfLRPZOx4m98Zi63ESMIUhcWKkVvGddF2hMzJ9oNPoBB2HR LrquK7av2wKj/V4KddZXN2v5enSZ3olKKkLncn6i8OKacyRiPyrSc+QijyUrniX/B72T cv+9gM17EE07743b+QbFAgV5fFCgAIrQ6gcPvGgCXwvx3uzk/2XccXb4yWr7SRlGCzwa Mvq9Mz/DWZEZzp56++EZ5s8DSATe2k4zh+Et8w1ReNRrqDc+aOKVG5nYDEar9S5em7uY O319dDNAKjw2xuTsAllIcUzubYlUlP7thbMgSspoJi9ppyAo980aafjeXisadnQOsKj+ GHpw== X-Gm-Message-State: AOAM5330VczhMc2KA4jOaDymOwIbHLuMhK2xRcV70/AXFBoR3vnd8RX2 cUo5Rj/9hNeayY3WBkBY++u9/z6iCbTqUqZcV5w= X-Google-Smtp-Source: ABdhPJwGm0E5IrOyoWiHIdVuB2ubCbv9nAM1U73re/fe0kHvackB7JamNXOYoBqltDF2U1wbUJRTwG+1L9NXcxYDN84= X-Received: by 2002:aca:4c0f:: with SMTP id z15mr2190051oia.44.1614929943855; Thu, 04 Mar 2021 23:39:03 -0800 (PST) MIME-Version: 1.0 References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> In-Reply-To: <83a6riysnv.fsf@gnu.org> From: Pip Cet Date: Fri, 5 Mar 2021 07:38:27 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Eli Zaretskii Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Daniel Colascione , eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Fri, Mar 5, 2021 at 7:19 AM Eli Zaretskii wrote: > > From: Pip Cet > > Date: Fri, 5 Mar 2021 02:30:13 +0000 > > Cc: Eli Zaretskii , Paul Eggert , 46881@debbugs.gnu.org > > > > > It looks fine, but wouldn't dumping to a FILE* (with internal buffering) > > > do the same basic thing in a simpler way? > > > > I initially set out to do that, but decided against it. We don't just > > write sequentially (when FILE I/O helps, a little), we also have the > > seek-and-fixup phase, and it didn't seem any simpler at that point.. > > I'm not sure I understand: what's wrong with fseek? Nothing, assuming you're fine with the current performance. Many libcs aren't going to be smart enough to avoid I/O when you fseek through a "large" file and write a word here and there, and my suspicion is that would include glibc. Also, we're not currently using fseek-and-write anywhere in Emacs. We're talking about a file which Emacs is going to have to keep in memory anyway, when reading the dump. The only case in which there might be a problem is if the build machine has significantly less available memory than the machine we intend to run on, and I just don't think that's going to happen. Pip From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 02:55:07 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 07:55:07 +0000 Received: from localhost ([127.0.0.1]:60916 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI5Ih-00072J-6G for submit@debbugs.gnu.org; Fri, 05 Mar 2021 02:55:07 -0500 Received: from eggs.gnu.org ([209.51.188.92]:52276) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI5Ie-00071k-UJ for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 02:55:05 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53251) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lI5IX-0000OO-QA; Fri, 05 Mar 2021 02:54:57 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1971 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lI5IX-0005oi-6t; Fri, 05 Mar 2021 02:54:57 -0500 Date: Fri, 05 Mar 2021 09:54:41 +0200 Message-Id: <83zgzixcfy.fsf@gnu.org> From: Eli Zaretskii To: Pip Cet In-Reply-To: (message from Pip Cet on Fri, 5 Mar 2021 07:38:27 +0000) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, dancol@dancol.org, eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Pip Cet > Date: Fri, 5 Mar 2021 07:38:27 +0000 > Cc: Daniel Colascione , eggert@cs.ucla.edu, 46881@debbugs.gnu.org > > > I'm not sure I understand: what's wrong with fseek? > > Nothing, assuming you're fine with the current performance. Many libcs > aren't going to be smart enough to avoid I/O when you fseek through a > "large" file and write a word here and there, and my suspicion is that > would include glibc. Could we benchmark the two implementations instead of acting on suspicions? In general, I'd prefer not to reinvent the wheel, and trust modern libc's that they are efficient enough in handling buffered streams, unless we have hard evidence to the contrary. If nothing else, it would prevent people asking, like Daniel did, why didn't we use stdio in the first place. > Also, we're not currently using fseek-and-write anywhere in Emacs. I don't see why this would be important. Since we open the file in binary mode, fseek should work correctly even on non-Posix systems. Am I missing something? > We're talking about a file which Emacs is going to have to keep in > memory anyway, when reading the dump. The only case in which there > might be a problem is if the build machine has significantly less > available memory than the machine we intend to run on, and I just > don't think that's going to happen. You are thinking about memory consumption, while I am thinking how to avoid implementing our own private buffered streams. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 04:35:32 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 09:35:33 +0000 Received: from localhost ([127.0.0.1]:32871 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI6rs-000144-ML for submit@debbugs.gnu.org; Fri, 05 Mar 2021 04:35:32 -0500 Received: from mail-out.m-online.net ([212.18.0.9]:52540) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI6rq-00013u-7X for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 04:35:31 -0500 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4DsMzN49hKz1qtdy; Fri, 5 Mar 2021 10:35:27 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4DsMzM5MPvz1qqkq; Fri, 5 Mar 2021 10:35:27 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id IM447b6YijDw; Fri, 5 Mar 2021 10:35:27 +0100 (CET) X-Auth-Info: fhSJwioBJAvJNsQ/GmLU0kYqA/+5GVM3Jh8ffWYZkuSrPplrSvCCijjRsl11t7pP Received: from igel.home (ppp-46-244-183-40.dynamic.mnet-online.de [46.244.183.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA; Fri, 5 Mar 2021 10:35:27 +0100 (CET) Received: by igel.home (Postfix, from userid 1000) id 76A0F2C362B; Fri, 5 Mar 2021 10:35:26 +0100 (CET) From: Andreas Schwab To: Pip Cet Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> X-Yow: Someone is DROOLING on my collar!! Date: Fri, 05 Mar 2021 10:35:26 +0100 In-Reply-To: (Pip Cet's message of "Fri, 5 Mar 2021 07:38:27 +0000") Message-ID: <87y2f254f5.fsf@igel.home> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.4 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Eli Zaretskii , eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.4 (-) On Mär 05 2021, Pip Cet wrote: > We're talking about a file which Emacs is going to have to keep in > memory anyway, when reading the dump. While reading the dump, you only have the data once, and you don't have to realloc the whole data. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 04:42:31 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 09:42:31 +0000 Received: from localhost ([127.0.0.1]:32881 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI6yd-0001Eq-69 for submit@debbugs.gnu.org; Fri, 05 Mar 2021 04:42:31 -0500 Received: from mail-oi1-f174.google.com ([209.85.167.174]:39773) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI6yb-0001Ee-Or for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 04:42:30 -0500 Received: by mail-oi1-f174.google.com with SMTP id z126so1838416oiz.6 for <46881@debbugs.gnu.org>; Fri, 05 Mar 2021 01:42:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=D3uqr5W55Z7Yrg+ynSUew8pX3D7ASLuo5MTdlD9SVPM=; b=RSlcyhA3KSavK/a4uEbd203l9l7vUkhNKSgsrc6etmpQQ+L7lHmM9S+YFF+aox7qAR 35V0d2XNYHr6gwUnZFITfvresFoRJLPuywZ9yE9vBL7Dx5h6EdPorQbNivJJUiwXqVUJ 7UQCAj6COYZadZ0cSUnHxwJTKYcgMabF1kKheubEtwhPmgWSgsJk8XwMRNj1+NyUwe07 JXfmGPE0a8vsUlUDgXd1vUSpktafMObcOluVAJKOr0Rg2MXzhK0QXjTqIxoOAzLbW2PE Zm75rYpSCydTu/IfReVqiJ+EbvkQVZGoh057s9uvakjYuF9oCcrUB+gRTkbPxLTc0HUi JmCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=D3uqr5W55Z7Yrg+ynSUew8pX3D7ASLuo5MTdlD9SVPM=; b=LWSgVTU5uDX/WHiyd/XbZytmYMaDcHiZHG+kKaKY+axeAFnZT8YGnRks1xXbb4a/Gv glxaE6kvhgbmQaNuDr4WGabXNq9fV8U4sG47/tdrITCFSOxLgAsuSIUZQ9PHSYX6x7cU yo0aNXaIUZjs/3lQxZgYIxmxILGd4qWtC27EUdReoVG1dral8zpC31qj6u97rVxO5/vA 1kKMgIUoO9Cw80IAmPfz9HCVGxiIGmpeUvXwn0gn1gP8QdN5QGLxe0GKUspWiiZ5+bMk PX3k7DE+AI/m8/oRhR0NzDG48wYFMkBqmtviB9ZV1z5QdC3TWqt7YQx1z+sE+pcG8/jy eDhA== X-Gm-Message-State: AOAM530urZZFlfUaK1g899kqexxg7OIO2lAGY2rXheX9O8VLLV0OhZJs QHEhPNeB56a0zb33ffhhEWJwcDzbxQYSE5frFvY5SFIIl19qww== X-Google-Smtp-Source: ABdhPJxc1FMLpBtHwhRXFjg5qKRPjNLZRZFJ0HlksfD53dIJjuwYWXgkPMOuzXEIpJHywifToyTZxBp4zZLFqLn3OVY= X-Received: by 2002:aca:d905:: with SMTP id q5mr6185981oig.30.1614937343876; Fri, 05 Mar 2021 01:42:23 -0800 (PST) MIME-Version: 1.0 References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <87y2f254f5.fsf@igel.home> In-Reply-To: <87y2f254f5.fsf@igel.home> From: Pip Cet Date: Fri, 5 Mar 2021 09:41:47 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Andreas Schwab Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Eli Zaretskii , eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Fri, Mar 5, 2021 at 9:35 AM Andreas Schwab wrote= : > On M=C3=A4r 05 2021, Pip Cet wrote: > > > We're talking about a file which Emacs is going to have to keep in > > memory anyway, when reading the dump. > > While reading the dump, you only have the data once, and you don't have > to realloc the whole data. Correct. I think a build memory usage of 28 MB for a 10 MB dump file is something we can live with... Pip From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 04:55:17 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 09:55:17 +0000 Received: from localhost ([127.0.0.1]:32905 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI7Az-0001Zv-1N for submit@debbugs.gnu.org; Fri, 05 Mar 2021 04:55:17 -0500 Received: from mail-oo1-f53.google.com ([209.85.161.53]:43607) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI7Aw-0001Zh-OS for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 04:55:15 -0500 Received: by mail-oo1-f53.google.com with SMTP id x19so280470ooj.10 for <46881@debbugs.gnu.org>; Fri, 05 Mar 2021 01:55:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=/pUVpAbPc7Snr4jWcgI+Fr34P94CvOSjnbZxL7DVfXU=; b=k4Dnu2pWWSsfRJn16TNN3tfWhtXAxXhvCo+knS4NNTUuSu43WRATaf7Kgh0W9k0ydH 24CIXmVIK794htiv2bPmOsvX7ZbuRY7y8kPWgFNg+/bE8XbdTIe2FIcGBDbupOWailTz c+CXt37E6BHifUh62oApnwbrZtnlqDqPDaXf9EAY3CXU0LzRrtjP9c8pfniYzIo2gm4Z kRC5Kux5sRMHksbPR1ZcQA0VwS8c3UagF/OhFFtTwfDL5Hyn1+4FEGWfsG95Exio6beC ex4hErZWpAeleZ3YYMrNr1oshOyM2LJioTEqaVo64Nkt0OkafA3LKZGmIP2WgoV+zsuN CdzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/pUVpAbPc7Snr4jWcgI+Fr34P94CvOSjnbZxL7DVfXU=; b=jnD5Io+GgbdOLO8sUnhekX9cOoCKMZ7c+NIWq2i+5C+8baZZ9oH1wh5QX5A9yRTiw6 chkdEh+Ofbcj/EX8r5lm1xxFi9Huq5t+OBhqHvp0McZjmlQwT90zEF4ZIFt0V0HhPefU dtJr9Qwec1xRVbyLXPzJsP4dug8mQhG2pFz+DDOAWY7CNYIrYAceWovWyYRta0FMNzP/ ypt7wjObOljvqM9LF8fw6LEqJ8QiA1L+ZiFHjcrXvTMN+dbV3DFQC+1ETfTkL4hSXNvX AYooUwhMsBXikUNB7KgXYRSJS6GmL3jpVaNIKSixV5oQPWXkU/p+vuzozQb3SBUY5D/+ v5sw== X-Gm-Message-State: AOAM532dvDTA1CKuVp70nSJ3ueI1eJDIymzUyNY9vN4c0coRMdkAz5fk perIGSQVFHZy0XgygY7gQGlvelnwc4L/fL3G1Sg= X-Google-Smtp-Source: ABdhPJzwUz0HRGSnxn97A5/I2FLppqHNkracf205PzStwH++5E9gyq/88OUef5GOlI5zuICJfNzWCwag/vRDs9B+ZJk= X-Received: by 2002:a4a:2a0a:: with SMTP id k10mr7039762oof.88.1614938109021; Fri, 05 Mar 2021 01:55:09 -0800 (PST) MIME-Version: 1.0 References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> In-Reply-To: <83zgzixcfy.fsf@gnu.org> From: Pip Cet Date: Fri, 5 Mar 2021 09:54:32 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Eli Zaretskii Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Daniel Colascione , eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Fri, Mar 5, 2021 at 7:55 AM Eli Zaretskii wrote: > > From: Pip Cet > > Date: Fri, 5 Mar 2021 07:38:27 +0000 > > Cc: Daniel Colascione , eggert@cs.ucla.edu, 46881@debbugs.gnu.org > > > > > I'm not sure I understand: what's wrong with fseek? > > > > Nothing, assuming you're fine with the current performance. Many libcs > > aren't going to be smart enough to avoid I/O when you fseek through a > > "large" file and write a word here and there, and my suspicion is that > > would include glibc. > > Could we benchmark the two implementations instead of acting on > suspicions? Sure. My patch: real 0m1.988s user 0m1.916s sys 0m0.073s fwrite-based patch: real 0m3.576s user 0m2.571s sys 0m1.006s This is as I expected: glibc just isn't doing a very good job for this buffered stream. > In general, I'd prefer not to reinvent the wheel, and trust modern > libc's that they are efficient enough in handling buffered streams, > unless we have hard evidence to the contrary. We do, now. > If nothing else, it > would prevent people asking, like Daniel did, why didn't we use stdio > in the first place. I think it's a very good question (in fact, the brach I'm working on is called pdumper-fwrite because I decided only after creating it that all the seeking would hurt performance too much). I'll try including a comment explaining why. > > Also, we're not currently using fseek-and-write anywhere in Emacs. > > I don't see why this would be important. Because the stream returned by emacs_fopen might not be generally seekable? > Since we open the file in > binary mode, fseek should work correctly even on non-Posix systems. I guess I should have used emacs_fopen :-) > > We're talking about a file which Emacs is going to have to keep in > > memory anyway, when reading the dump. The only case in which there > > might be a problem is if the build machine has significantly less > > available memory than the machine we intend to run on, and I just > > don't think that's going to happen. > > You are thinking about memory consumption, while I am thinking how to > avoid implementing our own private buffered streams. By preparing the data in memory and writing it in one go, which doesn't require any of the major complications of implementing buffered streams. Pip From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 05:23:56 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 10:23:56 +0000 Received: from localhost ([127.0.0.1]:32938 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI7ci-0002Hi-2k for submit@debbugs.gnu.org; Fri, 05 Mar 2021 05:23:56 -0500 Received: from mx.sdf.org ([205.166.94.24]:58264) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI7cg-0002HZ-BT for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 05:23:55 -0500 Received: from mab (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id 125ANjC3003950 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits) verified NO); Fri, 5 Mar 2021 10:23:45 GMT From: Andrea Corallo To: Pip Cet Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> Date: Fri, 05 Mar 2021 10:23:45 +0000 In-Reply-To: (Pip Cet's message of "Fri, 5 Mar 2021 09:54:32 +0000") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Eli Zaretskii , eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Pip Cet writes: [...] > By preparing the data in memory and writing it in one go, which > doesn't require any of the major complications of implementing > buffered streams. Preparing data in memory might also be seen as a small step in the direction of having pdumper as a generic de/serializer. IMO would be helpful for a number of tasks (native compiler included). Andrea From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 07:07:29 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 12:07:29 +0000 Received: from localhost ([127.0.0.1]:33068 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI9Ev-00053A-3t for submit@debbugs.gnu.org; Fri, 05 Mar 2021 07:07:29 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50112) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI9Et-00052z-Ad for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 07:07:27 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35305) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lI9En-0008Le-8j; Fri, 05 Mar 2021 07:07:21 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1570 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lI9Ed-0006Mh-IQ; Fri, 05 Mar 2021 07:07:19 -0500 Date: Fri, 05 Mar 2021 14:06:55 +0200 Message-Id: <83v9a5yfc0.fsf@gnu.org> From: Eli Zaretskii To: Pip Cet In-Reply-To: (message from Pip Cet on Fri, 5 Mar 2021 09:54:32 +0000) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, dancol@dancol.org, eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Pip Cet > Date: Fri, 5 Mar 2021 09:54:32 +0000 > Cc: Daniel Colascione , eggert@cs.ucla.edu, 46881@debbugs.gnu.org > > My patch: > > real 0m1.988s > user 0m1.916s > sys 0m0.073s > > fwrite-based patch: > > real 0m3.576s > user 0m2.571s > sys 0m1.006s 30% slowdown and 1.5 sec absolute time difference doesn't sound bad enough to me to justify a homemade solution. I say let's go with stdio. > > > Also, we're not currently using fseek-and-write anywhere in Emacs. > > > > I don't see why this would be important. > > Because the stream returned by emacs_fopen might not be generally seekable? I don't see how that could happen. > > Since we open the file in > > binary mode, fseek should work correctly even on non-Posix systems. > > I guess I should have used emacs_fopen :-) Yes, of course. Especially as with fopen there are problems with non-ASCII file names on MS-Windows. > By preparing the data in memory and writing it in one go, which > doesn't require any of the major complications of implementing > buffered streams. There are no complications I can see, not in our sources. (And you don't actually write it in one go anyway, see emacs_full_write.) So let's go with the stdio solution, please. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 07:49:59 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 12:49:59 +0000 Received: from localhost ([127.0.0.1]:33153 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI9u2-0008Bl-Tn for submit@debbugs.gnu.org; Fri, 05 Mar 2021 07:49:59 -0500 Received: from quimby.gnus.org ([95.216.78.240]:34262) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lI9u0-0008BW-Q9 for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 07:49:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=gjm4us0t1R5jS+aoIrtqLbN4JhaTd6Exl9H4KfBtOeA=; b=IlorhVDZ7Mz2WTNUybESlvSpgM OZusR4NN6B2nLAv5WHth2zbobOmDklqRzB7F3gB+k7qFTVkelfgMGfCNuaWXiDg5f0dOWDBHZBTym Kx4zm1r1/ZRIEAqZXIBKDNRDMaNYZQHthu8zQ1zFWlmrWQ5GMhC0WQhk9hOV9n0iJNdY=; Received: from cm-84.212.220.105.getinternet.no ([84.212.220.105] helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lI9tn-0006O0-TM; Fri, 05 Mar 2021 13:49:48 +0100 From: Lars Ingebrigtsen To: Eli Zaretskii Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> <83v9a5yfc0.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAG1BMVEU+RoM3R4Uscq4n hsItS41TWpOwr8o1iL7///9ogBk0AAAAAWJLR0QIht6VegAAAAd0SU1FB+UDBQwvMinyOvoAAAGs SURBVDjLdZNLcoMwDIbJDWJad49c2nWtCex7A+iYPcngrBtm8PUr+QFOQkVMHH2W5N+xikI824FG sQfYikMJoJAGBNNK00sciwO8VgDvJXxDJQDKqvScQPVcgdLQgL0KIPIIabp9YITMAUgjDZuggI6/ AxBgognZmYknlFCJwtTR3wljhlGGKQNng9E6Y6eYlXQMGRjsFLNSDbusgFLR43MxaLaIt8EOIabc wEQChxX4VBd7ZUDirJkGS+6+zoCVspNm9AA8aC6uZeAPy1jeWAAfF2xjcSKSgTqFiBk9CEc7rGC5 IPpdxTM3D0AkQARjKpy9jPX/6vHU+4gZFzvmQNes3HrQZeBH16lGY/Ob0OsoEPFj+ge0d8Ak0GI7 5qBOOtoZfzN/qTOQ39MXDAKXc3vDrwyoFSCizgAmQLtCteUq6ecqMA9R92AN4QCsVRTIVm1+ijim CDLuoFJl4KxQaepP1BCXRHDzLas2N6Vl0MzeC/TB1jlHY44ANT6YB8uVlrlb7P8KPmnaMXDpTse2 5A7rWaDzpoC2xbVoD5omfBlSd6zd7ht3H3RH8QdZYeMm+rfuuwAAACV0RVh0ZGF0ZTpjcmVhdGUA MjAyMS0wMy0wNVQxMjo0Nzo1MCswMDowMCUNay8AAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDMt MDVUMTI6NDc6NTArMDA6MDBUUNOTAAAAAElFTkSuQmCC X-Now-Playing: Brigid Mae Power's _Head Above the Water_: "The Blacksmith" Date: Fri, 05 Mar 2021 13:49:42 +0100 In-Reply-To: <83v9a5yfc0.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 05 Mar 2021 14:06:55 +0200") Message-ID: <877dmlsr2x.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Eli Zaretskii writes: > 30% slowdown and 1.5 sec absolute time difference doesn't sound bad > enough to me to justify a homemade solution. I say let's go with > stdio. Seems significant to me -- we're building Emacs a lot, and this bit can't be parallelised. And the savings in electricity alone should make us go for the most efficient solution. Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, eggert@cs.ucla.edu, Pip Cet X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: > 30% slowdown and 1.5 sec absolute time difference doesn't sound bad > enough to me to justify a homemade solution. I say let's go with > stdio. Seems significant to me -- we're building Emacs a lot, and this bit can't be parallelised. And the savings in electricity alone should make us go for the most efficient solution. There doesn't seem to be any significant drawbacks to doing it the efficient way, either. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 08:16:58 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 13:16:58 +0000 Received: from localhost ([127.0.0.1]:33212 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIAK9-00049d-Mq for submit@debbugs.gnu.org; Fri, 05 Mar 2021 08:16:58 -0500 Received: from mail-oi1-f175.google.com ([209.85.167.175]:35934) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIAK8-00044Q-Dh for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 08:16:57 -0500 Received: by mail-oi1-f175.google.com with SMTP id j1so2467027oiw.3 for <46881@debbugs.gnu.org>; Fri, 05 Mar 2021 05:16:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=H0SXET0ERFvOY8lTy7BnLVCeaYnyZz1ClNI5Aa6Rr1k=; b=KHBoAln4dkiPDt1K7dk/CD05wWfrGFrKkpTKjHkQPYDBZjKKFnNi6P3oRVe/MvrNjA XSzX4gLRvsqDDSqlvSScbcn7ZVmEwMUl/qEam6y7ytwRSTDiCJB6ZGAdgQDVQr/X3anG fsbeL4Iun4Lit+SgWujb8uKwamzSUx8u00y5tL9uGpA88mMrI7tcptdMj8szJuhxYeO/ FN6Ue5nux35oPs+5G9b/rA7Q9Wp70x+E6JnNzeymQaxXWipY+jHdPOuJURRM9hSDOSuL vXE8HrRq1A2vH9rOj6lxeiNKA37W+Wf4obLz/hY4NpsyCehw3KjzJl7iKIMXrZrBX/jD V06Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=H0SXET0ERFvOY8lTy7BnLVCeaYnyZz1ClNI5Aa6Rr1k=; b=Uts56fRDR4fyEDNKdFLptg5fpO+5oOqjcifcbcvZB0U2Q/Br6jNInQ5q+/5aIWLY3N 3vKI4TiJUkbeEiifnkHLHuuhp8fqEgCadE1EGgMPV3UpHnsJH/hT3+xv+xKR0prGgOHm L+9EAsUs0CC/uacUGO/c+2TFqXyINHKj2vgEI7HkZoTEo+8ViE0HGNa6LA20MIHcfqct yhSACMjy+gpRi+f1AHkoOyWfMS0+rE/Y9RbJstuHvWr8H50AxLiYgyrqgLTRN4EKzQ76 rfiFnlcxVNOfoDbxxX67n5EQcGYnRuqCF2XqvEz0wKpA2234uZMK2EPitqVRkyQSIDMn oZDA== X-Gm-Message-State: AOAM530epCuOWzRjHlEGLHr5X8tiMp5X4JdbS7E/XZGrSa5V9Z1XwWpg ZFwQrwZsK4jFOR+HzR4uC20jfz527cY07gO2Gx0= X-Google-Smtp-Source: ABdhPJyrccCB5s04d5/bgzi+YG0ag4UF7eDxUZlwZGS/HcavT4D2asqz4WCtxIsxj8i5S6vrhF1jlW0+QMshPDXyH1M= X-Received: by 2002:aca:4c0f:: with SMTP id z15mr3194359oia.44.1614950210855; Fri, 05 Mar 2021 05:16:50 -0800 (PST) MIME-Version: 1.0 References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> <83v9a5yfc0.fsf@gnu.org> In-Reply-To: <83v9a5yfc0.fsf@gnu.org> From: Pip Cet Date: Fri, 5 Mar 2021 13:16:14 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Eli Zaretskii Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Daniel Colascione , eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Fri, Mar 5, 2021 at 12:07 PM Eli Zaretskii wrote: > > From: Pip Cet > > Date: Fri, 5 Mar 2021 09:54:32 +0000 > > Cc: Daniel Colascione , eggert@cs.ucla.edu, 46881@debbugs.gnu.org > > > > My patch: > > > > real 0m1.988s > > user 0m1.916s > > sys 0m0.073s > > > > fwrite-based patch: > > > > real 0m3.576s > > user 0m2.571s > > sys 0m1.006s > > 30% slowdown and 1.5 sec absolute time difference doesn't sound bad > enough to me to It's a 30% slowdown of the entire dump process, including the CPU-intensive part which loads Emacs. I think you get a better idea of the performance difference from the "sys" numbers above. And the absolute time difference is more than that, because Emacs is dumped twice during each build; the first dump file is about 2.5 times the size of the ultimate dump file, so my guess (as I said before, unfortunately Intel decided to make this system not have a predictable CPU clock, so I can't really run good benchmarks) is we're talking about 4.5 seconds here. > justify a homemade solution. "Create a buffer in memory and do all the IO at once" is such an old solution that even the GNU Coding Standards explicitly recommend it (albeit for input files): You could keep the entire input file in memory and scan it there instead of using stdio >I say let's go with stdio. Maybe setbuffer(3) could help us here? I could run some benchmarks for that if the idea isn't out of the question. > > > > Also, we're not currently using fseek-and-write anywhere in Emacs. > > > > > > I don't see why this would be important. > > > > Because the stream returned by emacs_fopen might not be generally seekable? > > I don't see how that could happen. It has, to me, but I'm willing to accept I did some inadvisable things first. > > By preparing the data in memory and writing it in one go, which > > doesn't require any of the major complications of implementing > > buffered streams. > > There are no complications I can see, not in our sources. (And you > don't actually write it in one go anyway, see emacs_full_write.) Er, precisely. I was the one saying there are no complications, so we shouldn't let the idea of "implementing our own buffered streams" scare us, because that is a complicated project but it's also not what we are doing. > So let's go with the stdio solution, please. Should I add a sync after every seek to make absolutely certain, rather than merely likely, this will destroy someone's flash chip one day? Pip From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 08:24:20 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 13:24:20 +0000 Received: from localhost ([127.0.0.1]:33222 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIARH-0004ob-Vo for submit@debbugs.gnu.org; Fri, 05 Mar 2021 08:24:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:36162) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIARF-0004oJ-MB for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 08:24:18 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40742) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lIAR9-0000qE-TM; Fri, 05 Mar 2021 08:24:11 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2333 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lIAR9-0004Ny-5j; Fri, 05 Mar 2021 08:24:11 -0500 Date: Fri, 05 Mar 2021 15:23:55 +0200 Message-Id: <83pn0dybro.fsf@gnu.org> From: Eli Zaretskii To: Lars Ingebrigtsen In-Reply-To: <877dmlsr2x.fsf@gnus.org> (message from Lars Ingebrigtsen on Fri, 05 Mar 2021 13:49:42 +0100) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> <83v9a5yfc0.fsf@gnu.org> <877dmlsr2x.fsf@gnus.org> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, eggert@cs.ucla.edu, pipcet@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Lars Ingebrigtsen > Cc: Pip Cet , 46881@debbugs.gnu.org, eggert@cs.ucla.edu > Date: Fri, 05 Mar 2021 13:49:42 +0100 > > Eli Zaretskii writes: > > > 30% slowdown and 1.5 sec absolute time difference doesn't sound bad > > enough to me to justify a homemade solution. I say let's go with > > stdio. > > Seems significant to me -- we're building Emacs a lot, and this bit > can't be parallelised. And the savings in electricity alone should make > us go for the most efficient solution. > > There doesn't seem to be any significant drawbacks to doing it the > efficient way, either. Fine, let's do it that way, then. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 09:03:16 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 14:03:16 +0000 Received: from localhost ([127.0.0.1]:33311 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIB2y-0007xJ-EV for submit@debbugs.gnu.org; Fri, 05 Mar 2021 09:03:16 -0500 Received: from mail-ot1-f50.google.com ([209.85.210.50]:40889) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIB2x-0007x5-41 for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 09:03:15 -0500 Received: by mail-ot1-f50.google.com with SMTP id b8so1858097oti.7 for <46881@debbugs.gnu.org>; Fri, 05 Mar 2021 06:03:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=cWHWfVt8D63BMHYJ7E3DoVq79QE51UbB2DmOOJdPdk4=; b=EcB8NcerdTKF89sdzpDXoTaYfRwFVtXRRfoJ9r0iuxlybQIKKHC3hVOAX0rzHyadQi 1xZ7a0ze/zgIKqIB39s7TTMf0N2uGvlrwP3HlBEAgPppl8U3S3GbBnVZXfTIWStjjZVc XO/RUr3bReQFuVjevCPInXDrCFZ68yyu4ttfQG0MzX78InwuHf1tLDP4ekBfHAJ/P8jE +JQfA1vhm9vDolRB5UqGtazMS+oIzTNnO/ajCKTuAR56z07ZKKuEOk51Z7vIQAB2W6mZ rhSESwSQl6rPllK6ZcgcQFyK/apiMEwT3vZANzVgY24+IR/wf86z+b1DXS0weFzRjmLR LWAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=cWHWfVt8D63BMHYJ7E3DoVq79QE51UbB2DmOOJdPdk4=; b=fDXaYRaLVy/+qGD0hS4DhXixrt2ceUpKYxV7y1hwP5wt75vY6GuNl0scw4UoRyHLW6 I3Qq6WxDITx/j48Fha9OCkREE1oF9bNyt/XoEs2qd1MMRVvuSnZCef0NfOco/8dwYtpR ozJC8b1ffD5eUzZsXnhRZwwIyWbiJnuW5h6Py1WzIYyH50bIPybmlQHFr9dmlq/NXLDo THsISsKuUPaU6FIEBleOokMl+12nFPD/se5mWPQejB9/1zsaO7/bcC4I3dShV/NbKoUf pQNjsoKsUMBtCn3Bs0RwCoWgoVq5JtBLH+WK6DRBc1gQ5Fpq8zyrAvh595s8uRVFD+l2 YU1g== X-Gm-Message-State: AOAM530/CTYwRm37wDW8rQXg+TGs/3DrDLLjJO77uelEyX0VUQSXrp02 6UH0e+AEz5KnZl3Fih2+bN+1E6N6PrWVfX+Y680= X-Google-Smtp-Source: ABdhPJzrAtMGz1K9b8uS5l7rcbwl5k/M8D+3xPeBXjvS0XvTRzIeWd0Sd9363g8dsnsYypGhOhsMbXKQiu0f+3AFkdk= X-Received: by 2002:a05:6830:11d4:: with SMTP id v20mr7987913otq.287.1614952989425; Fri, 05 Mar 2021 06:03:09 -0800 (PST) MIME-Version: 1.0 References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> <83v9a5yfc0.fsf@gnu.org> In-Reply-To: From: Pip Cet Date: Fri, 5 Mar 2021 14:02:33 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Eli Zaretskii Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Daniel Colascione , eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Fri, Mar 5, 2021 at 1:16 PM Pip Cet wrote: > >I say let's go with stdio. > > Maybe setbuffer(3) could help us here? I could run some benchmarks for > that if the idea isn't out of the question. Actually, calling setbuffer() with a large buffer will make glibc reread the entire file on every fseek(), rendering the dump so slow I gave up and interrupted it. However, there's open_memstream: that would have the added advantage of actually making glibc write out the entire file in one go, so it seems to shave a few extra milliseconds off the build. (However however, glibc's memstreams are somewhat broken. I'll file a bug report if there isn't one already...) Still, that way we could use stdio today, leave the buffering to glibc, and hopefully be able to switch trivially to a "open this file but keep it in memory" combined memstream/FILE* one day. Pip From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 09:13:54 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 14:13:54 +0000 Received: from localhost ([127.0.0.1]:33319 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIBDG-0008DG-Ce for submit@debbugs.gnu.org; Fri, 05 Mar 2021 09:13:54 -0500 Received: from dancol.org ([96.126.100.184]:49432) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIBDE-0008D8-C5 for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 09:13:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=zPPgF/7eO2Uj4xyZhkMnlfvQDA2sfcVuI8ZtqjIQlqg=; b=mnVacmzD3CI0dP5qMP1oJ7Twtf zV7Lyk6yDfrvgZa9T9iRCFlb29rQw2068rCTAlkjDcmkYialZOrxU6bCaVisSPQgg0suBiR8tXfaL 4wrm5FWaWB/X0OuPr2/rRbRVMTovtX0Me3zGltCrKpVS38Qn2sVLyK9B1GZjNVphBCVXxvMvR3qED d0H9rVAzkPtaB+S+3DpR6SjYBzF3ceTghdpjGkumsaXoDzpHY24h5qi6RvseThYQ2gtWT7SR788VG EWQWSpFDhH3zAXLy5W79lp0zSZihiVEXbDrUEW8SHaLUw5RLOvFaCnWwT9IRK9242bIiFNXLLSPi4 l0notuZA==; Received: from [97.104.73.87] (port=49570 helo=[192.168.1.148]) by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1lIBDC-0001sT-Jv; Fri, 05 Mar 2021 06:13:50 -0800 Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Pip Cet , Eli Zaretskii References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> <83v9a5yfc0.fsf@gnu.org> From: Daniel Colascione Message-ID: <810bd1fb-dc9c-e65b-66cc-09324859569e@dancol.org> Date: Fri, 5 Mar 2021 09:13:49 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 3/5/21 9:02 AM, Pip Cet wrote: > On Fri, Mar 5, 2021 at 1:16 PM Pip Cet wrote: >>> I say let's go with stdio. >> Maybe setbuffer(3) could help us here? I could run some benchmarks for >> that if the idea isn't out of the question. > Actually, calling setbuffer() with a large buffer will make glibc > reread the entire file on every fseek(), rendering the dump so slow I > gave up and interrupted it. > > However, there's open_memstream: that would have the added advantage > of actually making glibc write out the entire file in one go, so it > seems to shave a few extra milliseconds off the build. > > (However however, glibc's memstreams are somewhat broken. I'll file a > bug report if there isn't one already...) > > Still, that way we could use stdio today, leave the buffering to > glibc, and hopefully be able to switch trivially to a "open this file > but keep it in memory" combined memstream/FILE* one day. You could also use fopencookie to make your own stdio stream that does the right thing while still using the stdio abstraction in the part of the code actually doing the writing. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 09:56:07 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 14:56:08 +0000 Received: from localhost ([127.0.0.1]:35135 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIBs7-0003N2-Cz for submit@debbugs.gnu.org; Fri, 05 Mar 2021 09:56:07 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37114) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIBs4-0003Me-O8 for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 09:56:05 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:42627) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lIBrx-00025H-OY; Fri, 05 Mar 2021 09:55:58 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4265 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lIBrw-000169-Vg; Fri, 05 Mar 2021 09:55:57 -0500 Date: Fri, 05 Mar 2021 16:55:43 +0200 Message-Id: <83k0qly7io.fsf@gnu.org> From: Eli Zaretskii To: Pip Cet In-Reply-To: (message from Pip Cet on Fri, 5 Mar 2021 14:02:33 +0000) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> <83v9a5yfc0.fsf@gnu.org> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, dancol@dancol.org, eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Pip Cet > Date: Fri, 5 Mar 2021 14:02:33 +0000 > Cc: Daniel Colascione , eggert@cs.ucla.edu, 46881@debbugs.gnu.org > > However, there's open_memstream That's glibc-only, AFAIK. Not portable enough for us. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 05 10:13:18 2021 Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 15:13:19 +0000 Received: from localhost ([127.0.0.1]:35147 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIC8k-0003mG-Gg for submit@debbugs.gnu.org; Fri, 05 Mar 2021 10:13:18 -0500 Received: from mail-ot1-f44.google.com ([209.85.210.44]:37170) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIC8g-0003m1-BZ for 46881@debbugs.gnu.org; Fri, 05 Mar 2021 10:13:17 -0500 Received: by mail-ot1-f44.google.com with SMTP id g8so2095752otk.4 for <46881@debbugs.gnu.org>; Fri, 05 Mar 2021 07:13:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=OQTY154O0EtOUQUT7wNx89gDq8dxgiLLXu6sCbODzzk=; b=oGc8CEfJH5tRP/RiLKVBo6Ijs/VujGqcx598ZIRWuHWbZPpMMrqMdXZ56BOMoGTqKa +Oseq3JtiTWwFAHRTqiTd+pJXriL9c97rKbFXZ1UXRmA6Mw1+xo9RUwTiXqr0XFxdKXL 5ZkWeKixKwfrx/FhA24/dxb9si+IKbSoX0/tFkgBXyQZSeCP3AgC6VYnJVL0ZcByDE09 gkyEFoxOe8ElXDwxrkvqCg3FDoXZ76qQDG8xVbis5ydGcdPTUC1U8uryJ8bPk3kzQdwP 9OFgBOOVuIBTNLmu/TQc1haKGj4BtpMzXEgaIk6UYQrwIIxnuZIEJhF3XKTZC68yHUW1 a1gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=OQTY154O0EtOUQUT7wNx89gDq8dxgiLLXu6sCbODzzk=; b=Twhf5Dvq50xmqwrLTEviVgBmGqfl0JE+aCFoQoV8KJ/P3C7X6soFLvXGonz1aCQiuN 7D8JCXNpP38SpSBz8kX5s40socYhK5lY/8/tmAF38KqQmKBD7cvtNMFCLmVFKTYg99nB Sz9H0FdR3/AGwt4Cm2KL4THxU51HHia3dhPelqUKw7NLCrH9ubUD4EcUabK4DRrHF7H3 4k5T1sIFt/3tBzOPijVs1p6RhgbBOqbVt3+QwL9ZusnJ2J9eAvj1CBnCogQCXBMXZuiz wf6xplOw4JqctAuCot2XEe4ttnMeMUtNnyb5WRv44w0lKt5gK6NIamfTpl6NKipjVEZr 8u+Q== X-Gm-Message-State: AOAM531EhcKloSoguuFe+OF3Mj/uSWVRKG6qwNWEvIJinIMqEwVAlDiF Dve01Ndn1NyPD0GL4de80jYN1oU0EEol/ObNGcU= X-Google-Smtp-Source: ABdhPJzCJOV5QJXeztbeX45TAXrhTvu5bmySqL3Pxf+hLit6HLeBxjbyMSlfFl5hYuZ5CBmLAPVl8pE8Gdih1pNV6x0= X-Received: by 2002:a05:6830:1e51:: with SMTP id e17mr8383116otj.292.1614957188753; Fri, 05 Mar 2021 07:13:08 -0800 (PST) MIME-Version: 1.0 References: <83r1kw6b06.fsf@gnu.org> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@dancol.org> <83a6riysnv.fsf@gnu.org> <83zgzixcfy.fsf@gnu.org> <83v9a5yfc0.fsf@gnu.org> <83k0qly7io.fsf@gnu.org> In-Reply-To: <83k0qly7io.fsf@gnu.org> From: Pip Cet Date: Fri, 5 Mar 2021 15:12:31 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: Eli Zaretskii Content-Type: multipart/mixed; boundary="000000000000904de305bccb854b" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Daniel Colascione , eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --000000000000904de305bccb854b Content-Type: text/plain; charset="UTF-8" On Fri, Mar 5, 2021 at 2:56 PM Eli Zaretskii wrote: > > From: Pip Cet > > Date: Fri, 5 Mar 2021 14:02:33 +0000 > > Cc: Daniel Colascione , eggert@cs.ucla.edu, 46881@debbugs.gnu.org > > > > However, there's open_memstream > > That's glibc-only, AFAIK. Not portable enough for us. POSIX.1-2008. Not portable enough to require, certainly, but portable enough to use? Pip --000000000000904de305bccb854b Content-Type: text/x-patch; charset="UTF-8"; name="0001-Use-stdio-memstreams-if-available-for-pdumper-Bug-46.patch" Content-Disposition: attachment; filename="0001-Use-stdio-memstreams-if-available-for-pdumper-Bug-46.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_klwfpc7y0 RnJvbSBiNjAwYzQ0NGUwNDZkMzg4OGY1NjgzOWRiZTcwZDZkMjA5YTZiYTI5IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBQaXAgQ2V0IDxwaXBjZXRAZ21haWwuY29tPgpEYXRlOiBUdWUs IDIgTWFyIDIwMjEgMjA6Mzg6MjMgKzAwMDAKU3ViamVjdDogW1BBVENIXSBVc2Ugc3RkaW8sIG1l bXN0cmVhbXMgaWYgYXZhaWxhYmxlIGZvciBwZHVtcGVyIChCdWcjNDY4ODEpCgoqIGNvbmZpZ3Vy ZS5hYzogQ2hlY2sgZm9yIG9wZW5fbWVtc3RyZWFtLgoqIHNyYy9wZHVtcGVyLmMgKGR1bXBfdW53 aW5kX2NsZWFudXApOiBIYW5kbGUgbWVtc3RyZWFtcy4KKGR1bXBfc2Vlayk6IFJlY29yZCBtYXhp bXVtIG9mZnNldCBmb3IgbWVtc3RyZWFtcy4gVXNlIHN0ZGlvLgooc3RydWN0IGR1bXBfY29udGV4 dCk6IEFkZCBidWZmZXIgaW5mb3JtYXRpb24gZm9yIG1lbXN0cmVhbXMuCihGZHVtcF9lbWFjc19w b3J0YWJsZSk6IFVzZSBtZW1zdHJlYW1zIGlmIGF2YWlsYWJsZS4KKGR1bXBfd3JpdGUpOiBVc2Ug c3RkaW8uCihGZHVtcF9lbWFjc19wb3J0YWJsZSk6IFVzZSBtZW1zdHJlYW1zIGlmIGF2YWlsYWJs ZS4KLS0tCiBjb25maWd1cmUuYWMgIHwgIDEgKwogc3JjL3BkdW1wZXIuYyB8IDY0ICsrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystLS0tLS0tLS0tLQogMiBmaWxlcyBjaGFu Z2VkLCA1MSBpbnNlcnRpb25zKCspLCAxNCBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9jb25m aWd1cmUuYWMgYi9jb25maWd1cmUuYWMKaW5kZXggMTFhMDZhMzliZWUzZi4uNDIyNWFlYTk2MzVj ZSAxMDA2NDQKLS0tIGEvY29uZmlndXJlLmFjCisrKyBiL2NvbmZpZ3VyZS5hYwpAQCAtNDU1NCw2 ICs0NTU0LDcgQEAgQUNfREVGVU4KIGRubCBBQ19DSEVDS19GVU5DU19PTkNFIHdvdWxkbuKAmXQg YmUgcmlnaHQgZm9yIHNucHJpbnRmLCB3aGljaCBuZWVkcwogZG5sIHRoZSBjdXJyZW50IENGTEFH UyBldGMuCiBBQ19DSEVDS19GVU5DUyhzbnByaW50ZikKK0FDX0NIRUNLX0ZVTkNTKG9wZW5fbWVt c3RyZWFtKQogCiBkbmwgQ2hlY2sgZm9yIGdsaWIuICBUaGlzIGRpZmZlcnMgZnJvbSBvdGhlciBs aWJyYXJ5IGNoZWNrcyBpbiB0aGF0CiBkbmwgRW1hY3MgbmVlZCBub3QgbGluayB0byBnbGliIHVu bGVzcyBzb21lIG90aGVyIGxpYnJhcnkgaXMgYWxyZWFkeQpkaWZmIC0tZ2l0IGEvc3JjL3BkdW1w ZXIuYyBiL3NyYy9wZHVtcGVyLmMKaW5kZXggMzM3NzQyZmRhNGFkZS4uYTUyNGY4NzMwMGI1MSAx MDA2NDQKLS0tIGEvc3JjL3BkdW1wZXIuYworKysgYi9zcmMvcGR1bXBlci5jCkBAIC00ODIsOCAr NDgyLDE0IEBAIGR1bXBfZmluZ2VycHJpbnQgKGNoYXIgY29uc3QgKmxhYmVsLAogICBib29sIGJs b2NrZWRfcmFsbG9jOwogI2VuZGlmCiAKLSAgLyogRmlsZSBkZXNjcmlwdG9yIGZvciBkdW1wZmls ZTsgPCAwIGlmIGNsb3NlZC4gICovCi0gIGludCBmZDsKKyAgLyogRmlsZSBoYW5kbGUgZm9yIGR1 bXBmaWxlOyBOVUxMIGlmIGNsb3NlZC4gICovCisgIEZJTEUgKmZpbGU7CisjaWZkZWYgSEFWRV9P UEVOX01FTVNUUkVBTQorICAvKiBidWYgaXMgbm9uLU5VTEwgaWYgZmlsZSByZWZlcnJlZCB0byBh IG1lbXN0cmVhbSBhbmQgaGFzIGJlZW4gY2xvc2VkLiAqLworICBjaGFyICpidWY7CisgIHNpemVf dCBidWZfc2l6ZTsKKyAgc2l6ZV90IG1heF9vZmZzZXQ7CisjZW5kaWYKICAgLyogTmFtZSBvZiBk dW1wIGZpbGUgLS0tIHVzZWQgZm9yIGVycm9yIHJlcG9ydGluZy4gICovCiAgIExpc3BfT2JqZWN0 IGR1bXBfZmlsZW5hbWU7CiAgIC8qIEN1cnJlbnQgb2Zmc2V0IGluIGR1bXAgZmlsZS4gICovCkBA IC03NDcsOSArNzUzLDE2IEBAIGR1bXBfd3JpdGUgKHN0cnVjdCBkdW1wX2NvbnRleHQgKmN0eCwg Y29uc3Qgdm9pZCAqYnVmLCBkdW1wX29mZiBuYnl0ZSkKICAgZWFzc2VydCAobmJ5dGUgPT0gMCB8 fCBidWYgIT0gTlVMTCk7CiAgIGVhc3NlcnQgKGN0eC0+b2JqX29mZnNldCA9PSAwKTsKICAgZWFz c2VydCAoY3R4LT5mbGFncy5kdW1wX29iamVjdF9jb250ZW50cyk7Ci0gIGlmIChlbWFjc193cml0 ZSAoY3R4LT5mZCwgYnVmLCBuYnl0ZSkgPCBuYnl0ZSkKLSAgICByZXBvcnRfZmlsZV9lcnJvciAo IkNvdWxkIG5vdCB3cml0ZSB0byBkdW1wIGZpbGUiLCBjdHgtPmR1bXBfZmlsZW5hbWUpOwotICBj dHgtPm9mZnNldCArPSBuYnl0ZTsKKyAgd2hpbGUgKG5ieXRlKQorICAgIHsKKyAgICAgIHNpemVf dCByZXMgPSBmd3JpdGUgKGJ1ZiwgMSwgbmJ5dGUsIGN0eC0+ZmlsZSk7CisgICAgICBpZiAocmVz ID09IDApCisJcmVwb3J0X2ZpbGVfZXJyb3IgKCJDb3VsZCBub3Qgd3JpdGUgdG8gZHVtcCBmaWxl IiwKKwkJCSAgIGN0eC0+ZHVtcF9maWxlbmFtZSk7CisgICAgICBidWYgPSAoY2hhciAqKWJ1ZiAr IHJlczsKKyAgICAgIGN0eC0+b2Zmc2V0ICs9IHJlczsKKyAgICAgIG5ieXRlIC09IHJlczsKKyAg ICB9CiB9CiAKIHN0YXRpYyBMaXNwX09iamVjdApAQCAtODI4LDEwICs4NDEsMTMgQEAgZHVtcF90 YWlscV9wb3AgKHN0cnVjdCBkdW1wX3RhaWxxICp0YWlscSkKIHN0YXRpYyB2b2lkCiBkdW1wX3Nl ZWsgKHN0cnVjdCBkdW1wX2NvbnRleHQgKmN0eCwgZHVtcF9vZmYgb2Zmc2V0KQogeworI2lmZGVm IEhBVkVfT1BFTl9NRU1TVFJFQU0KKyAgaWYgKGN0eC0+b2Zmc2V0ID4gY3R4LT5tYXhfb2Zmc2V0 KQorICAgIGN0eC0+bWF4X29mZnNldCA9IGN0eC0+b2Zmc2V0OworI2VuZGlmCiAgIGVhc3NlcnQg KGN0eC0+b2JqX29mZnNldCA9PSAwKTsKLSAgaWYgKGxzZWVrIChjdHgtPmZkLCBvZmZzZXQsIFNF RUtfU0VUKSA8IDApCi0gICAgcmVwb3J0X2ZpbGVfZXJyb3IgKCJTZXR0aW5nIGZpbGUgcG9zaXRp b24iLAotICAgICAgICAgICAgICAgICAgICAgICBjdHgtPmR1bXBfZmlsZW5hbWUpOworICBpZiAo ZnNlZWsgKGN0eC0+ZmlsZSwgb2Zmc2V0LCBTRUVLX1NFVCkgIT0gMCkKKyAgICByZXBvcnRfZmls ZV9lcnJvciAoIlNldHRpbmcgZmlsZSBwb3NpdGlvbiIsIGN0eC0+ZHVtcF9maWxlbmFtZSk7CiAg IGN0eC0+b2Zmc2V0ID0gb2Zmc2V0OwogfQogCkBAIC0zNTEzLDggKzM1MjksMjUgQEAgZHVtcF9k cmFpbl91c2VyX3JlbWVtYmVyZWRfZGF0YV9jb2xkIChzdHJ1Y3QgZHVtcF9jb250ZXh0ICpjdHgp CiBkdW1wX3Vud2luZF9jbGVhbnVwICh2b2lkICpkYXRhKQogewogICBzdHJ1Y3QgZHVtcF9jb250 ZXh0ICpjdHggPSBkYXRhOwotICBpZiAoY3R4LT5mZCA+PSAwKQotICAgIGVtYWNzX2Nsb3NlIChj dHgtPmZkKTsKKyNpZmRlZiBIQVZFX09QRU5fTUVNU1RSRUFNCisgIGlmIChjdHgtPmZpbGUpCisg ICAgeworICAgICAgZnNlZWsgKGN0eC0+ZmlsZSwgY3R4LT5tYXhfb2Zmc2V0LCBTRUVLX1NFVCk7 CisgICAgICBmY2xvc2UgKGN0eC0+ZmlsZSk7CisgICAgfQorICBpZiAoY3R4LT5idWYpCisgICAg eworICAgICAgZWFzc2VydCAoY3R4LT5tYXhfb2Zmc2V0IDw9IGN0eC0+YnVmX3NpemUpOworICAg ICAgY3R4LT5maWxlID0gZW1hY3NfZm9wZW4gKFNTREFUQSAoY3R4LT5kdW1wX2ZpbGVuYW1lKSwg InciKTsKKyAgICAgIGlmIChjdHgtPmZpbGUgPT0gTlVMTCkKKwlyZXBvcnRfZmlsZV9lcnJvciAo IkNvdWxkIG5vdCBvcGVuIGR1bXAgZmlsZSIsCisJCQkgICBjdHgtPmR1bXBfZmlsZW5hbWUpOwor ICAgICAgY3R4LT5vZmZzZXQgPSAwOworICAgICAgZHVtcF93cml0ZSAoY3R4LCBjdHgtPmJ1Ziwg Y3R4LT5tYXhfb2Zmc2V0KTsKKyAgICB9CisjZW5kaWYKKyAgaWYgKGN0eC0+ZmlsZSkKKyAgICBm Y2xvc2UgKGN0eC0+ZmlsZSk7CiAjaWZkZWYgUkVMX0FMTE9DCiAgIGlmIChjdHgtPmJsb2NrZWRf cmFsbG9jKQogICAgIHJfYWxsb2NfaW5oaWJpdF9idWZmZXJfcmVsb2NhdGlvbiAoMCk7CkBAIC0z OTUyLDcgKzM5ODUsNyBAQCBERUZVTiAoImR1bXAtZW1hY3MtcG9ydGFibGUiLAogCiAgIHN0cnVj dCBkdW1wX2NvbnRleHQgY3R4X2J1ZiA9IHswfTsKICAgc3RydWN0IGR1bXBfY29udGV4dCAqY3R4 ID0gJmN0eF9idWY7Ci0gIGN0eC0+ZmQgPSAtMTsKKyAgY3R4LT5maWxlID0gTlVMTDsKIAogICBj dHgtPm9iamVjdHNfZHVtcGVkID0gbWFrZV9lcV9oYXNoX3RhYmxlICgpOwogICBkdW1wX3F1ZXVl X2luaXQgKCZjdHgtPmR1bXBfcXVldWUpOwpAQCAtNDAxMiw5ICs0MDQ1LDEyIEBAIERFRlVOICgi ZHVtcC1lbWFjcy1wb3J0YWJsZSIsCiAgIGN0eC0+b2xkX3Byb2Nlc3NfZW52aXJvbm1lbnQgPSBW cHJvY2Vzc19lbnZpcm9ubWVudDsKICAgVnByb2Nlc3NfZW52aXJvbm1lbnQgPSBRbmlsOwogCi0g IGN0eC0+ZmQgPSBlbWFjc19vcGVuIChTU0RBVEEgKGZpbGVuYW1lKSwKLSAgICAgICAgICAgICAg ICAgICAgICAgIE9fUkRXUiB8IE9fVFJVTkMgfCBPX0NSRUFULCAwNjY2KTsKLSAgaWYgKGN0eC0+ ZmQgPCAwKQorI2lmZGVmIEhBVkVfT1BFTl9NRU1TVFJFQU0KKyAgY3R4LT5maWxlID0gb3Blbl9t ZW1zdHJlYW0gKCZjdHgtPmJ1ZiwgJmN0eC0+YnVmX3NpemUpOworI2VuZGlmCisgIGlmIChjdHgt PmZpbGUgPT0gTlVMTCkKKyAgICBjdHgtPmZpbGUgPSBlbWFjc19mb3BlbiAoU1NEQVRBIChmaWxl bmFtZSksICJ3KyIpOworICBpZiAoY3R4LT5maWxlID09IE5VTEwpCiAgICAgcmVwb3J0X2ZpbGVf ZXJyb3IgKCJPcGVuaW5nIGR1bXAgb3V0cHV0IiwgZmlsZW5hbWUpOwogICB2ZXJpZnkgKHNpemVv ZiAoY3R4LT5oZWFkZXIubWFnaWMpID09IHNpemVvZiAoZHVtcF9tYWdpYykpOwogICBtZW1jcHkg KCZjdHgtPmhlYWRlci5tYWdpYywgZHVtcF9tYWdpYywgc2l6ZW9mIChkdW1wX21hZ2ljKSk7Ci0t IAoyLjMwLjEKCg== --000000000000904de305bccb854b-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 15 05:26:13 2021 Received: (at 46881) by debbugs.gnu.org; 15 Jun 2021 09:26:13 +0000 Received: from localhost ([127.0.0.1]:48414 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt5Kn-0003De-8I for submit@debbugs.gnu.org; Tue, 15 Jun 2021 05:26:13 -0400 Received: from mail210c50.megamailservers.eu ([91.136.10.220]:46744 helo=mail194c50.megamailservers.eu) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt5Kk-0003DU-UR for 46881@debbugs.gnu.org; Tue, 15 Jun 2021 05:26:12 -0400 X-Authenticated-User: mattiase@bredband.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=megamailservers.eu; s=maildub; t=1623749152; bh=xlqopOLGa1Kpw6Z5Ty1x73eQCuFRJr2xSqe8murz5uw=; h=From:Subject:Date:To:From; b=AEp98MISMqq4wf1W9iFEIxeuMLmBV0XUDb+Hjd4CEl2IVsR0e75V/fw8SLh5kDzyE N32RuymlO3dcHo+5zLFKtIVn/saiskBQo8rGj7Vqt/LBF27+6seKwpaybbYSd4yxom R88VCPlD+A5o5u7F/q1dPUJA/WVkz5arxY6uJHAg= Feedback-ID: mattiase@acm.or Received: from stanniol.lan (c-b952e353.032-75-73746f71.bbcust.telenor.se [83.227.82.185]) (authenticated bits=0) by mail194c50.megamailservers.eu (8.14.9/8.13.1) with ESMTP id 15F9Pm8M016357; Tue, 15 Jun 2021 09:25:49 +0000 From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.21\)) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls Message-Id: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> Date: Tue, 15 Jun 2021 11:25:47 +0200 To: Pip Cet , Eli Zaretskii , 46881@debbugs.gnu.org, Daniel Colascione , Paul Eggert X-Mailer: Apple Mail (2.3445.104.21) X-CTCH-RefID: str=0001.0A742F24.60C87220.000F, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 X-CTCH-VOD: Unknown X-CTCH-Spam: Unknown X-CTCH-Score: 0.000 X-CTCH-Rules: X-CTCH-Flags: 0 X-CTCH-ScoreCust: 0.000 X-CSC: 0 X-CHA: v=2.3 cv=QJQWuTDL c=1 sm=1 tr=0 a=von4qPfY+hyqc0zmWf0tYQ==:117 a=von4qPfY+hyqc0zmWf0tYQ==:17 a=kj9zAlcOel0A:10 a=M51BFTxLslgA:10 a=mDV3o1hIAAAA:8 a=19wse0ghbwVjaFCDfdsA:9 a=CjuIK1q_8ugA:10 a=_FVE-zBwftR9WsbkzFJk:22 X-Origin-Country: SE X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 46881 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Any reason not to apply the patch from = https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D46881#20 right away? = I've been using it locally for quite some time with very good results. There seems to be a consensus for it, and while there may be even better = solutions, having this in place now won't hurt. From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 15 08:58:51 2021 Received: (at 46881) by debbugs.gnu.org; 15 Jun 2021 12:58:51 +0000 Received: from localhost ([127.0.0.1]:48670 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt8eZ-0004k1-F6 for submit@debbugs.gnu.org; Tue, 15 Jun 2021 08:58:51 -0400 Received: from dancol.org ([96.126.100.184]:42262) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt8eW-0004jr-Ri for 46881@debbugs.gnu.org; Tue, 15 Jun 2021 08:58:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:To:Subject:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=5ELWFXtdjzJFHxBC+pp2tV98laO91WViKFwETfJI9U4=; b=U7PQzIxWELmeIQ8MypIo2LICnJ 5vMPGtBIolmY/0pQp8dOyL4gcZ1ie1m7+BaxKhM8Nf0ju4xcbpPyyVMnIexiDTZ5ix7+WZ9Wmo08X QlnHepsYGCKvjNxBOw69bHq3OFc6tqBw5tO+ORViNP8BWDE9ZKCoY+A6BMGxbcf+AlLeLov7gtEqz f8d9qY7oYN4e6ifppn/fzco2GakfiVYRm1CShbrf6LfCrVuP+gRywqx5n29xD9RxwCt30Bgmy7r0v /lgsJCoUrAcX3F+aVch5ardjgcV+cbDr4jyVmPxC5vblukYmFxfGsNfvX51uMKV3CiSv0IdThSY0a fGnyjMtA==; Received: from [216.9.110.2] (port=11217 helo=[172.31.104.204]) by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1lt8eU-0000Rd-BO; Tue, 15 Jun 2021 05:58:46 -0700 Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: =?UTF-8?Q?Mattias_Engdeg=c3=a5rd?= , Pip Cet , Eli Zaretskii , 46881@debbugs.gnu.org, Paul Eggert References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> From: Daniel Colascione Message-ID: <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> Date: Tue, 15 Jun 2021 05:58:26 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 46881 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.1 (-) On 6/15/21 2:25 AM, Mattias Engdegård wrote: > Any reason not to apply the patch from https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46881#20 right away? I've been using it locally for quite some time with very good results. > > There seems to be a consensus for it, and while there may be even better solutions, having this in place now won't hurt. I thought we had consensus for an fwrite based approach? From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 15 09:06:41 2021 Received: (at 46881) by debbugs.gnu.org; 15 Jun 2021 13:06:41 +0000 Received: from localhost ([127.0.0.1]:48689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt8m9-000796-FN for submit@debbugs.gnu.org; Tue, 15 Jun 2021 09:06:41 -0400 Received: from eggs.gnu.org ([209.51.188.92]:54238) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt8m6-00078s-Gu for 46881@debbugs.gnu.org; Tue, 15 Jun 2021 09:06:40 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52630) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lt8lx-00082e-3w; Tue, 15 Jun 2021 09:06:31 -0400 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2694 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lt8lw-0007MF-LP; Tue, 15 Jun 2021 09:06:28 -0400 Date: Tue, 15 Jun 2021 16:06:28 +0300 Message-Id: <83o8c746qz.fsf@gnu.org> From: Eli Zaretskii To: Daniel Colascione , Lars Ingebrigtsen In-Reply-To: <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> (message from Daniel Colascione on Tue, 15 Jun 2021 05:58:26 -0700) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, mattiase@acm.org, eggert@cs.ucla.edu, pipcet@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Daniel Colascione > Date: Tue, 15 Jun 2021 05:58:26 -0700 > > On 6/15/21 2:25 AM, Mattias Engdegård wrote: > > > Any reason not to apply the patch from https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46881#20 right away? I've been using it locally for quite some time with very good results. > > > > There seems to be a consensus for it, and while there may be even better solutions, having this in place now won't hurt. > I thought we had consensus for an fwrite based approach? That's what I preferred (still do), but Lars thought that the slightly faster version without stdio was preferable. From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 15 09:18:11 2021 Received: (at 46881) by debbugs.gnu.org; 15 Jun 2021 13:18:11 +0000 Received: from localhost ([127.0.0.1]:48696 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt8xF-0007PW-Hp for submit@debbugs.gnu.org; Tue, 15 Jun 2021 09:18:11 -0400 Received: from quimby.gnus.org ([95.216.78.240]:49644) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt8xA-0007Ov-QJ for 46881@debbugs.gnu.org; Tue, 15 Jun 2021 09:18:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=DYHobWWNGbKaPPK0PVyEPj5BP5GcDGXWwNLksaJ/G48=; b=dJjqT5CVHKJSg5oF4OCZQVS/bz sVMHo0B/FaqHEEUOA+Y9v5dHXJ2xwMf8IxTPuKGmi7a6WVf0a8l8cLEn6DLia+O45k00Pa46j0JHN +By8qUlSvROt904mvVLLNFPHkgEwIYHS+mCP+eDtCO8itM/SrgvX8N8uGbu6BZjIoKqc=; Received: from cm-84.212.220.105.getinternet.no ([84.212.220.105] helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lt8wz-00074u-5L; Tue, 15 Jun 2021 15:17:56 +0200 From: Lars Ingebrigtsen To: Eli Zaretskii Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> <83o8c746qz.fsf@gnu.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAMAAABg3Am1AAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAM1BMVEX+/v6inZpdW1qZ cVtSQzyqh3TX1NDIlXTRrJPr6efdxrclJCU1NDQaGRkgHh99gIH///9QJp/gAAAAAWJLR0QQlbIN LAAAAAd0SU1FB+UGDw0RDXjOCZkAAAGBSURBVEjHlZXLosIwCEQDIU8I/v/fXpJWe1eCLNTFHJlp CE3pFGBGwlxqClbNrfdMecSB3nujEtWnWkorJYcdpdSbeSphRynNZtXj+jRaaf2HBttTiUfeHUrO DcItBqCFmMgYYgbKktk74FrK0wdwMQsUgP1D0QeITWeuEiqLiGuqsjCvCWCk6ZfrCVSEFxAkOSgE IhghlIZ9GeCGOJmtYB6A2QtBRyVwDLG6nurRm2wcubiewBztJrS9Ke9H6wICasQUvQD+fo9wKe0u 1mYDIF4IVMUD6BWFfGCVA9zZ9x98B1jP1L0Bgz1g+/4AAuwBujUfgKq4AOEDqAFOBlj0RGCVSj8B LKEO+g+Y4gwT/FPvAh8gpA+0XjXn70DNZb6eg6vV3cm9zWc0uA7wLnXNtZ4TsPFmWzL+8ps17W1h +88mwxPfRabF8xEE7OgW/QrItuRusbtg3RcusLuvJ6X3bQjq03jPavgdRHrtvfCb+j3hvwJxS/c0 UVSf5jqbzD+3P3rvF5p8A/xkAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTA2LTE1VDEzOjE3OjEz KzAwOjAwhUJMdgAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0wNi0xNVQxMzoxNzoxMyswMDowMPQf 9MoAAAAASUVORK5CYII= X-Now-Playing: David Bowie's _Reality (1)_: "She'll Drive the Big Car" Date: Tue, 15 Jun 2021 15:17:51 +0200 In-Reply-To: <83o8c746qz.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 15 Jun 2021 16:06:28 +0300") Message-ID: <87wnqvuv0g.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Eli Zaretskii writes: > That's what I preferred (still do), but Lars thought that the slightly > faster version without stdio was preferable. Well, it's 30% faster in a single-threaded phase of the compilation, so it seems like a significant improvement to me. Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, mattiase@acm.org, Daniel Colascione , pipcet@gmail.com, eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Eli Zaretskii writes: > That's what I preferred (still do), but Lars thought that the slightly > faster version without stdio was preferable. Well, it's 30% faster in a single-threaded phase of the compilation, so it seems like a significant improvement to me. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 15 09:26:02 2021 Received: (at 46881) by debbugs.gnu.org; 15 Jun 2021 13:26:02 +0000 Received: from localhost ([127.0.0.1]:48707 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt94r-0007bG-KO for submit@debbugs.gnu.org; Tue, 15 Jun 2021 09:26:02 -0400 Received: from dancol.org ([96.126.100.184]:42264) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt94p-0007b0-QR for 46881@debbugs.gnu.org; Tue, 15 Jun 2021 09:26:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Subject: References:In-Reply-To:Message-ID:Date:CC:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=z+Ypz9m60RMlzptZT7oFKmCZnqvdMapbNpbOTqV93Lo=; b=Xzg/5xZANLLWUw/CComA/0yrFB snCT3QyS2xXzDQpKqf91EucfMi4P9j6uyYzFlA2SKg+p8q6rnjTwX4FiJcO7VBScH3ptzwxSMLK8z qYjibaU/cbrzdNWXbfuFTNRDCAtD9NgKG7lCfSuDpkxzj47hWplL7fHOKeoykpWDU2YU3k6gRg3LO QZFQfyZGBDrr3ISCz5mM63ph3gj/SL/9M+7r4yBVsCj01aEfChn1E1/zaATo2LW4l0wZrrLKYiVab LrvSkegmFRDXFAGAqA7QqzD4RyCu22K1bP+JFlqBmeGNBiNH8rMP3CvvtWyulYlbX0L70NiTUiK/1 WB70ZjHg==; Received: from 83.sub-174-194-196.myvzw.com ([174.194.196.83]:6315 helo=[100.111.184.186]) by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256) (Exim 4.89) (envelope-from ) id 1lt94h-0000du-R6; Tue, 15 Jun 2021 06:25:51 -0700 From: Daniel Colascione To: Lars Ingebrigtsen , Eli Zaretskii Date: Tue, 15 Jun 2021 06:25:50 -0700 Message-ID: <17a0fd97f30.2816.cc5b3318d7e9908e2c46732289705cb0@dancol.org> In-Reply-To: <87wnqvuv0g.fsf@gnus.org> References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> <83o8c746qz.fsf@gnu.org> <87wnqvuv0g.fsf@gnus.org> User-Agent: AquaMail/1.29.2-1810 (build: 102900008) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="us-ascii" Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, mattiase@acm.org, eggert@cs.ucla.edu, pipcet@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On June 15, 2021 6:18:11 AM Lars Ingebrigtsen wrote: > Eli Zaretskii writes: > >> That's what I preferred (still do), but Lars thought that the slightly >> faster version without stdio was preferable. > > Well, it's 30% faster in a single-threaded phase of the compilation, so > it seems like a significant improvement to me. > > -- > (domestic pets only, the antidote for overdose, milk.) > bloggy blog: http://lars.ingebrigtsen.n The in-memory version is 30% faster than the FILE* version or 30% faster than the current write () version? From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 15 09:30:51 2021 Received: (at 46881) by debbugs.gnu.org; 15 Jun 2021 13:30:51 +0000 Received: from localhost ([127.0.0.1]:48715 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt99W-0007iz-Rk for submit@debbugs.gnu.org; Tue, 15 Jun 2021 09:30:51 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59324) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt99V-0007im-6S for 46881@debbugs.gnu.org; Tue, 15 Jun 2021 09:30:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53646) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lt99O-0006Xb-GC; Tue, 15 Jun 2021 09:30:42 -0400 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4241 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lt99K-0004Y8-PW; Tue, 15 Jun 2021 09:30:41 -0400 Date: Tue, 15 Jun 2021 16:30:36 +0300 Message-Id: <83lf7b45mr.fsf@gnu.org> From: Eli Zaretskii To: Daniel Colascione In-Reply-To: <17a0fd97f30.2816.cc5b3318d7e9908e2c46732289705cb0@dancol.org> (message from Daniel Colascione on Tue, 15 Jun 2021 06:25:50 -0700) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> <83o8c746qz.fsf@gnu.org> <87wnqvuv0g.fsf@gnus.org> <17a0fd97f30.2816.cc5b3318d7e9908e2c46732289705cb0@dancol.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, mattiase@acm.org, larsi@gnus.org, eggert@cs.ucla.edu, pipcet@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Daniel Colascione > CC: , , <46881@debbugs.gnu.org>, > Date: Tue, 15 Jun 2021 06:25:50 -0700 > > >> That's what I preferred (still do), but Lars thought that the slightly > >> faster version without stdio was preferable. > > > > Well, it's 30% faster in a single-threaded phase of the compilation, so > > it seems like a significant improvement to me. > > > > -- > > (domestic pets only, the antidote for overdose, milk.) > > bloggy blog: http://lars.ingebrigtsen.n > > The in-memory version is 30% faster than the FILE* version or 30% faster > than the current write () version? The former, see https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46881#56 From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 15 11:32:36 2021 Received: (at 46881) by debbugs.gnu.org; 15 Jun 2021 15:32:36 +0000 Received: from localhost ([127.0.0.1]:50526 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltB3M-0005Qr-EU for submit@debbugs.gnu.org; Tue, 15 Jun 2021 11:32:36 -0400 Received: from mail213c50.megamailservers.eu ([91.136.10.223]:56820 helo=mail194c50.megamailservers.eu) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltB3K-0005Qd-0K for 46881@debbugs.gnu.org; Tue, 15 Jun 2021 11:32:35 -0400 X-Authenticated-User: mattiase@bredband.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=megamailservers.eu; s=maildub; t=1623771136; bh=jKUgJbgGV10dKGsDRDNYs9ZwHsXBJKUj1/ZdRhzgPOA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From; b=gz3Yx4vAiv8RoQXqcIySolLZpNG0fY7IdXinn41FWAGYP5h3MiaQG4BoIaPB5hrOJ CX3MPQCnXfHa5Axf9LTO+wgvqa1hWHIxOZgzs72DMG1hoifVtNOp5VbJzy9jg2xxJ8 PrTcNAfI8yDhxKNUC8yWBU95ew+RbElkykbDrmo8= Feedback-ID: mattiase@acm.or Received: from stanniol.lan (c-b952e353.032-75-73746f71.bbcust.telenor.se [83.227.82.185]) (authenticated bits=0) by mail194c50.megamailservers.eu (8.14.9/8.13.1) with ESMTP id 15FFWC5o018365; Tue, 15 Jun 2021 15:32:14 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.21\)) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: <87wnqvuv0g.fsf@gnus.org> Date: Tue, 15 Jun 2021 17:32:12 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> <83o8c746qz.fsf@gnu.org> <87wnqvuv0g.fsf@gnus.org> To: Lars Ingebrigtsen X-Mailer: Apple Mail (2.3445.104.21) X-CTCH-RefID: str=0001.0A742F1C.60C8C800.0032, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 X-CTCH-VOD: Unknown X-CTCH-Spam: Unknown X-CTCH-Score: 0.000 X-CTCH-Rules: X-CTCH-Flags: 0 X-CTCH-ScoreCust: 0.000 X-CSC: 0 X-CHA: v=2.3 cv=QJQWuTDL c=1 sm=1 tr=0 a=von4qPfY+hyqc0zmWf0tYQ==:117 a=von4qPfY+hyqc0zmWf0tYQ==:17 a=kj9zAlcOel0A:10 a=M51BFTxLslgA:10 a=OocQHUDgAAAA:8 a=VYphYFrhStWdGqm6pPwA:9 a=CjuIK1q_8ugA:10 a=xUZTl98r3Qw_uB5NK3jt:22 X-Origin-Country: SE X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Eli Zaretskii , Daniel Colascione , pipcet@gmail.com, eggert@cs.ucla.edu X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) 15 juni 2021 kl. 15.17 skrev Lars Ingebrigtsen : > Well, it's 30% faster in a single-threaded phase of the compilation, = so > it seems like a significant improvement to me. The patch seems to be good in all respects -- simple, low risk, better = performance, no compatibility trouble. And nothing prevents us from = replacing it with something better later on, should the need arise. All set then? From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 15 18:44:38 2021 Received: (at 46881) by debbugs.gnu.org; 15 Jun 2021 22:44:38 +0000 Received: from localhost ([127.0.0.1]:51011 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltHnS-0005ue-5g for submit@debbugs.gnu.org; Tue, 15 Jun 2021 18:44:38 -0400 Received: from dancol.org ([96.126.100.184]:42266) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltHnP-0005uT-Ev for 46881@debbugs.gnu.org; Tue, 15 Jun 2021 18:44:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=q2MdeO7b6Uu1eC2hVrmfcO7ZxpYm8vf1GQAAmEY9db0=; b=UO2WnpvVoNIblh+CXk2VL/HcVI imOf83Mog4g4gFTrU4OqBzSTrPaluUHHZCTPNlGcuhEymZp588oiqhsRe9yAuv6FZ2lxSCz7U92Iz iZj94y+XKNsRrNIEqKsS9jPAc2pcMsutGr/pJaiIQXbcFwR3N7mjRItmkH4BKgXC/FJdrSt66xlKW wKeOLREol8LPw4qlPJuagdxE7TKbqffBqUlQGH2C016snSqDqtf09FNOtfBgLjjxhzTuAE8v3l26G Hims8qpiJLQeBr9u8kWU/4KaDsBE8R8bQm4pW91Lq5akbUZ70E/qOkajJknXuP1ovkaV14JOPF6fV 21jCH7dA==; Received: from [2604:4080:1321:8020:2761:c5fe:e373:3ed] (port=41824) by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1ltHnN-0003bo-LJ; Tue, 15 Jun 2021 15:44:33 -0700 Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: =?UTF-8?Q?Mattias_Engdeg=c3=a5rd?= , Lars Ingebrigtsen References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> <83o8c746qz.fsf@gnu.org> <87wnqvuv0g.fsf@gnus.org> From: Daniel Colascione Message-ID: Date: Tue, 15 Jun 2021 15:44:29 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org, Eli Zaretskii , eggert@cs.ucla.edu, pipcet@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.1 (-) On 6/15/21 8:32 AM, Mattias Engdegård wrote: > 15 juni 2021 kl. 15.17 skrev Lars Ingebrigtsen : >> Well, it's 30% faster in a single-threaded phase of the compilation, so >> it seems like a significant improvement to me. > The patch seems to be good in all respects -- simple, low risk, better performance, no compatibility trouble. And nothing prevents us from replacing it with something better later on, should the need arise. > > All set then? > Okay. I'm convinced. From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 16 04:00:52 2021 Received: (at 46881-done) by debbugs.gnu.org; 16 Jun 2021 08:00:52 +0000 Received: from localhost ([127.0.0.1]:51476 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltQTk-0007cG-A8 for submit@debbugs.gnu.org; Wed, 16 Jun 2021 04:00:52 -0400 Received: from mail211c50.megamailservers.eu ([91.136.10.221]:57942 helo=mail194c50.megamailservers.eu) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltQTh-0007c6-Tb for 46881-done@debbugs.gnu.org; Wed, 16 Jun 2021 04:00:51 -0400 X-Authenticated-User: mattiase@bredband.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=megamailservers.eu; s=maildub; t=1623830434; bh=+tSEpx5b805+cH2XGv7C09gI4qvwZ+AbXoasOmOXnAU=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From; b=Y4geP2GOA4kUnhZy2XrGcVk78cJN7DmYGhoVutmwX82q8lIPQrE2dJ1dA7S9Tf7rj +YVGrULgRh254hqf0jeyZWTblIs2d6AW9hZ7jxXHwVUqNsJVA7trtDEjdm46m3XrhL flRAhujcRoMy0JeLF9VfxYhJecjyEizsaJTsG6Mo= Feedback-ID: mattiase@acm.or Received: from stanniol.lan (c-b952e353.032-75-73746f71.bbcust.telenor.se [83.227.82.185]) (authenticated bits=0) by mail194c50.megamailservers.eu (8.14.9/8.13.1) with ESMTP id 15G80TOS018607; Wed, 16 Jun 2021 08:00:31 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.21\)) Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Wed, 16 Jun 2021 10:00:27 +0200 Content-Transfer-Encoding: 7bit Message-Id: References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> <83o8c746qz.fsf@gnu.org> <87wnqvuv0g.fsf@gnus.org> To: Daniel Colascione X-Mailer: Apple Mail (2.3445.104.21) X-CTCH-RefID: str=0001.0A742F22.60C9AFA2.0009, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 X-CTCH-VOD: Unknown X-CTCH-Spam: Unknown X-CTCH-Score: 0.000 X-CTCH-Rules: X-CTCH-Flags: 0 X-CTCH-ScoreCust: 0.000 X-CSC: 0 X-CHA: v=2.3 cv=QJQWuTDL c=1 sm=1 tr=0 a=von4qPfY+hyqc0zmWf0tYQ==:117 a=von4qPfY+hyqc0zmWf0tYQ==:17 a=kj9zAlcOel0A:10 a=M51BFTxLslgA:10 a=KxCsr730AAAA:8 a=6FYxFR_QqQ0eApocJXcA:9 a=CjuIK1q_8ugA:10 a=_wvLy7kD0tUA:10 a=_T5_SZ4BPdQ8elZ3MUhu:22 X-Origin-Country: SE X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 46881-done Cc: 46881-done@debbugs.gnu.org, Lars Ingebrigtsen , Paul Eggert , Eli Zaretskii , Pip Cet X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) 16 juni 2021 kl. 00.44 skrev Daniel Colascione : > Okay. I'm convinced. All right, pushed, and closing this bug. Thanks, Pip! From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 16 04:14:36 2021 Received: (at 46881-done) by debbugs.gnu.org; 16 Jun 2021 08:14:36 +0000 Received: from localhost ([127.0.0.1]:51491 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltQh1-0007vT-Rd for submit@debbugs.gnu.org; Wed, 16 Jun 2021 04:14:36 -0400 Received: from quimby.gnus.org ([95.216.78.240]:59960) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltQh0-0007vF-3j for 46881-done@debbugs.gnu.org; Wed, 16 Jun 2021 04:14:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID :In-Reply-To:Date:References:Subject:Cc:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=6mwm+O4urXDbcUJpM+V/XndKyqi3LRjUX37D9B2/hK4=; b=e3eSO7shgPGBHzwpL/I7iKhJ/W DvDUtVlZhnZeoBHDqtZoZX2vcCoQNFPS+kV5ZHYk4sxBUvN1dADtTIHvXGiFE0BcwyTd6leBfiX9i 5tv16D/suQdye20lQdaeTZF9sjtP0QePSM4C+hDGZpUNWj7z4hMxs6jlrhgCXXBKbfGc=; Received: from cm-84.212.220.105.getinternet.no ([84.212.220.105] helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ltQgp-0003Gh-A5; Wed, 16 Jun 2021 10:14:25 +0200 From: Lars Ingebrigtsen To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> <83o8c746qz.fsf@gnu.org> <87wnqvuv0g.fsf@gnus.org> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAG1BMVEX5+e/y8NZXSkTG u5p6cGydkXQnHCkNBhD///9PiCxjAAAAAWJLR0QIht6VegAAAAd0SU1FB+UGEAgODMwisrMAAAGT SURBVDjLvZLNVsIwEIXTwwu0etR9gHQLDHXtKQG2SqmuLZzp2orJ6zuZtDQF2XoXBebrzJ0fhPh/ RYnXNbkFkkuQxMNa59JSzui5zlYXYIH1QYgR1vZ1ACKFeBIRrO7sVxSCESIaSC2ibQYgIlDX1qm5 8FAVcty8DTLESKJy8UPsRqGe+3YVV/r0M4r4POBDtXXg1E3f7+I+ZY+4HT7YEmfYF185DsATg+9w nzB3n49+jP4ymgT3MF/Ys0kPWH7yWbBwVj7xbY2BG+uBlB4cdQEkAmWndwY/AC2ATkvf1jXI2N0E YLcKMmwBSzVlsMR87QxzD4AuacYOfFgjxwAb35bJ6AXTOLA1dNi9qhs+Ff1wT86g71WNx50fhFRh 5cCzNUore8rrLq4lgyw1qHVqVAfob8elINMoJ5sGO9B5kBRWOf4FNoghoG5asENUiH18qluwRwwS KiWn3pxWXhvsHagXbldKrYIw1eKMIsulqpQr1hrTNSexy8hKreVQDpQABZT5BfHA2e+1S8v7jOSG xK3oL0Rh2UH/2mkWAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTA2LTE2VDA4OjE0OjEyKzAwOjAw mwwW5gAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0wNi0xNlQwODoxNDoxMiswMDowMOpRrloAAAAA SUVORK5CYII= X-Now-Playing: Sebastian's _Thirst_: "Doorman (feat. Syd)" Date: Wed, 16 Jun 2021 10:14:22 +0200 In-Reply-To: ("Mattias =?utf-8?Q?Engdeg=C3=A5rd=22's?= message of "Wed, 16 Jun 2021 10:00:27 +0200") Message-ID: <87h7hyql9d.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Mattias Engdegård writes: > All right, pushed, and closing this bug. > Thanks, Pip! Great; thanks. Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46881-done Cc: 46881-done@debbugs.gnu.org, Eli Zaretskii , Daniel Colascione , Pip Cet , Paul Eggert X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Mattias Engdeg=C3=A5rd writes: > All right, pushed, and closing this bug. > Thanks, Pip! Great; thanks. --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 16 04:17:28 2021 Received: (at 46881-done) by debbugs.gnu.org; 16 Jun 2021 08:17:28 +0000 Received: from localhost ([127.0.0.1]:51501 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltQjo-00080i-FQ for submit@debbugs.gnu.org; Wed, 16 Jun 2021 04:17:28 -0400 Received: from mail-oi1-f171.google.com ([209.85.167.171]:34588) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltQjj-00080R-Gz for 46881-done@debbugs.gnu.org; Wed, 16 Jun 2021 04:17:27 -0400 Received: by mail-oi1-f171.google.com with SMTP id u11so1705731oiv.1 for <46881-done@debbugs.gnu.org>; Wed, 16 Jun 2021 01:17:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=Qlf3qyQpQZokqpvwNXY7qHKz7v0zu0HJ9j8MBQQp6dk=; b=bK2YNAHYQkbjaLBWWQMpZX9OpwoFMlMAYqYc+W9YcbcpC99nUlTdLgI1FYdpfPZBzg /S8skgAKD7R0CB79bn/MvJKFvagyuMBSMj7upERlLVYpgsYJnT8NSa8Yw2+Dlj9PnJ/H +bLvfoFpd9pPhaVbKvgVPute674LWaCc8NMvv5A7RcVYtJ0JYjiY882pKlJbqYo2tqAw b7demPyChmDsQfAV8YcEdyTiy2RWxJIxmhFNsTUVhaeTyYuhYRWjplXpnyJhvuTTKrHV T9WYJwkSfhhZlu9YEX0q2zGPfZwiOV5wF8sm6ZDRrAdjQBFOMsxdUfdo3PLQKxotSy5I e0sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=Qlf3qyQpQZokqpvwNXY7qHKz7v0zu0HJ9j8MBQQp6dk=; b=RiJ9M3bSAyT0pq6hJhgfiywuJo7j4nM1mbm/KeuMQ+fuPQJimkhoddeYjCBjKHQvEL T9+MeeG1celuXFSj39FMJdKAsZUysdBfGWgNFxi1Wx8zY0T4MeXRdDDA+yiwRjNQwU1D 7ahWKscuz83sTENLKVhRGQDzcqf3/5knbxzVkwCQ68mglnoI8QVqt+7kkCamfgpX0j6q 2EdThtcXwU8cgAUJ6FLNhU6vXb0Tlv+QTQlMrspfWYXCWjPkNxF+XQx9pegHZ4UZCiCa hygn3NIuVWlGzK7BoqZHRDowA5Q9dwkEUAiP514HrvuWnhsm2vyoy5vK/CydWAbOo+2D viBg== X-Gm-Message-State: AOAM533kmx4NTSZT7P4vVrzJjqVeJLgdcP4EWaYM5tZLgLINSKB97WoM mnk6xbwCL6mpLXlOapvGnwdvFvpBexFfahzYamc= X-Google-Smtp-Source: ABdhPJyB9q8t8UE2ZVsYldMJFtwIaw2kao6pBnDFR0gvCC7OAqbia3UtqIVwM6SXhaUYs1ZeEVVSxR7ikwWMn6OCnzs= X-Received: by 2002:aca:fd0f:: with SMTP id b15mr2337753oii.44.1623831437985; Wed, 16 Jun 2021 01:17:17 -0700 (PDT) MIME-Version: 1.0 References: <48FDB44E-7C67-49F7-BD77-5DE99806B09F@acm.org> <7cf519e8-b3a5-f41b-f1aa-e270145df7ba@dancol.org> <83o8c746qz.fsf@gnu.org> <87wnqvuv0g.fsf@gnus.org> In-Reply-To: From: Pip Cet Date: Wed, 16 Jun 2021 08:16:41 +0000 Message-ID: Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls To: =?UTF-8?Q?Mattias_Engdeg=C3=A5rd?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 46881-done Cc: 46881-done@debbugs.gnu.org, Lars Ingebrigtsen , Daniel Colascione , Eli Zaretskii , Paul Eggert X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, Jun 16, 2021 at 8:00 AM Mattias Engdeg=C3=A5rd w= rote: > > 16 juni 2021 kl. 00.44 skrev Daniel Colascione : > > > Okay. I'm convinced. > > All right, pushed, and closing this bug. > Thanks, Pip! Thank you for following up on this! Sorry Emacs kind of lost priority for me right now, but I'll get back to it... From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 16 10:13:12 2021 Received: (at 46881) by debbugs.gnu.org; 16 Jun 2021 14:13:12 +0000 Received: from localhost ([127.0.0.1]:53151 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltWI4-0004ik-HW for submit@debbugs.gnu.org; Wed, 16 Jun 2021 10:13:12 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:44138) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltWI2-0004iU-1X for 46881@debbugs.gnu.org; Wed, 16 Jun 2021 10:13:10 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 26493440BCF; Wed, 16 Jun 2021 10:13:04 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 7857B440B26; Wed, 16 Jun 2021 10:13:02 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1623852782; bh=UmfcGJDnw+Od7Z2u37zwK9b1BDt0yZ7urQYfpbsXpXA=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=EzX9GACZKYAun4yuefIIJOGATn5gDIuv4FSG6M59EpbiWBx5a39NCj3Z1eNj3aSZa IrnhHWJVhQWuu9MaH2asxFLyX2/sq9v5I/p22KKnet+D3T8lND59rH39V6zdHXq+H+ IlGKdpT30rA6XESpKsR8oUErKY5Zmdo6B+i/qjC+Bv8SY3Ze/c7EWvCtjFtCHbLJLK Okjgmul+r/E2qEVE5bGDVE7bgW6eBPNqpg2F5UDyqzKbTDhGLX/R8PWPn/YtGEzhA5 yoGpkmTQt6jK5fy67qBufnMLMvd/JwpPGWM046E5+WvsM6v5h1LikAbDKAxEj8O3an G+PywWXJePsMw== Received: from alfajor (69-196-163-239.dsl.teksavvy.com [69.196.163.239]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3CD2D12083D; Wed, 16 Jun 2021 10:13:02 -0400 (EDT) From: Stefan Monnier To: Pip Cet Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls Message-ID: References: Date: Wed, 16 Jun 2021 10:13:01 -0400 In-Reply-To: (Pip Cet's message of "Tue, 2 Mar 2021 20:33:42 +0000") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.033 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 46881 Cc: 46881@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Pip Cet [2021-03-02 20:33:42] wrote: > Playing around with the WebAssembly port, I noticed that pdumper, in > creating the dump file, makes way too many syscalls: it uses > emacs_write(), not fwrite(), so these calls translate to actual > syscalls and context switches. On immature systems (or in special > circumstances like a device mounted synchronously), Thanks for this patch. For the little story, this inefficiency showed up on one of my Thinkpads running GNU/Linux with a plain old ext4 partition mounted in the most standard way (no synchronous mount or other funny business): https://serverfault.com/questions/996495/writes-throttled-to-500kb-s The way this manifested itself is that after some uptime individual writes to the SSD became very slow. For most operations, this was completely invisible, but it was quite noticeable during Emacs's dump which sometimes took several minutes (while all the rest of the compilation (including the "load" part of the dump)) progressed at (apparently) usual speeds. Stefan From unknown Mon Aug 18 18:01:01 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 15 Jul 2021 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator