From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 02 17:03:17 2015 Received: (at submit) by debbugs.gnu.org; 2 Dec 2015 22:03:17 +0000 Received: from localhost ([127.0.0.1]:36636 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a4FUb-0002Bb-0l for submit@debbugs.gnu.org; Wed, 02 Dec 2015 17:03:17 -0500 Received: from eggs.gnu.org ([208.118.235.92]:50898) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a4FUZ-0002BU-8c for submit@debbugs.gnu.org; Wed, 02 Dec 2015 17:03:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a4FUY-0003U5-1S for submit@debbugs.gnu.org; Wed, 02 Dec 2015 17:03:14 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:58974) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a4FUX-0003U1-VH for submit@debbugs.gnu.org; Wed, 02 Dec 2015 17:03:13 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60885) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a4FUW-0000iu-TL for bug-guix@gnu.org; Wed, 02 Dec 2015 17:03:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a4FUS-0003SO-SL for bug-guix@gnu.org; Wed, 02 Dec 2015 17:03:12 -0500 Received: from mout.gmx.net ([212.227.15.15]:50475) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a4FUS-0003SG-Je for bug-guix@gnu.org; Wed, 02 Dec 2015 17:03:08 -0500 Received: from [192.168.1.241] ([77.8.107.219]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0Llm9w-1adaF12vuS-00ZMTa for ; Wed, 02 Dec 2015 23:03:07 +0100 To: bug-guix@gnu.org From: Florian Paul Schmidt Subject: failed builds due to exceeding max-silent-time not marked as failed in db X-Enigmail-Draft-Status: N1110 Message-ID: <565F6A9B.9050406@gmx.net> Date: Wed, 2 Dec 2015 23:03:07 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:Pi34p7XkZsMSwEQB43UYQQ3o9rtgEnztDD/kGWp7W3NC3Wm+xpQ 7NBWgzarQsfz2YWJXFBwSLQL4Y2x24vFAPnDe6GPAr7frSPEY6RfabZMNpoLtVc+sjPB2FQ 7TcwTT5juVg9Hd/R1PQrBWomEdZT3wofUg2Kh9gBxkM+QhafYmDjJIsCggKkvcDguY1Hyid pCKktcCUdAQyWHwzFX5cg== X-UI-Out-Filterresults: notjunk:1;V01:K0:/dtX6KSkjLY=:w+mmWozu7+7MSKz+AwLP60 1kjaraD4CGX+c1yH3HAClvkPNgI9Oor73350vjlLpG2MYRwdks5zSKnMtlSbZINKjOR6n8pAH qob37qDGqAXsAVt4dRFU/eGq/x3Uc8J3iVc8OXLy4vkLDHVXkz8HL3FSGNBT41IDeLhAPg8X6 Ov9d0HzkzR+SFsVRuOiXllhnQFysoRCE4loTPwIBAhYmjpMYvNlCga3aioySlqc8V1b/ieOWZ MkPrdYagsbgJJIh9xk+1L2Dm9j1+4xX2sTHDneM+iQbqbrkg+rUdTb0iQaoRiLebwww9exC72 joWcQBnlP+UfXJRAPbijQo97DhYaTuiBv5aF4xfSf8dnZJa5z1Q5J8BcqoYLTmXkVlQKnajrs FjJ3KX+vCrWW1WpaECHDHhiBRCEWCPRpMftvZzatwdqq7J7oOjhUr4ZEh8KdKURXikspCuZEh PLpHZi0E0Lq6VvHUoMBuWi8yTpy3T10IPqGdNVvx/TIbKt3QhK7G6PWB2jG1XQrKCY3Vswqxy EAaC3qtH9QDBrzbncbd697GFc2o7RLqV2OIt/vRkGr+9fTjtihln3Hqwnubo1pT/sL/hnBWZz rpslvYu52r/Eio4SP75bfMctQpx7vsJqR6a6kquJnynC3iEwMczc5+34MLHr2Mpt1XkEajFVN Ag+pWjHS3AEE1lOEZPvRTuhKbqt0ki8JLngfcXJ/I5A9WQNNQniT+IwHkhGIZPkQ3dsOYZoRn Ovcl5NrclfqqoGz5a0kNRlFGR6EeFOrfpAcUlAq0Hku39JpFy9zl3kYGGs6eik3R8g2kqj50+ bbOE11U X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.1 (----) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, on my system bulding the derivation for the package tbb (version 4.3.2) does not complete due to exceeding the max-silent-time default value of 3600 seconds (one hour). It seems that in this case the path is not marked as failed in the sqlite3 db /var/guix/db/db.sqlite in the table FailedPaths. This is quite annoying since it seems that several packages depend on it causing the derivation to be built several times (each taking over an hour to fail). The guix daemon is running with the --cache-failures option and I would expect the second run of for n in `guix package -A | cut -f1`; do guix build --no-substitutes "$n" || true; done to be mostly a NOOP, since all failures from the first run should be cached. And even in the first run I wouldn't expect failed dependencies to be tried to build again. Contrary to this on this box even the second run of this takes about half a day or so to complete ;) Flo P.S.: FYI: The thing that takes over an hour to run is ./test_atomic.exe - -- https://fps.io -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJWX2qaAAoJEA5f4Coltk8ZnasH/jOg+E0Y/CDxw5SGgcJN0Q6K TYo41AVz0u9tLJEVYW4ZW9Z7A3UL5OTB+03LwC1zT7iDtFzU6a7BzaW2N3gP+GGi Tx+Rq0z7ZIHEF1t71YFtPOAIpuyxwl1yMnRo0kd8BVsrNu843ITI4w+kzGV4tcP1 l9uDf7c+WQ8MFhoMDUqjW5ufIb3zy6yKk1GDXw14xZ8laeiE8hrXFE2LFV4WCxzP VMPDgHBlPF6pAKLYpWSpL2RtL/WxO9tYIYpQ16EW7GjOouCy2ObT+1CJ75kSIOie DZ/RLUSxa39amDFwii5liR+ETgvz3FCoBAcyI5AP/76uMToub1z3S1PNt58EnsE= =Hivd -----END PGP SIGNATURE----- From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 04 17:41:08 2015 Received: (at submit) by debbugs.gnu.org; 4 Dec 2015 22:41:08 +0000 Received: from localhost ([127.0.0.1]:39470 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a4z2J-0001YH-AB for submit@debbugs.gnu.org; Fri, 04 Dec 2015 17:41:07 -0500 Received: from eggs.gnu.org ([208.118.235.92]:55690) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1a4z1y-0001X7-68 for submit@debbugs.gnu.org; Fri, 04 Dec 2015 17:41:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a4z1w-0006VI-Cc for submit@debbugs.gnu.org; Fri, 04 Dec 2015 17:40:45 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:50031) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a4z1w-0006VC-9M for submit@debbugs.gnu.org; Fri, 04 Dec 2015 17:40:44 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37447) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a4z1u-0004St-Jd for bug-guix@gnu.org; Fri, 04 Dec 2015 17:40:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a4z1r-0006UQ-52 for bug-guix@gnu.org; Fri, 04 Dec 2015 17:40:42 -0500 Received: from mout.gmx.net ([212.227.15.19]:54701) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a4z1q-0006U3-Qx for bug-guix@gnu.org; Fri, 04 Dec 2015 17:40:39 -0500 Received: from [192.168.1.247] ([77.2.71.66]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0MSMKx-1ZgcVr2tEx-00TXtt for ; Fri, 04 Dec 2015 23:40:36 +0100 Subject: Re: bug#22078: failed builds due to exceeding max-silent-time not marked as failed in db To: bug-guix@gnu.org References: <565F6A9B.9050406@gmx.net> From: Florian Paul Schmidt Message-ID: <56621663.4080007@gmx.net> Date: Fri, 4 Dec 2015 23:40:35 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <565F6A9B.9050406@gmx.net> Content-Type: multipart/mixed; boundary="------------070807090901070700040402" X-Provags-ID: V03:K0:r5MLxTxsPibWfF3KUXMJVlA5Y1xJ0A6z7guihJ70CJnnOO8wb4H kc6Mj5WcR0Puo+BGD33bAKyY7XPwIWjSDUbjaDa0084xV3KqDf/v9L7ReAl6LJ2jLzpV3ea E46wueVq8PEQovoQilIFulAePpYZC1jX6oTt+c3lEszONcvI4VHdzuhv5+apEkDeWTanfKQ SQNhswWeuAonwGZprG12A== X-UI-Out-Filterresults: notjunk:1;V01:K0:EHzlBkKsxNw=:y6hgRxUllKH3ZifUNQoxly owxEgG03GRyaw/glsmw1MuapD020D40UJD9G3nwA/Sv5Y2LQlfWIvnvM2sEXsTN4u+S+YIRCU UTX07boBLygCu4mUijPCuq2Egvm7rePO+2TGwwbZICoMfyysZLFJSYuudUypUX4rMN+YjtKYS Yl46sSu/SO44kbtkUTMiMh+t6+sMNY7mApnp3hkng2hziFjo5e3Tj3AWcN0cOpNwWclXI3sN0 2MTca6KmpyrPw6q4YsQyga232cnHGZOgBABRqW9GiVMgjvanJUxpA4GeOpATh/P0jVHk1s0n9 g+8/Wp2+Lk72Mo2BFAHMEIKCxdVzzHNi9kj+wen9ypCoBVoFpSV3CzKehpzEdOntJxOMItvuu 0y6G/X9RFP86ifqYwBBa6RkKJpc543F1ujmM38XYpTB+kxwmwoiXw52MAudoS0aaEB1G6pW8Y dn54tiZ7CAZ4fuHUSw/s+wzx/XgaEbbGmJfYHhXaDTMpSBLnPcPR7Kxevyv+j6vNBYxVBX84z 3oDJ4GeoBVo/waRQ7e09NJSowI14oaRgeYNLg0oz77lRg4uytXX8gM2m1GKU4G+rEdADEL7zB +u9HQizOgCp33jmV/WNmJH/KyKz0fUNaMC5OfAMJdnEnXssOb1u5mk5q7A9Tay7HdB237x1EV yhm2hEI9eT6pA/ky3ypJ3Ye28iyYs6iR1fN4LlqNQPrQeaU7LmSoeEQD+GCp+T2tTOciqbOl7 bcT5nr6MfFR5JE62j6I6YcMQ99hUq7P21SQ0skzCf544+W+kgsiyrOHcJlE= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.1 (----) This is a multi-part message in MIME format. --------------070807090901070700040402 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Attached is a first stab at fixing this. There are additional options to guix-daemons now: --cache-failures cache build failures --cache-hook-failures cache build failures due to hook failures (depends on cache-failures) --cache-timeout-failures cache build failures due to timeouts (depends on cache-failures) Patch compiles, but is yet untested since the system I need it has gone away for the time being.. Flo On 12/02/2015 11:03 PM, Florian Paul Schmidt wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > > Hi, > > on my system bulding the derivation for the package tbb (version > 4.3.2) does not complete due to exceeding the max-silent-time default > value of 3600 seconds (one hour). > > It seems that in this case the path is not marked as failed in the > sqlite3 db > > /var/guix/db/db.sqlite > > in the table FailedPaths. This is quite annoying since it seems that > several packages depend on it causing the derivation to be built > several times (each taking over an hour to fail). > > The guix daemon is running with the --cache-failures option and I > would expect the second run of > > for n in `guix package -A | cut -f1`; do guix build --no-substitutes > "$n" || true; done > > to be mostly a NOOP, since all failures from the first run should be > cached. And even in the first run I wouldn't expect failed > dependencies to be tried to build again. Contrary to this on this box > even the second run of this takes about half a day or so to complete ;) > > Flo > > P.S.: FYI: The thing that takes over an hour to run is > > ./test_atomic.exe > > > - -- > https://fps.io > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQEcBAEBCAAGBQJWX2qaAAoJEA5f4Coltk8ZnasH/jOg+E0Y/CDxw5SGgcJN0Q6K > TYo41AVz0u9tLJEVYW4ZW9Z7A3UL5OTB+03LwC1zT7iDtFzU6a7BzaW2N3gP+GGi > Tx+Rq0z7ZIHEF1t71YFtPOAIpuyxwl1yMnRo0kd8BVsrNu843ITI4w+kzGV4tcP1 > l9uDf7c+WQ8MFhoMDUqjW5ufIb3zy6yKk1GDXw14xZ8laeiE8hrXFE2LFV4WCxzP > VMPDgHBlPF6pAKLYpWSpL2RtL/WxO9tYIYpQ16EW7GjOouCy2ObT+1CJ75kSIOie > DZ/RLUSxa39amDFwii5liR+ETgvz3FCoBAcyI5AP/76uMToub1z3S1PNt58EnsE= > =Hivd > -----END PGP SIGNATURE----- > > > --------------070807090901070700040402 Content-Type: text/plain; charset=UTF-8; name="0001-guix-daemon-cache-more-failures-if-requested.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="0001-guix-daemon-cache-more-failures-if-requested.patch" RnJvbSAzZTM3NmY3ZDIyYTYyYzE5NDkxZDgzMGMzNDE4MmYyZjQ4MjhmMGEzIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBGbG9yaWFuIFBhdWwgU2NobWlkdCA8bWlzdGEudGFw YXNAZ214Lm5ldD4KRGF0ZTogRnJpLCA0IERlYyAyMDE1IDIzOjM3OjEzICswMTAwClN1Ympl Y3Q6IFtQQVRDSF0gZ3VpeC1kYWVtb246IGNhY2hlIG1vcmUgZmFpbHVyZXMgaWYgcmVxdWVz dGVkCgotLS0KIG5peC9saWJzdG9yZS9idWlsZC5jYyAgICAgICAgIHwgIDggKysrKysrKysK IG5peC9saWJzdG9yZS9nbG9iYWxzLmNjICAgICAgIHwgIDQgKysrKwogbml4L2xpYnN0b3Jl L2dsb2JhbHMuaGggICAgICAgfCAgNiArKysrKysKIG5peC9uaXgtZGFlbW9uL2d1aXgtZGFl bW9uLmNjIHwgMTIgKysrKysrKysrKysrCiA0IGZpbGVzIGNoYW5nZWQsIDMwIGluc2VydGlv bnMoKykKCmRpZmYgLS1naXQgYS9uaXgvbGlic3RvcmUvYnVpbGQuY2MgYi9uaXgvbGlic3Rv cmUvYnVpbGQuY2MKaW5kZXggZWZlMWFiMi4uNDg5MzZmOSAxMDA2NDQKLS0tIGEvbml4L2xp YnN0b3JlL2J1aWxkLmNjCisrKyBiL25peC9saWJzdG9yZS9idWlsZC5jYwpAQCAtMTQ4Mywx MiArMTQ4MywyMCBAQCB2b2lkIERlcml2YXRpb25Hb2FsOjpidWlsZERvbmUoKQogICAgICAg ICAgICAgaWYgKHNldHRpbmdzLnByaW50QnVpbGRUcmFjZSkKICAgICAgICAgICAgICAgICBw cmludE1zZyhsdmxFcnJvciwgZm9ybWF0KCJAIGJ1aWxkLWZhaWxlZCAlMSUgLSB0aW1lb3V0 IikgJSBkcnZQYXRoKTsKICAgICAgICAgICAgIHdvcmtlci50aW1lZE91dCA9IHRydWU7CisK KyAgICAgICAgICAgIGlmIChzZXR0aW5ncy5jYWNoZUZhaWx1cmUgJiYgc2V0dGluZ3MuY2Fj aGVUaW1lb3V0RmFpbHVyZSkKKyAgICAgICAgICAgICAgICBmb3JlYWNoIChEZXJpdmF0aW9u T3V0cHV0czo6aXRlcmF0b3IsIGksIGRydi5vdXRwdXRzKQorICAgICAgICAgICAgICAgICAg ICB3b3JrZXIuc3RvcmUucmVnaXN0ZXJGYWlsZWRQYXRoKGktPnNlY29uZC5wYXRoKTsKICAg ICAgICAgfQogCiAgICAgICAgIGVsc2UgaWYgKGhvb2sgJiYgKCFXSUZFWElURUQoc3RhdHVz KSB8fCBXRVhJVFNUQVRVUyhzdGF0dXMpICE9IDEwMCkpIHsKICAgICAgICAgICAgIGlmIChz ZXR0aW5ncy5wcmludEJ1aWxkVHJhY2UpCiAgICAgICAgICAgICAgICAgcHJpbnRNc2cobHZs RXJyb3IsIGZvcm1hdCgiQCBob29rLWZhaWxlZCAlMSUgLSAlMiUgJTMlIikKICAgICAgICAg ICAgICAgICAgICAgJSBkcnZQYXRoICUgc3RhdHVzICUgZS5tc2coKSk7CisKKyAgICAgICAg ICAgIGlmIChzZXR0aW5ncy5jYWNoZUZhaWx1cmUgJiYgc2V0dGluZ3MuY2FjaGVIb29rRmFp bHVyZSkKKyAgICAgICAgICAgICAgICBmb3JlYWNoIChEZXJpdmF0aW9uT3V0cHV0czo6aXRl cmF0b3IsIGksIGRydi5vdXRwdXRzKQorICAgICAgICAgICAgICAgICAgICB3b3JrZXIuc3Rv cmUucmVnaXN0ZXJGYWlsZWRQYXRoKGktPnNlY29uZC5wYXRoKTsKICAgICAgICAgfQogCiAg ICAgICAgIGVsc2UgewpkaWZmIC0tZ2l0IGEvbml4L2xpYnN0b3JlL2dsb2JhbHMuY2MgYi9u aXgvbGlic3RvcmUvZ2xvYmFscy5jYwppbmRleCAwN2YyM2Q0Li43ODI5YzFjIDEwMDY0NAot LS0gYS9uaXgvbGlic3RvcmUvZ2xvYmFscy5jYworKysgYi9uaXgvbGlic3RvcmUvZ2xvYmFs cy5jYwpAQCAtNDgsNiArNDgsOCBAQCBTZXR0aW5nczo6U2V0dGluZ3MoKQogICAgIGNvbXBy ZXNzTG9nID0gdHJ1ZTsKICAgICBtYXhMb2dTaXplID0gMDsKICAgICBjYWNoZUZhaWx1cmUg PSBmYWxzZTsKKyAgICBjYWNoZVRpbWVvdXRGYWlsdXJlID0gZmFsc2U7CisgICAgY2FjaGVI b29rRmFpbHVyZSA9IGZhbHNlOwogICAgIHBvbGxJbnRlcnZhbCA9IDU7CiAgICAgY2hlY2tS b290UmVhY2hhYmlsaXR5ID0gZmFsc2U7CiAgICAgZ2NLZWVwT3V0cHV0cyA9IGZhbHNlOwpA QCAtMTU4LDYgKzE2MCw4IEBAIHZvaWQgU2V0dGluZ3M6OnVwZGF0ZSgpCiAgICAgX2dldChj b21wcmVzc0xvZywgImJ1aWxkLWNvbXByZXNzLWxvZyIpOwogICAgIF9nZXQobWF4TG9nU2l6 ZSwgImJ1aWxkLW1heC1sb2ctc2l6ZSIpOwogICAgIF9nZXQoY2FjaGVGYWlsdXJlLCAiYnVp bGQtY2FjaGUtZmFpbHVyZSIpOworICAgIF9nZXQoY2FjaGVUaW1lb3V0RmFpbHVyZSwgImJ1 aWxkLWNhY2hlLXRpbWVvdXQtZmFpbHVyZSIpOworICAgIF9nZXQoY2FjaGVIb29rRmFpbHVy ZSwgImJ1aWxkLWNhY2hlLWhvb2stZmFpbHVyZSIpOwogICAgIF9nZXQocG9sbEludGVydmFs LCAiYnVpbGQtcG9sbC1pbnRlcnZhbCIpOwogICAgIF9nZXQoY2hlY2tSb290UmVhY2hhYmls aXR5LCAiZ2MtY2hlY2stcmVhY2hhYmlsaXR5Iik7CiAgICAgX2dldChnY0tlZXBPdXRwdXRz LCAiZ2Mta2VlcC1vdXRwdXRzIik7CmRpZmYgLS1naXQgYS9uaXgvbGlic3RvcmUvZ2xvYmFs cy5oaCBiL25peC9saWJzdG9yZS9nbG9iYWxzLmhoCmluZGV4IGMxN2UxMGQuLmJmODY2NmEg MTAwNjQ0Ci0tLSBhL25peC9saWJzdG9yZS9nbG9iYWxzLmhoCisrKyBiL25peC9saWJzdG9y ZS9nbG9iYWxzLmhoCkBAIC0xNzAsNiArMTcwLDEyIEBAIHN0cnVjdCBTZXR0aW5ncyB7CiAg ICAgLyogV2hldGhlciB0byBjYWNoZSBidWlsZCBmYWlsdXJlcy4gKi8KICAgICBib29sIGNh Y2hlRmFpbHVyZTsKIAorICAgIC8qIFdoZXRoZXIgdG8gY2FjaGUgdGltZW91dCBmYWlsdXJl cyAqLworICAgIGJvb2wgY2FjaGVUaW1lb3V0RmFpbHVyZTsKKworICAgIC8qIFdoZXRoZXIg dG8gY2FjaGUgaG9vayBmYWlsdXJlcyAqLworICAgIGJvb2wgY2FjaGVIb29rRmFpbHVyZTsK KyAgCiAgICAgLyogSG93IG9mdGVuIChpbiBzZWNvbmRzKSB0byBwb2xsIGZvciBsb2Nrcy4g Ki8KICAgICB1bnNpZ25lZCBpbnQgcG9sbEludGVydmFsOwogCmRpZmYgLS1naXQgYS9uaXgv bml4LWRhZW1vbi9ndWl4LWRhZW1vbi5jYyBiL25peC9uaXgtZGFlbW9uL2d1aXgtZGFlbW9u LmNjCmluZGV4IDE5MzQ0ODcuLmY2MTNkZTkgMTAwNjQ0Ci0tLSBhL25peC9uaXgtZGFlbW9u L2d1aXgtZGFlbW9uLmNjCisrKyBiL25peC9uaXgtZGFlbW9uL2d1aXgtZGFlbW9uLmNjCkBA IC04MCw2ICs4MCw4IEBAIGJ1aWxkcyBkZXJpdmF0aW9ucyBvbiBiZWhhbGYgb2YgaXRzIGNs aWVudHMuIik7CiAjZGVmaW5lIEdVSVhfT1BUX05PX0JVSUxEX0hPT0sgMTQKICNkZWZpbmUg R1VJWF9PUFRfR0NfS0VFUF9PVVRQVVRTIDE1CiAjZGVmaW5lIEdVSVhfT1BUX0dDX0tFRVBf REVSSVZBVElPTlMgMTYKKyNkZWZpbmUgR1VJWF9PUFRfQ0FDSEVfVElNRU9VVF9GQUlMVVJF UyAxNworI2RlZmluZSBHVUlYX09QVF9DQUNIRV9IT09LX0ZBSUxVUkVTIDE4CiAKIHN0YXRp YyBjb25zdCBzdHJ1Y3QgYXJncF9vcHRpb24gb3B0aW9uc1tdID0KICAgewpAQCAtMTA0LDYg KzEwNiwxMCBAQCBzdGF0aWMgY29uc3Qgc3RydWN0IGFyZ3Bfb3B0aW9uIG9wdGlvbnNbXSA9 CiAgICAgICBuXygiZG8gbm90IHVzZSB0aGUgJ2J1aWxkIGhvb2snIikgfSwKICAgICB7ICJj YWNoZS1mYWlsdXJlcyIsIEdVSVhfT1BUX0NBQ0hFX0ZBSUxVUkVTLCAwLCAwLAogICAgICAg bl8oImNhY2hlIGJ1aWxkIGZhaWx1cmVzIikgfSwKKyAgICB7ICJjYWNoZS10aW1lb3V0LWZh aWx1cmVzIiwgR1VJWF9PUFRfQ0FDSEVfVElNRU9VVF9GQUlMVVJFUywgMCwgMCwKKyAgICAg IG5fKCJjYWNoZSBidWlsZCBmYWlsdXJlcyBkdWUgdG8gdGltZW91dHMgKGRlcGVuZHMgb24g Y2FjaGUtZmFpbHVyZXMpIikgfSwKKyAgICB7ICJjYWNoZS1ob29rLWZhaWx1cmVzIiwgR1VJ WF9PUFRfQ0FDSEVfSE9PS19GQUlMVVJFUywgMCwgMCwKKyAgICAgIG5fKCJjYWNoZSBidWls ZCBmYWlsdXJlcyBkdWUgdG8gaG9vayBmYWlsdXJlcyAoZGVwZW5kcyBvbiBjYWNoZS1mYWls dXJlcykiKSB9LAogICAgIHsgImxvc2UtbG9ncyIsIEdVSVhfT1BUX0xPU0VfTE9HUywgMCwg MCwKICAgICAgIG5fKCJkbyBub3Qga2VlcCBidWlsZCBsb2dzIikgfSwKICAgICB7ICJkaXNh YmxlLWxvZy1jb21wcmVzc2lvbiIsIEdVSVhfT1BUX0RJU0FCTEVfTE9HX0NPTVBSRVNTSU9O LCAwLCAwLApAQCAtMTg5LDYgKzE5NSwxMiBAQCBwYXJzZV9vcHQgKGludCBrZXksIGNoYXIg KmFyZywgc3RydWN0IGFyZ3Bfc3RhdGUgKnN0YXRlKQogICAgIGNhc2UgR1VJWF9PUFRfQ0FD SEVfRkFJTFVSRVM6CiAgICAgICBzZXR0aW5ncy5jYWNoZUZhaWx1cmUgPSB0cnVlOwogICAg ICAgYnJlYWs7CisgICAgY2FzZSBHVUlYX09QVF9DQUNIRV9USU1FT1VUX0ZBSUxVUkVTOgor ICAgICAgc2V0dGluZ3MuY2FjaGVUaW1lb3V0RmFpbHVyZSA9IHRydWU7CisgICAgICBicmVh azsKKyAgICBjYXNlIEdVSVhfT1BUX0NBQ0hFX0hPT0tfRkFJTFVSRVM6CisgICAgICBzZXR0 aW5ncy5jYWNoZUhvb2tGYWlsdXJlID0gdHJ1ZTsKKyAgICAgIGJyZWFrOwogICAgIGNhc2Ug R1VJWF9PUFRfSU1QRVJTT05BVEVfTElOVVhfMjY6CiAgICAgICBzZXR0aW5ncy5pbXBlcnNv bmF0ZUxpbnV4MjYgPSB0cnVlOwogICAgICAgYnJlYWs7Ci0tIAoyLjUuMAoK --------------070807090901070700040402-- From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 09 15:52:38 2015 Received: (at 22078) by debbugs.gnu.org; 9 Dec 2015 20:52:38 +0000 Received: from localhost ([127.0.0.1]:38118 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1a6lj3-0001a1-Ll for submit@debbugs.gnu.org; Wed, 09 Dec 2015 15:52:37 -0500 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:44895) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1a6liy-0001Zp-EH for 22078@debbugs.gnu.org; Wed, 09 Dec 2015 15:52:36 -0500 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 4E69320ED1; Wed, 9 Dec 2015 14:57:17 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute2.internal (MEProxy); Wed, 09 Dec 2015 14:57:17 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=famulari.name; h= cc:content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=1tkxw OUBH3saNl+s+dsLEk9mzA0=; b=JCMtLTlHbxn09OE4Z3fYhJLx4lmvCrjfMwJMd +sbpWX/CZ2hUTGdyGkc0LGSfosNCtGpG7qATXMJSKvk628rgEwtwJ1GxuHL4qtOu rDGyew9kH/9tPrLj2psLJ6FyZafL3tdB7C9hGwjoB83z9AEjIAkeLdwy4ez0D2u7 nhsAso= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=smtpout; bh=1tkxwOUBH3saNl+s+dsLEk9mzA0=; b=D34DC OZlNrsXUrxACXlyQTbmqWfzyaM4SDUoueO+jd75+QAVYdQ7QF9Upu4uvkMkhyruI 4zHcKo/4u2qYF+7sCCfq/HMB72N+SANJ6G9FhjAu7aaxbsb3bBzTq80Fc1/gpZca K3CVs+EqIYttNcDvm5sBkhkceiWrni4w/sOt9w= X-Sasl-enc: EZi3bg6cFIszn9Fe4/w+S9MRU6R625eLEh9ZI80FYryB 1449691036 Received: from localhost (c-69-249-5-231.hsd1.pa.comcast.net [69.249.5.231]) by mail.messagingengine.com (Postfix) with ESMTPA id EC12D680190; Wed, 9 Dec 2015 14:57:16 -0500 (EST) Date: Wed, 9 Dec 2015 14:57:20 -0500 From: Leo Famulari To: Florian Paul Schmidt Subject: Re: bug#22078: failed builds due to exceeding max-silent-time not marked as failed in db Message-ID: <20151209195720.GA18503@jasmine> References: <565F6A9B.9050406@gmx.net> <56621663.4080007@gmx.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <56621663.4080007@gmx.net> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 22078 Cc: 22078@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Fri, Dec 04, 2015 at 11:40:35PM +0100, Florian Paul Schmidt wrote: > Attached is a first stab at fixing this. There are additional options to > guix-daemons now: > > --cache-failures cache build failures > --cache-hook-failures cache build failures due to hook failures > (depends > on cache-failures) > --cache-timeout-failures cache build failures due to timeouts > (depends > on cache-failures) I see the value of this. I wonder how it would change the semantics of the store? I think the current failure caching system is only based on the hash of the dependency graph. Is that correct? That reinforces the determinism of the system. It seems impossible to make build hook and timeout failures deterministic. Both depend on things that can't be observed by the daemon or the builder. So if these failures are cached, the caching should be enabled separately from the memoized failures. I think that if we do decide to cache timeout failures, the value of "--timeout" and "--max-silent-time" should be included. I have a slow machine that can't build certain packages with the default "--max-silent-time" even when nothing is "competing" for the system's resources. But of course there are other factors that affect this like other processes on the system so it's not deterministic. > Patch compiles, but is yet untested since the system I need it has gone away > for the time being.. Have you tested the patch yet? > > Flo > > On 12/02/2015 11:03 PM, Florian Paul Schmidt wrote: > >-----BEGIN PGP SIGNED MESSAGE----- > >Hash: SHA256 > > > > > >Hi, > > > >on my system bulding the derivation for the package tbb (version > >4.3.2) does not complete due to exceeding the max-silent-time default > >value of 3600 seconds (one hour). > > > >It seems that in this case the path is not marked as failed in the > >sqlite3 db > > > >/var/guix/db/db.sqlite > > > >in the table FailedPaths. This is quite annoying since it seems that > >several packages depend on it causing the derivation to be built > >several times (each taking over an hour to fail). > > > >The guix daemon is running with the --cache-failures option and I > >would expect the second run of > > > >for n in `guix package -A | cut -f1`; do guix build --no-substitutes > >"$n" || true; done > > > >to be mostly a NOOP, since all failures from the first run should be > >cached. And even in the first run I wouldn't expect failed > >dependencies to be tried to build again. Contrary to this on this box > >even the second run of this takes about half a day or so to complete ;) > > > >Flo > > > >P.S.: FYI: The thing that takes over an hour to run is > > > >./test_atomic.exe > > > > > >- -- https://fps.io > >-----BEGIN PGP SIGNATURE----- > >Version: GnuPG v2 > > > >iQEcBAEBCAAGBQJWX2qaAAoJEA5f4Coltk8ZnasH/jOg+E0Y/CDxw5SGgcJN0Q6K > >TYo41AVz0u9tLJEVYW4ZW9Z7A3UL5OTB+03LwC1zT7iDtFzU6a7BzaW2N3gP+GGi > >Tx+Rq0z7ZIHEF1t71YFtPOAIpuyxwl1yMnRo0kd8BVsrNu843ITI4w+kzGV4tcP1 > >l9uDf7c+WQ8MFhoMDUqjW5ufIb3zy6yKk1GDXw14xZ8laeiE8hrXFE2LFV4WCxzP > >VMPDgHBlPF6pAKLYpWSpL2RtL/WxO9tYIYpQ16EW7GjOouCy2ObT+1CJ75kSIOie > >DZ/RLUSxa39amDFwii5liR+ETgvz3FCoBAcyI5AP/76uMToub1z3S1PNt58EnsE= > >=Hivd > >-----END PGP SIGNATURE----- > > > > > > > > From 3e376f7d22a62c19491d830c34182f2f4828f0a3 Mon Sep 17 00:00:00 2001 > From: Florian Paul Schmidt > Date: Fri, 4 Dec 2015 23:37:13 +0100 > Subject: [PATCH] guix-daemon: cache more failures if requested > > --- > nix/libstore/build.cc | 8 ++++++++ > nix/libstore/globals.cc | 4 ++++ > nix/libstore/globals.hh | 6 ++++++ > nix/nix-daemon/guix-daemon.cc | 12 ++++++++++++ > 4 files changed, 30 insertions(+) > > diff --git a/nix/libstore/build.cc b/nix/libstore/build.cc > index efe1ab2..48936f9 100644 > --- a/nix/libstore/build.cc > +++ b/nix/libstore/build.cc > @@ -1483,12 +1483,20 @@ void DerivationGoal::buildDone() > if (settings.printBuildTrace) > printMsg(lvlError, format("@ build-failed %1% - timeout") % drvPath); > worker.timedOut = true; > + > + if (settings.cacheFailure && settings.cacheTimeoutFailure) > + foreach (DerivationOutputs::iterator, i, drv.outputs) > + worker.store.registerFailedPath(i->second.path); > } > > else if (hook && (!WIFEXITED(status) || WEXITSTATUS(status) != 100)) { > if (settings.printBuildTrace) > printMsg(lvlError, format("@ hook-failed %1% - %2% %3%") > % drvPath % status % e.msg()); > + > + if (settings.cacheFailure && settings.cacheHookFailure) > + foreach (DerivationOutputs::iterator, i, drv.outputs) > + worker.store.registerFailedPath(i->second.path); > } > > else { > diff --git a/nix/libstore/globals.cc b/nix/libstore/globals.cc > index 07f23d4..7829c1c 100644 > --- a/nix/libstore/globals.cc > +++ b/nix/libstore/globals.cc > @@ -48,6 +48,8 @@ Settings::Settings() > compressLog = true; > maxLogSize = 0; > cacheFailure = false; > + cacheTimeoutFailure = false; > + cacheHookFailure = false; > pollInterval = 5; > checkRootReachability = false; > gcKeepOutputs = false; > @@ -158,6 +160,8 @@ void Settings::update() > _get(compressLog, "build-compress-log"); > _get(maxLogSize, "build-max-log-size"); > _get(cacheFailure, "build-cache-failure"); > + _get(cacheTimeoutFailure, "build-cache-timeout-failure"); > + _get(cacheHookFailure, "build-cache-hook-failure"); > _get(pollInterval, "build-poll-interval"); > _get(checkRootReachability, "gc-check-reachability"); > _get(gcKeepOutputs, "gc-keep-outputs"); > diff --git a/nix/libstore/globals.hh b/nix/libstore/globals.hh > index c17e10d..bf8666a 100644 > --- a/nix/libstore/globals.hh > +++ b/nix/libstore/globals.hh > @@ -170,6 +170,12 @@ struct Settings { > /* Whether to cache build failures. */ > bool cacheFailure; > > + /* Whether to cache timeout failures */ > + bool cacheTimeoutFailure; > + > + /* Whether to cache hook failures */ > + bool cacheHookFailure; > + > /* How often (in seconds) to poll for locks. */ > unsigned int pollInterval; > > diff --git a/nix/nix-daemon/guix-daemon.cc b/nix/nix-daemon/guix-daemon.cc > index 1934487..f613de9 100644 > --- a/nix/nix-daemon/guix-daemon.cc > +++ b/nix/nix-daemon/guix-daemon.cc > @@ -80,6 +80,8 @@ builds derivations on behalf of its clients."); > #define GUIX_OPT_NO_BUILD_HOOK 14 > #define GUIX_OPT_GC_KEEP_OUTPUTS 15 > #define GUIX_OPT_GC_KEEP_DERIVATIONS 16 > +#define GUIX_OPT_CACHE_TIMEOUT_FAILURES 17 > +#define GUIX_OPT_CACHE_HOOK_FAILURES 18 > > static const struct argp_option options[] = > { > @@ -104,6 +106,10 @@ static const struct argp_option options[] = > n_("do not use the 'build hook'") }, > { "cache-failures", GUIX_OPT_CACHE_FAILURES, 0, 0, > n_("cache build failures") }, > + { "cache-timeout-failures", GUIX_OPT_CACHE_TIMEOUT_FAILURES, 0, 0, > + n_("cache build failures due to timeouts (depends on cache-failures)") }, > + { "cache-hook-failures", GUIX_OPT_CACHE_HOOK_FAILURES, 0, 0, > + n_("cache build failures due to hook failures (depends on cache-failures)") }, > { "lose-logs", GUIX_OPT_LOSE_LOGS, 0, 0, > n_("do not keep build logs") }, > { "disable-log-compression", GUIX_OPT_DISABLE_LOG_COMPRESSION, 0, 0, > @@ -189,6 +195,12 @@ parse_opt (int key, char *arg, struct argp_state *state) > case GUIX_OPT_CACHE_FAILURES: > settings.cacheFailure = true; > break; > + case GUIX_OPT_CACHE_TIMEOUT_FAILURES: > + settings.cacheTimeoutFailure = true; > + break; > + case GUIX_OPT_CACHE_HOOK_FAILURES: > + settings.cacheHookFailure = true; > + break; > case GUIX_OPT_IMPERSONATE_LINUX_26: > settings.impersonateLinux26 = true; > break; > -- > 2.5.0 > From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 13 18:11:36 2015 Received: (at 22078) by debbugs.gnu.org; 13 Dec 2015 23:11:36 +0000 Received: from localhost ([127.0.0.1]:50712 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1a8Fnk-0002BP-Dr for submit@debbugs.gnu.org; Sun, 13 Dec 2015 18:11:36 -0500 Received: from eggs.gnu.org ([208.118.235.92]:36406) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1a8Fni-0002BD-NM for 22078@debbugs.gnu.org; Sun, 13 Dec 2015 18:11:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a8Fna-0008TU-9b for 22078@debbugs.gnu.org; Sun, 13 Dec 2015 18:11:29 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:47617) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a8Fna-0008TI-6B; Sun, 13 Dec 2015 18:11:26 -0500 Received: from reverse-83.fdn.fr ([80.67.176.83]:53318 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1a8FnZ-0000Lp-Gd; Sun, 13 Dec 2015 18:11:25 -0500 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Florian Paul Schmidt Subject: Re: bug#22078: failed builds due to exceeding max-silent-time not marked as failed in db References: <565F6A9B.9050406@gmx.net> <56621663.4080007@gmx.net> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 24 Frimaire an 224 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Mon, 14 Dec 2015 00:11:23 +0100 In-Reply-To: <56621663.4080007@gmx.net> (Florian Paul Schmidt's message of "Fri, 4 Dec 2015 23:40:35 +0100") Message-ID: <87a8peuep0.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 22078 Cc: 22078@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Florian Paul Schmidt skribis: > Attached is a first stab at fixing this. There are additional options > to guix-daemons now: > > --cache-failures cache build failures > --cache-hook-failures cache build failures due to hook failures > (depends > on cache-failures) > --cache-timeout-failures cache build failures due to timeouts > (depends > on cache-failures) OK. I=E2=80=99m unsure whether it makes sense to cache failures due to tim= eout because, by definition, they=E2=80=99re non-deterministic. Another problem is that clients can choose what the timeout is (both max-silent-time and absolute max-time), so it=E2=80=99d be easy for a clien= t to force a timeout failure; on a multi-user system, that would amount to a DoS attack. I=E2=80=99m not sure how to address these issues, so I=E2=80=99m rather in = favor of the status quo. WDYT? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 14 03:39:12 2015 Received: (at 22078) by debbugs.gnu.org; 14 Dec 2015 08:39:12 +0000 Received: from localhost ([127.0.0.1]:50879 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1a8Of1-0006eA-OC for submit@debbugs.gnu.org; Mon, 14 Dec 2015 03:39:11 -0500 Received: from mout.gmx.net ([212.227.17.21]:63470) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1a8Oez-0006dw-Px for 22078@debbugs.gnu.org; Mon, 14 Dec 2015 03:39:10 -0500 Received: from [192.168.1.241] ([93.130.28.71]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0Lp3Qu-1akmLj2uBQ-00esWk; Mon, 14 Dec 2015 09:39:03 +0100 Subject: Re: bug#22078: failed builds due to exceeding max-silent-time not marked as failed in db To: =?UTF-8?Q?Ludovic_Court=c3=a8s?= References: <565F6A9B.9050406@gmx.net> <56621663.4080007@gmx.net> <87a8peuep0.fsf@gnu.org> From: Florian Paul Schmidt X-Enigmail-Draft-Status: N1110 Message-ID: <566E8026.6070903@gmx.net> Date: Mon, 14 Dec 2015 09:39:02 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <87a8peuep0.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:h+lS5cIwoYwKUHrsI3vx/VBfcznRMrvY1doGja9NUdXXvSokLy3 lp+0BdUCgI9I9VqSG0ZuJLj05wmJAAjONsgI7rVW2ePFThJ6L6I221P9L7f/OjsPqvjqCGQ gOoVEMMaGRoUY0P1ZsW02nsWmKEuD+ZiZFM6nPM543QXyuVRb5BTb6Uimec8PslMbHt/Cek eD4KItgKTVa/Wjjtkj7aw== X-UI-Out-Filterresults: notjunk:1;V01:K0:GB4LDpoC9VY=:r8UaoZ3Qn/GH3lylrb7YPv CmGcjI5j1EBSh/zQxKrxKe75GKGCo0A5G0svmtiC/udE2wa5MkgC5t9eosKa01IevfYe/oGCe JprTASSv1bHNhlKmiyIzus+yMKdFP4YmYTng0c+mxh2oTp93N1hcHCruioBMVCA4r7tyYP/YB 4dk8BTIcqBps8e8oov5vRxM/oe5NRJF0/BjOexCp4VypQpfGS3ZNce185bglMpEEh+KSQ3AFo qGRvZQw72uwNBFczAeXDgnJ2zXKvCFw1nkw+kExIOViHrMIkIjSya4IDugYFlOnu8WfTm5WNj OPrn45TZlON0cqUM5SV53knGAq0pylnsmnLNcTs5ZZCTKb9QmSpoqXOF2RJ0kdANZ8v/HJaFE 2XxH7t0tT1XDxgTc+eiDQRrPpccayqvkRZRfGXa8us+dXljjtROtqjysm4lZC+6iCg6CL2IPo MQQJE7v/zwViIj1lYlnpByiX4xVBXQohBsilKGjcOstxscm0AxaeHJcfKwSF4+hUpEoaJrEgc rqUPFz0hmkBgg8ravrYdX5LPWmzVEmB7jBsfblnDS/SkgRT6HZvUwXjV3O4MgLYEtX6aqpMau n7A0HZB+HiSbGrBGqCU6pknUBPk9kWJ1RC58pmHXdDaS5MRkEcz32fqU51iF27lLPqKRPfFpu 8G69XB+bqJ7aa5n1lwa3jNKfExHyWXo/AZuYvfaFV+rxjRgXXARI5I0vA9PYqzy8EomHy2aXh vG4E9+cShE47ebwOBZrUtsBKqxWY1KAzKw1CJjhosBz9x3IbjCLWUF1S2wXZ4Mf0QQj/A6p67 KxoXt5I X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 22078 Cc: 22078@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 14.12.2015 00:11, Ludovic Courtès wrote: > OK. I’m unsure whether it makes sense to cache failures due to > timeout because, by definition, they’re non-deterministic. Except for cases where they are deterministic (Consider a buggy package that has a testcase that reduces to while (true) { } that is not optimized away). They very seldom are though. Ayways: I'm not proposing to make any of this the default. > Another problem is that clients can choose what the timeout is > (both max-silent-time and absolute max-time), so it’d be easy for a > client to force a timeout failure; on a multi-user system, that > would amount to a DoS attack. You mean a user just builds all packages with a timeout that's impossible to fulfill? And consequently all their failures will be cached and if then another user tries to build them they just get the cached failure? That points out another (though more contrived) flaw indeed: Even without caching failures a package might be nondeterministic for some reason (bugs always happen). A user who knows how to trigger the failure (assuming it's depending on something under the user's control) then could DOS that particular build. In general it would probably be good to have a way of resetting the cached failures in the db. Maybe --check does almost this: If a failed derivation gets built again with --check will the subsequent success overwrite the failed one and remove the entry from the FailedPaths table? Or will --check just happily report that the build is nondeterministic? > > I’m not sure how to address these issues, so I’m rather in favor of > the status quo. I found that the changes I made don't seem to work correctly anyways. So LNGTMUAC (let's not get that merged under any circumstances). Flo - -- https://fps.io -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJWboAmAAoJEA5f4Coltk8Zhe4H/2B8jFpvjyTCn87eRoPCVjYV 3bHgjl/WXByrei93l65q+TY+IxFxA66p1Q9GV/cBoj7k/gkFxylUamaqw0wbQEbm yohD0G7YnKpywXCLp1pwJeFeBUGmAe/F0Fw4G45OGcAeIQ7AbDZRHmYq4KRe9x1q i0n96plAirsy5zvBY88bdZU8Fbc4c8pm1Mw2e8B9i3EEWjwcXh8UWeuerTKHhMK4 KNtxgX+Wnx05ZmzOnM3yJKOM8qgujW4peYhVJl3SRAMv/5kLFVCOOUC3XbsijdMM 8ny68tXgE5pNtfHsGko/rqwBT/LQ0C94zO+ggkitd51sgLFKXMRdt+j9pmDLfS0= =ZFzw -----END PGP SIGNATURE----- From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 14 11:40:42 2015 Received: (at 22078-done) by debbugs.gnu.org; 14 Dec 2015 16:40:42 +0000 Received: from localhost ([127.0.0.1]:51765 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1a8WAv-0002qq-3Q for submit@debbugs.gnu.org; Mon, 14 Dec 2015 11:40:42 -0500 Received: from eggs.gnu.org ([208.118.235.92]:35427) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1a8WAp-0002qZ-Qw for 22078-done@debbugs.gnu.org; Mon, 14 Dec 2015 11:40:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a8WAh-0006mS-Jr for 22078-done@debbugs.gnu.org; Mon, 14 Dec 2015 11:40:26 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34514) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a8WAh-0006mN-GW; Mon, 14 Dec 2015 11:40:23 -0500 Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:48670 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1a8WAg-0005zL-Vy; Mon, 14 Dec 2015 11:40:23 -0500 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Florian Paul Schmidt Subject: Re: bug#22078: failed builds due to exceeding max-silent-time not marked as failed in db References: <565F6A9B.9050406@gmx.net> <56621663.4080007@gmx.net> <87a8peuep0.fsf@gnu.org> <566E8026.6070903@gmx.net> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 24 Frimaire an 224 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Mon, 14 Dec 2015 17:40:21 +0100 In-Reply-To: <566E8026.6070903@gmx.net> (Florian Paul Schmidt's message of "Mon, 14 Dec 2015 09:39:02 +0100") Message-ID: <87mvtdx9u2.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 22078-done Cc: 22078-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Florian Paul Schmidt skribis: > On 14.12.2015 00:11, Ludovic Court=C3=A8s wrote: > >> OK. I=E2=80=99m unsure whether it makes sense to cache failures due to >> timeout because, by definition, they=E2=80=99re non-deterministic. > > Except for cases where they are deterministic (Consider a buggy > package that has a testcase that reduces to while (true) { } that is > not optimized away). They very seldom are though. Ayways: I'm not > proposing to make any of this the default. Yes. >> Another problem is that clients can choose what the timeout is >> (both max-silent-time and absolute max-time), so it=E2=80=99d be easy fo= r a >> client to force a timeout failure; on a multi-user system, that >> would amount to a DoS attack. > > You mean a user just builds all packages with a timeout that's > impossible to fulfill? And consequently all their failures will be > cached and if then another user tries to build them they just get the > cached failure? Right. > That points out another (though more contrived) flaw indeed: > > Even without caching failures a package might be nondeterministic for > some reason (bugs always happen). A user who knows how to trigger the > failure (assuming it's depending on something under the user's > control) then could DOS that particular build. That=E2=80=99s very unlikely because builds are performed under a separate = UID, in a container. > In general it would probably be good to have a way of resetting the > cached failures in the db. One can do: guix gc --clear-failures $(guix gc --list-failures) > Maybe --check does almost this: If a failed derivation gets built > again with --check will the subsequent success overwrite the failed > one and remove the entry from the FailedPaths table? Or will --check > just happily report that the build is nondeterministic? Good question. I guess --check would just do nothing, but I haven=E2=80=99t checked. >> I=E2=80=99m not sure how to address these issues, so I=E2=80=99m rather = in favor of >> the status quo. > > I found that the changes I made don't seem to work correctly anyways. > So LNGTMUAC (let's not get that merged under any circumstances). Heh, OK. :-) In general, I expect there should be very few packages that get stuck forever (like Chicken currently), and it=E2=80=99s obviously a bug to fix. = So I guess we can simply live. with the possibility that occasionally your machine will be trying to build Chicken and fail again. ;-) You can always choose a smaller timeout anyway. Thanks, Ludo=E2=80=99. From unknown Tue Jun 17 22:11:09 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 12 Jan 2016 12:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator