From unknown Sat Aug 09 04:54:35 2025 X-Loop: help-debbugs@gnu.org Subject: bug#79077: host_execute and non-zero exit status Resent-From: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= Original-Sender: "Debbugs-submit" Resent-CC: bug-dejagnu@gnu.org Resent-Date: Wed, 23 Jul 2025 07:33:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 79077 X-GNU-PR-Package: dejagnu X-GNU-PR-Keywords: To: 79077@debbugs.gnu.org X-Debbugs-Original-To: bug-dejagnu@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.17532559298999 (code B ref -1); Wed, 23 Jul 2025 07:33:01 +0000 Received: (at submit) by debbugs.gnu.org; 23 Jul 2025 07:32:09 +0000 Received: from localhost ([127.0.0.1]:47904 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ueTxQ-0002L4-O8 for submit@debbugs.gnu.org; Wed, 23 Jul 2025 03:32:09 -0400 Received: from lists.gnu.org ([2001:470:142::17]:58846) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1ueTxJ-0002JF-1A for submit@debbugs.gnu.org; Wed, 23 Jul 2025 03:32:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ueTww-0003k0-O0 for bug-dejagnu@gnu.org; Wed, 23 Jul 2025 03:31:42 -0400 Received: from mail-yw1-x1135.google.com ([2607:f8b0:4864:20::1135]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ueTwu-0007FB-Vk for bug-dejagnu@gnu.org; Wed, 23 Jul 2025 03:31:38 -0400 Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-70e75f30452so41804687b3.2 for ; Wed, 23 Jul 2025 00:31:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753255894; x=1753860694; darn=gnu.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=u5wjU1vo4cQ+1wG9KjRXRFfRSy9RkNlq9jqO+RW7IJk=; b=CfbTzeAuJHE5DlV/NsCq80ndhKz8KZBKqStpFWPxS5VTIVUgieB9fW4aTpzXCI3c1k F3P7EAzmITtu+i1UbweGjmnNPPQQrzc8Pb3mvHSGEmVU5xJ9gY8Zp7B2kT4MTC1apW/F JMB4fYl+7hhO7bMh8YCO7jNW7sUiaKngNqDntxxnwDc6qBwi9Np/4GCW2aXko29BXOp4 z0efEctpo9TDNmUEx7xr/rtGxTjBk5f+5L/HRIE0y7Sw15K7eZnQNInhGkOITlMsiDta F/WHIm23UUOZckO31cu03IlOBX3/fTDviToxQRhimsS9RUz41NB9u2jnLwrbJgmUP4fD SY3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753255894; x=1753860694; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=u5wjU1vo4cQ+1wG9KjRXRFfRSy9RkNlq9jqO+RW7IJk=; b=S03ugFFWPZdtdydJcIyGmM5GI4GUamUZxp/2M97ra6hLOH0nUiFcJTfoPytO1NM4gw ECv1ugqdtnBfCQzD5xrDFjPNE/eIjmfPDI7ucZ2pnaiRX77GbAiQlJLrpqbVatWE0NkL PnAJegEGFTRwvXkrD+juVsZBq86DHmtotO27HE+chUk6OFabaV8kjcvuohFa8ydrsrgy 6w/JB4GsXdhuyOm/9U160QDc/MZi7TIIEVQaqwLqrlSKt3tzmUszhKoFfvVTxOiDqKNU 3NrQiO/HKu5p62AQMl5sWxexoxjkpXy6VuWGJqQehcqp7ifqsFZCW23PpNPfnt/9FaKL FBTg== X-Gm-Message-State: AOJu0Yz5j20b/SbAkUM8V+gdIsrrU+fpfk1nr82tI0kzTqnIOpq1apPf AbkFQSnfvqXDWnNbhPyI/bzfIx9mA3DiANH8VE/xAuk2FoZIY6wny8inCl5rt6zAThbgByyqkua NOlrBD1sMjGWxf+UpZRvtrWZbxTXllJ+/Idlg X-Gm-Gg: ASbGncu640A17fS7hRLQcrhLjAQw+wwDIVnTlv2RaICDsmyE9bINvP7p2QJ9Zgy76lz 1IGs3/PtNFrNPXVI4hMak3Je/ZqcM5oj0I5r5JvdfcRhAqMLpO2qnvN7oimLadU6kl0YwcwQD3L wmbikY3M69I8tTnTc+wxQf9nx72zesM10/ZF8P6t6jYzP2KS5skb/iIsD/YmMOqY6NzHKbM+NPk Tb3k1wj X-Google-Smtp-Source: AGHT+IFYxs0w7ifIKpURWgQ5gprAAhJYX+w+RFsWDVYkBJg+NvHj6w9Z+8fIuF2NAGwAAR4v9NT6aoG4Q//wWBe6Xzk= X-Received: by 2002:a05:690c:350f:b0:712:c14a:a388 with SMTP id 00721157ae682-719b4208afemr23206557b3.7.1753255893924; Wed, 23 Jul 2025 00:31:33 -0700 (PDT) MIME-Version: 1.0 From: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= Date: Wed, 23 Jul 2025 09:31:21 +0200 X-Gm-Features: Ac12FXzLYfgx09WCS1gmCKTVVzcr3k994ua9gsq7lrQiKgq8LJydTyjmKvPrxPg Message-ID: Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::1135; envelope-from=marc.nieper@gmail.com; helo=mail-yw1-x1135.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi, The host_execute procedure in dejagnu.exp (see [1]) doesn't seem to check the exit status of the executed test programs. A test program that simply aborts (e.g. using the C function of the same name) won't cause any testsuite failures. This seems brittle and like a misfeature. What is the supposed way to deal with this? Thanks, Marc -- [1] https://www.gnu.org/software/dejagnu/manual/Running-unit-tests.html From unknown Sat Aug 09 04:54:35 2025 X-Loop: help-debbugs@gnu.org Subject: bug#79077: host_execute and non-zero exit status Resent-From: Jacob Bachmeyer Original-Sender: "Debbugs-submit" Resent-CC: bug-dejagnu@gnu.org Resent-Date: Thu, 24 Jul 2025 03:10:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 79077 X-GNU-PR-Package: dejagnu X-GNU-PR-Keywords: To: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= , 79077@debbugs.gnu.org Reply-To: jcb62281@gmail.com Received: via spool by 79077-submit@debbugs.gnu.org id=B79077.175332655026667 (code B ref 79077); Thu, 24 Jul 2025 03:10:01 +0000 Received: (at 79077) by debbugs.gnu.org; 24 Jul 2025 03:09:10 +0000 Received: from localhost ([127.0.0.1]:53429 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uemKU-0006w1-Aq for submit@debbugs.gnu.org; Wed, 23 Jul 2025 23:09:10 -0400 Received: from mail-yb1-xb36.google.com ([2607:f8b0:4864:20::b36]:43385) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1uemKR-0006vX-8Q for 79077@debbugs.gnu.org; Wed, 23 Jul 2025 23:09:07 -0400 Received: by mail-yb1-xb36.google.com with SMTP id 3f1490d57ef6-e8d2c331bb5so1277856276.1 for <79077@debbugs.gnu.org>; Wed, 23 Jul 2025 20:09:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753326541; x=1753931341; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:reply-to:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=4kw/WIQvdEfFOuVi71KtlKqgPOtlOQkKjuprohqk4xs=; b=MU3yFgGzU/YnqgB2feSyFq755vwBr/pCVfh62zM0zknIeMp4JzsCqMMnGgNmDY6h9O j2gspsnKxEWY7TAih/sku7a9PSsIwYrmvvBw269fRXYx4ptqb5DZqIyd+2DOMiQyk4ii d7oQ6cCXG3Ic2uDV065Z1qpEmZA3IwV6yKB2p53YCifNrL41o7e1eQLGM6uf1SXt1uHi 3zII/9jQ1atGqCzl8b4ufXulVYZzuCmTo8iNHtRcfeVpaBLTT3LbGrZ2zY5KnNvw6ANm gOM3PumzTt5w72BjfcGYxguao/vs/lbbPMVOttoa0AdSletsRV3Pj/23Vv42xZGlSyWC hidA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753326541; x=1753931341; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:reply-to:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4kw/WIQvdEfFOuVi71KtlKqgPOtlOQkKjuprohqk4xs=; b=bSfRNt5brpR65UGinMqC+8093aYRdv5Dvhgx2gE9uUGl8T9dIa00DBNqlJbhVAPJvj Hj2w//WtUjQbLQunYa1trHgxfq8veucR1bZT3EkuemW7JNhhfLF9JIsy2iWKG0WZv4FQ DJ1P/Jgz76TAPSeV5rFrsQxrMXsJFmCXQGKqZiV/11hxKNAyRyp3cm1DSKqTpFMH8P1c /WZ5hjjk3EilVIDSG+/lZjOJv/q1xLhTMOtVJ5ehE5R6QShKLqnq+ycqQRsT1CFthCOh 6y7ZoFeP51CYbRH+lryEqzaVjN9gWJ1NzTYqbF2XatRMEztlEx6lkYE0g3UvH0/idItc 1pBg== X-Forwarded-Encrypted: i=1; AJvYcCWNMyOut2goJtjoUhk5/4Re/5OWJMI9EhBiZz+9gACSNy6etWDgd7D8R+jTXg11wDsa7KfltQ==@debbugs.gnu.org X-Gm-Message-State: AOJu0Yw9OsuTQpKj7g57CrM2KZk3o4cx90jlM6dSofJgrnMzPr6QQ2O1 ZClTohooz+2XLPLpnNf+4gmHI1HhPmJyKXkFMiBL61XTBGELWv9lsjrA X-Gm-Gg: ASbGncshzaJICBT3M+MTPjBcnP65RqIl4NFyLe8rtykObkUkIBKqhwBPuR1Ty3DUT0f a9qdZushbn7NMAa+MticJRqf1XrdjA4UTFzDpRimg+PsY044ebz2oZEmsh6yksmT6eEUD5rxk68 eIpoY7GFyZxqEsi9wIz+6W5ZIEl844+RP4FOZxSbSRWffCCKqkp3VZJZmMQeZ/th78rPhhuO3zN MIiUO9GegLd2v4gebHDlspmGiSzztW6jLmXYnK0d8YluBWHAHkjhM7AyBFY+kIdbIrdKb3ipsqW +Iu2gdb1qBxPNTptDCID1HnSyC89cPF+73UxG73rBzhQjaT8mYlBZBYFLV+650P1JITK45+P21b VBx60cv1k+aSRtI24FRo= X-Google-Smtp-Source: AGHT+IGe5bY/EAi1ek/Yf7SeyHSjCucwEHKuUC/S5QHW6bB/xHUkN8txh4kI/270NO03mVDSbsjgng== X-Received: by 2002:a05:690c:a87:b0:712:c295:d012 with SMTP id 00721157ae682-719ce8ae9e4mr3612477b3.13.1753326541370; Wed, 23 Jul 2025 20:09:01 -0700 (PDT) Received: from [127.0.0.1] ([70.133.144.35]) by smtp.gmail.com with ESMTPSA id 00721157ae682-719cb81cafcsm1589877b3.46.2025.07.23.20.08.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jul 2025 20:09:00 -0700 (PDT) Message-ID: <402791ec-705d-4065-800a-a8c27f19a144@gmail.com> Date: Wed, 23 Jul 2025 22:08:43 -0500 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird References: Content-Language: en-US From: Jacob Bachmeyer In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On 7/23/25 02:31, Marc Nieper-Wißkirchen wrote: > Hi, > > The host_execute procedure in dejagnu.exp (see [1]) doesn't seem to > check the exit status of the executed test programs. A test program > that simply aborts (e.g. using the C function of the same name) won't > cause any testsuite failures. This seems brittle and like a > misfeature. > > What is the supposed way to deal with this? The host_execute procedure is intended for running unit test programs that speak a special DejaGnu protocol and ignoring exit codes from unit test programs is intended.  The DejaGnu unit testing protocol does not depend on the exit code of the unit test program at all, because that exit code might not be available in all environments. Instead, DejaGnu expects an explicit "END" token from the unit test program to indicate that the program has reached its intended completion.  A warning is produced if this token is not observed; perhaps a future version of DejaGnu should insert an UNRESOLVED result like we currently do when a Tcl test script aborts? -- Jacob From unknown Sat Aug 09 04:54:35 2025 X-Loop: help-debbugs@gnu.org Subject: bug#79077: host_execute and non-zero exit status Resent-From: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= Original-Sender: "Debbugs-submit" Resent-CC: bug-dejagnu@gnu.org Resent-Date: Thu, 24 Jul 2025 05:58:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 79077 X-GNU-PR-Package: dejagnu X-GNU-PR-Keywords: To: jcb62281@gmail.com Cc: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= , 79077@debbugs.gnu.org Received: via spool by 79077-submit@debbugs.gnu.org id=B79077.17533366735663 (code B ref 79077); Thu, 24 Jul 2025 05:58:01 +0000 Received: (at 79077) by debbugs.gnu.org; 24 Jul 2025 05:57:53 +0000 Received: from localhost ([127.0.0.1]:54087 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ueoxk-0001TH-H5 for submit@debbugs.gnu.org; Thu, 24 Jul 2025 01:57:52 -0400 Received: from mail-yb1-xb2a.google.com ([2607:f8b0:4864:20::b2a]:56770) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1ueoxh-0001Sr-R3 for 79077@debbugs.gnu.org; Thu, 24 Jul 2025 01:57:50 -0400 Received: by mail-yb1-xb2a.google.com with SMTP id 3f1490d57ef6-e8d7122c3a5so473962276.1 for <79077@debbugs.gnu.org>; Wed, 23 Jul 2025 22:57:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753336664; x=1753941464; darn=debbugs.gnu.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=GcetkvgBXU5ZmqneulzVjRI0jvxtztb573MmJnPNYWs=; b=KFXfN9Jb4vguWVfF6Xdu5VtLrywmc7MwJAD5V5K6sQZOg9IaKM3sgeK1bz8M9e9vwt IPq7vFRID9ElC33LOxALrVdIwmWSWWA5+hERUYZToxrzdDTdVx6RKc8QyXYPPiPJrC4V eN9W95MewKkjmPnuNrItF96wdDLiBpmdK+8Bb/qCiQYKuZFhLX4i/0wuXhhy1mWkyAc/ R6U4IoSxox37BDaWS/Y586hMf8yBuUcp58humXErWdWklM6ykNt/Z9HyPVxUsmUkT9FQ jMhaR3ZmNfrNSvD6QenCaGHTUJWA0PkgEQwYGnzmMvtRBr6mufLbdatOwCXFVWSc0ac8 P18g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753336664; x=1753941464; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GcetkvgBXU5ZmqneulzVjRI0jvxtztb573MmJnPNYWs=; b=dWmdUSAq57x9phlGqoJzNXuxMrXkZN2AHYI5GS40sgZq0fK0JqtT8Wrao8dFVGFcgo w8vssD+9jjt5lqTpmqxEWSguuERS7RXWfejr8aFRaNPc3qBvPenLyixhbL3/Ps6SIdz7 M988UxJL3quTEcjc7PA/wTvJhTCV3M4KC9lkZLQ+QGEdVAwnFGSEFhprrE9wvNl1RzX4 UnLrAy+hN+Ma3oSDoDOUqXuNu1osUgEa6kvr8arBVFMr8GBT5DDZU4UyL14l/alQNgHW 6YbgxuoBg2JK3SyBxCINamCsvEuvejxbSgxdLsKZHwKyjCo0HSg0xZTjsoO/8zVkfaQx +0NQ== X-Forwarded-Encrypted: i=1; AJvYcCXG9k5zdJPbKy3nYHQGwtl63qVTk4wkVnSefNiZ1yFjnM8CGkFZADuIAk0UIvThwrQSX4STBw==@debbugs.gnu.org X-Gm-Message-State: AOJu0YwieOIkUeRqltP5n1XfFD3cQcdBSo8F8G0iCjPv/ufRfN5U39Ho gA2uYhNy8Zs+tPdVCPfccD9/qw4SLyvTyfUMwTHmdUqvRgVck/hKs/CxmLpkGFLakECp1106BFM PMBOJEm8xQ4ALVARTDs1DrCwkxfwIXxY= X-Gm-Gg: ASbGnct2zfJvY65dkXUESd2aag13v4ovHcb3i0XHr5R5ZF92XV8EqLqbrFAQWMPmC6J fz+ic8NtmauYXAu5xeKzDZnNF1WzQeZGxbzh3SeDLSLaxgMxL4m43UQlfPZR2ytwk0UZZW8ZH43 oCNBgiizi6Mmt/HiN0+9s/B86FnHd1EbYXd4COreH+VCrD5XEBPbrznExTsF+UuE8WJHJjadaC5 5q1id6/j1NjsjUm/5k= X-Google-Smtp-Source: AGHT+IHqAzmv5Xn/0fBt65cHuog946jk6gBoxYsTs3sbRWoCSqPJEuAwvD9d++DC+NDm+2vCojIDV4vWygwIC0W8A2Q= X-Received: by 2002:a05:6902:12c7:b0:e8d:717f:a510 with SMTP id 3f1490d57ef6-e8dc58aff6emr6022137276.11.1753336663541; Wed, 23 Jul 2025 22:57:43 -0700 (PDT) MIME-Version: 1.0 References: <402791ec-705d-4065-800a-a8c27f19a144@gmail.com> In-Reply-To: <402791ec-705d-4065-800a-a8c27f19a144@gmail.com> From: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= Date: Thu, 24 Jul 2025 07:57:31 +0200 X-Gm-Features: Ac12FXyz78E-aE-6X_r8cvC66XBAP7l1DgAI6Fquel8hbeszhkf-ihcHXSQEHoE Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Jacob, Thank you for your quick response! Am Do., 24. Juli 2025 um 05:09 Uhr schrieb Jacob Bachmeyer : > > On 7/23/25 02:31, Marc Nieper-Wi=C3=9Fkirchen wrote: > > Hi, > > > > The host_execute procedure in dejagnu.exp (see [1]) doesn't seem to > > check the exit status of the executed test programs. A test program > > that simply aborts (e.g. using the C function of the same name) won't > > cause any testsuite failures. This seems brittle and like a > > misfeature. > > > > What is the supposed way to deal with this? > > The host_execute procedure is intended for running unit test programs > that speak a special DejaGnu protocol and ignoring exit codes from unit > test programs is intended. The DejaGnu unit testing protocol does not > depend on the exit code of the unit test program at all, because that > exit code might not be available in all environments. I am sorry if I wasn't clear enough in my previous email. I didn't mean that the exit code should be part of the actual protocol, only that running the testcase should be considered unsuccessful if the program exits with a failure code (in a POSIX system; in some other environment, there may be other indications for failure of execution). > Instead, DejaGnu expects an explicit "END" token from the unit test > program to indicate that the program has reached its intended > completion. A warning is produced if this token is not observed; > perhaps a future version of DejaGnu should insert an UNRESOLVED result > like we currently do when a Tcl test script aborts? I think inserting an UNRESOLVED result would be more appropriate. Otherwise, there would be no formal failure result if a unit test breaks due to, say, a segfault. libgccjit uses a heavily patched version of host_execute ([1]) that also handles tests run under Valgrind; perhaps some of the ideas from that version can be incorporated into DejaGnu proper. -- Marc [1] https://gcc.gnu.org/git/?p=3Dgcc.git;a=3Dblob;f=3Dgcc/testsuite/jit.dg/= jit.exp;h=3D57b133b6d8c6ba7424d4a97bef1ba34264d469be;hb=3DHEAD#l89 > > > -- Jacob > > From unknown Sat Aug 09 04:54:35 2025 X-Loop: help-debbugs@gnu.org Subject: bug#79077: host_execute and non-zero exit status Resent-From: Jacob Bachmeyer Original-Sender: "Debbugs-submit" Resent-CC: bug-dejagnu@gnu.org Resent-Date: Fri, 25 Jul 2025 02:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 79077 X-GNU-PR-Package: dejagnu X-GNU-PR-Keywords: To: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= Cc: 79077@debbugs.gnu.org Reply-To: jcb62281@gmail.com Received: via spool by 79077-submit@debbugs.gnu.org id=B79077.175340965726939 (code B ref 79077); Fri, 25 Jul 2025 02:15:02 +0000 Received: (at 79077) by debbugs.gnu.org; 25 Jul 2025 02:14:17 +0000 Received: from localhost ([127.0.0.1]:59755 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uf7wu-00070Q-Ll for submit@debbugs.gnu.org; Thu, 24 Jul 2025 22:14:17 -0400 Received: from mail-yw1-x112a.google.com ([2607:f8b0:4864:20::112a]:60901) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1uf7wq-0006zm-VO for 79077@debbugs.gnu.org; Thu, 24 Jul 2025 22:14:14 -0400 Received: by mail-yw1-x112a.google.com with SMTP id 00721157ae682-7115e32802bso14230977b3.1 for <79077@debbugs.gnu.org>; Thu, 24 Jul 2025 19:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753409647; x=1754014447; darn=debbugs.gnu.org; h=in-reply-to:from:content-language:references:cc:to:subject:reply-to :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=U+/8uVFIyJvZ31bWjD4JfBa0KKV0yqsi+m7f15XuGoc=; b=ImYKmDJOT1Am+f9HhlXY4rkV/RxpW/CIAkX8WdghqrPemyju9xzjUQ4v7HVSRFebbR idAXM2aJ+uJh5QHY4ZMTih4baR3cgYUZp+A1p+xXURzn2ZPmpMnoBh2EovkAC3SidTpd g3cN/qNSmUOeXgIOKBq9qHcLKhNa/wWW/PjfHoztDAyrSjIGQopXSWcTgUbb+JZs2ZcC 4Rx3p03lQGGyZGOmvuoOooqmeeZTjTwKS7U2g9+JQYXaS4n13me/glQQs/7GLaK16y7u r47xM1C2d7gDMXlg+i5YEbNSLiFMksemD9aWcCHcvbic8cLt/JWspDvZbJ4i9s1MS9a4 o0+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753409647; x=1754014447; h=in-reply-to:from:content-language:references:cc:to:subject:reply-to :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=U+/8uVFIyJvZ31bWjD4JfBa0KKV0yqsi+m7f15XuGoc=; b=NTiWBoBjEYwhX1y8VgyH0gFdCK0Jw4onOvvknOWoqWnBeuXIlfDrF+D/YwJTQnlE04 rNW1gM/j2QVZHFDnuzBixn57uuejhwBo1x1PZwo8HAb/HhMBolaXyEXjSvLweXAKvKhm CwHAEVSqwV84FYrx+O/mhvCYfNGUplCI10AdMdOLcEMx9wMJeYpcCe7EAS81DawQ1x22 YjR/cVAu7VZ62n8cPJQv/O+Z/stdk/V4mFOVwdERiESwmx1ma0Wggm+yqeBSF6Rf9yyD 1rqgqYnJCrGDWCOrPf4OieiOaaDmxZ7bRqEAN0+6C2O0+82qzLrD6cfc61959FmFL5NR jZHg== X-Gm-Message-State: AOJu0YzLGbSdhNKkq3xvu5BBmaZPmfD8gb10UTKqRnkz6XJwak9Sj2fp ByVc47InWLg7h0SS59yAhhKm2Nh2W2emBBSjEuZ/if/QzKh1tdmfE0QG X-Gm-Gg: ASbGncu64MptSDdnDqHt+rM1htnSkQVHtDsGyQhZUMbjLBb/m6/AuoF4vPo8xPXz/X8 KT4fYi8p4Q2jH6Jns8Pdoe3pjNHw+b9to10J3AXmiwEMZ3MNv6FINDT3Wo69xF2jzJNQJu1jPMz 7t83genXVKvFcyisSPdfViv50qsSuXFvF+UNmwzE7yxZMoPRjDribS4STWUX8dWcgsmXcfycAMx WvLMs2GX/Q3XsSmejqwR3nmbfbwln4gPHIBZYQ36EOebvINhPlKogEPmIejtChIk/UyJyAPbaAT yUTKO7JHGZsTc8RIRdM5gc746NirCAhUcO0XzjuYbswFJQNhQ9bPQK4QBiXy8QrWY7iaxI6AveW P5v8FX7dwRj8/olQ9BUg= X-Google-Smtp-Source: AGHT+IFNIVN3f9a7G9SQ1fJ6BED7tuZGCfQnKUwhKeeHN0Nl3y3Yomroy0lBtGo6kTiDdlmmUAZUUw== X-Received: by 2002:a05:690c:6112:b0:717:b390:1af9 with SMTP id 00721157ae682-719e3407a1emr3670517b3.19.1753409646775; Thu, 24 Jul 2025 19:14:06 -0700 (PDT) Received: from [127.0.0.1] ([70.133.144.35]) by smtp.gmail.com with ESMTPSA id 00721157ae682-719cb91e9d9sm6577107b3.54.2025.07.24.19.14.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jul 2025 19:14:06 -0700 (PDT) Content-Type: multipart/alternative; boundary="------------ojfszxGOoYe5QkouL0herLWs" Message-ID: <88f57f1c-2273-4a6c-9af6-6eaf726463c9@gmail.com> Date: Thu, 24 Jul 2025 21:13:53 -0500 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird References: <402791ec-705d-4065-800a-a8c27f19a144@gmail.com> Content-Language: en-US From: Jacob Bachmeyer In-Reply-To: X-Spam-Score: 0.5 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) This is a multi-part message in MIME format. --------------ojfszxGOoYe5QkouL0herLWs Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 7/24/25 00:57, Marc Nieper-Wißkirchen wrote: > Hi Jacob, > > Thank you for your quick response! You are welcome. Is Nieper-Wisskirchen a proper ASCII transliteration of your name for a thanks in the ChangeLog? > Am Do., 24. Juli 2025 um 05:09 Uhr schrieb Jacob Bachmeyer: >> On 7/23/25 02:31, Marc Nieper-Wißkirchen wrote: >>> Hi, >>> >>> The host_execute procedure in dejagnu.exp (see [1]) doesn't seem to >>> check the exit status of the executed test programs. A test program >>> that simply aborts (e.g. using the C function of the same name) won't >>> cause any testsuite failures. This seems brittle and like a >>> misfeature. >>> >>> What is the supposed way to deal with this? >> The host_execute procedure is intended for running unit test programs >> that speak a special DejaGnu protocol and ignoring exit codes from unit >> test programs is intended. The DejaGnu unit testing protocol does not >> depend on the exit code of the unit test program at all, because that >> exit code might not be available in all environments. > I am sorry if I wasn't clear enough in my previous email. I didn't > mean that the exit code should be part of the actual protocol, only > that running the testcase should be considered unsuccessful if the > program exits with a failure code (in a POSIX system; in some other > environment, there may be other indications for failure of execution). But that would make the exit code part of the protocol.  DejaGnu generally supports running tests on "remote" target boards and a target connected over a serial line is unlikely to return an exit code.  The only way to be sure that a dependency on the exit code will not creep in is to ignore the exit code. Admittedly, DejaGnu does not yet properly support running unit tests on remote targets. >> Instead, DejaGnu expects an explicit "END" token from the unit test >> program to indicate that the program has reached its intended >> completion. A warning is produced if this token is not observed; >> perhaps a future version of DejaGnu should insert an UNRESOLVED result >> like we currently do when a Tcl test script aborts? > I think inserting an UNRESOLVED result would be more appropriate. > Otherwise, there would be no formal failure result if a unit test > breaks due to, say, a segfault. The explicit "END" token is a relatively recent addition (added after the last release).  I was reluctant to outright require it (in case there are any testsuites out there with independent unit test protocol implementations) but I now agree that a warning from the test framework is not enough when a unit test bombs out early. An initial solution has been pushed to Savannah on the PR79077 branch. DejaGnu can now be run directly from a Git checkout, you should be able to simply pass RUNTEST=/full/name/of/working/tree/runtest on the "make check" command line. > libgccjit uses a heavily patched version of host_execute ([1]) that > also handles tests run under Valgrind; perhaps some of the ideas from > that version can be incorporated into DejaGnu proper. That heavily patched version has problems that have since been fixed upstream.  I would encourage you to try getting rid of it. You should be able to move the Valgrind support into a wrapper around host_execute, since host_execute has accepted arguments since 1.6.3; you can test precisely by matching [info body host_execute] against "The arguments are".  Note that the name of the executable needs to be absolute; the DejaGnu utility proc "which" will search PATH for its argument. A first draft (untested) off the top of my head: proc wrap_host_execute {args} { global env set run_under_valgrind [info exists env(RUN_UNDER_VALGRIND)] set exec_args $args set executable [lindex $args 0] set arguments [lrange $args 1 end] if { $run_under_valgrind } { set valgrind_logfile "${executable}.valgrind.txt" set valgrind_params [which "valgrind"] lappend valgrind_params "--leak-check=full" lappend valgrind_params "--log-file=${valgrind_logfile}" set exec_args [eval [list linsert $exec_args 0] $valgrind_params] } set reported_error [eval [list host_execute] $exec_args] if { $run_under_valgrind } { upvar 2 name name parse_valgrind_logfile $name $valgrind_logfile xfail } return $reported_error } You should probably instead use a VALGRIND global and have one of the initialization files default it to [which "valgrind"] if RUN_UNDER_VALGRIND is set in the environment.  I would recommend taking VALGRIND from the first of:     (1)  value given on DejaGnu command line ([info exists VALGRIND])     (2)  environment variable VALGRIND ($::env(VALGRIND) in Tcl)     (3)  default by searching PATH ([which "valgrind"]) A first draft (untested) for that initialization code off the top of my head: if { ![info exists VALGRIND] && [info exists ::env(RUN_UNDER_VALGRIND)] } { if { [info exists ::env(VALGRIND)] } { set VALGRIND $::env(VALGRIND) } else { set VALGRIND [which "valgrind"] } } The code does nothing if VALGRIND is already set because the framework will set it as a Tcl global if it is given on the command line.  A framework-provided value can also originate from multi-pass testing and vary between passes, perhaps to compare results from different Valgrind versions. > [1]https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/testsuite/jit.dg/jit.exp;h=57b133b6d8c6ba7424d4a97bef1ba34264d469be;hb=HEAD#l89 -- Jacob --------------ojfszxGOoYe5QkouL0herLWs Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
On 7/24/25 00:57, Marc Nieper-Wißkirchen wrote:
Hi Jacob,

Thank you for your quick response!

You are welcome.

Is Nieper-Wisskirchen a proper ASCII transliteration of your name for a thanks in the ChangeLog? 

Am Do., 24. Juli 2025 um 05:09 Uhr schrieb Jacob Bachmeyer <jcb62281@gmail.com>:

On 7/23/25 02:31, Marc Nieper-Wißkirchen wrote:

Hi,

The host_execute procedure in dejagnu.exp (see [1]) doesn't seem to
check the exit status of the executed test programs. A test program
that simply aborts (e.g. using the C function of the same name) won't
cause any testsuite failures. This seems brittle and like a
misfeature.

What is the supposed way to deal with this?

The host_execute procedure is intended for running unit test programs
that speak a special DejaGnu protocol and ignoring exit codes from unit
test programs is intended.  The DejaGnu unit testing protocol does not
depend on the exit code of the unit test program at all, because that
exit code might not be available in all environments.

I am sorry if I wasn't clear enough in my previous email. I didn't
mean that the exit code should be part of the actual protocol, only
that running the testcase should be considered unsuccessful if the
program exits with a failure code (in a POSIX system; in some other
environment, there may be other indications for failure of execution).

But that would make the exit code part of the protocol.  DejaGnu generally supports running tests on "remote" target boards and a target connected over a serial line is unlikely to return an exit code.  The only way to be sure that a dependency on the exit code will not creep in is to ignore the exit code.

Admittedly, DejaGnu does not yet properly support running unit tests on remote targets.

Instead, DejaGnu expects an explicit "END" token from the unit test
program to indicate that the program has reached its intended
completion.  A warning is produced if this token is not observed;
perhaps a future version of DejaGnu should insert an UNRESOLVED result
like we currently do when a Tcl test script aborts?

I think inserting an UNRESOLVED result would be more appropriate.
Otherwise, there would be no formal failure result if a unit test
breaks due to, say, a segfault.

The explicit "END" token is a relatively recent addition (added after the last release).  I was reluctant to outright require it (in case there are any testsuites out there with independent unit test protocol implementations) but I now agree that a warning from the test framework is not enough when a unit test bombs out early.

An initial solution has been pushed to Savannah on the PR79077 branch. DejaGnu can now be run directly from a Git checkout, you should be able to simply pass RUNTEST=/full/name/of/working/tree/runtest on the "make check" command line. 

libgccjit uses a heavily patched version of host_execute ([1]) that
also handles tests run under Valgrind; perhaps some of the ideas from
that version can be incorporated into DejaGnu proper.

That heavily patched version has problems that have since been fixed upstream.  I would encourage you to try getting rid of it.

You should be able to move the Valgrind support into a wrapper around host_execute, since host_execute has accepted arguments since 1.6.3; you can test precisely by matching [info body host_execute] against "The arguments are".  Note that the name of the executable needs to be absolute; the DejaGnu utility proc "which" will search PATH for its argument.

A first draft (untested) off the top of my head:

proc wrap_host_execute {args} {
    global env

    set run_under_valgrind [info exists env(RUN_UNDER_VALGRIND)]

    set exec_args $args
    set executable [lindex $args 0]
    set arguments [lrange $args 1 end]

    if { $run_under_valgrind } {
	set valgrind_logfile "${executable}.valgrind.txt"
	set valgrind_params [which "valgrind"]
	lappend valgrind_params "--leak-check=full"
	lappend valgrind_params "--log-file=${valgrind_logfile}"
	set exec_args [eval [list linsert $exec_args 0] $valgrind_params]
    }

    set reported_error [eval [list host_execute] $exec_args]

    if { $run_under_valgrind } {
	upvar 2 name name
	parse_valgrind_logfile $name $valgrind_logfile xfail
    }

    return $reported_error
}

You should probably instead use a VALGRIND global and have one of the initialization files default it to [which "valgrind"] if RUN_UNDER_VALGRIND is set in the environment.  I would recommend taking VALGRIND from the first of:

    (1)  value given on DejaGnu command line ([info exists VALGRIND])
    (2)  environment variable VALGRIND ($::env(VALGRIND) in Tcl)
    (3)  default by searching PATH ([which "valgrind"])

A first draft (untested) for that initialization code off the top of my head:

if { ![info exists VALGRIND] && [info exists ::env(RUN_UNDER_VALGRIND)] } {
    if { [info exists ::env(VALGRIND)] } {
	set VALGRIND $::env(VALGRIND)
    } else {
	set VALGRIND [which "valgrind"]
    }
}

The code does nothing if VALGRIND is already set because the framework will set it as a Tcl global if it is given on the command line.  A framework-provided value can also originate from multi-pass testing and vary between passes, perhaps to compare results from different Valgrind versions.

[1] https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/testsuite/jit.dg/jit.exp;h=57b133b6d8c6ba7424d4a97bef1ba34264d469be;hb=HEAD#l89

-- Jacob
--------------ojfszxGOoYe5QkouL0herLWs-- From unknown Sat Aug 09 04:54:35 2025 X-Loop: help-debbugs@gnu.org Subject: bug#79077: host_execute and non-zero exit status Resent-From: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= Original-Sender: "Debbugs-submit" Resent-CC: bug-dejagnu@gnu.org Resent-Date: Fri, 25 Jul 2025 11:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 79077 X-GNU-PR-Package: dejagnu X-GNU-PR-Keywords: To: jcb62281@gmail.com Cc: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= , 79077@debbugs.gnu.org Received: via spool by 79077-submit@debbugs.gnu.org id=B79077.175344305617399 (code B ref 79077); Fri, 25 Jul 2025 11:31:02 +0000 Received: (at 79077) by debbugs.gnu.org; 25 Jul 2025 11:30:56 +0000 Received: from localhost ([127.0.0.1]:33391 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ufGdb-0004Uv-IR for submit@debbugs.gnu.org; Fri, 25 Jul 2025 07:30:56 -0400 Received: from mail-yb1-xb35.google.com ([2607:f8b0:4864:20::b35]:44406) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1ufGdX-00046I-4q for 79077@debbugs.gnu.org; Fri, 25 Jul 2025 07:30:53 -0400 Received: by mail-yb1-xb35.google.com with SMTP id 3f1490d57ef6-e8dd92e3eeeso1706364276.1 for <79077@debbugs.gnu.org>; Fri, 25 Jul 2025 04:30:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753443045; x=1754047845; darn=debbugs.gnu.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=I5Ls55YLOnQWU0xFFMbW1IQOLVUk1JZ+hGGzotzA0j4=; b=YpVlqpLbOZTuaQttWGFI/rIpYbI39dbaihO4zyOwWFOF8hlL21+8+vyB5XnyafB+0L 9Unugl499niCpyvJf+354twKbOsB6Hohs5CV3TYmgLQAcrZBsF06l2qK1o6m2x9JZarC kCT5osOCY3xPcrHHU/2PvuBi877t6++u6XtQ9S3dygGctX+npg1PdNbX6Lkf0it7tPFc tnueYOUSm9tDr4gsUCqC3iFJx1t60N/GSGzTVtHdMxojT3xH4oT/ODZPSuSLzoKKPO93 KvitpBecuQKeDSZe4boLN2+bXCEGJ2EIFxTsyI2AqqOmP1Ipr5oXx/ufa+aWCP7PhbhP QXlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753443045; x=1754047845; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I5Ls55YLOnQWU0xFFMbW1IQOLVUk1JZ+hGGzotzA0j4=; b=h+iqa2Nwb4ZfeJV+JFmqZVwYjxNZZ1/ClNe6iozC24Y+f0oPraaf2qz6fBxi/EFGG/ NgYxRkT0r7VaATehnD6VhWEcVH7pbg0MmDvqFTRKZXE5NZNVWKiNuaEjtgvxKYEbyFI9 XVz2Fw1mQuIB9D02MI7uf0hSFK4NlNeyA9SKsvsKS53T+ci2hxpK+3CPPNBECbft8Kf7 rRphiV8rQFNyt1EuM4Sagjek19VzHqa7b4WWaqDuoh9FbFL6uqdgaIrhOJVT7BU0Pq5a 2IpKEVBKV2D974mBcxr06GATcaRRrQVrYIynSAPAf485/BPeEA0InOKcJubIkqA5BdXA CYgw== X-Forwarded-Encrypted: i=1; AJvYcCV/LVKBddNz3PN3tJcU1qsARPk+8Du0GDTK72VkBHOyR/nJFq1Kg40XqqlGhlELH6zaCGX4ag==@debbugs.gnu.org X-Gm-Message-State: AOJu0YwxScBvvabymwssF6E9lexGctb1rWwu7d6/PNfy0xMn/3tTBx9z 8cI68BAtouK10hhzL7XxSR+ddXBL3uwMZyIzQFBt/VtWRDcClf/vFDxg9JRqXPY38oSC6VfGcyG sTP1tumO2jR/YstxtQJ7PVLp5GUbkPHE= X-Gm-Gg: ASbGnct0kIJopPXBeBeiY+hIB0By5Ev+xAk8x1keOcd5Ktg2XQZRwtF+dMVaSIhZsTp KVei1fO9A8AHq/TNVCR8Zbqs0Jbnobh8+q9Hst9/tNXMYTDnRQSkrq4ukZkMJJNT5yLSdQ3GD5I mRM+mR0Z5nHF8gYyLCwnxoikT66V+RbIY8d76+DtLe65HrW3qcOQtmTajbcDT4PN/4MY/fSS4xE oYtcnUP X-Google-Smtp-Source: AGHT+IGXl2oGZi7wu2R5qwPCaCLo6XqI1ZUxjB6PPMGt/mPaZZxmM1jerRHifCNdlU61yNHAtv2OTqwn7pPY484JZi8= X-Received: by 2002:a05:6902:726:b0:e8d:6e7e:7149 with SMTP id 3f1490d57ef6-e8df113f508mr1501559276.13.1753443044945; Fri, 25 Jul 2025 04:30:44 -0700 (PDT) MIME-Version: 1.0 References: <402791ec-705d-4065-800a-a8c27f19a144@gmail.com> <88f57f1c-2273-4a6c-9af6-6eaf726463c9@gmail.com> In-Reply-To: <88f57f1c-2273-4a6c-9af6-6eaf726463c9@gmail.com> From: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= Date: Fri, 25 Jul 2025 13:30:33 +0200 X-Gm-Features: Ac12FXy51K_dP_p0oyqTjXNLDRX599GkKlF8YBoqI2-KSqbpvx-oiNqmm71kq-Q Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, Am Fr., 25. Juli 2025 um 04:14 Uhr schrieb Jacob Bachmeyer : [...] > Is Nieper-Wisskirchen a proper ASCII transliteration of your name for a t= hanks in the ChangeLog? Yes, exactly. > Am Do., 24. Juli 2025 um 05:09 Uhr schrieb Jacob Bachmeyer : > > On 7/23/25 02:31, Marc Nieper-Wi=C3=9Fkirchen wrote: > > Hi, > > The host_execute procedure in dejagnu.exp (see [1]) doesn't seem to > check the exit status of the executed test programs. A test program > that simply aborts (e.g. using the C function of the same name) won't > cause any testsuite failures. This seems brittle and like a > misfeature. > > What is the supposed way to deal with this? > > The host_execute procedure is intended for running unit test programs > that speak a special DejaGnu protocol and ignoring exit codes from unit > test programs is intended. The DejaGnu unit testing protocol does not > depend on the exit code of the unit test program at all, because that > exit code might not be available in all environments. > > I am sorry if I wasn't clear enough in my previous email. I didn't > mean that the exit code should be part of the actual protocol, only > that running the testcase should be considered unsuccessful if the > program exits with a failure code (in a POSIX system; in some other > environment, there may be other indications for failure of execution). > > But that would make the exit code part of the protocol. DejaGnu generall= y supports running tests on "remote" target boards and a target connected o= ver a serial line is unlikely to return an exit code. The only way to be s= ure that a dependency on the exit code will not creep in is to ignore the e= xit code. With the recent addition of the "END" token, this is probably moot; without, how would have DejaGnu handled a test running remotely, say over a serial line, where the connection drops early? I saw a non-zero exit code as an analogue of such a dropped connection, which could be expressed by an "UNRESOLVED" result. > > Admittedly, DejaGnu does not yet properly support running unit tests on r= emote targets. > > Instead, DejaGnu expects an explicit "END" token from the unit test > program to indicate that the program has reached its intended > completion. A warning is produced if this token is not observed; > perhaps a future version of DejaGnu should insert an UNRESOLVED result > like we currently do when a Tcl test script aborts? > > I think inserting an UNRESOLVED result would be more appropriate. > Otherwise, there would be no formal failure result if a unit test > breaks due to, say, a segfault. > > The explicit "END" token is a relatively recent addition (added after the= last release). I was reluctant to outright require it (in case there are = any testsuites out there with independent unit test protocol implementation= s) but I now agree that a warning from the test framework is not enough whe= n a unit test bombs out early. > > An initial solution has been pushed to Savannah on the PR79077 branch. De= jaGnu can now be run directly from a Git checkout, you should be able to si= mply pass RUNTEST=3D/full/name/of/working/tree/runtest on the "make check" = command line. That is great news. Thank you! Moreover, thank you very much for your ideas about the Valgrind mapper; very helpful, indeed. Best regards, Marc [...] From unknown Sat Aug 09 04:54:35 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Marc =?UTF-8?Q?Nieper-Wi=C3=9Fkirchen?= Subject: bug#79077: closed (Re: bug#79077: host_execute and non-zero exit status) Message-ID: References: <3aaa197b-b33a-4d0b-be49-45c920c97b87@gmail.com> X-Gnu-PR-Message: they-closed 79077 X-Gnu-PR-Package: dejagnu Reply-To: 79077@debbugs.gnu.org Date: Sat, 26 Jul 2025 04:22:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1753503722-25319-1" This is a multi-part message in MIME format... ------------=_1753503722-25319-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #79077: host_execute and non-zero exit status which was filed against the dejagnu package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 79077@debbugs.gnu.org. --=20 79077: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D79077 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1753503722-25319-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 79077-done) by debbugs.gnu.org; 26 Jul 2025 04:21:45 +0000 Received: from localhost ([127.0.0.1]:38624 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ufWPo-0006ZO-Fd for submit@debbugs.gnu.org; Sat, 26 Jul 2025 00:21:44 -0400 Received: from mail-oa1-x36.google.com ([2001:4860:4864:20::36]:47389) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1ufWPk-0006Z5-4W for 79077-done@debbugs.gnu.org; Sat, 26 Jul 2025 00:21:41 -0400 Received: by mail-oa1-x36.google.com with SMTP id 586e51a60fabf-2e95f0b6cb7so1303260fac.3 for <79077-done@debbugs.gnu.org>; Fri, 25 Jul 2025 21:21:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753503694; x=1754108494; darn=debbugs.gnu.org; h=in-reply-to:from:content-language:references:cc:to:subject:reply-to :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=UOUQ+IDR4A30QUEkLOCyVpnXj5Va/qebBzVI5HV8Tac=; b=E58LUCPMRCnW4QXUxbOgAFrCSJuqRr8uKAVfT833IHpkf0sO0VfVV+sEd+EnUtlEL7 zUkM3YUEWOA5TLEpwnIIVzWtXvUReU14lnjpjC/TqYmv0gJYfgJK7FomLBTOnmSPkumP jqe+/qLkDPI/FsmwrPdR1VoXsVfwCE7jWvqi3qGC9JGe9n6yS/SzA8inZkFHuoqn0FIK Uz6inXMZ6opLRvgtIa4kovfcAUDisrSzKbPIKYgWRUoWC9urLFrng0e7T34bvgSVw7x3 YRK/cp8xVwla85rBDCY+qU2GN5DYyN4/Yq8UHOGwvON/VbEIkgUFCVC5umXZH8z9Wmeo QKEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753503694; x=1754108494; h=in-reply-to:from:content-language:references:cc:to:subject:reply-to :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=UOUQ+IDR4A30QUEkLOCyVpnXj5Va/qebBzVI5HV8Tac=; b=PXRG3hq0ueBN1J1l8py6PnocJpak3AH+xP5hvoSWoan+BLGDja6egYUsO6ozhhKW3y ollRBY2Mbvu7h667Ydd52nU2JbtF3Td0IVNN9V9w6Bp7yRHcKizB+v7gjVVNWUPlnfdj C5fUSEjl5Qck9W5ZTN10sMZr4Ot03d2DInqnb30ff0YOlqGgOeJfJk/SALCWO3BMDudI 99cxb9ReQ2QOv+hcNUgc54cHHwx1Ea0LTHnEnAHuxuZao9v5Bh0p+hvqq5Y64/HMURN1 jyxt443xQC1IKWyC/+All+6l1P+VRzeE/E5CY5I2KGr+UtltuE9Ixa4VgjQhpw8sxPal vCBw== X-Gm-Message-State: AOJu0YwfA7Q1/5DK2+AjSN9keL1xy7P4svLHZmMDNlwSlJ+pgYNybdbC wnWXSMRX+ClpIZoom+B6z5cv5Wmjgpf/bGEJpFE6UFNbqImr+FHNmz2g X-Gm-Gg: ASbGncs5W7vc/4Teak7VQ/T9vWstpmPSnmTgA1vZsr7XWLghbIWMznvYlKo4VbCjeY2 6ZAKsJdOEVj/mB4sG4iVbEy8ery/ZGhSzHskz1UKvDmYd7enwjfyWn3oUcp/pLFE6cH4Dm95WWW iAvUDmg9ALrZdBv+iCB4VBYbMsxo3qw8FJyMXCiDcUxRt92LFuxuJp+vHpCqj1M4x2gdSvrbyBL B1nW+Q+bI8Q1sMYokW5P+L7yeJxpNI1+wRUpiSwqLcOFS9ij1U10BFvY32m2CnHgpqz2iLl52vB o0fZwbJId4yIR6vcX4Ho83OO6n72piGz4dpQn9r5jitwDdEZ8EEBTKQv65ynbEvh3Pz/Cvk8az9 ziYrB7i42GOITc1eSkCs= X-Google-Smtp-Source: AGHT+IEUIxgNw2/vJkgFQP9EKZz+KmZr1XivqujePsK3qW1Sbg8gI7rjqHWdSvTVQ68X7QFuqCAA+w== X-Received: by 2002:a05:6871:230b:b0:303:5cb:4296 with SMTP id 586e51a60fabf-30701aedafdmr2853373fac.30.1753503693803; Fri, 25 Jul 2025 21:21:33 -0700 (PDT) Received: from [127.0.0.1] ([70.133.144.35]) by smtp.gmail.com with ESMTPSA id 586e51a60fabf-30712a71ffdsm355882fac.0.2025.07.25.21.21.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jul 2025 21:21:32 -0700 (PDT) Content-Type: multipart/alternative; boundary="------------KxHWx8eLwPywVayIUMliwlbA" Message-ID: <3aaa197b-b33a-4d0b-be49-45c920c97b87@gmail.com> Date: Fri, 25 Jul 2025 23:21:19 -0500 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bug#79077: host_execute and non-zero exit status To: =?UTF-8?Q?Marc_Nieper-Wi=C3=9Fkirchen?= References: <402791ec-705d-4065-800a-a8c27f19a144@gmail.com> <88f57f1c-2273-4a6c-9af6-6eaf726463c9@gmail.com> Content-Language: en-US From: Jacob Bachmeyer In-Reply-To: X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 79077-done Cc: 79077-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: jcb62281@gmail.com Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) This is a multi-part message in MIME format. --------------KxHWx8eLwPywVayIUMliwlbA Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 7/25/25 06:30, Marc Nieper-Wißkirchen wrote: > Hi, > > Am Fr., 25. Juli 2025 um 04:14 Uhr schrieb Jacob Bachmeyer: > > [...] > >> Am Do., 24. Juli 2025 um 05:09 Uhr schrieb Jacob Bachmeyer: >> >> On 7/23/25 02:31, Marc Nieper-Wißkirchen wrote: >> [...] >> >> [...] DejaGnu generally supports running tests on "remote" target boards and a target connected over a serial line is unlikely to return an exit code. The only way to be sure that a dependency on the exit code will not creep in is to ignore the exit code. > With the recent addition of the "END" token, this is probably moot; > without, how would have DejaGnu handled a test running remotely, say > over a serial line, where the connection drops early? I saw a non-zero > exit code as an analogue of such a dropped connection, which could be > expressed by an "UNRESOLVED" result. Previously, DejaGnu gave no indication that anything was wrong if a unit test program failed to complete, although older versions used a "Totals" line as an implicit end marker.  This was changed when the unit testing protocol was documented to avoid possible misfires if a unit test happens to emit a line beginning with "Totals"; the new "END" marker uses a syntax that is explicitly documented as reserved for unit test messages to the framework. >> [...] >> >> An initial solution has been pushed to Savannah on the PR79077 branch. DejaGnu can now be run directly from a Git checkout, you should be able to simply pass RUNTEST=/full/name/of/working/tree/runtest on the "make check" command line. > That is great news. Thank you! Moreover, thank you very much for your > ideas about the Valgrind mapper; very helpful, indeed. You are welcome. Since an improvement has been merged to Git master, this closes the bug report. -- Jacob --------------KxHWx8eLwPywVayIUMliwlbA Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
On 7/25/25 06:30, Marc Nieper-Wißkirchen wrote:
Hi,

Am Fr., 25. Juli 2025 um 04:14 Uhr schrieb Jacob Bachmeyer <jcb62281@gmail.com>:

[...]


Am Do., 24. Juli 2025 um 05:09 Uhr schrieb Jacob Bachmeyer <jcb62281@gmail.com>:

On 7/23/25 02:31, Marc Nieper-Wißkirchen wrote:
[...]

[...]  DejaGnu generally supports running tests on "remote" target boards and a target connected over a serial line is unlikely to return an exit code.  The only way to be sure that a dependency on the exit code will not creep in is to ignore the exit code.

With the recent addition of the "END" token, this is probably moot;
without, how would have DejaGnu handled a test running remotely, say
over a serial line, where the connection drops early? I saw a non-zero
exit code as an analogue of such a dropped connection, which could be
expressed by an "UNRESOLVED" result.

Previously, DejaGnu gave no indication that anything was wrong if a unit test program failed to complete, although older versions used a "Totals" line as an implicit end marker.  This was changed when the unit testing protocol was documented to avoid possible misfires if a unit test happens to emit a line beginning with "Totals"; the new "END" marker uses a syntax that is explicitly documented as reserved for unit test messages to the framework. 

[...]

An initial solution has been pushed to Savannah on the PR79077 branch. DejaGnu can now be run directly from a Git checkout, you should be able to simply pass RUNTEST=/full/name/of/working/tree/runtest on the "make check" command line.

That is great news. Thank you! Moreover, thank you very much for your
ideas about the Valgrind mapper; very helpful, indeed.

You are welcome.

Since an improvement has been merged to Git master, this closes the bug report.


-- Jacob


--------------KxHWx8eLwPywVayIUMliwlbA-- ------------=_1753503722-25319-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 23 Jul 2025 07:32:09 +0000 Received: from localhost ([127.0.0.1]:47904 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ueTxQ-0002L4-O8 for submit@debbugs.gnu.org; Wed, 23 Jul 2025 03:32:09 -0400 Received: from lists.gnu.org ([2001:470:142::17]:58846) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1ueTxJ-0002JF-1A for submit@debbugs.gnu.org; Wed, 23 Jul 2025 03:32:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ueTww-0003k0-O0 for bug-dejagnu@gnu.org; Wed, 23 Jul 2025 03:31:42 -0400 Received: from mail-yw1-x1135.google.com ([2607:f8b0:4864:20::1135]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ueTwu-0007FB-Vk for bug-dejagnu@gnu.org; Wed, 23 Jul 2025 03:31:38 -0400 Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-70e75f30452so41804687b3.2 for ; Wed, 23 Jul 2025 00:31:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753255894; x=1753860694; darn=gnu.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=u5wjU1vo4cQ+1wG9KjRXRFfRSy9RkNlq9jqO+RW7IJk=; b=CfbTzeAuJHE5DlV/NsCq80ndhKz8KZBKqStpFWPxS5VTIVUgieB9fW4aTpzXCI3c1k F3P7EAzmITtu+i1UbweGjmnNPPQQrzc8Pb3mvHSGEmVU5xJ9gY8Zp7B2kT4MTC1apW/F JMB4fYl+7hhO7bMh8YCO7jNW7sUiaKngNqDntxxnwDc6qBwi9Np/4GCW2aXko29BXOp4 z0efEctpo9TDNmUEx7xr/rtGxTjBk5f+5L/HRIE0y7Sw15K7eZnQNInhGkOITlMsiDta F/WHIm23UUOZckO31cu03IlOBX3/fTDviToxQRhimsS9RUz41NB9u2jnLwrbJgmUP4fD SY3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753255894; x=1753860694; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=u5wjU1vo4cQ+1wG9KjRXRFfRSy9RkNlq9jqO+RW7IJk=; b=S03ugFFWPZdtdydJcIyGmM5GI4GUamUZxp/2M97ra6hLOH0nUiFcJTfoPytO1NM4gw ECv1ugqdtnBfCQzD5xrDFjPNE/eIjmfPDI7ucZ2pnaiRX77GbAiQlJLrpqbVatWE0NkL PnAJegEGFTRwvXkrD+juVsZBq86DHmtotO27HE+chUk6OFabaV8kjcvuohFa8ydrsrgy 6w/JB4GsXdhuyOm/9U160QDc/MZi7TIIEVQaqwLqrlSKt3tzmUszhKoFfvVTxOiDqKNU 3NrQiO/HKu5p62AQMl5sWxexoxjkpXy6VuWGJqQehcqp7ifqsFZCW23PpNPfnt/9FaKL FBTg== X-Gm-Message-State: AOJu0Yz5j20b/SbAkUM8V+gdIsrrU+fpfk1nr82tI0kzTqnIOpq1apPf AbkFQSnfvqXDWnNbhPyI/bzfIx9mA3DiANH8VE/xAuk2FoZIY6wny8inCl5rt6zAThbgByyqkua NOlrBD1sMjGWxf+UpZRvtrWZbxTXllJ+/Idlg X-Gm-Gg: ASbGncu640A17fS7hRLQcrhLjAQw+wwDIVnTlv2RaICDsmyE9bINvP7p2QJ9Zgy76lz 1IGs3/PtNFrNPXVI4hMak3Je/ZqcM5oj0I5r5JvdfcRhAqMLpO2qnvN7oimLadU6kl0YwcwQD3L wmbikY3M69I8tTnTc+wxQf9nx72zesM10/ZF8P6t6jYzP2KS5skb/iIsD/YmMOqY6NzHKbM+NPk Tb3k1wj X-Google-Smtp-Source: AGHT+IFYxs0w7ifIKpURWgQ5gprAAhJYX+w+RFsWDVYkBJg+NvHj6w9Z+8fIuF2NAGwAAR4v9NT6aoG4Q//wWBe6Xzk= X-Received: by 2002:a05:690c:350f:b0:712:c14a:a388 with SMTP id 00721157ae682-719b4208afemr23206557b3.7.1753255893924; Wed, 23 Jul 2025 00:31:33 -0700 (PDT) MIME-Version: 1.0 From: =?UTF-8?Q?Marc_Nieper=2DWi=C3=9Fkirchen?= Date: Wed, 23 Jul 2025 09:31:21 +0200 X-Gm-Features: Ac12FXzLYfgx09WCS1gmCKTVVzcr3k994ua9gsq7lrQiKgq8LJydTyjmKvPrxPg Message-ID: Subject: host_execute and non-zero exit status To: bug-dejagnu@gnu.org Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::1135; envelope-from=marc.nieper@gmail.com; helo=mail-yw1-x1135.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi, The host_execute procedure in dejagnu.exp (see [1]) doesn't seem to check the exit status of the executed test programs. A test program that simply aborts (e.g. using the C function of the same name) won't cause any testsuite failures. This seems brittle and like a misfeature. What is the supposed way to deal with this? Thanks, Marc -- [1] https://www.gnu.org/software/dejagnu/manual/Running-unit-tests.html ------------=_1753503722-25319-1--