From unknown Tue Jun 17 22:27:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67392: dejagnu too noisy by default Resent-From: Tom Tromey Original-Sender: "Debbugs-submit" Resent-CC: bug-dejagnu@gnu.org Resent-Date: Thu, 23 Nov 2023 00:00:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 67392 X-GNU-PR-Package: dejagnu X-GNU-PR-Keywords: To: 67392@debbugs.gnu.org X-Debbugs-Original-To: bug-dejagnu@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.170069756222687 (code B ref -1); Thu, 23 Nov 2023 00:00:01 +0000 Received: (at submit) by debbugs.gnu.org; 22 Nov 2023 23:59:22 +0000 Received: from localhost ([127.0.0.1]:60062 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r5x7q-0005tq-3S for submit@debbugs.gnu.org; Wed, 22 Nov 2023 18:59:22 -0500 Received: from lists.gnu.org ([2001:470:142::17]:55500) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r5x7n-0005tc-Oc for submit@debbugs.gnu.org; Wed, 22 Nov 2023 18:59:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r5x7d-0006MR-W2 for bug-dejagnu@gnu.org; Wed, 22 Nov 2023 18:59:10 -0500 Received: from omta038.useast.a.cloudfilter.net ([44.202.169.37]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r5x7b-00021D-PH for bug-dejagnu@gnu.org; Wed, 22 Nov 2023 18:59:09 -0500 Received: from eig-obgw-6010a.ext.cloudfilter.net ([10.0.30.248]) by cmsmtp with ESMTPS id 5k1brvJaWWcCI5x7Zrtmdi; Wed, 22 Nov 2023 23:59:05 +0000 Received: from box5379.bluehost.com ([162.241.216.53]) by cmsmtp with ESMTPS id 5x7Yryu388vT05x7YrntJz; Wed, 22 Nov 2023 23:59:04 +0000 X-Authority-Analysis: v=2.4 cv=ffi+dmcF c=1 sm=1 tr=0 ts=655e95c8 a=ApxJNpeYhEAb1aAlGBBbmA==:117 a=ApxJNpeYhEAb1aAlGBBbmA==:17 a=OWjo9vPv0XrRhIrVQ50Ab3nP57M=:19 a=dLZJa+xiwSxG16/P+YVxDGlgEgI=:19 a=BNY50KLci1gA:10 a=Qbun_eYptAEA:10 a=UOaJ97vcZhxrPfGErDsA:9 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=tromey.com; s=default; h=Content-Type:MIME-Version:Message-ID:Date:Subject:To:From:Sender :Reply-To:Cc:Content-Transfer-Encoding:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=/XZGeiOSh4CaSHiucA+mHh9aFKvC+/7OimqA8JcYkDs=; b=NC0t/hDOVXTvyW3n6cOcAjdpOV 3qIiaz3RQUcT3wp56/SbIj6LMkno7z/FQ6r55p8VfI53bmuS/CAf5CPSx17kmExpwfysoIQn+JcRO PETs7/aLKb/vN1rGTJ2tIyt7l; Received: from [161.98.8.3] (port=51814 helo=prentzel) by box5379.bluehost.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96.2) (envelope-from ) id 1r5x7X-003VQn-2s; Wed, 22 Nov 2023 16:59:03 -0700 From: Tom Tromey X-Attribution: Tom Date: Wed, 22 Nov 2023 16:59:17 -0700 Message-ID: <87v89txru2.fsf@tromey.com> MIME-Version: 1.0 Content-Type: text/plain X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - box5379.bluehost.com X-AntiAbuse: Original Domain - gnu.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - tromey.com X-BWhitelist: no X-Source-IP: 161.98.8.3 X-Source-L: No X-Exim-ID: 1r5x7X-003VQn-2s X-Source: X-Source-Args: X-Source-Dir: X-Source-Sender: (prentzel) [161.98.8.3]:51814 X-Source-Auth: tom+tromey.com X-Email-Count: 2 X-Org: HG=bhshared;ORG=bluehost; X-Source-Cap: ZWx5bnJvYmk7ZWx5bnJvYmk7Ym94NTM3OS5ibHVlaG9zdC5jb20= X-Local-Domain: yes X-CMAE-Envelope: MS4xfPgRqlzw6MjCUfrmMrUTu3FfYBHXySYrg1cOaaVDkMvPLZkLr1YLGv8fb70PN8sWs06CCLpPmPnqOws39/KIkks23JZIIbmNglo7HnNeMDEodZuIqxMC sh7tqPF7vSyu837f0q6uW1kMFr9g0na3W1o/LOGMyXxtM3uvHmw4TpZm2IYAWzEBPVrJdqYCwm2RXQ== Received-SPF: permerror client-ip=44.202.169.37; envelope-from=tom@tromey.com; helo=omta038.useast.a.cloudfilter.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) I think runtest prints too much information to stdout by default. Consider the appended output. This comes from running a single gdb test. A little bit of this is gdb-specific (the "HIP" override line). But most of it is not. Most output is not really useful. In fact, pretty much the only lines I might normally be interested in are the "Running [...]" line and the result summary, so that if I "make -j8 check" I can easily see which tests are being run. (FWIW for -j8 even the summary isn't interesting because gdb's Makefile has to collate the results at the end anyway.) So, I think runtest should print a lot less text by default. Emitting it to the .log file is sufficient (although some could even be trimmed from there). If you don't like that, though, how about a command line option to make it be more quiet? Tom prentzel. runtest gdb.cp/local-static.exp WARNING: Couldn't find the global config file. Using /home/tromey/gdb/build/gdb/testsuite/../../../binutils-gdb/gdb/testsuite/lib/gdb.exp as tool init file. NOTE: Dejagnu's default_target_compile is missing support for HIP, using local override Test run by tromey on Wed Nov 22 16:53:00 2023 Native configuration is x86_64-pc-linux-gnu === gdb tests === Schedule of variations: unix Running target unix Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target. Using /usr/share/dejagnu/config/unix.exp as generic interface file for target. Using /home/tromey/gdb/build/gdb/testsuite/../../../binutils-gdb/gdb/testsuite/config/unix.exp as tool-and-target-specific interface file. Running /home/tromey/gdb/build/gdb/testsuite/../../../binutils-gdb/gdb/testsuite/gdb.cp/local-static.exp ... === gdb Summary === # of expected passes 407 /home/tromey/gdb/build/gdb/gdb version 15.0.50.20231121-git -nw -nx -q -iex "set height 0" -iex "set width 0" -data-directory /home/tromey/gdb/build/gdb/data-directory From unknown Tue Jun 17 22:27:47 2025 X-Loop: help-debbugs@gnu.org Subject: bug#67392: dejagnu too noisy by default Resent-From: Jacob Bachmeyer Original-Sender: "Debbugs-submit" Resent-CC: bug-dejagnu@gnu.org Resent-Date: Thu, 23 Nov 2023 04:04:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67392 X-GNU-PR-Package: dejagnu X-GNU-PR-Keywords: To: Tom Tromey Cc: 67392@debbugs.gnu.org Reply-To: jcb62281@gmail.com Received: via spool by 67392-submit@debbugs.gnu.org id=B67392.170071222318078 (code B ref 67392); Thu, 23 Nov 2023 04:04:02 +0000 Received: (at 67392) by debbugs.gnu.org; 23 Nov 2023 04:03:43 +0000 Received: from localhost ([127.0.0.1]:60318 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r60wI-0004hW-M0 for submit@debbugs.gnu.org; Wed, 22 Nov 2023 23:03:43 -0500 Received: from mail-oa1-x2c.google.com ([2001:4860:4864:20::2c]:46345) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r60wG-0004hI-Ad for 67392@debbugs.gnu.org; Wed, 22 Nov 2023 23:03:40 -0500 Received: by mail-oa1-x2c.google.com with SMTP id 586e51a60fabf-1f93d0cd2ddso319701fac.1 for <67392@debbugs.gnu.org>; Wed, 22 Nov 2023 20:03:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700712210; x=1701317010; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:references:subject:cc:to :mime-version:user-agent:reply-to:from:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=CmJY4vfZxR8hw0MnI+Yqxhercx8rz4fMgML172Iz5OY=; b=Lw1mpw8vg5GvfimsPZIzL/pHdVquVS/Io1Bw23K9JGYbMJ6wPpzEZAu30yD0oPGdbU 4A+gkFTKn+u/7UVygeWp/GcWvNM61yPWaVsMmlpP4kyIfgpjoSIoRaFHI9FfGrGABvND jrQYs81eTZ3ADI0qIkPuk/RLMoqyfFlC7WH3VhzKmVou43iLppsUEGXQb3B66I6Qo+yy Bg9pUIRYA3P0Yjb8QnBS0zlRslPHPq9uCymSZiaXkM6g3KcAm4SnP9eOINQgWfn5BYDt STUSMhqiigvg5rFqzfhsk4buWc5U0GZNJAu9ymljhONFW9bvgfwkKd4lAiWHqfMSjpPo 7U/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700712210; x=1701317010; h=content-transfer-encoding:in-reply-to:references:subject:cc:to :mime-version:user-agent:reply-to:from:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CmJY4vfZxR8hw0MnI+Yqxhercx8rz4fMgML172Iz5OY=; b=HJYuaDxiRA7hkHd1rh3ojX5RS8HAfnFLkbfjtUsuEfzrj1CLxJehTqPv967PlWX/v2 5C5UhVjylARPBooo4UoMeUWpygfATtDOPiJRfLxJffW0e+0880zgGPTxAZWwNNycOWRd 2l0sDUmDgN/ID0tey34PdHlQF5U6FdK10/iJBTfP8cVSEiChWRyqZY7EbkdJfVHg8/ls hiUWBZE8CiyHRRVr49sGQbUCYJ/Y44W0dUatQdGNgQJ+BcoAa0MWzjaNUs9JXRZy9+NP pZVE2B/llKdSuN24vwfa4NZayHEiWlGlP9eIBs0hjAYAlPurQ81MaSlE+VYlSseehnDA kS/w== X-Gm-Message-State: AOJu0YwdpRheUpJmeNhhSQPnMHMAXSXHZYN17qo/y/PMYaytPUUWR+NM Hmrm/bUzL2oYGgzhht3d77kMUxJBOYA= X-Google-Smtp-Source: AGHT+IGhvVYxvv3e4q8gN7IEJyQsEKkJpHf4F5mxB23QoThmQB0VSgBOBR8dz+C8boOBIfhCwsO0uA== X-Received: by 2002:a05:6870:9d8c:b0:1f5:ecc1:4896 with SMTP id pv12-20020a0568709d8c00b001f5ecc14896mr5913045oab.7.1700712210431; Wed, 22 Nov 2023 20:03:30 -0800 (PST) Received: from [127.0.0.1] ([70.133.144.43]) by smtp.gmail.com with ESMTPSA id pi20-20020a056871d11400b001e11ad88f7csm127202oac.30.2023.11.22.20.03.29 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 22 Nov 2023 20:03:30 -0800 (PST) Message-ID: <655ECF0F.6020008@gmail.com> Date: Wed, 22 Nov 2023 22:03:27 -0600 From: Jacob Bachmeyer User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.22) Gecko/20090807 MultiZilla/1.8.3.4e SeaMonkey/1.1.17 Mnenhy/0.7.6.0 MIME-Version: 1.0 References: <87v89txru2.fsf@tromey.com> In-Reply-To: <87v89txru2.fsf@tromey.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -1.7 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.7 (--) Tom Tromey wrote: > I think runtest prints too much information to stdout by default. > > Consider the appended output. This comes from running a single gdb > test. > > A little bit of this is gdb-specific (the "HIP" override line). But > most of it is not. > What is "HIP"? I have to warn you, again, that monkey-patching the framework like that is not supported and is likely to break in a future release, probably around the time that native parallel testing is introduced but possibly before then. (You will, of course, be able to define a procedure named "default_target_compile" in the tool-init context but it may not be the "default_target_compile" that the framework calls anymore.) Also, default_target_compile is slated to be entirely rewritten before the next release, so whatever matching you are using may also break. Further, default_target_compile is not intended as a customization point, but rather as a default for targets that do not have their own ${target}_compile procedures defined in the site configuration. The documented API call is target_compile, which only calls default_target_compile if ${target}_compile does not exist. Monkeypatching default_target_compile will break at any site that actually has target-specific compile procedures. There are plans to provide for extensibility; current Git master has most of a spec strings module in lib/specs.exp that is planned to be used as part of a table-driven new default_target_compile; an API for testsuites to insert additional values will be provided. The tentative plan is to add categories to the layers mechanism: site, local, builtin, and fallback. The "site" category is reserved for site-specific configuration and, since the site configuration files can determine what testsuite is about to run, is the highest priority. The "local" category allows testsuites to override the standard behavior. The "builtin" category will contain the information currently hardwired into default_target_compile. Lastly, the "fallback" category exists explicitly to allow testsuites to extend DejaGnu as GDB did with Go and Rust, but without causing upstream changes to mysteriously vanish when testing GDB. > Most output is not really useful. In fact, pretty much the only lines I > might normally be interested in are the "Running [...]" line and the > result summary, so that if I "make -j8 check" I can easily see which > tests are being run. (FWIW for -j8 even the summary isn't interesting > because gdb's Makefile has to collate the results at the end anyway.) > The future native parallel testing support will take care of that. As long as the GDB Makefiles do not pass the relevant option (probably also -j) to runtest, the parallel testing infrastructure will not be used. > So, I think runtest should print a lot less text by default. Emitting > it to the .log file is sufficient (although some could even be trimmed > from there). > The largish block makes finding a previous run easier when rolling through terminal scrollback. It visually stands out from the "Running ..." lines very eye-catchingly. > If you don't like that, though, how about a command line option to make > it be more quiet? > The question here is what should such an option actually do? Which messages should be suppressed? > prentzel. runtest gdb.cp/local-static.exp > WARNING: Couldn't find the global config file. > Using /home/tromey/gdb/build/gdb/testsuite/../../../binutils-gdb/gdb/testsuite/lib/gdb.exp as tool init file. > NOTE: Dejagnu's default_target_compile is missing support for HIP, using local override > Test run by tromey on Wed Nov 22 16:53:00 2023 > Native configuration is x86_64-pc-linux-gnu > > === gdb tests === > > Schedule of variations: > unix > > Running target unix > Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target. > Using /usr/share/dejagnu/config/unix.exp as generic interface file for target. > Using /home/tromey/gdb/build/gdb/testsuite/../../../binutils-gdb/gdb/testsuite/config/unix.exp as tool-and-target-specific interface file. > Running /home/tromey/gdb/build/gdb/testsuite/../../../binutils-gdb/gdb/testsuite/gdb.cp/local-static.exp ... > > === gdb Summary === > > # of expected passes 407 > /home/tromey/gdb/build/gdb/gdb version 15.0.50.20231121-git -nw -nx -q -iex "set height 0" -iex "set width 0" -data-directory /home/tromey/gdb/build/gdb/data-directory > DejaGnu is specifically intended to support complex testing environments, so please consider a user with multiple test targets, of different architectures, possibly connected using different interfaces, all managed from one workstation. That said, perhaps some different defaults may be appropriate in the narrow case of running one specific test script, locally. While writing this, I had an idea for a possible new feature: a "oneshot" testing mode, perhaps available with a --oneshot option, that will set up the testsuite and run exactly one script, with minimal output, perhaps as little as the summary lines and ${tool}_version output. Implementing this right now would require extensive changes to runtest, so it will sit on my TODO list until I find an elegant way to do it, but the goal would be something like: 8<------ prentzel. runtest --oneshot gdb.cp/local-static.exp # of expected passes 407 /home/tromey/gdb/build/gdb/gdb version 15.0.50.20231121-git -nw -nx -q -iex "set height 0" -iex "set width 0" -data-directory /home/tromey/gdb/build/gdb/data-directory 8<------ The intended use of --oneshot is for rapid testing while developing a feature or chasing a bug, so logs would probably also be different in oneshot mode but this requires more thought. I presume that your site.exp is setting tool to "gdb"? -- Jacob