Subject: Re: bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case

From: Jacob Bachmeyer <[email-masked]>
Date: Tue, 30 Jun 2020 18:49:11 -0500

Pedro Alves wrote:
> On 6/30/20 5:40 AM, Jacob Bachmeyer wrote:
>   
>> Pedro Alves wrote:
>>     
>>> On 6/26/20 11:37 PM, Jacob Bachmeyer wrote:
>>>  
>>>       
>>>> Pedro Alves wrote:
>>>>    
>>>>         
>>>>> And again, people run the GDB testsuite in parallel mode, so your
>>>>> assumption that the testsuite would stop immediately anyway doesn't
>>>>> hold.  One of the parallel runtest invocations bombs
>>>>> out, while the others keep running, so by the time you get
>>>>> back the prompt, other testcases already executed, and you don't
>>>>> see the ERROR in the current terminal anymore.
>>>>>       
>>>>>           
>>>> This is a fundamental problem with slicing up the testsuite external to DejaGnu and running multiple instances of runtest -- output that is supposed to be "last at the end of the run" can be hidden away by other ongoing subset runs.
>>>>     
>>>>         
>>> That's like arguing against parallelization, and saying that everyone
>>> should run the testsuite in serial mode, and be OK with it taking 2 hours
>>> instead of 30 mins to complete a run...
>>>
>>> The parallel mode aggregates the summaries from the different runtest
>>> invocations as a single summary at the end,
>>>
>>> https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=contrib/dg-extract-results.py;h=30aa68771d491451f72d6dbd18f6d5ac339451b5;hb=HEAD
>>>
>>> so I don't see what problem here is unsolvable.  Again, the issue is that Tcl
>>> ERRORs aren't tallied up on the DejaGnu summary, so the aggregated summary
>>> doesn't contain those either.
>>>   
>>>       
>> The current version of the fix for this issue (commit 61dc0cafad8845b3c668940ed2e574bd503d410f on temporary branch PR41918) generates separator lines containing at least 10 hyphens around each error when repeating the errors after the summary.  The leading separator line currently begins with "ERROR: " due to limitations of the internal interfaces, but plans are to eventually fix that and collapse the separator lines between adjacent error reports if multiple Tcl errors occur in a run.  Each error report begins with "ERROR: in testcase " and ends with a Tcl backtrace and a line of at least 10 hyphens.  Is this something the GDB results aggregation tool can be extended to handle?
>>     
>
> Your guess is as good as mine, since I never touched that code.
>   

After briefly looking through that script, I expect that it will not be 
particularly difficult, although I also note that the script counts the 
error messages generated by DejaGnu's ::unknown procedure, so 
lib/framework.exp:unknown cannot be withdrawn as useless until 1.7.

-- Jacob