GNU bug report logs - #72536
[PATCH] tap: new `--merge-prefix' option to prefix stderr before merging

Previous Next

Full log

Message #62 received at 72536 <at> debbugs.gnu.org (full text, mbox):

From: Richard Hansen <rhansen <at> rhansen.org>
To: Karl Berry <karl <at> freefriends.org>, Jacob Bachmeyer <jcb62281 <at> gmail.com>
Cc: 72536 <at> debbugs.gnu.org
Subject: Re: [PATCH v3] tap: new `--stderr-prefix' option to prefix test
 script's stderr
Date: Fri, 21 Feb 2025 19:20:40 -0500

[Message part 1 (text/plain, inline)]

On 2/20/25 18:23, Karl Berry wrote:
>> (a) not piping stderr through the prefixing function when the prefix 
>> is the empty string, and (b) avoiding stdout buffering when stderr 
>> is prefixed.
> 
> Those sound desirable in any case. Especially the first. 
> It's highly desirable for existing behavior to remain the same, as much 
> as possible.

OK, will do.

> But what about the new test you wrote?  Isn't any test that a) outputs 
> to both stdout and stderr and b) uses the new merge prefix going to be 
> subject to the race condition?

To be clear, the only negative effect of the race condition is that 
stderr lines might appear out of order relative to stdout lines, even 
when --merge is passed.  (All stderr lines are still processed in order 
relative to other stderr lines, and the same goes for the stdout lines 
relative to other stdout lines.)  That reordering shouldn't break any 
user's TAP tests because they shouldn't be outputting TAP commands to 
stderr anyway (and certainly not if stderr lines are prefixed).

The reordering might break some of the tests in Automake that test the 
TAP driver itself, but only if the tests care about line ordering 
between stderr and stdout.  The new test I added doesn't care, so it's 
fine.  The t/tap-merge-stdout-stderr.sh test does care about order, 
which is why it broke.

I think the only major downside to the reordering is that it is harder 
for users to understand logs when trying to figure out a bug in their 
own code (like it is with --no-merge).  Users can remove --stderr-prefix 
(once I make the changes proposed in my previous email) or they can run 
the test program directly (not via the TAP driver) to get a more 
accurate understanding of output timing.

The order of processing (and logging) should not differ from the order 
of the test program's write(2) calls by much.  Out-of-order processing 
only happens if the test program writes to stdout then stderr but the 
kernel wakes up the TAP driver's thread/process that is reading stderr 
before it wakes up the TAP driver's thread/process that is reading 
stdout.  (Or vice-versa.)

As I mentioned before, the out-of-order processing can also happen with 
select/poll (the kernel might not wake up the reading thread/process 
until after the test program writes to both stdout and stderr), but 
select/poll is considerably less expensive than context switches (IIUC) 
so the odds of in-order processing are better.

>> Users that require stronger guarantees can wrap their test program 
>> in a custom helper utility rather than rely on Automake's TAP driver 
>> to do the prefixing and merging.
> 
> That feels like asking an awful lot of the users.

I'm hoping most users won't care much about minor reordering; that 
having --stderr-prefix is better than not having it even if it has some 
reordering issues.  But maybe not---maybe it would be better to omit the 
feature if it doesn't quite meet users' expectations.

I can make it an error to specify both --stderr-prefix and --merge so 
that users don't expect in-order processing when they pass 
--stderr-prefix.  Later, if further testing shows that --stderr-prefix 
and --merge together isn't too bad, we can remove the restriction.

> If we have to resort 
> to that, it sounds like something we should show an example of. And 
> maybe use for the new test, if that's possible.

I've been frustrated enough by shell's lack of select/poll that I'm 
tempted to create a new utility to help with cases like this.  For example:

    fdmux --mode=line "1:[stdout] %l" "2:[stderr] %l" -- foo arg

would run 'foo arg1', select/poll its FDs 1 and 2, prefix stdout lines 
with "[stdout] " and stderr lines with "[stderr] ", and write the 
prefixed lines to stdout.

If a utility like that was widely available then devs could wrap their 
test programs in it and not bother with the TAP driver's --stderr-prefix 
and --merge options.  The problem is that there's a near 0% chance such 
a utility would be added to POSIX, which means it wouldn't be widely 
available.  A pure shell+awk script like Automake's TAP driver can run 
just about everywhere.

-Richard

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.