Subject: sed conditional branching

From: Erik Popp <[email-masked]>
Date: Sat, 14 Mar 2015 19:47:59 +0000 (UTC)

A couple of days ago, I sent a bug report to [email-masked], but it was late enough at night that I did a bad job of explaining what was wrong.  I also forgot to include the word "sed" in the subject line.

Here's take 2:

I'm using sed to filter a log, before passing it to the next part of a shell script

It appears that sed's "t" command doesn't work properly under some situations

The log file's format is almost identical to JSON:

{
property: "value",
property2: "value",
...
lastproperty: "value"
}

{
property_of_next_entry: "value",
...

The sed script does 2 things:

Block any invalid entries

Transform the log's data into a format that's easy for a shell script to process:
property_name,value

sed options:
sed -nrf log-filter.sed

sed version:
GNU sed 4.2.2

uname -a
Linux yum 3.13.0-40-generic #69-Ubuntu SMP Thu Nov 13 17:56:26 UTC 2014 i686 athlon i686 GNU/Linux

There are 4 commands that are relevant:

Trim white space from both sides

s/^[ ]*|[ ]*$//g

If the line is a curly brace, jump to success handler

/^[{}]$/ b success

If the line is in the correct format for a JSON property: "value" pair, rearrange the property and value into a format that's easy for a shell script to process

s/^([A-Za-z_\-]+):[ ]*"([A-Za-z0-9 _\-\.\:\/]*)",?$/\1,\2/

If the above substitution succeeds, jump to success handler

t success

The script is structured so that it should print nothing unless the script branches to a success handler.  After the above-mentioned commands, there's a "d" command.  The success handler is below that.

The first 3 above-mentioned components work properly individually.  Of particular relevance is that if I comment out the white space remover, and also the curly brace handler, so that the first command that runs is the JSON property verifier, the "t" command works properly.  Input that matches it causes the "t" command to jump to the success handler, while input that doesn't causes it to fall through to the "d" command, as designed.

If I run these 4 commands in sequence, the "t" command always succeeds, even if the substitution right above it fails.  It's as if on the back end of the "t" command, sed checks if any substitution in the whole script has succeed on this line, and branches if any did.  Since the white space remover line "succeeds" pretty much all the time, this would explain the symptom.