GNU bug report logs - #6554
[coreutils] Additional suffix for split

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 6554 in the body.
You can then email your comments to 6554 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Peng Yu <pengyu.ut <at> gmail.com>
Cc: bug-coreutils <bug-coreutils <at> gnu.org>, coreutils <at> gnu.org
Subject: Re: [coreutils] Additional suffix for split
Date: Fri, 02 Jul 2010 13:51:44 -0600

[Message part 1 (text/plain, inline)]

[adding bug-coreutils, to create a bug id to track this by]

On 06/15/2010 09:23 PM, Peng Yu wrote:
> I need to add an additional common suffix to the files splited by
> split. Right now, I have to use mv to do so. But I feel it is
> convenient to have an option to add the suffix. Is this feature going
> to be considered to be added in the future?

Hmm, considering that we recently taught mktemp how to honor suffixes,
it does indeed sound like this might be a useful feature addition.  It's
probably not going to be my highest priority to write such a patch, but
I'll gladly review any patch written by someone else.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #8 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Peng Yu <pengyu.ut <at> gmail.com>
To: Eric Blake <eblake <at> redhat.com>
Cc: 6554 <at> debbugs.gnu.org, coreutils <at> gnu.org
Subject: Re: bug#6554: [coreutils] Additional suffix for split
Date: Sun, 13 Feb 2011 13:28:48 -0600

On Fri, Jul 2, 2010 at 2:51 PM, Eric Blake <eblake <at> redhat.com> wrote:
> [adding bug-coreutils, to create a bug id to track this by]
>
> On 06/15/2010 09:23 PM, Peng Yu wrote:
>> I need to add an additional common suffix to the files splited by
>> split. Right now, I have to use mv to do so. But I feel it is
>> convenient to have an option to add the suffix. Is this feature going
>> to be considered to be added in the future?
>
> Hmm, considering that we recently taught mktemp how to honor suffixes,
> it does indeed sound like this might be a useful feature addition.  It's
> probably not going to be my highest priority to write such a patch, but
> I'll gladly review any patch written by someone else.

I'm wondering if this feature has been added to 'split'.
-- 
Regards,
Peng

Message #11 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Jérémy Compostella <jeremy.compostella <at> gmail.com>
To: Eric Blake <eblake <at> redhat.com>, Peng Yu <pengyu.ut <at> gmail.com>
Cc: 6554 <at> debbugs.gnu.org, coreutils <at> gnu.org
Subject: [PATCH] split: Additional suffix for split (bug#6554)
Date: Fri, 27 Jan 2012 18:28:39 +0100

[Message part 1 (text/plain, inline)]

Eric, Peng, others,

> On 06/15/2010 09:23 PM, Peng Yu wrote:
> > I need to add an additional common suffix to the files splited by
> > split. Right now, I have to use mv to do so. But I feel it is
> > convenient to have an option to add the suffix. Is this feature going
> > to be considered to be added in the future?
> 
> Hmm, considering that we recently taught mktemp how to honor suffixes,
> it does indeed sound like this might be a useful feature addition.  It's
> probably not going to be my highest priority to write such a patch, but
> I'll gladly review any patch written by someone else.

I'm interested in this feature too so I wrote the attached patch which
implements it by adding the "--suffix=SUFF" option.

As this is my first contribution to this project, I'm not conformable
with coreutils coding standard or documentation needs, so feel free to
comment the submitted patch. I will take into account whatever you ask
for.

Cheers,

Jérémy
---

[0001-split-Additional-suffix-for-split-bug-6554.patch (text/x-diff, inline)]

From cc808cd440c0f578a4f2f36345f8f1bc48ebe0ae Mon Sep 17 00:00:00 2001
From: Jeremy Compostella <jeremy.compostella <at> gmail.com>
Date: Fri, 27 Jan 2012 18:14:34 +0100
Subject: [PATCH] split: Additional suffix for split (bug#6554)

Add support to an additionnal suffix with the new '--suffix" option.

Signed-off-by: Jeremy Compostella <jeremy.compostella <at> gmail.com>
---
 src/split.c |   24 ++++++++++++++++++++++--
 1 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/src/split.c b/src/split.c
index 5fbce0e..c86a014 100644
--- a/src/split.c
+++ b/src/split.c
@@ -80,6 +80,13 @@ static size_t suffix_length;
 /* Alphabet of characters to use in suffix.  */
 static char const *suffix_alphabet = "abcdefghijklmnopqrstuvwxyz";
 
+/* Length of fixed suffix. */
+static size_t fixed_suffix_len = 0;
+
+/* Fixed suffix to append to OUTFILE right after the dynamic
+   suffix.  */
+static char const *fixed_suffix;
+
 /* Name of input file.  May be "-".  */
 static char *infile;
 
@@ -110,7 +117,8 @@ enum
 {
   VERBOSE_OPTION = CHAR_MAX + 1,
   FILTER_OPTION,
-  IO_BLKSIZE_OPTION
+  IO_BLKSIZE_OPTION,
+  SUFFIX_OPTION
 };
 
 static struct option const longopts[] =
@@ -122,6 +130,7 @@ static struct option const longopts[] =
   {"elide-empty-files", no_argument, NULL, 'e'},
   {"unbuffered", no_argument, NULL, 'u'},
   {"suffix-length", required_argument, NULL, 'a'},
+  {"suffix", required_argument, NULL, SUFFIX_OPTION},
   {"numeric-suffixes", no_argument, NULL, 'd'},
   {"filter", required_argument, NULL, FILTER_OPTION},
   {"verbose", no_argument, NULL, VERBOSE_OPTION},
@@ -193,6 +202,8 @@ Mandatory arguments to long options are mandatory for short options too.\n\
 "), stdout);
       fprintf (stdout, _("\
   -a, --suffix-length=N   use suffixes of length N (default %d)\n\
+      --suffix=SUFF       append SUFF to each output filename. SUFF must\n\
+                          not contain slash\n\
   -b, --bytes=SIZE        put SIZE bytes per output file\n\
   -C, --line-bytes=SIZE   put at most SIZE bytes of lines per output file\n\
   -d, --numeric-suffixes  use numeric suffixes instead of alphabetic\n\
@@ -237,13 +248,15 @@ next_file_name (void)
       /* Allocate and initialize the first file name.  */
 
       size_t outbase_length = strlen (outbase);
-      size_t outfile_length = outbase_length + suffix_length;
+      size_t outfile_length = outbase_length + suffix_length + fixed_suffix_len;
       if (outfile_length + 1 < outbase_length)
         xalloc_die ();
       outfile = xmalloc (outfile_length + 1);
       outfile_mid = outfile + outbase_length;
       memcpy (outfile, outbase, outbase_length);
       memset (outfile_mid, suffix_alphabet[0], suffix_length);
+      if (fixed_suffix_len)
+        memcpy (outfile_mid + suffix_length, fixed_suffix, fixed_suffix_len);
       outfile[outfile_length] = 0;
       sufindex = xcalloc (suffix_length, sizeof *sufindex);
 
@@ -1036,6 +1049,13 @@ main (int argc, char **argv)
           }
           break;
 
+        case SUFFIX_OPTION:
+          {
+            fixed_suffix = optarg;
+            fixed_suffix_len = strlen (fixed_suffix);
+          }
+          break;
+
         case 'b':
           if (split_type != type_undef)
             FAIL_ONLY_ONE_WAY ();
-- 
1.7.2.5

Message #14 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jérémy Compostella
 <jeremy.compostella <at> gmail.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sat, 28 Jan 2012 14:48:54 +0000

On 01/27/2012 05:28 PM, Jérémy Compostella wrote:
> Eric, Peng, others,
> 
>> On 06/15/2010 09:23 PM, Peng Yu wrote:
>>> I need to add an additional common suffix to the files splited by
>>> split. Right now, I have to use mv to do so. But I feel it is
>>> convenient to have an option to add the suffix. Is this feature going
>>> to be considered to be added in the future?
>>
>> Hmm, considering that we recently taught mktemp how to honor suffixes,
>> it does indeed sound like this might be a useful feature addition.  It's
>> probably not going to be my highest priority to write such a patch, but
>> I'll gladly review any patch written by someone else.
> 
> I'm interested in this feature too so I wrote the attached patch which
> implements it by adding the "--suffix=SUFF" option.
> 
> As this is my first contribution to this project, I'm not conformable
> with coreutils coding standard or documentation needs, so feel free to
> comment the submitted patch. I will take into account whatever you ask
> for.
> 
> Cheers,
> 
> Jérémy
> ---

Thanks a lot for working on this.
Could you give a real world example where
you find this useful, just for the record.

I should note that --suffix is incompatible
with a variable length generated suffix.
I.E. one that would allow for arbitrary sized input:
http://lists.gnu.org/archive/html/bug-coreutils/2009-09/msg00220.html
Though I guess that functionality could still be
provided if required, by specifying --suffix-length=auto
and having that mutually exclusive with --suffix.

As for the patch, it seems to work :)

t$ seq 10 > file.txt
t$ ../split -n10 --suffix=.txt file.txt file.
t$ l
-rw-rw-r--. 1 padraig 21 Jan 28 14:06 file.txt
-rw-rw-r--. 1 padraig  3 Jan 28 14:06 file.aj.txt
-rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ai.txt
-rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ah.txt
-rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ag.txt
-rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.af.txt
-rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ae.txt
-rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ad.txt
-rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ac.txt
-rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ab.txt
-rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.aa.txt

We'd need some corresponding documentation in doc/coreutils.texi,
and an entry in NEWS.

Also even thought the patch is small, you'd need to start
the copyright assignment process for a new parameter to `split`.

cheers,
Pádraig.

Message #17 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Jérémy Compostella <jeremy.compostella <at> gmail.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sun, 29 Jan 2012 00:57:59 +0100

[Message part 1 (text/plain, inline)]

 Pádraig Brady wrotes:
> Thanks a lot for working on this.

> Could you give a real world example where
> you find this useful, just for the record.
It's useful in some cases where the chunk files type is known. As for
example, when I split a file which is a concatenation of several bitmap
files of the same size. Automatically append the appropriate suffix
"type" is time saving and comfortable. Another example is when the split
does not alter the file "type" as in the example you provided below,
input and output files are "txt" files. Having the appropriate suffix
automatically appended to the chunk file names make these files easily
available to applications which care about the file suffix.

> I should note that --suffix is incompatible
> with a variable length generated suffix.
> I.E. one that would allow for arbitrary sized input:
> http://lists.gnu.org/archive/html/bug-coreutils/2009-09/msg00220.html
> Though I guess that functionality could still be
> provided if required, by specifying --suffix-length=auto
> and having that mutually exclusive with --suffix.
First, after having carefully read the thread you are talking about I
did not find any implementation of this in the current coreutils
repository. Anyway, this feature is interesting and I would be glad to
implement it if needs too but in another commit.

Second, I don't get your point. Why it should not be possible to append
a fixed length suffix to output files with a variable length dynamic
suffix ? I do not figure out why it's incompatible. What do I
misunderstood ?

For example, the following hypothetical split command call looks
perfectly compatible:
$ split -n10 --suffix-length=auto --suffix=.txt file.txt file.
$ ls
  file.aa.txt
  file.ab.txt
  file.ac.txt
  file.ad.txt
  file.ae.txt
  file.af.txt
  file.ag.txt
  file.ah.txt
  file.ai.txt
  file.aj.txt
  [...]
  file.yv.txt
  file.yw.txt
  file.yx.txt
  file.yy.txt
  file.yz.txt
  file.zaaa.txt
  file.zaab.txt
  file.zaac.txt
  file.zaad.txt
  file.zaae.txt
  file.zaaf.txt

> As for the patch, it seems to work :)
> 
> t$ seq 10 > file.txt
> t$ ../split -n10 --suffix=.txt file.txt file.
> t$ l
> -rw-rw-r--. 1 padraig 21 Jan 28 14:06 file.txt
> -rw-rw-r--. 1 padraig  3 Jan 28 14:06 file.aj.txt
> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ai.txt
> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ah.txt
> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ag.txt
> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.af.txt
> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ae.txt
> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ad.txt
> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ac.txt
> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ab.txt
> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.aa.txt

> We'd need some corresponding documentation in doc/coreutils.texi,
> and an entry in NEWS.
I added the corresponding documentation in both of files. I've done my
best but feel free to comment my additions.

> Also even thought the patch is small, you'd need to start
> the copyright assignment process for a new parameter to `split`.
I filled out and sent my coreutils assignment form.

Cheers,

Jérémy
---

[0001-split-Additional-suffix-for-split-bug-6554.patch (text/x-diff, inline)]

From 587e15898a947cf22539bf7f782b7963284dc1e4 Mon Sep 17 00:00:00 2001
From: Jeremy Compostella <jeremy.compostella <at> gmail.com>
Date: Fri, 27 Jan 2012 18:14:34 +0100
Subject: [PATCH] split: Additional suffix for split (bug#6554)

Add support to an additionnal suffix with the new `--suffix=SUFF'
option.  SUFF is appended to each output filename right after the
dynamic suffix.

Signed-off-by: Jeremy Compostella <jeremy.compostella <at> gmail.com>
---
 NEWS               |    5 +++++
 doc/coreutils.texi |    6 ++++++
 src/split.c        |   24 ++++++++++++++++++++++--
 3 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index 2b0926f..398405d 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,11 @@ GNU coreutils NEWS                                    -*- outline -*-
 
 * Noteworthy changes in release ?.? (????-??-??) [?]
 
+** New features
+
+  split now accepts the new --suffix=SUFF option. When SUFF is
+  specified, SUFF is append to each output filenames right after the
+  dynamic suffix.
 
 * Noteworthy changes in release 8.15 (2012-01-06) [stable]
 
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 0d3b739..9ccadf5 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -3083,6 +3083,12 @@ and so can be a pipe for example.
 @opindex --suffix-length
 Use suffixes of length @var{length}.  The default @var{length} is 2.
 
+@itemx --suffix=@var{suff}
+@opindex -a
+@opindex --suffix-length
+Append @var{suff} to each output filename right after the dynamic
+suffix. @var{suff} must not contain slash.
+
 @item -d
 @itemx --numeric-suffixes
 @opindex -d
diff --git a/src/split.c b/src/split.c
index 5fbce0e..fa7d30a 100644
--- a/src/split.c
+++ b/src/split.c
@@ -80,6 +80,13 @@ static size_t suffix_length;
 /* Alphabet of characters to use in suffix.  */
 static char const *suffix_alphabet = "abcdefghijklmnopqrstuvwxyz";
 
+/* Length of fixed suffix. */
+static size_t fixed_suffix_len = 0;
+
+/* Fixed suffix to append to OUTFILE right after the dynamic
+   suffix.  */
+static char const *fixed_suffix;
+
 /* Name of input file.  May be "-".  */
 static char *infile;
 
@@ -110,7 +117,8 @@ enum
 {
   VERBOSE_OPTION = CHAR_MAX + 1,
   FILTER_OPTION,
-  IO_BLKSIZE_OPTION
+  IO_BLKSIZE_OPTION,
+  SUFFIX_OPTION
 };
 
 static struct option const longopts[] =
@@ -122,6 +130,7 @@ static struct option const longopts[] =
   {"elide-empty-files", no_argument, NULL, 'e'},
   {"unbuffered", no_argument, NULL, 'u'},
   {"suffix-length", required_argument, NULL, 'a'},
+  {"suffix", required_argument, NULL, SUFFIX_OPTION},
   {"numeric-suffixes", no_argument, NULL, 'd'},
   {"filter", required_argument, NULL, FILTER_OPTION},
   {"verbose", no_argument, NULL, VERBOSE_OPTION},
@@ -193,6 +202,8 @@ Mandatory arguments to long options are mandatory for short options too.\n\
 "), stdout);
       fprintf (stdout, _("\
   -a, --suffix-length=N   use suffixes of length N (default %d)\n\
+      --suffix=SUFF       append SUFF to each output filename right after\n\
+                          the dynamic suffix. SUFF must not contain slash\n\
   -b, --bytes=SIZE        put SIZE bytes per output file\n\
   -C, --line-bytes=SIZE   put at most SIZE bytes of lines per output file\n\
   -d, --numeric-suffixes  use numeric suffixes instead of alphabetic\n\
@@ -237,13 +248,15 @@ next_file_name (void)
       /* Allocate and initialize the first file name.  */
 
       size_t outbase_length = strlen (outbase);
-      size_t outfile_length = outbase_length + suffix_length;
+      size_t outfile_length = outbase_length + suffix_length + fixed_suffix_len;
       if (outfile_length + 1 < outbase_length)
         xalloc_die ();
       outfile = xmalloc (outfile_length + 1);
       outfile_mid = outfile + outbase_length;
       memcpy (outfile, outbase, outbase_length);
       memset (outfile_mid, suffix_alphabet[0], suffix_length);
+      if (fixed_suffix_len)
+        memcpy (outfile_mid + suffix_length, fixed_suffix, fixed_suffix_len);
       outfile[outfile_length] = 0;
       sufindex = xcalloc (suffix_length, sizeof *sufindex);
 
@@ -1036,6 +1049,13 @@ main (int argc, char **argv)
           }
           break;
 
+        case SUFFIX_OPTION:
+          {
+            fixed_suffix = optarg;
+            fixed_suffix_len = strlen (fixed_suffix);
+          }
+          break;
+
         case 'b':
           if (split_type != type_undef)
             FAIL_ONLY_ONE_WAY ();
-- 
1.7.2.5

Message #20 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jérémy Compostella
 <jeremy.compostella <at> gmail.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sun, 29 Jan 2012 00:14:40 +0000

On 01/28/2012 11:57 PM, Jérémy Compostella wrote:
>  Pádraig Brady wrotes:
>> Thanks a lot for working on this.
> 
>> Could you give a real world example where
>> you find this useful, just for the record.
> It's useful in some cases where the chunk files type is known. As for
> example, when I split a file which is a concatenation of several bitmap
> files of the same size. Automatically append the appropriate suffix
> "type" is time saving and comfortable. Another example is when the split
> does not alter the file "type" as in the example you provided below,
> input and output files are "txt" files. Having the appropriate suffix
> automatically appended to the chunk file names make these files easily
> available to applications which care about the file suffix.

Right, thanks for the clarification.

>> I should note that --suffix is incompatible
>> with a variable length generated suffix.
>> I.E. one that would allow for arbitrary sized input:
>> http://lists.gnu.org/archive/html/bug-coreutils/2009-09/msg00220.html
>> Though I guess that functionality could still be
>> provided if required, by specifying --suffix-length=auto
>> and having that mutually exclusive with --suffix.
> First, after having carefully read the thread you are talking about I
> did not find any implementation of this in the current coreutils
> repository. Anyway, this feature is interesting and I would be glad to
> implement it if needs too but in another commit.

Yes, this feature was removed from coreutils.
Anyway something to worry about later.

> Second, I don't get your point. Why it should not be possible to append
> a fixed length suffix to output files with a variable length dynamic
> suffix ? I do not figure out why it's incompatible. What do I
> misunderstood ?

To allow processing the split files in order.
If the supplied suffix is overlapping part of a generated suffix,
then there is no way to subsequently sort the parts.

> For example, the following hypothetical split command call looks
> perfectly compatible:
> $ split -b10 --suffix-length=auto --suffix=.txt file.txt file.
> $ ls
>   file.aa.txt
>   [...]
>   file.yz.txt
>   file.zaaa.txt

This example is usually OK as '.' will sort before letters.
If one has a --suffix without a '.' though, then we've problems.
This is just an edge case worth noting.

>> As for the patch, it seems to work :)
>>
>> t$ seq 10 > file.txt
>> t$ ../split -n10 --suffix=.txt file.txt file.
>> t$ l
>> -rw-rw-r--. 1 padraig 21 Jan 28 14:06 file.txt
>> -rw-rw-r--. 1 padraig  3 Jan 28 14:06 file.aj.txt
>> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ai.txt
>> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ah.txt
>> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ag.txt
>> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.af.txt
>> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ae.txt
>> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ad.txt
>> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ac.txt
>> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ab.txt
>> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.aa.txt
> 
>> We'd need some corresponding documentation in doc/coreutils.texi,
>> and an entry in NEWS.
> I added the corresponding documentation in both of files. I've done my
> best but feel free to comment my additions.
> 
>> Also even thought the patch is small, you'd need to start
>> the copyright assignment process for a new parameter to `split`.
> I filled out and sent my coreutils assignment form.

Excellent thanks.

Pádraig.

Message #23 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Jérémy Compostella <jeremy.compostella <at> gmail.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sun, 29 Jan 2012 13:26:28 +0100

On 01/28/2012 00:00 AM, Pádraig Brady wrote:
> On 01/28/2012 11:57 PM, Jérémy Compostella wrote:
[...]
> >> I should note that --suffix is incompatible
> >> with a variable length generated suffix.
> >> I.E. one that would allow for arbitrary sized input:
> >> http://lists.gnu.org/archive/html/bug-coreutils/2009-09/msg00220.html
> >> Though I guess that functionality could still be
> >> provided if required, by specifying --suffix-length=auto
> >> and having that mutually exclusive with --suffix.
> > First, after having carefully read the thread you are talking about I
> > did not find any implementation of this in the current coreutils
> > repository. Anyway, this feature is interesting and I would be glad to
> > implement it if needs too but in another commit.
> 
> Yes, this feature was removed from coreutils.
> Anyway something to worry about later.
> 
> > Second, I don't get your point. Why it should not be possible to append
> > a fixed length suffix to output files with a variable length dynamic
> > suffix ? I do not figure out why it's incompatible. What do I
> > misunderstood ?
> 
> To allow processing the split files in order.
> If the supplied suffix is overlapping part of a generated suffix,
> then there is no way to subsequently sort the parts.
> 
> > For example, the following hypothetical split command call looks
> > perfectly compatible:
> > $ split -b10 --suffix-length=auto --suffix=.txt file.txt file.
> > $ ls
> >   file.aa.txt
> >   [...]
> >   file.yz.txt
> >   file.zaaa.txt
> 
> This example is usually OK as '.' will sort before letters.
> If one has a --suffix without a '.' though, then we've problems.
> This is just an edge case worth noting.
Thanks, I get it now. IMHO these options should not be mutual exclusives
but the documentation should mention the variable length dynamic suffix
algorithm and warn about this behavior. In that case the user will be
able to have the precise behavior he wants without a systematic
constraint.

> >> As for the patch, it seems to work :)
> >>
> >> t$ seq 10 > file.txt
> >> t$ ../split -n10 --suffix=.txt file.txt file.
> >> t$ l
> >> -rw-rw-r--. 1 padraig 21 Jan 28 14:06 file.txt
> >> -rw-rw-r--. 1 padraig  3 Jan 28 14:06 file.aj.txt
> >> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ai.txt
> >> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ah.txt
> >> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ag.txt
> >> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.af.txt
> >> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ae.txt
> >> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ad.txt
> >> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ac.txt
> >> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.ab.txt
> >> -rw-rw-r--. 1 padraig  2 Jan 28 14:06 file.aa.txt
> > 
> >> We'd need some corresponding documentation in doc/coreutils.texi,
> >> and an entry in NEWS.
> > I added the corresponding documentation in both of files. I've done my
> > best but feel free to comment my additions.
> > 
> >> Also even thought the patch is small, you'd need to start
> >> the copyright assignment process for a new parameter to `split`.
> > I filled out and sent my coreutils assignment form.
> 
> Excellent thanks.

Jeremy

Message #26 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Jérémy Compostella <jeremy.compostella <at> gmail.com>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Mon, 06 Feb 2012 14:55:24 +0100

[Message part 1 (text/plain, inline)]

Pádraig, all,

I took into account general comments on my commits. I attached the
improved patch for feature. Improvements are:
- Add by file description,
- I put back accents on my name (Jérémy instead of Jeremy),
- I referenced from who this feature was requested.

Cheers,

Jérémy
---

[0001-split-Additional-suffix-for-split-bug-6554.patch (text/x-diff, inline)]

From 81e12245be60ef4e9b06ab18fa9e92bce81ea63c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Compostella?= <jeremy.compostella <at> gmail.com>
Date: Fri, 27 Jan 2012 18:14:34 +0100
Subject: [PATCH] split: Additional suffix for split (bug#6554)

Add support to an additionnal suffix with the new `--suffix=SUFF'
option.  SUFF is appended to each output filename right after the
dynamic suffix.

* src/split.c (next_file_name): Append fixed_suffix to outfile.
(main): Handle --suffix new option.
* NEWS (New features): Mention it.
* doc/coreutils.texi (split invocation): Mention it.
* tests/split/suffix: New file. --suffix option tests.
* tests/Makefile.am (TESTS): Add it.
Requested by Peng Yu.
---
 NEWS               |    6 ++++++
 doc/coreutils.texi |    6 ++++++
 src/split.c        |   24 ++++++++++++++++++++++--
 tests/Makefile.am  |    1 +
 tests/split/suffix |   41 +++++++++++++++++++++++++++++++++++++++++
 5 files changed, 76 insertions(+), 2 deletions(-)
 create mode 100755 tests/split/suffix

diff --git a/NEWS b/NEWS
index 9eebbf6..fa056e1 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,12 @@ GNU coreutils NEWS                                    -*- outline -*-
 
 * Noteworthy changes in release ?.? (????-??-??) [?]
 
+** New features
+
+  split now accepts the new --suffix=SUFF option. When SUFF is
+  specified, SUFF is append to each output filenames right after the
+  dynamic suffix.
+
 ** Bug fixes
 
   mv now lets you move a symlink onto a same-inode destination file that
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 52838e7..db12dbb 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -3083,6 +3083,12 @@ and so can be a pipe for example.
 @opindex --suffix-length
 Use suffixes of length @var{length}.  The default @var{length} is 2.
 
+@itemx --suffix=@var{suff}
+@opindex -a
+@opindex --suffix-length
+Append @var{suff} to each output filename right after the dynamic
+suffix. @var{suff} must not contain slash.
+
 @item -d
 @itemx --numeric-suffixes
 @opindex -d
diff --git a/src/split.c b/src/split.c
index 1d0310c..af532fa 100644
--- a/src/split.c
+++ b/src/split.c
@@ -80,6 +80,13 @@ static size_t suffix_length;
 /* Alphabet of characters to use in suffix.  */
 static char const *suffix_alphabet = "abcdefghijklmnopqrstuvwxyz";
 
+/* Length of fixed suffix. */
+static size_t fixed_suffix_len = 0;
+
+/* Fixed suffix to append to OUTFILE right after the dynamic
+   suffix.  */
+static char const *fixed_suffix;
+
 /* Name of input file.  May be "-".  */
 static char *infile;
 
@@ -110,7 +117,8 @@ enum
 {
   VERBOSE_OPTION = CHAR_MAX + 1,
   FILTER_OPTION,
-  IO_BLKSIZE_OPTION
+  IO_BLKSIZE_OPTION,
+  SUFFIX_OPTION
 };
 
 static struct option const longopts[] =
@@ -122,6 +130,7 @@ static struct option const longopts[] =
   {"elide-empty-files", no_argument, NULL, 'e'},
   {"unbuffered", no_argument, NULL, 'u'},
   {"suffix-length", required_argument, NULL, 'a'},
+  {"suffix", required_argument, NULL, SUFFIX_OPTION},
   {"numeric-suffixes", no_argument, NULL, 'd'},
   {"filter", required_argument, NULL, FILTER_OPTION},
   {"verbose", no_argument, NULL, VERBOSE_OPTION},
@@ -193,6 +202,8 @@ Mandatory arguments to long options are mandatory for short options too.\n\
 "), stdout);
       fprintf (stdout, _("\
   -a, --suffix-length=N   use suffixes of length N (default %d)\n\
+      --suffix=SUFF       append SUFF to each output filename right after\n\
+                          the dynamic suffix. SUFF must not contain slash\n\
   -b, --bytes=SIZE        put SIZE bytes per output file\n\
   -C, --line-bytes=SIZE   put at most SIZE bytes of lines per output file\n\
   -d, --numeric-suffixes  use numeric suffixes instead of alphabetic\n\
@@ -237,13 +248,15 @@ next_file_name (void)
       /* Allocate and initialize the first file name.  */
 
       size_t outbase_length = strlen (outbase);
-      size_t outfile_length = outbase_length + suffix_length;
+      size_t outfile_length = outbase_length + suffix_length + fixed_suffix_len;
       if (outfile_length + 1 < outbase_length)
         xalloc_die ();
       outfile = xmalloc (outfile_length + 1);
       outfile_mid = outfile + outbase_length;
       memcpy (outfile, outbase, outbase_length);
       memset (outfile_mid, suffix_alphabet[0], suffix_length);
+      if (fixed_suffix_len)
+        memcpy (outfile_mid + suffix_length, fixed_suffix, fixed_suffix_len);
       outfile[outfile_length] = 0;
       sufindex = xcalloc (suffix_length, sizeof *sufindex);
 
@@ -1036,6 +1049,13 @@ main (int argc, char **argv)
           }
           break;
 
+        case SUFFIX_OPTION:
+          {
+            fixed_suffix = optarg;
+            fixed_suffix_len = strlen (fixed_suffix);
+          }
+          break;
+
         case 'b':
           if (split_type != type_undef)
             FAIL_ONLY_ONE_WAY ();
diff --git a/tests/Makefile.am b/tests/Makefile.am
index a94aaa2..28a7ef6 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -254,6 +254,7 @@ TESTS =						\
   misc/sort-NaN-infloop				\
   split/filter					\
   split/suffix-length				\
+  split/suffix					\
   split/b-chunk					\
   split/fail					\
   split/lines					\
diff --git a/tests/split/suffix b/tests/split/suffix
new file mode 100755
index 0000000..8282359
--- /dev/null
+++ b/tests/split/suffix
@@ -0,0 +1,41 @@
+#!/bin/sh
+# show that 'split --suffix=SUFF' works.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ split
+
+printf '1\n2\n3\n4\n5\n' > in || framework_failure_
+
+split --lines=2 --suffix=.txt in > out || fail=1
+cat <<\EOF > exp-1
+1
+2
+EOF
+cat <<\EOF > exp-2
+3
+4
+EOF
+cat <<\EOF > exp-3
+5
+EOF
+
+compare exp-1 xaa.txt || fail=1
+compare exp-2 xab.txt || fail=1
+compare exp-3 xac.txt || fail=1
+
+Exit $fail
-- 
1.7.2.5

Message #29 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Jérémy Compostella <jeremy.compostella <at> gmail.com>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sat, 18 Feb 2012 13:52:35 +0100

[Message part 1 (text/plain, inline)]

All,

I'm continuing with this feature implementation. Getting advantage on my
newly acquired experience on this project I made some changes in my
implementation.

Now I have a concern about the option name "--suffix". I think the
--suffix-length, --numeric-suffixes and the new --suffix are confusing
for the user since the two first are related to the count suffix and the
last one is an additional suffix appended to the whole output file
name. I think it would be clearer to replace "--suffix" with something
like "--additional-suffix". What do you think ?

Cheers,

Jérémy

[Message part 2 (text/html, inline)]

Message #32 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jérémy Compostella
 <jeremy.compostella <at> gmail.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sat, 18 Feb 2012 14:24:41 +0000

On 02/18/2012 12:52 PM, Jérémy Compostella wrote:
> All,
> 
> I'm continuing with this feature implementation. Getting advantage on my
> newly acquired experience on this project I made some changes in my
> implementation.
> 
> Now I have a concern about the option name "--suffix". I think the
> --suffix-length, --numeric-suffixes and the new --suffix are confusing
> for the user since the two first are related to the count suffix and the
> last one is an additional suffix appended to the whole output file
> name. I think it would be clearer to replace "--suffix" with something
> like "--additional-suffix". What do you think ?

That's a good point. --additional-suffix is better.

Note csplit has a more general --suffix-format option
(which can be shortened to --suffix).
So one can do: --suffix="%02d.txt".
However that's only possible, because csplit
supports only numeric suffixes.

However, this is starting to make me think
this marginal feature isn't worth it.
For the record, summarising the disadvantages so far:

 Backwards incompat with other systems (including older coreutils).
 Some complexity in understanding/describing various suffix options.
 Interoperability issue with possible future auto suffix len feature.

cheers,
Pádraig.

Message #35 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Jérémy Compostella <jeremy.compostella <at> gmail.com>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sat, 18 Feb 2012 15:51:28 +0100

[Message part 1 (text/plain, inline)]

> That's a good point. --additional-suffix is better.
>
> Note csplit has a more general --suffix-format option
> (which can be shortened to --suffix).
> So one can do: --suffix="%02d.txt".
> However that's only possible, because csplit
> supports only numeric suffixes.
>
> However, this is starting to make me think
> this marginal feature isn't worth it.
> For the record, summarising the disadvantages so far:
>
>  Backwards incompat with other systems (including older coreutils).
>  Some complexity in understanding/describing various suffix options.
With the new "--additional-suffix" it looks clear to me.
>  Interoperability issue with possible future auto suffix len feature.
If I remember well the issue is related with the alphabetic file
order. This issue could be easily handled by the user.

IMHO, this feature is maybe marginal but it is very convenient to have
it. Anyway, it would be great to state on it as I need to know if I
should let it go or not. It's always hard to trash work but that's the
game ! :)

Cheers,

Jérémy

[Message part 2 (text/html, inline)]

Message #38 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jérémy Compostella
 <jeremy.compostella <at> gmail.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sat, 18 Feb 2012 14:57:31 +0000

On 02/18/2012 02:51 PM, Jérémy Compostella wrote:
>> That's a good point. --additional-suffix is better.
>> 
>> Note csplit has a more general --suffix-format option
>> (which can be shortened to --suffix).
>> So one can do: --suffix="%02d.txt".
>> However that's only possible, because csplit
>> supports only numeric suffixes.
>> 
>> However, this is starting to make me think
>> this marginal feature isn't worth it.
>> For the record, summarising the disadvantages so far:
>> 
>>  Backwards incompat with other systems (including older coreutils).
>>  Some complexity in understanding/describing various suffix options.
> With the new "--additional-suffix" it looks clear to me.
>>  Interoperability issue with possible future auto suffix len feature.
> If I remember well the issue is related with the alphabetic file
> order. This issue could be easily handled by the user.
> 
> IMHO, this feature is maybe marginal but it is very convenient to have
> it. Anyway, it would be great to state on it as I need to know if I
> should let it go or not. It's always hard to trash work but that's the
> game ! :)

Sure, please go ahead since you've done most of the work,
and others have expressed a desire for this.
I'm just stating the disadvantages for the record,
and perhaps a prompt to others to voice any
reservations they may have.
We always have to be wary when adding new stuff.

cheers,
Pádraig.

Message #41 received at 6554 <at> debbugs.gnu.org (full text, mbox):

From: Jérémy Compostella <jeremy.compostella <at> gmail.com>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 6554 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sun, 19 Feb 2012 13:54:30 +0100

[Message part 1 (text/plain, inline)]

Pádraig,

I replaced --suffix with --additional-suffix and made some cleaning.

Feel free to comment the attached patch.

Cheers,

Jérémy

[Message part 2 (text/html, inline)]

[0001-split-additional-suffix-for-split-bug-6554.patch (text/x-patch, attachment)]

Message #46 received at 6554-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jérémy Compostella
 <jeremy.compostella <at> gmail.com>
Cc: Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	6554-done <at> debbugs.gnu.org, Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sun, 19 Feb 2012 18:16:20 +0000

On 02/19/2012 12:54 PM, Jérémy Compostella wrote:
> Pádraig,
> 
> I replaced --suffix with --additional-suffix and made some cleaning.
> 
> Feel free to comment the attached patch.

Logic is perfect again.

I adjusted the help output a little like:

-  -a, --suffix-length=N   use suffixes of length N (default %d)\n\
-      --additional-suffix=SUFFIX  append an additional SUFFIX to output file\n\
-                                       names. SUFFIX must not contain slash.\n\
+  -a, --suffix-length=N   generate suffixes of length N (default %d)\n\
+      --additional-suffix=SUFFIX  append an additional SUFFIX to file names.\n\

I'll push later on.

cheers,
Pádraig.

Message #47 received at 6554-done <at> debbugs.gnu.org (full text, mbox):

From: Jérémy Compostella <jeremy.compostella <at> gmail.com>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	6554-done <at> debbugs.gnu.org, Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sun, 19 Feb 2012 19:30:16 +0100

[Message part 1 (text/plain, inline)]

> On 02/19/2012 12:54 PM, Jérémy Compostella wrote:
> > Pádraig,
> >
> > I replaced --suffix with --additional-suffix and made some cleaning.
> >
> > Feel free to comment the attached patch.
>
> Logic is perfect again.
Thanks :)

> I adjusted the help output a little like:
>
> -  -a, --suffix-length=N   use suffixes of length N (default %d)\n\
> -      --additional-suffix=SUFFIX  append an additional SUFFIX to output
file\n\
> -                                       names. SUFFIX must not contain
slash.\n\
> +  -a, --suffix-length=N   generate suffixes of length N (default %d)\n\
> +      --additional-suffix=SUFFIX  append an additional SUFFIX to file
names.\n\
I'm just wondering why did you removed the slash warning ? This warning
is maybe too much in the command usage but IMHO the man page should
mention it. Moreover, the mktemp mention this "restriction" on its
--suffix option this way.

Cheers,

Jérémy

[Message part 2 (text/html, inline)]

Message #48 received at 6554-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jérémy Compostella
 <jeremy.compostella <at> gmail.com>
Cc: Eric Blake <eblake <at> redhat.com>, coreutils <at> gnu.org,
	6554-done <at> debbugs.gnu.org, Peng Yu <pengyu.ut <at> gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Sun, 19 Feb 2012 20:16:52 +0000

On 02/19/2012 06:30 PM, Jérémy Compostella wrote:
>> On 02/19/2012 12:54 PM, Jérémy Compostella wrote:
>> > Pádraig,
>> >
>> > I replaced --suffix with --additional-suffix and made some cleaning.
>> >
>> > Feel free to comment the attached patch.
>> 
>> Logic is perfect again.
> Thanks :)
> 
>> I adjusted the help output a little like:
>> 
>> -  -a, --suffix-length=N   use suffixes of length N (default %d)\n\
>> -      --additional-suffix=SUFFIX  append an additional SUFFIX to output file\n\
>> -                                       names. SUFFIX must not contain slash.\n\
>> +  -a, --suffix-length=N   generate suffixes of length N (default %d)\n\
>> +      --additional-suffix=SUFFIX  append an additional SUFFIX to file names.\n\
> I'm just wondering why did you removed the slash warning ? This warning
> is maybe too much in the command usage but IMHO the man page should
> mention it. Moreover, the mktemp mention this "restriction" on its
> --suffix option this way.

I thought it wasn't worth mentioning, especially since
it messed up the output alignment a bit.
Who is going to use a slash?
the .0001% who do will immediately get an error anyway.

cheers,
Pádraig.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.