Subject: bug#60419: closed (Re: bug#19615: make dist tarball contains owner/group information from the build system)

From: [email-masked] (GNU bug Tracking System)
Date: Fri, 30 Jun 2023 01:02:03 +0000

Your bug report

#19615: Allow user-defined 'tar' options

which was filed against the automake package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
19615: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=19615
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#19615: make dist tarball contains owner/group information from the build system
From: Karl Berry <[email-masked]>
To: [email-masked], [email-masked], [email-masked]
Cc: [email-masked], [email-masked]
Date: Thu, 29 Jun 2023 19:01:11 -0600
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi Dimitrios and all,

Back on this bug from 2015 and earlier (yikes, sorry):
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=19615

    TAR_OPTIONS = --owner=0 --group=0
    I expected that something like this would be the default though.

I agree it would be desirable, in theory.  But we can't make it the
default because it's not portable.  It would also be a big change from
existing practice.

I think adding it in projects' Makefile.am is as good as it gets.
Certainly much simpler than trying to determine whether tar supports
--owner/--group at "make dist" time, and then providing options as to
whether to use them or not. Or which options to use. Etc.

Thus, instead of making any change to the code (sorry again), I merely
added a note to the manual about it.

    pavel> The fix was promised back in year 2008. I hope we'll see it. 

If one of the past or current Automake maintainers has code to deal with
this, I hope they'll say.

    http://comments.gmane.org/gmane.comp.sysutils.automake.bugs/4340

404. I didn't try to guess which bug-automake message that might refer to.

    bogdan> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=60419
    ... a patch that allows Automake users to provide their own 
    command-line options that will be passed to the 'tar' utility when
    ...
    -  [am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -'],
    +  [am__tar='$${TAR-tar} c $${TAR_OPTIONS} -hof - "$$tardir"' am__untar='$${TAR-tar} x $${UNTAR_OPTIONS} -f -'],

Unfortunately this sort of change is not portable. tar option parsing is
a strange and wonderful area. Non-dash options ("c") can't be mixed with
dash options (-hof). Conceivably we could switch to all dash options
(... -chof - $${TAR_OPTIONS} ...), but Automake currently goes to a lot
of trouble to support even the oldest tars, which don't support dash
options at all. And I'm not sure dash options (with older tars) can be
separated into different groups.

Although such ancient-tar support is probably not important nowadays,
more fundamentally, I don't see anything to be gained. Either GNU tar is
being used, in which case it already supports the TAR_OPTIONS envvar
itself, or it's not, in which case either there are different options or
no options at all for the job, and something different has to be
done. Automake can't solve that problem, as far as I can see. For
instance, trying to chown the unpacked tree before tarring would fail.

    This "bug report" can be merged with 19615 or simply deleted.

My attempts to use debbugs to merge the reports failed for no clear
reason. Closing both bugs separately. --thanks, karl.

-----------------------------------------------------------------------------
doc: discuss setting uid/gid information in tarballs.

In response to https://bugs.gnu.org/19615.

* doc/automake.texi (Basics of Distribution): give
example of specifying the TAR_OPTIONS (environment)
variable used by GNU tar.
diff --git a/doc/automake.texi b/doc/automake.texi
index a56e7f8ee..1bf74cd73 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8621,12 +8621,30 @@ More precisely, the gzipped @code{tar} file is named

 @c See automake #9822.
 @vindex TAR
-You can set the environment variable @code{TAR} to override the tar
-program used; it defaults to @code{tar}.  @xref{The Types of
-Distributions}, for how to generate other kinds of archives.
+You can set the environment (or @code{Makefile.am}) variable @code{TAR}
+to override the tar program used; it defaults to @code{tar}.
+@xref{The Types of Distributions}, for how to generate other kinds of
+archives.

-For the most part, the files to distribute are automatically found by
-Automake:
+@vindex TAR_OPTIONS
+With GNU tar, you can also set the environment (or @code{Makefile.am})
+variable @code{TAR_OPTIONS} to pass options to @code{tar}.  One common
+case for this is wanting to avoid using the local user's uid and gid
+in the tar file, or the uid being larger than is supported by the tar
+format (not uncommon nowadays). This can be done with, for example>
+
+@example
+TAR_OPTIONS = --owner=0 --group=0
+export TAR_OPTIONS
+@end example
+
+@noindent
+The @code{export} (a GNU make feature) is necessary to pass the
+variable in the environment to the @code{tar} invocation.
+(For more discussion, see @url{https://bugs.gnu.org/19615}.)
+
+For the most part, the files to distribute are automatically
+found by Automake:

 @itemize @bullet
 @item
@@ -11376,13 +11394,12 @@ time may be overridden: @code{make V=1} will produce verbose output,
 @code{make V=0} less verbose output.

 Unfortunately, if @code{V} is assigned a value other than 0 or 1,
-errors will result. This is problematic when a third-party program or
+errors will result.  This is problematic when a third-party program or
 library is built in the same tree and also uses the make variable
-@code{V}, with different values. The best workaround is probably to
+@code{V}, with different values.  The best workaround is probably to
 set @code{AM_V_P=true} (or similar), either on the make command line
-or in the @code{V}-using project's @code{Makefile.am}. For more
-discussion:
-@url{https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20077}.
+or in the @code{V}-using project's @code{Makefile.am}.  (For more
+discussion, see @url{https://bugs.gnu.org/20077}.)
 @end itemize

 @cindex default verbosity for silent rules

compile finished at Thu Jun 29 17:59:48 2023

Subject: Allow user-defined 'tar' options
From: Bogdan <[email-masked]>
To: [email-masked]
Date: Thu, 29 Dec 2022 20:02:33 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

Hello again.

This time a patch that allows Automake users to provide their own 
command-line options that will be passed to the 'tar' utility when 
creating distribution packages.

This indirectly addresses bug#19615. The defect probably can't be 
fixed in a portable and secure way, so Automake can at least allow the 
user to specify the right options for him/her.

Feel free to rename the options as needed (like prefix/suffix with 
"AM" or something).

Regards,
Bogdan Drozdowski

-- 
Regards - Bogdan ('bogdro') D.                 (GNU/Linux & FreeDOS)
X86 assembly (DOS, GNU/Linux):    http://bogdro.evai.pl/index-en.php
Soft(EN): http://bogdro.evai.pl/soft  http://bogdro.evai.pl/soft4asm
www.Xiph.org  www.TorProject.org  www.LibreOffice.org  www.GnuPG.org

From 81c1e187fd98b3a4cec9a15fd3c4d1fd0471ad49 Mon Sep 17 00:00:00 2001
From: Bogdan Drozdowski <>
Date: Thu, 29 Dec 2022 19:46:41 +0100
Subject: [PATCH] Allow user-defined options for tar

---
 doc/automake.texi    |  6 +++++-
 m4/tar.m4            | 20 +++++++++---------
 t/list-of-tests.mk   |  1 +
 t/tar-opts-envvar.sh | 48 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 64 insertions(+), 11 deletions(-)
 create mode 100644 t/tar-opts-envvar.sh

diff --git a/doc/automake.texi b/doc/automake.texi
index 48744e24f..d53caef07 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8622,7 +8622,11 @@ More precisely, the gzipped @code{tar} file is named
 @c See automake #9822.
 @vindex TAR
 You can set the environment variable @code{TAR} to override the tar
-program used; it defaults to @code{tar}.  @xref{The Types of
+program used; it defaults to @code{tar}.
+You can set the environment variable @code{TAR_OPTIONS} to specify
+additional options to pass to the tar program when creating packages
+and @code{UNTAR_OPTIONS} for additional options to pass when unpacking.
+@xref{The Types of
 Distributions}, for how to generate other kinds of archives.
 
 For the most part, the files to distribute are automatically found by
diff --git a/m4/tar.m4 b/m4/tar.m4
index 8f4d2f213..ca00555eb 100644
--- a/m4/tar.m4
+++ b/m4/tar.m4
@@ -29,7 +29,7 @@ AC_SUBST([AMTAR], ['$${TAR-tar}'])
 _am_tools='gnutar m4_if([$1], [ustar], [plaintar]) pax cpio none'
 
 m4_if([$1], [v7],
-  [am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -'],
+  [am__tar='$${TAR-tar} c $${TAR_OPTIONS} -hof - "$$tardir"' am__untar='$${TAR-tar} x $${UNTAR_OPTIONS} -f -'],
 
   [m4_case([$1],
     [ustar],
@@ -81,17 +81,17 @@ m4_if([$1], [v7],
       for _am_tar in tar gnutar gtar; do
         AM_RUN_LOG([$_am_tar --version]) && break
       done
-      am__tar="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$$tardir"'
-      am__tar_="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$tardir"'
-      am__untar="$_am_tar -xf -"
+      am__tar="$_am_tar $${TAR_OPTIONS} --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$$tardir"'
+      am__tar_="$_am_tar $${TAR_OPTIONS} --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$tardir"'
+      am__untar="$_am_tar -x $${UNTAR_OPTIONS} -f -"
       ;;
     plaintar)
       # Must skip GNU tar: if it does not support --format= it doesn't create
       # ustar tarball either.
       (tar --version) >/dev/null 2>&1 && continue
-      am__tar='tar chf - "$$tardir"'
-      am__tar_='tar chf - "$tardir"'
-      am__untar='tar xf -'
+      am__tar='tar c $${TAR_OPTIONS} -hf - "$$tardir"'
+      am__tar_='tar c $${TAR_OPTIONS} -hf - "$tardir"'
+      am__untar='tar x $${UNTAR_OPTIONS} -f -'
       ;;
     pax)
       am__tar='pax -L -x $1 -w "$$tardir"'
@@ -99,9 +99,9 @@ m4_if([$1], [v7],
       am__untar='pax -r'
       ;;
     cpio)
-      am__tar='find "$$tardir" -print | cpio -o -H $1 -L'
-      am__tar_='find "$tardir" -print | cpio -o -H $1 -L'
-      am__untar='cpio -i -H $1 -d'
+      am__tar='find "$$tardir" -print | cpio $${CPIO_OPTIONS} -o -H $1 -L'
+      am__tar_='find "$tardir" -print | cpio $${CPIO_OPTIONS} -o -H $1 -L'
+      am__untar='cpio $${UNCPIO_OPTIONS} -i -H $1 -d'
       ;;
     none)
       am__tar=false
diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk
index d82cf9c4d..1b2d91cfe 100644
--- a/t/list-of-tests.mk
+++ b/t/list-of-tests.mk
@@ -1207,6 +1207,7 @@ t/tags-lisp-space.sh \
 t/tags-pr12372.sh \
 t/tar-ustar.sh \
 t/tar-pax.sh \
+t/tar-opts-envvar.sh \
 t/tar-opts-errors.sh \
 t/tar-ustar-id-too-high.sh \
 t/tar-override.sh \
diff --git a/t/tar-opts-envvar.sh b/t/tar-opts-envvar.sh
new file mode 100644
index 000000000..73836edbc
--- /dev/null
+++ b/t/tar-opts-envvar.sh
@@ -0,0 +1,48 @@
+#! /bin/sh
+# Copyright (C) 2004-2022 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+# Check if passing options from environment to 'tar' works.
+
+. test-init.sh
+
+echo AC_OUTPUT >> configure.ac
+
+# not all shells like 'export var=value'...
+TAR_OPTIONS='--numeric-owner --test-tar-opt'
+export TAR_OPTIONS
+
+UNTAR_OPTIONS='--numeric-owner --test-untar-opt'
+export UNTAR_OPTIONS
+
+: > Makefile.am
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+./configure
+
+# debug:
+#grep am__tar Makefile
+#grep am__untar Makefile
+
+grep am__tar Makefile | grep '$${TAR_OPTIONS}' || exit 1
+grep am__untar Makefile | grep '$${UNTAR_OPTIONS}' || exit 2
+
+# 'tar' is expected to fail with invalid options, hence this:
+($MAKE dist-gzip 2>&1 || true) | grep test-tar-opt || exit 3
+($MAKE distcheck 2>&1 || true) | grep test-untar-opt || exit 4
+
+:
-- 
2.35.1