Package: automake;
Reported by: Peter Johansson <trojkan <at> gmail.com>
Date: Mon, 18 Aug 2014 00:31:01 UTC
Severity: normal
Tags: patch
Done: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18286 in the body.
You can then email your comments to 18286 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-automake <at> gnu.org
:bug#18286
; Package automake
.
(Mon, 18 Aug 2014 00:31:01 GMT) Full text and rfc822 format available.Peter Johansson <trojkan <at> gmail.com>
:bug-automake <at> gnu.org
.
(Mon, 18 Aug 2014 00:31:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Peter Johansson <trojkan <at> gmail.com> To: bug-automake <at> gnu.org Subject: distcheck fails to detect missing files Date: Mon, 18 Aug 2014 10:30:11 +1000
Hi, I have this snippet in my 'Makefile.am' include am/test_data.am $(srcdir)/am/test_data.am: $(srcdir)/test/data.txt $(srcdir)/am/test_data.am.in cd $(srcdir) && $(SHELL) bootstrap am/test_data.am It first includes (at Automake time) a snippet, which is generated while bootstrapping. Now I forgot to include $(srcdir)/test/data.txt $(srcdir)/am/test_data.am.in in the distball, which caused the embarrassing make: *** No rule to make target `am/test_data.am.in', needed by `am/test_data.am'. Stop. The surprising thing is that distcheck does not detect this problem, but 'make distcheck' passes through without problem. The reason for this is probably because while $(srcdir)/test/data.txt does not exist, make also looks in $(srcdir)/$(srcdir)/test/data.txt and that file exists as it is part of my development directory. If I change my Makefile.am to $(srcdir)/am/test_data.am: test/data.txt am/test_data.am.in cd $(srcdir) && $(SHELL) bootstrap am/test_data.am 'make distcheck' will detect the problem, but I have this habit of type $(srcdir) for things that are supposed to be in $(srcdir). Is that a bad habit? A fix could be to let distcheck not work in $(distdir) but instead create 'tmp_/$(distdir)', builddir 'tmp_/$(distdir)/$(builddir)', and installdir 'tmp_/$(distdir)/inst_' which would hide the development files from the distcheck. What do you think? Thanks, Peter -- Peter Johansson
bug-automake <at> gnu.org
:bug#18286
; Package automake
.
(Mon, 22 Dec 2014 19:31:01 GMT) Full text and rfc822 format available.Message #8 received at 18286 <at> debbugs.gnu.org (full text, mbox):
From: Stefano Lattarini <stefano.lattarini <at> gmail.com> To: Peter Johansson <trojkan <at> gmail.com>, 18286 <at> debbugs.gnu.org Cc: Nicola Fontana <ntd <at> entidi.it>, "automake-patches <at> gnu.org" <automake-patches <at> gnu.org> Subject: Re: bug#18286: distcheck fails to detect missing files Date: Mon, 22 Dec 2014 20:30:33 +0100
Reference: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18286 On 08/18/2014 02:30 AM, Peter Johansson wrote: > Hi, > > I have this snippet in my 'Makefile.am' > > > include am/test_data.am > > $(srcdir)/am/test_data.am: $(srcdir)/test/data.txt $(srcdir)/am/test_data.am.in > cd $(srcdir) && $(SHELL) bootstrap am/test_data.am > > > It first includes (at Automake time) a snippet, which is generated while bootstrapping. Now I forgot to include $(srcdir)/test/data.txt $(srcdir)/am/test_data.am.in in the distball, which caused the embarrassing > > make: *** No rule to make target `am/test_data.am.in', needed by `am/test_data.am'. Stop. > > The surprising thing is that distcheck does not detect this problem, > but 'make distcheck' passes through without problem. > This issue has been reported already in the past: http://lists.gnu.org/archive/html/automake/2006-09/msg00008.html http://lists.gnu.org/archive/html/automake/2013-01/msg00049.html (BTW, sorry Nicola for missing your report and your fix for so long!) > The reason for > this is probably because while $(srcdir)/test/data.txt does not exist, > make also looks in $(srcdir)/$(srcdir)/test/data.txt and that file exists > as it is part of my development directory. If I change my Makefile.am to > > $(srcdir)/am/test_data.am: test/data.txt am/test_data.am.in > cd $(srcdir) && $(SHELL) bootstrap am/test_data.am > > 'make distcheck' will detect the problem, but I have this habit of type > $(srcdir) for things that are supposed to be in $(srcdir). Is that a bad > habit? > It can sometimes lead to strange interactions, if not used consistently; see http://lists.gnu.org/archive/html/automake/2006-09/msg00008.html and follow-up messages. But the issue you are experiencing is not due to that, but rather to a bug/limitation of the current 'distcheck' implementation. > A fix could be to let distcheck not work in $(distdir) but instead create > 'tmp_/$(distdir)', builddir 'tmp_/$(distdir)/build_', and installdir > 'tmp_/$(distdir)/inst_' which would hide the development files from the > distcheck. > > What do you think? > Seems like a viable plan. In such a setup, relatively to $(builddir), $(srcdir) will be '..', while the "developement" copy of the source tree will be in '../..'; so the incomplete tarball should be diagnosed. Definitely a nice fix to have for Automake 1.15... Let's see if I can make it happen.
bug-automake <at> gnu.org
:bug#18286
; Package automake
.
(Tue, 23 Dec 2014 19:25:01 GMT) Full text and rfc822 format available.Message #11 received at 18286 <at> debbugs.gnu.org (full text, mbox):
From: Stefano Lattarini <stefano.lattarini <at> gmail.com> To: automake-patches <at> gnu.org Cc: Nicola Fontana <ntd <at> entidi.it>, 18286 <at> debbugs.gnu.org, Peter Johansson <trojkan <at> gmail.com> Subject: [PATCH 2/2] dist: fix bug#18286 "distcheck fails to detect missing files" Date: Tue, 23 Dec 2014 20:23:58 +0100
BTW, this issue had been already reported in the past: http://lists.gnu.org/archive/html/automake/2006-09/msg00008.html http://lists.gnu.org/archive/html/automake/2013-01/msg00049.html "make distcheck" could sometimes fail to detect missing files in the distribution tarball, especially in those cases where both the generated files and their dependencies are explicitly in $(srcdir). An important example of this are *generated* makefile fragments included at Automake time in Makefile.am. A basic example: # -*- Makefile.am -*- $(srcdir)/fragment.am: $(srcdir)/data.txt $(srcdir)/preproc.sh cd $(srcdir) && $(SHELL) preproc.sh <data.txt >fragment.am include $(srcdir)/fragment.am ... If the use forgot to add data.txt and/or preproc.sh in the distribution tarball, "make distcheck" would have erroneously succeeded! The reason is that, while $(srcdir)/data.txt does not exist, make also looks in $(srcdir)/$(srcdir)/data.txt, and in the distcheck-issued VPATH build where $(srcdir) is '..', that file exists, as it is part of the original development directory. * t/distdir.am (distcheck): Adjust to have the build directory be '$(distdir)/_build/sub' rather than just '$(distdir)/_build'. Thanks Nicola Fontana for the suggestion. * t/distcheck-pr18286.sh: Enhance and tighten a little. * t/list-of-tests.mk (XFAIL_TESTS): Remove 't/distcheck-pr18286.sh', as it's now passing. * t/subdir-am-cond.sh: Adjust to avoid a fully spurious failure due to the new distcheck semantics. * t/subdir-ac-subst.sh: Likewise. * t/dejagnu-relative-srcdir.sh: Likewise. * t/txinfo-builddir.sh: Likewise. * NEWS: Update. Helped-by: Nicola Fontana <ntd <at> entidi.it> Helped-by: Peter Johansson <trojkan <at> gmail.com> Signed-off-by: Stefano Lattarini <stefano.lattarini <at> gmail.com> --- NEWS | 16 ++++++++++++++++ THANKS | 1 + lib/am/distdir.am | 14 +++++++++----- t/dejagnu-relative-srcdir.sh | 8 ++++---- t/distcheck-pr18286.sh | 8 ++++++++ t/list-of-tests.mk | 1 - t/subdir-ac-subst.sh | 16 +++++++++------- t/subdir-am-cond.sh | 16 +++++++++------- t/txinfo-builddir.sh | 4 ++-- 9 files changed, 58 insertions(+), 26 deletions(-) diff --git a/NEWS b/NEWS index 1fae7ad..8f2d2a7 100644 --- a/NEWS +++ b/NEWS @@ -115,6 +115,22 @@ New in 1.14.2: something like "... overrides Automake target '$(srcdir)/foo.am". This bug is now fixed. + - Automake bug#18286: "make distcheck" could sometimes fail to detect + missing files in the distribution tarball, especially in those cases + where both the generated files and their dependencies are explicitly + in $(srcdir). An important example of this are *generated* makefile + fragments included at Automake time in Makefile.am; e.g.: + + ... + $(srcdir)/fragment.am: $(srcdir)/data.txt $(srcdir)/preproc.sh + cd $(srcdir) && $(SHELL) preproc.sh <data.txt >fragment.am + include $(srcdir)/fragment.am + ... + + If the use forgot to add data.txt and/or preproc.sh in the distribution + tarball, "make distcheck" would have erroneously succeeded! This issue + is now fixed. + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ New in 1.14.1: diff --git a/THANKS b/THANKS index 6be803e..62f22f8 100644 --- a/THANKS +++ b/THANKS @@ -281,6 +281,7 @@ Nathanael Nerode neroden <at> twcny.rr.com Nelson H. F. Beebe beebe <at> math.utah.edu Nicholas Wourms nwourms <at> netscape.net Nick Bowler nbowler <at> elliptictech.com +Nicola Fontana ntd <at> entidi.it Nicolas Joly njoly <at> pasteur.fr Nicolas Thiery nthiery <at> Icare.mines.edu NightStrike nightstrike <at> gmail.com diff --git a/lib/am/distdir.am b/lib/am/distdir.am index a8ad63c..6d9d42f 100644 --- a/lib/am/distdir.am +++ b/lib/am/distdir.am @@ -1,5 +1,5 @@ ## automake - create Makefile.in from Makefile.am -## Copyright (C) 2001-2013 Free Software Foundation, Inc. +## Copyright (C) 2001-2014 Free Software Foundation, Inc. ## This program is free software; you can redistribute it and/or modify ## it under the terms of the GNU General Public License as published by @@ -430,7 +430,7 @@ distcheck: dist ## can make our new subdirs. chmod -R a-w $(distdir) chmod u+w $(distdir) - mkdir $(distdir)/_build $(distdir)/_inst + mkdir $(distdir)/_build $(distdir)/_build/sub $(distdir)/_inst ## Undo the write access. chmod a-w $(distdir) ## With GNU make, the following command will be executed even with "make -n", @@ -451,8 +451,12 @@ distcheck: dist ## Parallel BSD make may not start a new shell for each command in a recipe, ## so be sure to 'cd' back to the original directory after this. && am__cwd=`pwd` \ - && $(am__cd) $(distdir)/_build \ - && ../configure \ +## If we merely used '$(distdir)/_build' here, "make distcheck" could +## sometimes fail to detect missing files in the distribution tarball, +## especially in those cases where both the generated files and their +## dependencies are explicitly in $(srcdir). See automake bug#18286. + && $(am__cd) $(distdir)/_build/sub \ + && ../../configure \ ?GETTEXT? --with-included-gettext \ ## Additional flags for configure. $(AM_DISTCHECK_CONFIGURE_FLAGS) \ @@ -461,7 +465,7 @@ distcheck: dist ## and --prefix values, so don't allow them to be overridden by the user or ## the developer. That used to be allowed, and caused issues in practice ## (in corner-case usages); see automake bug#14991. - --srcdir=.. --prefix="$$dc_install_base" \ + --srcdir=../.. --prefix="$$dc_install_base" \ && $(MAKE) $(AM_MAKEFLAGS) \ && $(MAKE) $(AM_MAKEFLAGS) dvi \ && $(MAKE) $(AM_MAKEFLAGS) check \ diff --git a/t/dejagnu-relative-srcdir.sh b/t/dejagnu-relative-srcdir.sh index 6168421..34e6710 100644 --- a/t/dejagnu-relative-srcdir.sh +++ b/t/dejagnu-relative-srcdir.sh @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2011-2013 Free Software Foundation, Inc. +# Copyright (C) 2011-2014 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -43,7 +43,7 @@ END cat > env.test/env.exp << 'END' set env_srcdir $env(srcdir) send_user "env_srcdir: $env_srcdir\n" -if { [ regexp "^\.\.?$" $env_srcdir ] } { +if { [ regexp "^\.(\./\.\.)?$" $env_srcdir ] } { pass "test_env_src" } else { fail "test_env_src" @@ -52,13 +52,13 @@ END cat > tcl.test/tcl.exp << 'END' send_user "tcl_srcdir: $srcdir\n" -if { [ regexp "^\.\.?$" $srcdir ] } { +if { [ regexp "^\.(\./\.\.)?$" $srcdir ] } { pass "test_tcl_src" } else { fail "test_tcl_src" } send_user "tcl_orig_srcdir: $orig_srcdir\n" -if { [ regexp "^\.\.?$" $orig_srcdir ] } { +if { [ regexp "^\.(\./\.\.)?$" $orig_srcdir ] } { pass "test_tcl_orig_src" } else { fail "test_tcl_orig_src" diff --git a/t/distcheck-pr18286.sh b/t/distcheck-pr18286.sh index b107735..fe89612 100644 --- a/t/distcheck-pr18286.sh +++ b/t/distcheck-pr18286.sh @@ -57,6 +57,14 @@ $AUTOMAKE -a $MAKE check +# Oops, we failed to distribute some required files! run_make -e FAIL -M distcheck +$FGREP '../../test_data.txt' output + +# But if we distribute them, everything will be OK. +echo 'EXTRA_DIST = test_data.txt gen-testdata.sh' >> Makefile.am + +using_gmake || $MAKE Makefile +$MAKE distcheck : diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk index ad96326..f97f2f6 100644 --- a/t/list-of-tests.mk +++ b/t/list-of-tests.mk @@ -31,7 +31,6 @@ t/pm/Version3.pl XFAIL_TESTS = \ t/all.sh \ t/cond17.sh \ -t/distcheck-pr18286.sh \ t/gcj6.sh \ t/override-conditional-2.sh \ t/override-conditional-pr13940.sh \ diff --git a/t/subdir-ac-subst.sh b/t/subdir-ac-subst.sh index d3f50bc..b3ede8c 100644 --- a/t/subdir-ac-subst.sh +++ b/t/subdir-ac-subst.sh @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002-2013 Free Software Foundation, Inc. +# Copyright (C) 2002-2014 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,7 +20,7 @@ . test-init.sh -cat >>configure.ac <<'END' +cat >> configure.ac <<'END' if test "$want_opt" = yes; then MAYBE_OPT=opt else @@ -31,7 +31,7 @@ AC_CONFIG_FILES([src/Makefile opt/Makefile]) AC_OUTPUT END -cat >Makefile.am << 'END' +cat > Makefile.am <<'END' SUBDIRS = src $(MAYBE_OPT) DIST_SUBDIRS = src opt @@ -45,14 +45,16 @@ DIST_SUBDIRS = src opt # We rely on 'distcheck' to run 'check-local' and use # 'sanity1' and 'sanity2' as evidences that test-build was run. +test_rootdir = $(top_builddir)/../../.. + test-build: all test -f src/result if test -n "$(MAYBE_OPT)"; then \ test -f opt/result || exit 1; \ - : > $(top_builddir)/../../sanity2 || exit 1; \ + : > $(test_rootdir)/sanity2 || exit 1; \ else \ test ! -f opt/result || exit 1; \ - : > $(top_builddir)/../../sanity1 || exit 1; \ + : > $(test_rootdir)/sanity1 || exit 1; \ fi test-dist: distdir @@ -66,7 +68,7 @@ mkdir src opt : > src/source : > opt/source -cat >src/Makefile.am << 'END' +cat > src/Makefile.am <<'END' EXTRA_DIST = source all-local: result CLEANFILES = result @@ -76,7 +78,7 @@ result: source END # We want in opt/ the same Makefile as in src/. Let's exercise 'include'. -cat >opt/Makefile.am << 'END' +cat > opt/Makefile.am <<'END' include ../src/Makefile.am END diff --git a/t/subdir-am-cond.sh b/t/subdir-am-cond.sh index 8a36bc3..69345d1 100644 --- a/t/subdir-am-cond.sh +++ b/t/subdir-am-cond.sh @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002-2013 Free Software Foundation, Inc. +# Copyright (C) 2002-2014 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,13 +20,13 @@ . test-init.sh -cat >>configure.ac <<'END' +cat >> configure.ac <<'END' AM_CONDITIONAL([COND_OPT], [test "$want_opt" = yes]) AC_CONFIG_FILES([src/Makefile opt/Makefile]) AC_OUTPUT END -cat >Makefile.am << 'END' +cat > Makefile.am <<'END' if COND_OPT MAYBE_OPT = opt endif @@ -42,16 +42,18 @@ SUBDIRS = src $(MAYBE_OPT) # We rely on 'distcheck' to run 'check-local' and use # 'sanity1' and 'sanity2' as evidences that test-build was run. +test_rootdir = $(top_builddir)/../../.. + if COND_OPT test-build: all test -f src/result test -f opt/result - : > $(top_builddir)/../../sanity2 + : > $(test_rootdir)/sanity2 else test-build: all test -f src/result test ! -f opt/result - : > $(top_builddir)/../../sanity1 + : > $(test_rootdir)/sanity1 endif test-dist: distdir @@ -65,7 +67,7 @@ mkdir src opt : > src/source : > opt/source -cat >src/Makefile.am << 'END' +cat > src/Makefile.am <<'END' EXTRA_DIST = source all-local: result CLEANFILES = result @@ -75,7 +77,7 @@ result: source END # We want in opt/ the same Makefile as in src/. Let's exercise 'include'. -cat >opt/Makefile.am << 'END' +cat > opt/Makefile.am <<'END' include ../src/Makefile.am END diff --git a/t/txinfo-builddir.sh b/t/txinfo-builddir.sh index 42d4112..50e3417 100644 --- a/t/txinfo-builddir.sh +++ b/t/txinfo-builddir.sh @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2012-2013 Free Software Foundation, Inc. +# Copyright (C) 2012-2014 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -43,7 +43,7 @@ CLEANFILES = mu.info # to think 'version.texi' has been created... check-local: test ! -e mu.info - test -f ../mu.info + test -f $(srcdir)/mu.info END mkdir subdir -- 2.1.3
bug-automake <at> gnu.org
:bug#18286
; Package automake
.
(Tue, 23 Dec 2014 19:25:02 GMT) Full text and rfc822 format available.Message #14 received at 18286 <at> debbugs.gnu.org (full text, mbox):
From: Stefano Lattarini <stefano.lattarini <at> gmail.com> To: automake-patches <at> gnu.org Cc: Nicola Fontana <ntd <at> entidi.it>, 18286 <at> debbugs.gnu.org, Peter Johansson <trojkan <at> gmail.com> Subject: [PATCH 1/2] tests: expose automake bug#18286 "distcheck fails to detect missing files" Date: Tue, 23 Dec 2014 20:23:57 +0100
* t/distcheck-pr18286.sh: New test, still XFAILing. * t/list-of-tests.mk: Add it. Signed-off-by: Stefano Lattarini <stefano.lattarini <at> gmail.com> --- t/distcheck-pr18286.sh | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++ t/list-of-tests.mk | 2 ++ 2 files changed, 64 insertions(+) create mode 100644 t/distcheck-pr18286.sh diff --git a/t/distcheck-pr18286.sh b/t/distcheck-pr18286.sh new file mode 100644 index 0000000..b107735 --- /dev/null +++ b/t/distcheck-pr18286.sh @@ -0,0 +1,62 @@ +#! /bin/sh +# Copyright (C) 2014 Free Software Foundation, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2, or (at your option) +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +# Ensure "make distcheck" detects all missing files, without getting +# confused by the fact that they exists in the "original" source tree +# from which "make distcheck" is run. See automake bug#18286. + +. test-init.sh + +echo AC_OUTPUT >> configure.ac + +cat > Makefile.am <<'END' +$(srcdir)/test_data.am: $(srcdir)/test_data.txt $(srcdir)/gen-testdata.sh + cd $(srcdir) && $(SHELL) gen-testdata.sh <test_data.txt >test_data.am + +include $(srcdir)/test_data.am + +check-local: + is $(testdata) == foo bar +END + +cat > test_data.txt <<'END' +foo +bar +END + + +cat > gen-testdata.sh <<'END' +#!/bin/sh +printf 'testdata = \\\n' +sed 's/$/ \\/' +echo '$(empty_string)' +END +chmod a+x gen-testdata.sh + +$sleep +./gen-testdata.sh <test_data.txt >test_data.am + +$ACLOCAL +$AUTOCONF +$AUTOMAKE -a + +./configure + +$MAKE check + +run_make -e FAIL -M distcheck + +: diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk index c07eb72..ad96326 100644 --- a/t/list-of-tests.mk +++ b/t/list-of-tests.mk @@ -31,6 +31,7 @@ t/pm/Version3.pl XFAIL_TESTS = \ t/all.sh \ t/cond17.sh \ +t/distcheck-pr18286.sh \ t/gcj6.sh \ t/override-conditional-2.sh \ t/override-conditional-pr13940.sh \ @@ -427,6 +428,7 @@ t/distcheck-no-prefix-or-srcdir-override.sh \ t/distcheck-override-infodir.sh \ t/distcheck-pr9579.sh \ t/distcheck-pr10470.sh \ +t/distcheck-pr18286.sh \ t/dmalloc.sh \ t/doc-parsing-buglets-colneq-subst.sh \ t/doc-parsing-buglets-tabs.sh \ -- 2.1.3
Stefano Lattarini <stefano.lattarini <at> gmail.com>
to control <at> debbugs.gnu.org
.
(Tue, 23 Dec 2014 19:27:02 GMT) Full text and rfc822 format available.Stefano Lattarini <stefano.lattarini <at> gmail.com>
to control <at> debbugs.gnu.org
.
(Tue, 23 Dec 2014 19:27:02 GMT) Full text and rfc822 format available.Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org
.
(Wed, 21 Jan 2015 12:24:03 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.