From 757021defd0cac3345821ba03720294ff65fd5ec Mon Sep 17 00:00:00 2001 From: Alexandre Duret-Lutz Date: Sun, 9 Feb 2003 20:59:10 +0000 Subject: [PATCH] * automake.texi (FAQ, CVS, maintainer-mode, wildcards) (distcleancheck): New nodes. --- ChangeLog | 5 + automake.texi | 488 +++++++++++++++++++++++++++++++++++++++++++++++++- stamp-vti | 2 +- version.texi | 2 +- 4 files changed, 492 insertions(+), 5 deletions(-) diff --git a/ChangeLog b/ChangeLog index 3d9b385b..644fd1f6 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2003-02-09 Alexandre Duret-Lutz + + * automake.texi (FAQ, CVS, maintainer-mode, wildcards) + (distcleancheck): New nodes. + 2003-02-06 Alexandre Duret-Lutz * automake.in (scan_autoconf_files): Don't complain that diff --git a/automake.texi b/automake.texi index 327ba284..bcef3742 100644 --- a/automake.texi +++ b/automake.texi @@ -134,6 +134,7 @@ This edition documents version @value{VERSION}. * Extending:: Extending Automake * Distributing:: Distributing the Makefile.in * API versioning:: About compatibility between Automake versions +* FAQ:: Frequently Asked Questions * Macro and Variable Index:: * General Index:: @end menu @@ -4544,7 +4545,9 @@ The above definition is not the default because it's usually an error if your Makefiles cause some distributed files to be rebuilt when the user build the package. (Think about the user missing the tool required to build the file; or if the required tool is built by your package, -consider the cross-compilation case where it can't be run.) +consider the cross-compilation case where it can't be run.) There is +a FAQ entry about this (@pxref{distcleancheck}), make sure you read it +before playing with @code{distcleancheck_listfiles}. @code{distcheck} also checks that the @code{uninstall} target works properly, both for ordinary and @samp{DESTDIR} builds. It does this @@ -5341,7 +5344,7 @@ have a special exception allowing you to distribute them with your package, regardless of the licensing you choose. -@node API versioning, Macro and Variable Index, Distributing, Top +@node API versioning, FAQ, Distributing, Top @chapter Automake API versioning New Automake releases usually include bug fixes and new features. @@ -5406,8 +5409,487 @@ If it turns out you need to use such a undocumented feature, contact @email{automake@@gnu.org} and try to get it documented and exercised by the test-suite. +@node FAQ, Macro and Variable Index, API versioning, Top +@chapter Frequently Asked Questions about Automake + +This chapters covers some questions that often come up on the mailing +lists. + +@menu +* CVS:: CVS and generated files +* maintainer-mode:: missing and AM_MAINTAINER_MODE +* wildcards:: Why doesn't Automake support wildcards? +* distcleancheck:: Files left in build directory after distclean +@end menu + +@node CVS, maintainer-mode, FAQ, FAQ +@section CVS and generated files + +@subsection Background: distributed generated files +@cindex generated files, distributed +@cindex rebuild rules + +Packages made with Autoconf and Automake ship with some generated +files like @file{configure} or @file{Makefile.in}. These files were +generated on the developer's host and are distributed so that +end-users do not have to install the maintainer tools required to +rebuild them. Other generated files like Lex scanners, Yacc parsers, +or Info documentation, are usually distributed on similar grounds. + +Automake output rules in @file{Makefile}s to rebuild these files. For +instance @command{make} will run @command{autoconf} to rebuild +@file{configure} whenever @file{configure.in} is changed. This makes +development safer by ensuring a @file{configure} is never out-of-date +with respect to @file{configure.in}. + +As generated files shipped in packages are up-to-date, and because +@command{tar} preserves timestamps, these rebuild rules are not +triggered when a user unpacks and builds a package. + +@subsection Background: CVS and timestamps +@cindex timestamps and CVS +@cindex CVS and timestamps + +Unless you use CVS keywords (in which case files must be updated at +commit time), CVS preserves timestamp during @code{cvs commit} and +@code{cvs import -d} operations. + +When you check out a file using @code{cvs checkout} its timestamp is +set to that of the revision which is being checked out. + +However, during @command{cvs update}, files will have the date of the +update, not the original timestamp of this revision. This is meant to +make sure that @command{make} notices sources files have been updated. + +This timestamp shift is troublesome when both sources and generated +files are kept under CVS. Because CVS processes files in alphabetical +order, @file{configure.in} will appear older than @file{configure} +after a @command{cvs update} that updates both files, even if +@file{configure} was newer than @file{configure.in} when it was +checked in. Calling @code{make} will then trigger a spurious rebuild +of @file{configure}. + +@subsection Living with CVS in Autoconfiscated projects +@cindex CVS and generated files +@cindex generated files and CVS + +There are basically two clans amongst maintainers: those who keep all +distributed files under CVS, including generated files, and those who +keep generated files @emph{out} of CVS. + +@subsubheading All files in CVS + +@itemize @bullet +@item +The CVS repository contains all distributed files so you know exactly +what is distributed, and you can checkout any prior version entirely. + +@item +Maintainers can see how generated files evolve (for instance you can +see what happens to your @file{Makefile.in}s when you upgrade Automake +and make sure they look OK). + +@item +Users do not need the autotools to build a checkout of the project, it +works just like a released tarball. + +@item +If users use @command{cvs update} to update their copy, instead of +@command{cvs checkout} to fetch a fresh one, timestamps will be +inaccurate. Some rebuild rules will be triggered and attempt to +run developer tools such as @command{autoconf} or @command{automake}. + +Actually, calls to such tools are all wrapped into a call to the +@command{missing} script discussed later (@pxref{maintainer-mode}). +@command{missing} will take care of fixing the timestamps when these +tools are not installed, so that the build can continue. + +@item +In distributed development, developers are likely to have different +version of the maintainer tools installed. In this case rebuilds +triggered by timestamp lossage will lead to spurious changes +to generated files. There are several solutions to this: + +@itemize +@item +All developers should use the same versions, so that the rebuilt files +are identical to files in CVS. (This starts to be difficult when each +project you work on uses different versions.) +@item +Or people use a script to fix the timestamp after a checkout (the GCC +folks have such a script). +@item +Or @file{configure.in} uses @code{AM_MAINTAINER_MODE}, which will +disable all these rebuild rules by default. This is further discussed +in @ref{maintainer-mode}. +@end itemize + +@item +Although we focused on spurious rebuilds, the converse can also +happen. CVS's timestamp handling can also let you think an +out-of-date file is up-to-date. + +For instance, suppose a developer has modified @file{Makefile.am} and +rebuilt @file{Makefile.in}, and then decide to do a last-minute change +to @file{Makefile.am} right before checking in both files (without +rebuilding @file{Makefile.in} to account for the change). + +This last change to @file{Makefile.am} make the copy of +@file{Makefile.in} out-of-date. Since CVS processes files +alphabetically, when another developer @code{cvs update} his or her +tree, @file{Makefile.in} will happen to be newer than +@file{Makefile.am}. This other developer will not see +@file{Makefile.in} is out-of-date. + +@end itemize + +@subsubheading Generated files out of CVS + +One way to get CVS and @code{make} working peacefully is to never +store generated files in CVS, i.e., do not CVS-control files which are +@code{Makefile} targets (or @emph{derived} files in Make terminology). + +This way developers are not annoyed by changes to generated files. It +does not matter if they all have different versions (assuming they are +compatible, of course). And finally, timestamps are not lost, changes +to sources files can't be missed as in the +@file{Makefile.am}/@file{Makefile.in} example discussed earlier. + +The drawback is that the CVS repository is not an exact copy of what +is distributed and that users now need to install various development +tools (maybe even specific versions) before they can build a checkout. +But, after all, CVS's job is versioning, not distribution. + +Allowing developers to use different versions of their tools can also +hide bugs during distributed development. Indeed, developers will be +using (hence testing) their own generated files, instead of the +generated files that will be released actually. The developer who +prepares the tarball might be using a version of the tool that +produces bogus output (for instance a non-portable C file), something +other developers could have noticed if they weren't using there own +versions of this tool. + +@subsection Third-party files +@cindex CVS and third-party files +@cindex third-party files and CVS + +Another class of files not discussed here (because they do not cause +timestamp issues) are files which are shipped with a package, but +maintained elsewhere. For instance tools like @command{gettextize} +and @command{autopoint} (from Gettext) or @command{libtoolize} (from +Libtool), will install or update files in your package. + +These files, whether they are kept under CVS or not, raise similar +concerns about version mismatch between developers' tools. The +Gettext manual has a section about this, see @ref{CVS Issues, CVS +Issues, Integrating with CVS, gettext, GNU gettext tools}. + +@node maintainer-mode, wildcards, CVS, FAQ +@section @command{missing} and @code{AM_MAINTAINER_MODE} + +@subsection @command{missing} +@cindex missing, purpose + +The @command{missing} script is a wrapper around several maintainer +tools, designed to warn users if a maintainer tool is required but +missing. Typical maintainer tools are @command{autoconf}, +@command{automake}, @command{bison}, etc. Because file generated by +these tools are shipped with the other sources of a package, these +tools shouldn't be required during a user build and they are not +checked for in @file{configure}. + +However, if for some reason a rebuild rule is triggered and involves a +missing tool, @command{missing} will notice it and warn the user. +Besides the warning, when a tool is missing, @command{missing} will +attempt to fix timestamps in a way which allow the build to continue. +For instance @command{missing} will touch @file{configure} if +@command{autoconf} is not installed. When all distributed files are +kept under CVS, this feature of @command{missing} allows user +@emph{with no maintainer tools} to build a package off CVS, bypassing +any timestamp inconsistency implied by @code{cvs update}. + +If the required tool is installed, @command{missing} will run it and +won't attempt to continue after failures. This is correct during +development: developers love fixing failures. However, users with +wrong versions of maintainer tools may get an error when the rebuild +rule is spuriously triggered, halting the build. This failure to let +the build continue is one of the arguments of the +@code{AM_MAINTAINER_MODE} advocators. + +@subsection @code{AM_MAINTAINER_MODE} +@cindex AM_MAINTAINER_MODE, purpose +@cvindex AM_MAINTAINER_MODE + +@code{AM_MAINTAINER_MODE} disables the so called "rebuild rules" by +default. If you have @code{AM_MAINTAINER_MODE} in +@file{configure.ac}, and run @code{./configure && make}, then +@command{make} will *never* attempt to rebuilt @file{configure}, +@file{Makefile.in}s, Lex or Yacc outputs, etc. I.e., this disables +build rules for files which are usually distributed and that users +should normally not have to update. + +If you run @code{./configure --enable-maintainer-mode}, then these +rebuild rules will be active. + +People use @code{AM_MAINTAINER_MODE} either because they do want their +users (or themselves) annoyed by timestamps lossage (@pxref{CVS}), or +because they simply can't stand the rebuild rules and prefer running +maintainer tools explicitly. + +@code{AM_MAINTAINER_MODE} also allows you to disable some custom build +rules conditionally. Some developers use this feature to disable +rules that need exotic tools that users may not have available. + +Several years ago François Pinard pointed out several arguments +against @code{AM_MAINTAINER_MODE}. Most of them relate to insecurity. +By removing dependencies you get non-dependable builds: change to +sources files can have no-effect on generated files and this can be +very confusing when unnoticed. He adds that security shouldn't be +reserved to maintainers (what @code{--enable-maintainer-mode} +suggests), on the contrary. If one user has to modify a +@file{Makefile.am}, then either @file{Makefile.in} should be updated +or a warning should be output (this is what Automake uses +@code{missing} for) but the last thing you want is that nothing +happens and the user doesn't notice it (this is what happens when +rebuild rules are disabled by @code{AM_MAINTAINER_MODE}). + +Jim Meyering, the inventor of the @code{AM_MAINTAINER_MODE} macro was +swayed by François's arguments, and got rid of +@code{AM_MAINTAINER_MODE} in all of his packages. + +Still many people continue to use @code{AM_MAINTAINER_MODE}, because +it helps them working on projects where all files are kept under CVS, +and because @command{missing} isn't enough if you have the wrong +version of the tools. + + +@node wildcards, distcleancheck, maintainer-mode, FAQ +@section Why doesn't Automake support wildcards? +@cindex wildcards + +Developers are lazy. They often would like to use wildcards in +@file{Makefile.am}s, so they don't need to remember they have to +update @file{Makefile.am}s every time they add, delete, or rename a +file. + +There are several objections to this: +@itemize +@item +When using CVS (or similar) developers need to remember they have to +run @code{cvs add} or @code{cvs rm} anyway. Updating +@file{Makefile.am} accordingly quickly becomes a reflex. + +Conversely, if your application doesn't compile +because you forgot to add a file in @file{Makefile.am}, it will help +you remember to @code{cvs add} it. + +@item +Using wildcards makes easy to distribute files by mistake. For +instance some code a developer is experimenting with (a test case, +say) but which should not be part of the distribution. + +@item +Using wildcards it's easy to omit some files by mistake. For +instance one developer creates a new file, uses it at many places, +but forget to commit it. Another developer then checkout the +incomplete project and is able to run `make dist' successfully, +even though a file is missing. + +@item +Listing files, you control *exactly* what you distribute. +If some file that should be distributed is missing from your +tree, @code{make dist} will complain. Besides, you don't distribute +more than what you listed. + +@item +Finally it's really hard to @file{forget} adding a file to +@file{Makefile.am}, because if you don't add it, it doesn't get +compiled nor installed, so you can't even test it. +@end itemize + +Still, these are philosophic objections, and as such you may disagree, +or find enough value in wildcards to dismiss all of them. Before you +start writing a patch against Automake to teach it about wildcards, +let's see the the main technical issue: portability. + +Although @code{$(wildcard ...)} works with GNU @command{make}, it is +not portable to other @command{make} implementations. + +The only way Automake could support @command{$(wildcard ...)} is by +expending @command{$(wildcard ...)} when @command{automake} is run. +Resulting @file{Makefile.in}s would be portable since they would +list all files and not use @code{$(wildcard ...)}. However that +means developers need to remember they must run @code{automake} each +time they add, delete, or rename files. + +Compared to editing @file{Makefile.am}, this really little win. Sure, +it's easier and faster to type @code{automake; make} than to type +@code{emacs Makefile.am; make}. But nobody bothered enough to write a +patch add support for this syntax. Some people use scripts to +generated file lists in @file{Makefile.am} or in separate +@file{Makefile} fragments. + +Even if you don't care about portability, and are tempted to use +@code{$(wildcard ...)} anyway because you target only GNU Make, you +should know there are many places where Automake need to know exactly +which files should be processed. As Automake doesn't know how to +expand @code{$(wildcard ...)}, you cannot use it in these places. +@code{$(wildcard ...)} is a blackbox comparable to @code{AC_SUBST}ed +variables as far Automake is concerned. + +You can get warnings about @code{$(wildcard ...}) constructs using the +@code{-Wportability} flag. + +@node distcleancheck, , wildcards, FAQ +@section Files left in build directory after distclean +@cindex distclean, diagnostic +@cindex dependencies and distributed files +@trindex distclean +@trindex distcleancheck + +This is a diagnostic you might encounter while running @code{make +distcheck}. + +As explained in @ref{Dist}, @code{make distcheck} attempts to build +and check your package for errors like this one. + +@code{make distcheck} will perform a @code{VPATH} build of your +package, and then call @code{make distclean}. Files left in the build +directory after @code{make distclean} has run are listed after this +error. + +This diagnostic really covers two kinds of errors: + +@itemize @bullet +@item +files that are forgotten by distclean; +@item +distributed files that are erroneously rebuilt. +@end itemize + +The former left-over files are not distributed, so the fix is to mark +them for cleaning (@pxref{Clean}), this is obvious and doesn't deserve +more explanations. + +The latter bug is not always easy to understand and fix, so let's +proceed with an example. Suppose our package contains a program for +which we want to build a man page using @command{help2man}. GNU +@command{help2man} produces simple manual pages from the @code{--help} +and @code{--version} output of other commands (@pxref{Top, , Overview, +help2man, The Help2man Manual}). Because we don't want our users to +install @command{help2man}, we decide to distribute the generated man +page using the following setup. + +@example +# This Makefile.am is bogus. +bin_PROGRAMS = foo +foo_SOURCES = foo.c +dist_man_MANS = foo.1 + +foo.1: foo$(EXEEXT) + help2man --output=foo.1 ./foo$(EXEEXT) +@end example + +This will effectively distribute the man page. However, +@code{make distcheck} will fail with: + +@example +ERROR: files left in build directory after distclean: +./foo.1 +@end example + +Why was @file{foo.1} rebuilt? Because although distributed, +@file{foo.1} depends on a non-distributed built file: +@file{foo$(EXEEXT)}. @file{foo$(EXEEXT)} is built by the user, so it +will always appear to be newer than the distributed @file{foo.1}. + +@code{make distcheck} caught an inconsistency in our package. Our +intent was to distribute @file{foo.1} so users do not need installing +@command{help2man}, however since this our rule causes this file to be +always rebuilt, users @emph{do} need @command{help2man}. Either we +should ensure that @file{foo.1} is not rebuilt by users, or there is +no point in distributing @file{foo.1}. + +More generally, the rule is that distributed files should never depend +on non-distributed built files. If you distribute something +generated, distribute its sources. + +One way to fix the above example, while still distributing +@file{foo.1} is to not depend on @file{foo$(EXEEXT)}. For instance, +assuming @command{foo --version} and @command{foo --help} do not +change unless @file{foo.c} or @file{configure.ac} change, we could +write the following @file{Makefile.am}: + +@example +bin_PROGRAMS = foo +foo_SOURCES = foo.c +dist_man_MANS = foo.1 + +foo.1: foo.c $(top_srcdir)/configure.ac + $(MAKE) $(AM_MAKEFLAGS) foo$(EXEEXT) + help2man --output=foo.1 ./foo$(EXEEXT) +@end example + +This way, @file{foo.1} will not get rebuilt every time +@file{foo$(EXEEXT)} changes. The @command{make} call makes sure +@file{foo$(EXEEXT)} is up-to-date before @command{help2man}. Another +way to ensure this would be to use separate directories for binaries +and man pages, and set @code{SUBDIRS} so that binaries are built +before man pages. + +We could also decide not to distribute @file{foo.1}. In +this case it's fine to have @file{foo.1} dependent upon +@file{foo$(EXEEXT)}, since both will have to be rebuilt. +However it would be impossible to build the package in a +cross-compilation, because building @file{foo.1} involves +an @emph{execution} of @file{foo$(EXEEXT)}. + +Another context where such errors are common is when distributed files +are built by tools which are built by the package. The pattern is similar: + +@example +distributed-file: built-tools distributed-sources + build-command +@end example + +@noindent +should be changed to + +@example +distributed-file: distributed-sources + $(MAKE) $(AM_MAKEFLAGS) built-tools + build-command +@end example + +@noindent +or you could choose not to distribute @file{distributed-file}, if +cross-compilation does not matter. + +The points made through these examples are worth a summary: + +@cartouche +@itemize +@item +Distributed files should never depend upon non-distributed built +files. +@item +Distributed files should be distributed will all their dependencies. +@item +If a file is @emph{intended} be rebuilt by users, there is no point in +distributing it. +@end itemize +@end cartouche + +@vrindex distcleancheck_listfiles +For desperate cases, it's always possible to disable this check by +setting @code{distcleancheck_listfiles} as documented in @ref{Dist}. +Make sure you do understand the reason why @code{make distcheck} +complains before you do this. @code{distcleancheck_listfiles} is a +way to @emph{hide} errors, not to fix them. There is always better to +do. + @page -@node Macro and Variable Index, General Index, API versioning, Top +@node Macro and Variable Index, General Index, FAQ, Top @unnumbered Macro and Variable Index @printindex vr diff --git a/stamp-vti b/stamp-vti index fbdd1783..250cc3e4 100644 --- a/stamp-vti +++ b/stamp-vti @@ -1,4 +1,4 @@ -@set UPDATED 2 February 2003 +@set UPDATED 9 February 2003 @set UPDATED-MONTH February 2003 @set EDITION 1.7a @set VERSION 1.7a diff --git a/version.texi b/version.texi index fbdd1783..250cc3e4 100644 --- a/version.texi +++ b/version.texi @@ -1,4 +1,4 @@ -@set UPDATED 2 February 2003 +@set UPDATED 9 February 2003 @set UPDATED-MONTH February 2003 @set EDITION 1.7a @set VERSION 1.7a -- 2.43.5