Thanks for the report.

That macro is defined in m4/threadlib.m4, which is part of the released tarball.  

Could it be that your aclocal, for whatever reason, is not looking in the right directory?

It fails with autoreconf -if.

It seems to me that you have an older version of gnulib installed in the system.  So when you do autoreconf -if it brings in older versions of these .m4 files.  That explains this.

... but I don't have any gnulib installed, and with only -i autopoint still complains about a "locally modified" threadlib.m4... but locally modified respect to what :)

We digged a bit more.

autoreconf -i calls autopoint.
autopoint is in charge of copying in gettext support files.

autopoint looks in a gettext installed file:

 /usr/share/gettext/archive.dir.tar.xz

Which contains, among other things,

 -rw-r--r--    root/root      13567 gettext-0.18/m4/threadlib.m4

Then it tries to copy it over, but complains that the version we have in GNU poke is different.  This makes sense, because it is from a version from a more recent gnulib.

If you pass -f to autoreconf -i, the old threadlib.m4 from gettext replaces our gnulib threadlib.m4, and bang.

Who says that running autoreconf on an arbitrary GNU package's tarball is supported? In general, it is not supported.

> nixpkgs repository used autoreconf on previous release of poke to get support for new aarch64-darwin and other platforms to get recognized by configure.

The tarball now contains the newest config.guess and config.sub. You can ditch that autoreconf invocation.

(In reply to Bruno Haible from comment #5)
> Who says that running autoreconf on an arbitrary GNU package's tarball is
> supported? In general, it is not supported.

If it's unsupported on poke release tarballs then please feel free to close it as INVALID.

Having an equivalent procedure would be nice though. Downstreams have to patch m4 bits from time to time.

And optionally add an explicit error message if it's easy in configure.ac so people would not repeat the mistake again and again when next platform appears and they try `autoreconf -if` again.

> > nixpkgs repository used autoreconf on previous release of poke to get support for new aarch64-darwin and other platforms to get recognized by configure.
> 
> The tarball now contains the newest config.guess and config.sub. You can
> ditch that autoreconf invocation.

Sounds good. That is my current attempt.

(In reply to Sergei Trofimovich from comment #6)
> (In reply to Bruno Haible from comment #5)
> > Who says that running autoreconf on an arbitrary GNU package's tarball is
> > supported? In general, it is not supported.
> 
> If it's unsupported on poke release tarballs then please feel free to close
> it as INVALID.
> 
> Having an equivalent procedure would be nice though. Downstreams have to
> patch m4 bits from time to time.
> 
> And optionally add an explicit error message if it's easy in configure.ac so
> people would not repeat the mistake again and again when next platform
> appears and they try `autoreconf -if` again.
> 
> > > nixpkgs repository used autoreconf on previous release of poke to get support for new aarch64-darwin and other platforms to get recognized by configure.
> > 
> > The tarball now contains the newest config.guess and config.sub. You can
> > ditch that autoreconf invocation.
> 
> Sounds good. That is my current attempt.

If you can affect nixpkgs, I suggest adopting a practice of not regenerating build system files in autotools projects, and, if some are patched (which is often necessary, and completely understandable) only re-run the appropriate tools (the autoconf manual has a nifty, though incomplete, diagram about what produces what: https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/autoconf#Making-configure-Scripts).
Of course, if building from Git snapshots, you ought to run ``./bootstrap''.

You can safely bump config.{status,guess}, as both are just invocable programs that configure scripts run, and do not get inlined in any other files.

(In reply to Arsen Arsenović from comment #7)
> (In reply to Sergei Trofimovich from comment #6)
> > (In reply to Bruno Haible from comment #5)
> > > Who says that running autoreconf on an arbitrary GNU package's tarball is
> > > supported? In general, it is not supported.
> > 
> > If it's unsupported on poke release tarballs then please feel free to close
> > it as INVALID.
> > 
> > Having an equivalent procedure would be nice though. Downstreams have to
> > patch m4 bits from time to time.
> > 
> > And optionally add an explicit error message if it's easy in configure.ac so
> > people would not repeat the mistake again and again when next platform
> > appears and they try `autoreconf -if` again.
> > 
> > > > nixpkgs repository used autoreconf on previous release of poke to get support for new aarch64-darwin and other platforms to get recognized by configure.
> > > 
> > > The tarball now contains the newest config.guess and config.sub. You can
> > > ditch that autoreconf invocation.
> > 
> > Sounds good. That is my current attempt.
> 
> If you can affect nixpkgs, I suggest adopting a practice of not regenerating
> build system files in autotools projects, and, if some are patched (which is
> often necessary, and completely understandable) only re-run the appropriate
> tools (the autoconf manual has a nifty, though incomplete, diagram about
> what produces what:
> https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/
> autoconf#Making-configure-Scripts).

It's not clear to me: are you talking about poke specifically or in for autoconf-based build systems in general? In general I think I have some basic understanding how autoconf-based build systems are updated for simpler cases: I converted a few in the past from complex './bootstrap' scripts to be just 'autoreconf -if'. Specifically to support easy reconfiguration for release tarballs. Obviously 'poke' does something more complex that does not make it to Just Work.

> Of course, if building from Git snapshots, you ought to run ``./bootstrap''.

Not sure if it is relevant here. I'd prefer a release tarball not to rely on internet access to regenerate the build system to apply local patch as long as we have needed dependencies present. `./bootstrap` is not present in the release tarball either, thus it's probably not the way to go here.

> You can safely bump config.{status,guess}, as both are just invocable
> programs that configure scripts run, and do not get inlined in any other
> files.

Is it an advice for this specific case or for general case? I think specific case is not as relevant as files are fresh enough.

For general case your link says that 'autoreconf' is the tool that refreshes those files unless I'm misreading it. I'd rather not reimplement subset of autoconf, automake, or local .m4 macro logic by tracking what needs to be updated for individual files, to maintain file timestamps to avoid build system's regeneration loops and whatnot.

I did not get the clear answer above what is the supported way to regenerate build system in the tarball after the change. #comment5 implies autoreconf is not supported (and it is very surprising to me). I assume anything similar to that process is equally unsupported: be it 'autoconf' run, 'autoheader' regeneration or subset of file copies.

(In reply to Sergei Trofimovich from comment #8)
> (In reply to Arsen Arsenović from comment #7)
> > (In reply to Sergei Trofimovich from comment #6)
> > > (In reply to Bruno Haible from comment #5)
> > > > Who says that running autoreconf on an arbitrary GNU package's tarball is
> > > > supported? In general, it is not supported.
> > > 
> > > If it's unsupported on poke release tarballs then please feel free to close
> > > it as INVALID.
> > > 
> > > Having an equivalent procedure would be nice though. Downstreams have to
> > > patch m4 bits from time to time.
> > > 
> > > And optionally add an explicit error message if it's easy in configure.ac so
> > > people would not repeat the mistake again and again when next platform
> > > appears and they try `autoreconf -if` again.
> > > 
> > > > > nixpkgs repository used autoreconf on previous release of poke to get support for new aarch64-darwin and other platforms to get recognized by configure.
> > > > 
> > > > The tarball now contains the newest config.guess and config.sub. You can
> > > > ditch that autoreconf invocation.
> > > 
> > > Sounds good. That is my current attempt.
> > 
> > If you can affect nixpkgs, I suggest adopting a practice of not regenerating
> > build system files in autotools projects, and, if some are patched (which is
> > often necessary, and completely understandable) only re-run the appropriate
> > tools (the autoconf manual has a nifty, though incomplete, diagram about
> > what produces what:
> > https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/
> > autoconf#Making-configure-Scripts).
> 
> It's not clear to me: are you talking about poke specifically or in for
> autoconf-based build systems in general? In general I think I have some
> basic understanding how autoconf-based build systems are updated for simpler
> cases: I converted a few in the past from complex './bootstrap' scripts to
> be just 'autoreconf -if'. Specifically to support easy reconfiguration for
> release tarballs. Obviously 'poke' does something more complex that does not
> make it to Just Work.
I mean generally.  I'm not sure what exactly poke does that caused this error, but it's related to autopoint.  If autopoint isn't invoked, there's no break here.  Other such instances can easily happen when ``autoreconf -f'' is invoked, which is also why bootstrap/autogen.sh/autopull.sh are distributed in Git versions of various programs.

My (yet unconfirmed) suspicion is that threadlib.m4 that's pulled in via Gnulib has macros that the one distributed in Gettext (and that autopoint pulls in) doesn't, but autopoint notices that threadlib.m4 isn't what it would expect, and hence, tries to update it.  The now "updated" threadlib.m4 lacks macros that configure invokes, and so, breakage happens.

> > Of course, if building from Git snapshots, you ought to run ``./bootstrap''.
> 
> Not sure if it is relevant here. I'd prefer a release tarball not to rely on
> internet access to regenerate the build system to apply local patch as long
> as we have needed dependencies present. `./bootstrap` is not present in the
> release tarball either, thus it's probably not the way to go here.
Release tarballs do not need the network nor ``./bootstrap'', I was talking about Git clones.  Apologies - should've been clearer.

> > You can safely bump config.{status,guess}, as both are just invocable
> > programs that configure scripts run, and do not get inlined in any other
> > files.
> 
> Is it an advice for this specific case or for general case? I think specific
> case is not as relevant as files are fresh enough.
General case.  Gentoo automatically bumps config.{guess,sub}, for instance, which has only caused one problem so far to my awareness (in GMP, which wraps them with a custom script for detecting CPU features).

> For general case your link says that 'autoreconf' is the tool that refreshes
> those files unless I'm misreading it. I'd rather not reimplement subset of
> autoconf, automake, or local .m4 macro logic by tracking what needs to be
> updated for individual files, to maintain file timestamps to avoid build
> system's regeneration loops and whatnot.
> 
> I did not get the clear answer above what is the supported way to regenerate
> build system in the tarball after the change. #comment5 implies autoreconf
> is not supported (and it is very surprising to me). I assume anything
> similar to that process is equally unsupported: be it 'autoconf' run,
> 'autoheader' regeneration or subset of file copies.
``autoreconf -f'' specifically has a tendency to be overzealous.  In combination with -i, it caused some .m4s to be incorrectly overwritten, and hence caused the error above.  This happened as a result of autoreconf automatically invoking autopoint.  

As a test, I tried running autoreconf of various forms on fresh extracts of the Poke tarball.  Only the -fi case broke the build system, and the -i case stopped when autopoint noticed the difference, and refused to continue.

(continuation of my speculation above) ``autoreconf'' is unaware if the fact that somewhere in its steps, gnulib-tool is supposed to be invoked in order to generate the correct threadlib.m4, and so, it didn't invoke it.

Lets take a hypothetical of editing libpoke/Makefile.am because pvm-vm2.c breaks with some compiler flag or version, so you want to override the flags on it.  In that case, only the libpoke/Makefile.in is out of date, and so, you should re-run automake (without -f or -i should be fine too).  Similar logic applies for other files.

From what I'm hearing we can conclude that files are around and original description is misleading. From the pushback I gather 'autoreconf -if' is not supported and we'll need to resort to other hacks next time. Closing as INVALID. Apologies for the noise.