(new) pcre package for consideration

[Ronald Landheer-Cieslak]

Long mail.. comments in-line anyway..

I've broken it a bit into pieces (actually - I've just added some titles) 
to make it slightly more readable (for me, mostly).

On Mon, 5 May 2003, Charles Wilson wrote:
> Ronald Landheer-Cieslak wrote:
> Looks like a typo...
> 
> >>>ran cygcheck against cygpcreposix.dll and got:
That is a typo, it was run against cygpcreposix-0.dll

> >>>$ cygcheck cygpcreposix-0.dll 
> >>>Found: .\cygpcreposix-0.dll
> >>>cygpcreposix-0.dll
> >>>  D:\cygwin\bin\cygwin1.dll
> >>>    C:\WINNT\System32\KERNEL32.dll
> >>>      C:\WINNT\System32\ntdll.dll
> >>>  D:\cygwin\bin\cygpcre.dll
> 
> ^ no -0
> ALSO a typo? Or accurate transcription of the command output?
No typo: copy & paste.

> Given your reply below, I guess the first mismatch is a typo, and the
> second instance is NOT a typo.  If so, then you do have a problem.
In deed I do :|

> > The way I read these dependencies, cygpcreposix-0.dll depends on
> > cygpcre.dll and cygwin1.dll (which ...). cygpcre.dll will not exist
> > after installation - cygpcre-0.dll will.
> Well, if that's the case, then you have a packaging problem. 
In deed I do..

> You need to repackage things as described below, so as to avoid breaking
> other people's binaries that currently depend on cygpcre.dll.  However,
> the procedure below, while necessary, is not sufficient to solve the "my
> pcreposix dll is linking to the wrong pcre dll" problem.  See WAY below,
> for that.

Packaging procedure
------------------- 
> Create 'semi-new' pcre packages contain EXACTLY the same binaries as in 
> the current, official pcre (or, rebuilt binaries built using the OLD 
> non-libtool method, so that the semi-new binaries "match" the existing 
> ones).  It's probaby safest just to take the existing pcre package and 
> manually split up the binaries into multiple tarballs.  That way you're 
> assured drop-in compatibility.
>
> Put the dlls, cygpcre.dll + cygpcreposix.dll, into their own 'libpcre' 
> package.  Put everything else into the existing 'pcre' package.  The 
> pcre package should depend on 'libpcre'.  You could also split the 
> .la/.a/.dll.a and .h files into libpcre-devel, but that's probably overkill.
> Release those new packages.  They should be 'drop-in' replacements for 
> the existing pcre stuff.
> 
> Let simmer for approx 2 weeks.
> 
> Then, take your NEW libtool-enabled version of pcre, and put ITS 
> cygpcre-0.dll and cygpcreposix-0.dll into a NEW package, 'libpcre0' 
> (matching the '-0' version number on the DLLs).  Again, everything else 
> goes into the pcre (NOT pcre0) package.  If you chose, above, to split 
> the development stuff into a separate (e.g. third) -devel package, then 
> do so again here -- but it should be libpcre-devel NOT libpcre0-devel.
> 
> Then, release these new packages.
> 
> End result, your users will be able to have installed
> 
> libpcre (containing cygpcre.dll + cygpcreposix.dll)
> libpcre0 (continaing cygpcre-0.dll + cygpcreposix-0.dll)
> libpcre-devel (maybe. Contains devel stuff for libpcre0)
> pcre (contains everything else)
> 
> Thus, the "unversioned" pcre dlls can be installed alongside the new 
> stuff, and carried forward to support existing apps that use them. 
> HOWEVER, since the pcre (+ libpcre-devel?) packages contain ONLY stuff 
> for the new cygpcre-0 libraries, all NEW builds will properly link 
> against the NEW libraries.  Good.
I think this is slighty over-complicated, but only slightly.

A bzipped tarball with cygpcre.dll, cygpcre-0.dll, cygpcreposix.dll and 
cygpcreposix-0.dll is only 7 K larger than a bzipped tarball with only the 
unversioned DLLs. IOW, two seconds worth of downloading on a 4K/s modem. 
Putting both in a single tarball will make the libpcre0 package 
unnecessary. The binaries (pcretest, pcregrep) will be linked with the 
versioned versions, the devel files (good idea, IMHO) will link with the 
versioned versions as well, and the unversioned versions will disappear 
(with lots of flashing light in the announcement message, of course) after 
a couple of weeks.

That will allow the various users of pcre to catch up without breaking 
anything, and with the small size of pcre, we can keep the unversioned 
versions around quite a while if we want to..

> -------------------------------------
> Now, about the "Oops I linked it again" problem...
> 
> >>So I don't see what "problem" you're trying to fix.
> > 
> > I'm not trying to fix anything, I'm trying to avoid having to: I want
> > to be sure that the mere name of the cygpcre.dll file won't matter. It
> > usually does, and I think it might in this case as cygpcre-0.dll is in
> > the same dir as cygpcreposix-0.dll, and should thus be found when
> > looking for the DLL before the installed cygcpre.dll is (that is,
> > AFAIK, the way Windows resolves this).
> > Also, if I remove or rename cygpcre.dll (the installed one)  
> > cygpcreposix-0.dll has a broken dependency. I want to be sure that
> > doesn't happen if the libraries are installed.
> The problem, as you stated 3 emails ago but I missed, is that
> cygpcreposix-0.dll is being linked against cygpcre.dll NOT
> cygpcre-0.dll.  I'm starting to lose track, but I *thought* my patch
> fixed that.  I'll try again...

> >>libtool --finish simply fixes up some stuff in libpcre.la and possible
> >>relinks; on windows, it appears that it is almost never necessary.
> > That's great, but it seems that removing my old pcre installation from
> > Cygwin and replacing it with the one I made breaks anything that
> > depends on pcre - and it will stay broken when recompiled against
> > pcreposix.
> Again, because your new cygpcreposix-0.dll is fubared.
That's what I thought.

The thing is: because I was the onle one seeing the problem until now, I 
was beginning to think it might be a cockpit error "chez moi"..

> > For example: I've uninstalled pcre - on which grep depends - and put
> > pcre-4.2-1 in its place. grep is still broken (complains about not
> > finding a DLL), which would mean that by installing pcre-4.2-1 grep
> > will be broken if not relinked.
> Yes, this is the problem addressed by the packaging reorg outlined 
> above. 
I was kinda hopinh to avoid the package jugling (how do you write that 
anyway?) but I do agree it's a better idea to use Libtool if we can..

[snipped anecdote]

> > I don't know which packages use pcreposix, but compiling and linking a
> > very simple program against pcreposix will result in an executable
> > that can't run because cygcpre.dll isn't there.
> 
> ASSUMING you fix the "Oops my cygpcreposix-0.dll is linked wrong"  
> problem, the procedure described above means that you don't HAVE to know
> who is using your DLLs.  Although it's sometimes nice to do the
> following:
> 
> When splitting your package (e.g. gdbm-1.8.0-4 monolithic to compatible 
> 1.8.0-5 packages), search the setup.hints in the release heirarchy on 
> cygwin mirrors for 'requires" lines that include the monolithic package 
> ('gdbm' == 'pcre').   Send an email to the cygwin-apps mailing list, 
> like this one:
> 
> "ATTN: Apache, Perl, Python, and Exim maintainers"
> http://www.cygwin.com/ml/cygwin-apps/2003-03/msg00675.html
I'll do that.

[snipped (buried) the dead horse]

> > So, what I'd like to know is if there are any more things I may have
> > overlooked in the use of libtool, now that Gerrit's and your patches
> > have been applied, or whether this is a "doesn't work for me but works
> > for you"  type of situation.
> 
> Okay, now down to the brass tacks: why is cygpcreposix-0.dll getting
> linked to cygpcre.dll and not to cygpcre-0.dll.
> 
> Okay, after the build (but before 'pcre-4.2-1.sh install), I looked in 
> <srcdir>/.build/.libs:
> 
> $ cygcheck.exe cygpcreposix-0.dll
> Found: .\cygpcreposix-0.dll
> cygpcreposix-0.dll
>    .\cygpcre-0.dll
>      D:\cygwin\bin\cygwin1.dll
>        D:\WINNT\System32\KERNEL32.dll
>          D:\WINNT\System32\ntdll.dll
> 
> So, cygpcreposix-0.dll IS properly linked to cygpcre-0.dll.
I found that too, but..

> Next, I did the 'pcre-4.2-1.sh install' and looked in 
> <srcdir>/.inst/usr/bin.  And whaddaya know:
> 
> $ cygcheck cygpcreposix-0.dll
> Found: .\cygpcreposix-0.dll
> cygpcreposix-0.dll
>    D:\cygwin\bin\cygwin1.dll
>      D:\WINNT\System32\KERNEL32.dll
>        D:\WINNT\System32\ntdll.dll
>    D:\cygwin\bin\cygpcre.dll
I found that too :\

At least I'm not hallucinating - ye never can tell: I'm Dutch ;)

> While the relink of the executables didn't break them (e.g. the relinked
> .exe's still want cygpcre-0.dll NOT cygpcre.dll [*]), the relink of the
> posix dll DID break it.  Why?
> 
> [*] except in the case of pcretest -- which ITSELF links to 
> cygpcre-0.dll, but also to the newly broken cygpcreposix-0.dll which 
> wants cygpcre.dll.  So, pcretest now wants -0.dll directly, but .dll 
> indirectly == badness.
> 
> The problem seems to be that the relink cmd that is generated by libtool 
> contains this:
>     -L/usr/lib -L/tmp/pcre/pcre-4.2/.inst/usr/lib -lpcre
> Which measn that /usr/lib is searched before $inst_prefix_dir/usr/lib.
> 
> This is generated internal to libtool; it looks like a libtool bug to 
> me.  I'll try to put together a small testcase and submit it to the 
> libtool mailing list.
> 
> Now, on cygwin, I *think* that relinking is really unnecessary.  So, the 
> workaround -- until libtool gets fixed and 1.5.1 is released -- would 
> probably be something like the following HACK.
> 
> Modify the pcre-4.2-1.sh script so that at the end of build() [ or at 
> the beginning of install() ], you explicitly make a "backup copy" of 
> .build/.libs/cygpcreposix-0.dll.   (Yes, this is a total hack).  Then, 
> at the end of install(), you replace the newly installed (and newly 
> broken) cygpcreposix-0.dll with the saved "good" copy.
> 
> [Ack ach, cough cough. choke]  I know, I know, it's ugly -- but I really 
> think this is a libtool bug, and it might be a while before it gets 
> fixed since we'll want to wait for the 1.5.1 official release.
> 
> Sorry that I didn't have better news.
Actually, you showed that I'm not insane, which is nice to know ;)

I don't much like the hack, but if it's the only thing to do to work 
around this libtool bug, and if libtool 1.5.1 will not contain this bug, 
I'll just put it in a chunck in the patch I can later remove..

> --Chuck
> P.S. in the buildscript in prep(), after applying the patch, you should
>    'chmod +x ${srcdir}/reconf-cygwin.sh'
OK

> P.P.S. libtool no longer distributes automake files 'missing' 
> 'install-sh' or 'mkinstalldirs'.  in the the reconf-cygwin.sh script, 
> you should copy them from /usr/autotool/devel/share/automake-1.7/
OK

Thanks, Chuck, for your help.

I'll make a new version of pcre available asap (tomorow, probably).

Personally, because pcre is so very small, I think it's not really worth 
it to break it up into three (or more) packages: a single pcre package 
with two versions of the DLLs will increase download time by two seconds 
for people using modems (does anyone use less that a 4K/s modem nowadays?) 
Asking the cygwin@ list about it would create more trafic than two 
separate packages will spare, and it's even more transparent to the end 
user not splitting it up..

So, if nobody protests loudly, there'll be a single pcre package with two 
versions of each DLL in it for a while (either until the next release of 
pcre, or until the next full moon, whichever comes first ;)

rlc