Re: broken setup.hint files

[Charles Wilson <cwilson at ece dot gatech dot edu>]

Jan Nieuwenhuizen wrote:


>>Ummm...they are not broken.  You just dislike some stylistic choices...
>>
> 
> Well, setup.exe barfs on the.  Quoting from inilex.l:
> 
>    "sdesc:"		return SDESC;
>    "ldesc:"		return LDESC;
>    "category:"		return CATEGORY;
>    "requires:"		return REQUIRES;
> 
> So, after reading this and the setup.hint spec on cygwin.com, I
> implemented hinting, in gen-ini.sh and it *broke* setup.exe.  So, I
> considered it a bug, and wanted you to know about it.  But if you
> don't care, fine.


You are right: the *setup.ini* grammar as parsed by *setup.exe* requires 
  colons.  However, setup.ini is created on sourceware by a script 
called "upset".  And, the *setup.hint* grammar as parsed by upset 
doesn't require colons.


>>Also, blindly removing the prev/curr/test markers is not a good thing 
>>either.
>>
> 
> For me, it's *a lot* better than a barfing setup.exe.
> 
> 
>>Sometimes (when upset's automatic version parser fails)
>>
> 
> Who is `upset'?  


Ah -- upset is the setup.hint parser that creates setup.ini.  You seem 
to have written your own setup-ini builder and called it 'gen-ini.sh'. 
I am not going to "fix" my setup.hint files to make your parser happy -- 
the only parser I care about is the REAL one, upset.

> I haven't seen my version parser fail, but in general
> one should not provide the same information from two sources.  Which
> do you trust when they do not match?  Why not just have a sane
> archive, or fix setup.ini by hand if you don't like it?


Because us normal maintainers CANNOT edit setup.ini!  Only cgf can do 
that.  The *primary* information source is setup.hint -- therefore, if I 
need to override the version parser, it must be done in the setup.hint. 
  Also, us normal cygwin maintainers CANNOT force the upstream 
maintainers to use a sane naming scheme (see below).  And when upset 
creates setup.ini, it first uses the filenames of the archives -- but 
then setup.hint can be used to OVERRIDE it when necessary.

Here's a (real life) example:
openssh-2.9p2-3   (released on Jul 26, 2001)
openssh-2.9.9p2-1 (released on Sep 28, 2001)

if you sort this using normal (computer) lexical sorting, you end up with:

prev: openssh-2.9.9p2-1
curr: openssh-2.9p2-3

Which is clearly wrong -- and the fault lies not with Corinna, but with 
the OpenSSH group.

the new *recommendation* is to NOT specify curr/prev in setup.hint and 
rely on upset's parsing of archive names -- unless you need to override 
(cf. openssh above).  but it isn't a requirement,and not following the 
recommendation doesn't result in broken setup.ini's -- at least, when 
the REAL setup.ini generator (upset) is used.  If your parser doesn't 
work, that's not my problem.


> If a version parser fails, the offending package (filename) should be
> fixed, imo.


But sometimes, we don't have that luxury.  Also, since experimental 
packages -- or "real" packages based off of cvs -- are versioned like

foo-20010531 and not foo-1.3.5

What happens when you "graduate" from a cvs-based, dated version to a 
'real release' version?  2001xxxx is always > x.y.z

And, there is NO way for us normal maintainers to specify a test release 
other than specifying it in the hint.

If your parser can't handle that, your parser is broken.  Not my setup.hint.


>>the hints may just be old.
>>
> 
> Yes, I guess that they're old.  If old things don't get fixed, the get
> bitrot.


No, if old things are BROKEN, then they need to be fixed.  Since they 
are NOT broken, there is no need to rush out and repackage and "fix" 
everything.  I *will* update to the latest *RECOMMENDATION* -- but 
because it is a recommendation, and because nothing official is broken, 
I'm not going to rush it.  The only thing broken here is your gen-ini.sh 
script.

>>In my case, I will update a given setup.hint to follow that new 
>>*recommendation* (not requirement) the next time I update the package 
>>controlled by it, and not until then.
>>
> 
> Indeed, parts of the cygwin archive are a bit of a mess, but it's too
> bad if it's by principle, and not for want of time.


It's not principle or want of time.  AFAICT, there is no mess.  What 
specifically are you complaining about?  setup.hints that YOUR parser 
can't understand?  Again, not my problem.

 
>>If it ain't broken (and it ain't) then don't fix it.
>>
> 
> Well, sorry to bother you then.  I can keep kludging around this small
> thing too.


How about don't "kludge around it" -- try fixing your parser.  Here is 
the precursor to cgf's current upset.  I haven't used it in a while, but 
it ought to still work.

--Chuck

#!/usr/bin/perl
# -*- perl -*-

$ftptop = "/sourceware/ftp/anonftp/pub/cygwin";
if (@ARGV > 0) {
    $ftptop = shift;
}

$setupdirs = "contrib latest";
if (@ARGV > 0) {
    $setupdirs .= " " ."@ARGV"
}

$tmpfile = "/tmp/setup.ini.tmp";

open (TMPFILE, ">$tmpfile");
select (TMPFILE);

chdir $ftptop;

$setup_version = '';
open (S, "setup.exe");
while (<S>) {
    if (/%%% setup-version (\S+)/) {
        $setup_version = $1;
        last;
    }
}
close S;

open (F, "find $setupdirs -name '*.tar.[bg]z*' -print |");

while (<F>) {
    $file = $_;
    $file =~ s/[\r\n]+$//;
    ($subdir, $package) = $file =~ m@(.*)/(.*).tar.(gz|bz2)@;
    $src = '';
    if ($package =~ /-src$/) {
        $package =~ s/-src$//;
        $src = 'src';
    }
    $version = "-";
    if ($package =~ /-[0-9]/) {
        ($package, $version) = $package =~ m@(.*?)-([0-9].*)@;
    }

    #print "p=$package v=$version f=$file\n";
    if (!$saw_version{"$package $version"}) {
        $versions{$package} .= " " if $versions{$package};
        $versions{$package} .= $version;
        $saw_version{"$package $version"} = 1;
    }
    if ($src) {
        $source{"$package $version"} = $file;
    } else {
        $install{"$package $version"} = $file;
    }
}

close (F);

sub vsort {
    my ($a,$b) = @_;
    $a =~ s/(\d+)/sprintf("%08d", $1)/ge;
    $b =~ s/(\d+)/sprintf("%08d", $1)/ge;
    return $a cmp $b;
}

# for $v (sort {&vsort($a,$b)} keys %versions) {

sub one_file {
    my ($leader, $file) = @_;
    return unless $file;
    $sz = (stat($file))[7];
    print $leader, $file, " ", $sz, "\n";
}

print <<EOF;
# This file is automatically generated.  If you edit it, your
# edits will be discarded next time the file is generated.
# See http://cygwin.com/setup.html for details.
#
EOF

$now = time;
print "setup-timestamp: $now\n";
print "setup-version: $setup_version\n" if $setup_version;
print "\n";

sub push_version {
    my ($best, $next, $v) = @_;
    #print "push_versions $p $v\n";
    if (&vsort($type{"$p $best"}, $v) < 0) {
        $old = $type{"$p $best"};
        $type{"$p $best"} = $v;
        $v = $old;
    }
    if (&vsort($type{"$p $next"}, $v) < 0) {
        $type{"$p $next"} = $v;
    }
}

for $p (sort keys %versions) {
    for $v (split(' ', $versions{$p})) {
        if ($v =~ /\d\d\d\d\d\d/) {
            &push_version("test", "junk", $v);
        } else {
            &push_version("curr", "prev", $v);
        }
    }
    if ( ! $type{"$p prev"} && $type{"$p junk"}) {
        $type{"$p prev"} = $type{"$p junk"};
    }
    if ( ! $type{"$p curr"} && $type{"$p test"}) {
        $type{"$p curr"} = $type{"$p test"};
        $type{"$p test"} = undef;
    }
}

# Now look for overrides
open (F, "find $setupdirs -name setup.hint -print |");
while (<F>) {
    s/[\r\n]+$//;
    ($package) = m@.*/(.*)/setup\.hint$@;
    %tmp = ();
    $valid = 0;

    open (S, $_);
    while (<S>) {
        s/[\r\n]+$//;
        ($type, $version) = split(' ', $_, 2);
        if ($saw_version{"$package $version"}
            && $type =~ /^(prev|curr|test)$/) {
            $tmp{$type} = $version;
            $valid = 1;
        }
        if ($type eq "skip") {
            $skip{$package} = 1;
        }
        if ($type eq "sdesc") {
            $sdesc{$package} = $version;
        }
        if ($type eq "ldesc") {
            $ldesc{$package} = $version;
        }
    }
    close (S);

    if ($valid) {
        undef $type{"$package prev"};
        undef $type{"$package curr"};
        undef $type{"$package test"};
        for $t (keys %tmp) {
            $v = $tmp{$t};
            $type{"$package $t"} = $v;
        }
    }
}

sub one_version {
    my ($print, $type) = @_;
    $v = $type{"$p $type"};
    if ($install{"$p $v"} || $source{"$p $v"}) {
        print $print;
        print "version: $v\n" if $v ne "-";
        &one_file ("install: ", $install{"$p $v"});
        &one_file ("source:  ", $source{"$p $v"});
    }
}

for $p (sort keys %versions) {
    next if $skip{$p};
    print "@ $p\n";
    print "sdesc: ", quoted($sdesc{$p}), "\n" if $sdesc{$p};
    print "ldesc: ", quoted($ldesc{$p}), "\n" if $ldesc{$p};
    &one_version("", "curr");
    &one_version("[prev]\n", "prev");
    &one_version("[test]\n", "test");
    print "\n";
}

select (STDOUT);
close (TMPFILE);

open(NEW, $tmpfile);
open(OLD, "setup.ini");

$changed = 0;
while (1) {
    $new = <NEW>;
    $old = <OLD>;
    next if ($new =~ /setup-timestamp/ && $old =~ /setup-timestamp/);
    last if (! defined $new && ! defined $old);
    if ($new ne $old) {
        $changed = 1;
        last;
    }
}
close (NEW);
close (OLD);

if ($changed) {
    system "cp $tmpfile setup.ini";
}
unlink $tmpfile;

sub quoted {
    my $text = shift;
    return $text =~ /"/ ? $text : '"' . $text . '"';
}