noarching source packages

[Jon Turney]

On 27/04/2017 18:53, Achim Gratz wrote:
> Jon Turney writes:
>> Picking up the discussion from [1], I've been looking a bit at
>> noarching the source packages.
>>
>> So, the first problem is that we don't really have source packages.
>
> I'll use this occasion to raise the topic of the debuginfo packages
> again.  I still think we should change their naming convention (or
> alternatively the naming convention for the source packages) and a large

What is your reason for changing the name?

I was wondering if we need to explicitly identify debuginfo archives as 
a different kind of thing.  Currently, debuginfo packages work just like 
any other install archive, which is fine, except for perhaps they need a 
separate filter in setup.

> part of them is made up again of the source files, which should be
> separated out into noarch also.

Nice idea, but the practicalities seem complex (e.g. generated source 
files needs to be treated correctly). In any case, this would seem to be 
a piece of work which falls after noarching the sources.

>> calm would need updating to look for packages in src/ as well as
>> noarch/ and <arch>/, and to emit 'Source:' rather than 'source:' lines
>> in setup.ini when the source is an actual source package.
>
> I'd be hesitant to use yet another tree for this.  We already have way
> too many directories that make up the repo.

'too many'? why?

>> It's not quite clear how to deal with making source packages.  If we
>> do it when we make the binary package (as now), then there is the near
>> certainly that the source package made for a different arch will
>> differ, gratuitously.
>
> The only sane way is to mandate that the packages for all arches are
> built together so that you can package the sources only once during the
> packaging step.  Otherwise you either have to check that the contents

That would seem to require a cross-compilation environment for at least 
one cygwin arch, with all the dependencies available.

> (ignoring the metadata that _will_ differ) is identical between the
> source archives you've built seperately and then chose one of those for
> upload or you'll have to force a reproducible build of the source
> archive at least.
>
>> This also potentially loses information, as the maintainer might
>> adjust the .cygport to build on the 2nd architecture they try, but
>> those changes wouldn't be uploaded, (whereas currently the source
>> actually used for the build is uploaded)
>
> It's easy enough to branch that decision inside the cygport file and the
> only time I did that have passed now that the package content in both
> arches is almost identical.  So is anybody really doing that currently?

At the moment, nothing prevents SRC_URI and PATCH_URI depending on the 
ARCH, so we just don't know.

But this is more a question of workflow: nothing stops the maintainer 
going back and changing the source package, then just rebuilding one 
architecture.

The ideal solution would be a build service which accepts a source 
package and produces the install archives, but I don't see that 
happening anytime soon...

> But the real problem is that besides our own stuff some upstream sources
> are archful.

Examples?

>> Applied retroactively, it looks like this would save about 13G (out of
>> a total mirror size of approximately 97G), but it seems that there are
>> many source packages which (usually spuriously) differ between arches,
>> so that saving wouldn't be immediately realized.
>
> From my last dedup exercise (where my local Cygwin repo was around 80GB
> since I don't mirror some of the cross-compilation and KDE packages)
> doing the dedup on just the source and doc packages reduced the size of
> the repo by 30GB.  I'll note again that if it was possible to split off
> the noarch part of _all_ packages the gains would be larger than that.
> The way it would work is that setup.exe should accept both noarch and
> arch archives for the same package.  It would then proceed to first
> install the noarch and then the arch part if it finds both of them.
> Incidentally, this would keep the current tree structure intact and
> allow us to freely move packages from arch to noarch and vice versa
> between different releases with no manual intervention.

Great.  I look forward to reading the patches :)