This is the mail archive of the cygwin-apps mailing list for the Cygwin project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Dedup x86/x86_64 --> noarch

Jon Turney writes:
> I think I have implemented the changes to calm to support
> all-or-nothing noarch (i.e. where all packages produced from a source
> package must be noarch), so if you can nominate a suitable,
> unimportant perl package, we can test it with that, initially.

I'll have a look, there are quite a few packages that should have no
arch-dependent components by virtue of them being pure Perl.  In fact,
some of these were noarch while they were still in my private repo since
I kept the noarch alongside like Cygwin Ports has been doing.  You just
want to move them in the repo?

> (This wasn't quite as straightforward as just looking in another
> directory for packages, as the upload validation becomes more complex:
> we must check that consistent package sets result for both x86 and
> x86_64 before we can move noarch packages)

Sure.  For pure noarch this isn't much of a problem (or it never has
been with genini anyway).  Mixed packages might present a more
interesting challenge.  I think however, that if you simply treat
everything in noarch/<srcname>/ the same as <arch>/<srcname> and
coalesce these two directories logically, this problem should go away.

> To make full use of this, cygport upload will need a feature to upload
> noarch packages from dist/ to noarch/ rather than <arch>/.

I haven't looked into that, but I'm not using cygport for the upload

> I don't think this distinguishes between packages which are (or should
> be) marked ARCH="noarch" in the cygport, and those where the build
> products happen to be identical and can be deduped?

No, I just took the archives and ran with it.  In fact most of the big
packages that I had memory problems with would clearly be "noarch" from
the outset (fonts and some of texlive).

> I would guess that this saving is dominated by some very large,
> data-only noarch packages, but who knows?

Thosa packages exists, but the commonalities in source and debuginfo
packages are summarily larger.  The scripting languages provide another
set of common files that yould be split off into a noarch package that
is then a dependency for the arch-dependent stuff.  But that's not
possible to set up right now I take it.

> (Also, looking forward, perhaps cygport needs a separate command to
> build the source package, rather than building it for each arch and
> then deduping it?)

I'm not sure what Yaakov thinks of that, but I really wanted to move the
whole build under a single directory and indeed dedup on the packaging
step (where I can just hardlink in the fs rather than needing to copy in
memory).  I haven't looked into that yet, because the changes necessary
might be quite pervasive.  But for the moment deduping the archives
doesn't look too bad either, so for folks building on different machines
and not sharing the build resources that would be another option
(vs. unpacking the build results from one arch into the build directory
of another).

Another thing that cygport needs is a way to mark sub-packages as noarch
so that the dedup can be skipped for files that are known to be noarch
already (I think their correct designation should still be checked
w.r.t. the actual build results).  Again, I've not yet looked at what
that will need in terms of implementation.

+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]