Source retrieval
Enrico Weigelt
weigelt@metux.de
Sun May 25 15:26:00 GMT 2008
* Yann E. MORIN <yann.morin.1998@anciens.enib.fr> wrote:
Hi,
> The idea sounds interesting...
:)
> > Example:
> > http://sourcefarm.metux.de/robot/query-download-url/?package=binutils&version=2.18.0.0&encoding=tar.gz
>
> Well, here are a few suggestions:
> - default is to redirect to a randomly-picked URL from the list of mirrors
It already does :)
If you only want the URL instead of downloading, just do a raw HTTP request
(w/o following the redirect) and parse the reply header.
I'm also planning another bot which prints out a list of URLs.
(eg. if you want to walk through the whole list, in case of URLs fail)
> - allow retrieval of the canonical URL
The http redirector provides that canonical URL.
(maybe it could look a bit more convenient ;-o)
Clients that are interested in an canonical URL should only use this one,
and the actual locations behind are out of scope.
> - allow retrieval of the MD5sum for the package
Yep, already prepared that.
But: the scanner needs some way to get that MD5. Most projects provide them themselves,
so we'd first need to download all the tarballs (-> *HUGE* traffic :().
> - allow retrieval of the snapshots or pre-releases
Ugh, that's one thing I explicitly left out when designing CSDB - it should only
contain releases. But snaphots aren't releases and don't really fit into the
normalized versionspace. An possible solution could be giving them separate
package names (eg. w/ special prefix/suffix).
> Which would give the following API (only a suggestion):
>
> http://..../query-download-url/ : retrieve a package tarball. Parameters:
> - package, mandatory: the package name
> - version, mandatory: the package version
> - encoding, mandatory, the type of extension to retrieve, from the following
> list: {tar,tar.gz,tar.bz2} (others?)
Actually, it's not the filename suffix, but the encoding. By convention there's
exactly one name per encoding, eg. for *.tbz2 archives "tar.bz2" is used.
> - canon, optional: if not set, or set to 'no', will choose a URL amongst the
> mirrors; if set to 'yes', will redirect to the canonical URL only; if set
> to anything other than 'yes' or 'no', will invalidate the query.
I'd prefer to treat all mirrors equal (perhaps with an optionally weight).
Maybe we could add some geographical/topological information, so more intelligent
clients can choose the nearest mirror.
> New URL:
> http://..../query-md5-url/ : retrieve a package MD5 sum. Parameters:
> - the sames as for the above, having the same meaning.
ACK. That would be fine (as long as we *have* these information).
> > I could mirror it on some of my boxes and add a record to CSDB :)
>
> I guess you and I have the same problem. If our mirrors get used, then the
> bandwidth usage will rise to the point where we'd have almost no bandwidth
> left for our own use...
Right, so we should get more mirrors.
I'll try to aquire some spare resources from some of my customers.
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
--
For unsubscribe information see http://sourceware.org/lists.html#faq
More information about the crossgcc
mailing list