Bug 29769 - Feature proposal: cgit access rather than solely gitweb
Summary: Feature proposal: cgit access rather than solely gitweb
Status: RESOLVED FIXED
Alias: None
Product: sourceware
Classification: Unclassified
Component: Infrastructure (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: overseers mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-11-09 20:33 UTC by Arsen Arsenović
Modified: 2023-07-24 20:35 UTC (History)
4 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Arsen Arsenović 2022-11-09 20:33:00 UTC
$summary

cgit is nice and fast, and shouldn't cost us more.  We can probably provide a compatibility redirect by writing a simple script that takes gitweb-style query strings and translates them into a 301 to the appropriate cgit URL.
Comment 1 Mark Wielaard 2022-11-09 20:59:40 UTC
I have been playing with cgit already and I really like it.
It is indeed really fast, so cpu shouldn't be a problem.

But it does take some diskspace, my local cgit setup with various sourceware mirrors has a cgit cache of ~400MB. https://code.wildebeest.org/git/mirror/ But we have ~20G available, so that should not be a problem.

I have been looking at "overlaying" /cgit/ over /git/ and have URL rewriting "do the right thing", but it looks really hairy:
https://www.clearchain.com/blog/posts/cgit-upgrade-gitweb-retired

So maybe the best thing to do first is just have /cgit/ next to /git/
Comment 2 Arsen Arsenović 2022-11-10 09:19:28 UTC
(In reply to Mark Wielaard from comment #1)
> I have been looking at "overlaying" /cgit/ over /git/ and have URL rewriting
> "do the right thing", but it looks really hairy:
> https://www.clearchain.com/blog/posts/cgit-upgrade-gitweb-retired
> 
> So maybe the best thing to do first is just have /cgit/ next to /git/

Yes, that sounds fine, at least as a start.  (to repost from IRC) Rather than using rewrite rules to do translation, I think it should be fairly easy to write a CGI script that translates gitweb URLs to cgit URLs (e.g. https://git.savannah.gnu.org/gitweb/?p=poke.git;a=blob;f=README;h=fb5cb08b1fe9de1b15a5df8bbd7131535b5c23eb;hb=HEAD to https://git.savannah.gnu.org/cgit/poke.git/tree/README?h=HEAD or such; I haven't examined the whole format yet; savannah used as an example, they don't implement that)

Maybe such a thing even exists?  Unsure, will look around when I get some time.
Comment 3 Mark Wielaard 2022-11-13 21:09:01 UTC
Arsen wrote a perl cgi script that can rewrite gitweb URL path queries to cgit paths. We are looking to see if we can somehow overlay that on the /git/ base URL to multiplex over cgit and the gitweb redirector.

There is already one AliasMatch for the git-http-backend:

ScriptAliasMatch \
                 "(?x)^/git/(.*/(HEAD | \
                  info/refs | \
                  objects/(info/[^/]+ | \
                  [0-9a-f]{2}/[0-9a-f]{38} | \
                  pack/pack-[0-9a-f]{40}\.(pack|idx)) | \
                  git-(upload|receive)-pack))$" \
     /usr/libexec/git-core/git-http-backend/$1

This is the same as we would already need for cgit. So that one should stay as is.
Comment 4 Arsen Arsenović 2022-11-14 07:52:51 UTC
(In reply to Mark Wielaard from comment #3)
> ScriptAliasMatch \
>                  "(?x)^/git/(.*/(HEAD | \
>                   info/refs | \
>                   objects/(info/[^/]+ | \
>                   [0-9a-f]{2}/[0-9a-f]{38} | \
>                   pack/pack-[0-9a-f]{40}\.(pack|idx)) | \
>                   git-(upload|receive)-pack))$" \
>      /usr/libexec/git-core/git-http-backend/$1
Ah, yes, that alias is much more specific than what I thought that git-http-backend needs, so my concerns about it conflicting with cgit are void.

The aforementioned Perl script needs some code review (help appreciated!) but covers all gitweb actions AFAICT, and should be easier to extend and less prone to error due to unspecified query parameter order.  Some actions are simplified or ignore query parameters, or are just unmappable, so instead choose to redirect to the CGit project summary page instead.

Code: https://git.sr.ht/~arsen/cgitweb/tree/master/item/cgitweb.perl
Comment 5 Frank Ch. Eigler 2022-11-14 11:59:51 UTC
I agree that /cgit/ as a sibling makes more sense than shoehorning all the status quo via redirects.
Comment 6 Mark Wielaard 2023-02-27 11:53:31 UTC
As a first step there is now:

https://cygwin.com/cgit/
https://gcc.gnu.org/cgit/
https://sourceware.org/cgit/

The cygwin and sourceware repos are currently disambiguated through an explicit project-list because they have overlapping scan-paths. It would be nice to separate them more explicitly by using separate git repo root dirs.

It would also be nice to actually use Arsen's script to do redirects. cgit is nicer
looking, has easier to understand URLs and is a bit faster.
Comment 7 Andrew Pinski 2023-02-27 16:20:02 UTC
(In reply to Mark Wielaard from comment #6)
> The cygwin and sourceware repos are currently disambiguated through an
> explicit project-list because they have overlapping scan-paths. It would be
> nice to separate them more explicitly by using separate git repo root dirs.

The issue with seperating them (unless symbolic link works there) is that the cygwin-newlib repo is for both of them.
Comment 8 Arsen Arsenović 2023-02-27 21:14:11 UTC
(In reply to Andrew Pinski from comment #7)
> (In reply to Mark Wielaard from comment #6)
> > The cygwin and sourceware repos are currently disambiguated through an
> > explicit project-list because they have overlapping scan-paths. It would be
> > nice to separate them more explicitly by using separate git repo root dirs.
> 
> The issue with seperating them (unless symbolic link works there) is that
> the cygwin-newlib repo is for both of them.

those should work, my cgit instance also uses symlinks to control "exposure".

Mark: thanks!  this is quite nice :)
Comment 9 Mark Wielaard 2023-03-05 13:57:42 UTC
(In reply to Andrew Pinski from comment #7)
> (In reply to Mark Wielaard from comment #6)
> > The cygwin and sourceware repos are currently disambiguated through an
> > explicit project-list because they have overlapping scan-paths. It would be
> > nice to separate them more explicitly by using separate git repo root dirs.
> 
> The issue with seperating them (unless symbolic link works there) is that
> the cygwin-newlib repo is for both of them.

Thanks. Yes, symbolic links do work, that is actually how the cygwin related repos were kept under sourceware/git when they were (also) on the new cygwin domain. Corinna also said she liked the historic git repos kept/shared between sourceware/cygwin.

So I updated the list to keep all repos visible under /cgit/ that were visible under /git/ including cygwin-htdocs.git, newlib-cygwin.git, newlib-htdocs.git and the repos under /cygwin-apps/ (note that is only those that were historically there, https://cygwin.com/cgit/cygwin-apps/ lists a few more now)

See https://sourceware.org/cgit/ and https://cygwin.com/cgit/
Comment 10 Frank Ch. Eigler 2023-07-24 20:35:04 UTC
sourceware.org/cgit is installed and operating