Sourceware infrastructure after Cauldron - Forgejo experiment
Mark Wielaard
mark@klomp.org
Wed Sep 25 00:43:43 GMT 2024
There will be no drastic changes. We'll keep improving services to
make the (email-based) workflows better and more efficient. But we
will also do an experiment with Forgejo to let people try out a pull
request workflow. We don't know if the experiment will be successful,
nobody will be forced to participate, but volunteers to try it out
(and help with the setup and configuration) are more than welcome. It
might take up to a year to determine whether the experiment is a
success or a failure.
Attached are the Sourceware BoF discussion topics and notes taken by
Elena during the discussions (the notes might note make much sense to
people not there, but feel free to ask questions if something seems
unclear).
Jonathan Wakely posted about editor config, b4 setup and signing your
git commits:
https://inbox.sourceware.org/CACb0b4kOiiwbMuT8BpG30+mg=yq+DSM+1s_gO_5CehSCy5MAmg@mail.gmail.com/
https://inbox.sourceware.org/CACb0b4=uX+P4Wc5VtJGTrh=yMH+OMz9vnXuU5LAVRkz3-ZZzmw@mail.gmail.com/
https://inbox.sourceware.org/gcc/CACb0b4=KewuK-oXhny_ojwgxewFrib3RrUezvow0ZdMJox-LbA@mail.gmail.com/
Nick Alcock posted about hardware-backed git commit and tag signing
https://inbox.sourceware.org/87ttebeokf.fsf@esperi.org.uk/
(This would require an ssh server version update.)
Note that GCC also has a https://gcc.gnu.org/wiki/GitCookbook which
might be interesting not just for GCC hackers, but also for other
Sourceware projects.
Bradley Kuhn posted about a talk at LPC that covered some of the same
issues we discussed at Cauldron "A little GitLab won't hurt you"
https://inbox.sourceware.org/F8FB9E25-F496-4932-B359-930E269F3CB7@sfconservancy.org/
Joseph Myers wrote up his desired features for a system providing pull
request functionality. Which is a good guide to follow when we are
experimenting with Forgejo to see how close we can come to this ideal.
https://inbox.sourceware.org/b440cf28-674b-122a-160b-0fb96bbf989b@redhat.com/
At the Cauldron I tried to suggest we might want to resurrect the
gerrit experiment. But people really didn't seem to be enthousiastic
about it (one could say they were the opposite of enthousiastic). So I
talked to Sergio Durigan Junior who originally setup the gerrit
instance. And we agreed to reuse the VM for the Forgejo experiment.
That VM should have enough resources for the initial experiment. I'll
talk to the Sourceware PLC to discuss what resources are needed if we
want to roll this out for all Sourceware projects. We already made an
estimate for a larger gitolite server as part of the Security Vision
document: https://sourceware.org/sourceware-security-vision.html
Part of the Forgejo experiment will be making sure resource estimates
are correct.
If there are people who have some experience setting up/configuring a
Forgejo service please let us know your experiences.
-------------- next part --------------
= Bof: Sourceware infrastructure tips & tricks
Frank Ch. Eigler
overseer, systemtap, elfutils, bunsen, Red Hat
Christopher Faylor
overseer, Cygwin
Ian Kelling *
FSF tech-admin
Ian Lance Taylor
overseer, GCC steering committee, Google
Tom Tromey
GDB, AdaCore
Jon Turney
Cygwin
Mark J. Wielaard *
overseer, elfutils, valgrind, Red Hat
Elena Zannoni *
GDB, Oracle
* Here today, all on personal title
Affiliations are listed for identification purposes only
- Sourceware
- History and Roadmaps
https://sourceware.org/sourceware-25-roadmap.html
- Software Freedom Conservancy (SFC)
- Communication and contacts
- overseers, project admins
- mailinglists, bugzilla,
technical roadmaps, quarterly updates, open office hours
- (Hardware) sponsors
- Funds and how to spend it (see security-vision below)
- Forge experiment?
- Separate user gitolite server "cybersecurity plan"
- Gerrit
- Sourcehut
- Forgejo
- Success criteria?
- Risks? Mandatory features?
- Timeframe for the experiment?
- Mailing lists Mailman
- dmarc/dkim, (no) From-rewriting
(and SRS and SPF and VERP - for maximal email hygiene)
- Moderators and spam
- (python2) what should an upgrade look like?
- public-inbox
- imap, nntp, rss, git clone inbox.sourceware.org
- b4 config and usage
- Git
- cgit everywhere?
- project configs
- adacore hooks
- signed commits (also see cybersecurity)
- Bugzilla
- Account creation
- Two is better than one?
- Upgrading and local patches
- Wiki
- MoinMoin
- builder.sourceware.org
- hardware overview
i386, x86_64, ppc64le, s390x, ppc64, armhf, arm64, riscv
- kinds of builders
- pre-commit, try builders
- ci-builders finding regressions
- full-builders, storing test results (see bunsen below)
- autoregen (binutils-gdb, gcc)
- format checker (gdb, python)
- config grew a lot last year
help needed simplifying
- bunsen
- indexes repository of raw testsuite log files (sqlite + git)
- understands dejagnu, glibc, automake, autoconf styles
- use toolkit locally or centralized on sourceware
- live web interface at https://builder.sourceware.org/testruns/
- patchwork.sourceware.org
- git pw setup
- patchwork plus CI/CD
- Let's use those buildbot workers too
- snapshots.sourceware.org
new isolated machine/vm/container
takes over cron jobs
triggered by buildbot
can also trigger, manuals, code coverage, api docs, etc.
valgrind, libabigail, gnupoke, glibc, gdb, elfutils, dwarfstd, binutils
- Cybersecurity
- US Improving the Nation's Cybersecurity
Executive Order 14028
- EU Cyber Resilience Act (EU CRA)
- Secure Software Development Framework (SSDF, NIST SP 800-218)
- What can Sourceware do?
- signed-commit census report
- Sourceware Security Vision
https://sourceware.org/sourceware-security-vision.html
- Practical steps for projects and individual contributors
- Experiment: sourcehut
https://sr.ht/~sourceware/
A more webby git workflow alternative
git send-email without the email
- non-Sourceware, but useful, services
- BBB server (SFC)
- mattermost (OSUOSL)
- irc (libera.chat,oftc)
- gerrit server
- Software Heritage and archive.org mirrors
---
signed-commit census report
analyzing branch HEAD since 2024-08-13
/git/annobin.git 3 commits, 0 signed ( 0%), 1 committers, 0 signers ( 0%)
/git/binutils-gdb.git 246 commits, 2 signed ( 0%), 30 committers, 1 signers ( 3%)
/git/binutils-htdocs.git 1 commits, 0 signed ( 0%), 1 committers, 0 signers ( 0%)
/git/builder.git 13 commits, 0 signed ( 0%), 1 committers, 0 signers ( 0%)
/git/bunsen.git 6 commits, 6 signed (100%), 1 committers, 1 signers (100%)
/git/cygwin-htdocs.git 4 commits, 1 signed ( 25%), 3 committers, 1 signers ( 33%)
/git/debugedit.git 1 commits, 0 signed ( 0%), 1 committers, 0 signers ( 0%)
/git/elfutils.git 27 commits, 2 signed ( 7%), 3 committers, 1 signers ( 33%)
/git/elfutils-htdocs.git 1 commits, 0 signed ( 0%), 1 committers, 0 signers ( 0%)
/git/gcc.git 713 commits, 62 signed ( 8%), 92 committers, 6 signers ( 6%)
/git/gcc-wwwdocs.git 12 commits, 0 signed ( 0%), 7 committers, 0 signers ( 0%)
/git/glibc.git 78 commits, 0 signed ( 0%), 16 committers, 0 signers ( 0%)
/git/glibc-htdocs.git 3 commits, 0 signed ( 0%), 1 committers, 0 signers ( 0%)
/git/insight.git 4 commits, 0 signed ( 0%), 1 committers, 0 signers ( 0%)
/git/libabigail.git 57 commits, 0 signed ( 0%), 1 committers, 0 signers ( 0%)
/git/libabigail-tests.git 6 commits, 0 signed ( 0%), 2 committers, 0 signers ( 0%)
/git/lvm2.git 49 commits, 18 signed ( 36%), 4 committers, 1 signers ( 25%)
/git/newlib-cygwin.git 34 commits, 0 signed ( 0%), 3 committers, 0 signers ( 0%)
/git/systemtap.git 10 commits, 8 signed ( 80%), 3 committers, 1 signers ( 33%)
/git/valgrind.git 2 commits, 0 signed ( 0%), 2 committers, 0 signers ( 0%)
-------------- next part --------------
spdx identifiers in source code, is that happening?
fedora is using spdx identifier
agreed that this is more a project level question
sboms?
open chain
sbom mentioned in NIST as example, but unclear if that really
helps proof provenance of code.
topic for licensing bof
we should not spend time on this because we do ship the
source code, sboms are for people that do not ship source code
gitolite
Do we need to change the current hooks, incompatible with gitolite?
We do checks as part of git push
gentoo has these
Call out to another server, that server does something comes back
with a response Ian sware can provide VM where to run the githooks
solvable problem
Setting a forge will help with this need fewer people with special
access to the "real" repo
gerrit some projects have started using it but then hated it and
dropped it
Why would the forge impact the workflow of the review? why does it
necessary force the way you review patches?
Patches need to be rebased, and forge helps with that
patchwork CI bot does check and warns of need of rebase if patch no
longer applies.
Get structured info from unstructured info on mailing lists. We do not
have to move everything to do pull requests. not instantly. Joseph is
writing notes about what is desirable in a forge.
We should look at what we can do today w/o a forge, like b4
etc. Nobody knows that.
Richard E. no imminent move to a forge, we need to do a study to see
if it really meets our needs It's not an either/or.
experiment on sourcehut
Sends email from git, via the server. Reviewer send back comments by email.
The review goes to the mailing list. Solves the problem of the
initial submission (well formed git send-email like) People seem to
like that, it is well explained. It was used for another project.
Asking how to move a project over to the workflow.
We can set up a project on forgejo?
People asked to have a VM to experiment with the Forgejo, so
sourceware will create an VM for it.
Mark will start an email about this
Involved in clang discussions with email, not website, but everything
mirrored, and you do not need to deal with the web interface at all
mailman:
need to look for new mailing list software eventually
archive: using public inbox
FSF wants to upgrade to mailman3, and use inbox too
From-rewriting: look at lkml and Konstantin blog
public inbox preserves everything, not like mailman
Is there anything that shows the whole month? people not subscribed,
just reload the archives can we get webarchives with inbox?
Liked the way before mailman (ezml), easier to see the archives, and
had dates also liked reversed date order We want to have something to
massage the inbox archives, instead of keep using the old mailman etc
stuff for security (Ian)
builder
one big python file that needs simplifying, need help with that
patchwork
Need integration with buildbot for pre-commit CI
snapshots
gcc is not there yet
signed commits
Nick Alcock: knows how to easily sign commits, request to put it on
the gcc wiki
Also idea about having a comment at the commit time, printing
something like "please consider signing your commits, and this is
where the instructions are"
More information about the Libc-alpha
mailing list