CTI - Making a decision for glibc.

[Siddhesh Poyarekar]

On 2026-02-03 13:54, Zack Weinberg wrote:
> Again, I’m a very occasional contributor who is barely affected by any
> of the things that have come up in this thread, and will be barely
> affected by a move, if and when it happens.  But I’m just not seeing
> enough of a reason put forward to *bother* moving glibc.  A variety of
> problems with sourceware have been articulated, but all of them strike
> me as *easier* to fix in situ than by moving to new infrastructure.

The trouble is that technical arguments are easy to argue in an email 
thread and non-technical ones are even easier to take personally and 
hence tend to either get overlooked or are subject to personal 
interpretation.

> A desire for additional funding has been articulated, but the primary
> reason I’m aware of why money would be needed, is to pay developers
> to work more on glibc, and salaries should have no reason to be tied
> to hosting for the canonical VCS repository and bug tracker.
> 
> So, as I said last time around: Proponents, please *explain your
> reasons for seeking to move glibc*.  Not the criteria you chose
> for evaluating funding and/or hosting source, but the reasons
> why you were out there looking for funding and/or hosting sources
> in the first place!
> 
> Explain how you have tried and failed to fix the technical issues
> in situ; explain why it wouldn’t be *more work* to fix them in
> conjunction with a move.  Explain what you plan to do with the money,
> and why you consider “only if we also provide the hosting” an
> acceptable condition attached to the money.
> 
> Go all the way back to the beginning of your thought process.
> Be specific.  Be detailed.  Be candid.  Let the rest of us understand
> your thinking.  Because right now, I repeat, knowing only what I know,
> it seems to me that a move is *unnecessary*.

OK, I'll bite.  Let me share my personal perspective.  It's candid (as 
you requested) and it is opinionated and hopefully spells out why I 
strongly back the migration of GNU toolchain services.  Others may have 
a different view of all of these events and I can live with that.

Anyway, here we go:

My first discussions around infrastructure were when I migrated my 
patchwork instance (patchwork.sourceware.org used to be 
patchwork.siddhesh.in until around 2014 IIRC) over to sourceware and 
started maintaining it, I'd say around 2015/2016.  I was feeling the 
discomfort of maintaining patchwork (Mark was very helpful, but it was 
still quite a bit of learning for both of us) and I expressed the 
concern in one of the Cauldrons that there were barely a handful of us 
doing this work in our spare time and that doesn't seem scalable, nor 
the fact that we're not really full time system administrators (I guess 
I'm not a "real programmer").

This was apparently a known concern for a while, especially because the 
infrastructure story was struggling on multiple fronts: the number of 
active volunteer overseers were dwindling (2 then and still the same 
now) and the sourceware hardware that hosts core services for the GNU 
toolchain was (and still is) funded solely by Red Hat, hosted in a Red 
Hat datacentre.  It was 2 machines then (3 now) including backups, 
running *all* of the core infrastructure.  Everything ran on those 
machines, alongside each other.  Compromising one service would mean 
pretty much everything would be compromised.  I remember Carlos talking 
about service isolation (and how we can't really do it on sourceware 
because we don't have the hardware capacity to do that) as far back as 
our first conversation in 2016 when I complained about patchwork 
maintenance.

David and Carlos had been working on getting funding for GNU toolchain 
projects for some time (e.g. the GNU Toolchain Fund[1]), and they 
started looking for funding sources for the infrastructure as well. 
Unfortunately we had the pandemic disruption which sent a lot of things 
awry but closer to around 2021/2022 things started to coalesce around an 
LF/OpenSSF solution, which is when I was again involved.

As we were doing this, there was news of upcoming FEDRAMP and rumours of 
a similar EU legislation and ISTM it made an excellent formal framing of 
the concerns we already had.  FWIW, you say that those issues could be 
fixed in situ, but they were summarily dismissed when we raised them in 
2022.  Changes have happened for the better over the years since, but 
they have been limited because there's no funding and people are working 
in their spare time to support this.

Having a registered vendor account with large companies is one thing and 
getting those large companies to actually pay up for a cause is quite 
another.  I've seen Carlos and David run from pillar to post all these 
years, trying to drum up sponsors for everything from the Cauldron to 
the infrastructure, it's hard and I know for a fact that what we've 
ended up with is not a solution looking for a problem.  It is a solution 
that neatly fits a problem that we've discussed for over a decade.

To conclude, I wanted a FOSS-friendly hosting solution that has full 
time specialists taking care of the infrastructure for the GNU toolchain 
and the OpenSSF/LF-IT solution provides exactly that.  I do not think a 
part time volunteer based maintenance model is a viable solution for 
hosting the toolchain infrastructure in the long term, not in terms of 
reliability, nor financial viability nor funding.  This is especially 
true with a community that is slowly dwindling.  Our users are working 
around our deficiencies; wolfi and others use mirrors for their 
downstream distros because sourceware cannot handle that load and in 
some cases, has outright banned their IPs.  We have disruptions that we 
take in our stride because we're friends helping friends, and friends 
cannot be ungrateful to friends by saying that their work is not enough. 
  I love that, and I greatly respect the work and the expertise of the 
overseers, but I also see it as a threat to the viability of the project.

Sid

[1] https://gcc.gnu.org/wiki/GNUToolchainFund