[PATCH v2 3/3] scripts: Support custom Git URLs in build-many-glibcs.py

[Carlos O'Donell]

On 11/27/25 6:01 AM, Yury Khrustalev wrote:
> On Wed, Nov 26, 2025 at 12:14:01PM -0500, Carlos O'Donell wrote:
>> On 11/26/25 8:39 AM, Joseph Myers wrote:
>>> On Wed, 26 Nov 2025, Yury Khrustalev wrote:
>>>
>>>> On Wed, Nov 26, 2025 at 01:18:48PM +0000, Joseph Myers wrote:
>>>>> On Wed, 26 Nov 2025, Yury Khrustalev wrote:
>>>>>
>>>>>> If there is a need to do a clean checkout of sources, then I would just
>>>>>> remove the folder and re-run the 'checkout' step again. Why do we need
>>>>>> --replace-sources? It's rhetorical, since we already have it, we have
>>>>>> to deal with the consequences of maintaining it, but I think it was a
>>>>>> mistake to add it.
>>>>>
>>>>> A bot is supposed to keep running automatically, even when the default
>>>>> versions / branches change.
>>>>
>>>> Default is not changed in this version of the patch.
>>>
>>> The point is that the default versions *do* change over time, as new
>>> versions of the components are released, and so --replace-sources is key
>>> functionality to keep using the script, both with bots and with long-lived
>>> checkouts, as such changes occur.
> 
> It looks like the --replace-sources behaviour should be the default: if
> you happen to run the 'checkout' step, it is likely that you want your
> sources to be fixed-up to the expected state.

This is a design choice.

We are purposely being conservative here with the script, it has the potential
to really impact your network/CDN/VCS if you run this in a loop and the default
is to always pull from the network.

It enables a use case where you use minimal network/CDN and VCS resources
if the versions haven't changed, and you can be guaranteed you won't update
to a new version of the tools. If a new version is added in the script then
it will error out for you to inspect status and do an update if required.

If you want to always update, the you as a user of the script can just always
pass --replace-sources but realize this impacts the resources you're pulling
from.

> If the main use case is to support repeatedly running 'checkout' to pull
> updates in already existing working copy, then Git URL should just be
> ignored because we already have some state and just want to pull updates.
> We should have 'update-sources' action for this case. I think separation of
> concerns would help here.

Your changes introduce a security issue.

Suppose we change the URL in the script, because of a security issue.

The developer gets the notice, updates glibc, and restarts their build
but forgets to use --replace-sources.

This should fail safe.

We do not want existing bots to keep pulling from insecure or broken repos.

Instead we want their update to fail and the operator to be notified they
need to use --replace-sources.

If they were running with --replace-sources already, then they would just
continue onwards and adopt the change.

Again we do not default to --replace-sources because of the network/CDN
cost.

>> Example:
>>
>> $ GLIBC_GIT_MIRROR=https://gitlab.com/gnutools/glibc.git ~/src/glibc-review/scripts/build-many-glibcs.py . checkout
>> error: origin url has changed from https://sourceware.org/git/glibc.git to https://gitlab.com/gnutools/glibc.git, use --replace-sources to check out again
>>
>> $ GLIBC_GIT_MIRROR=https://gitlab.com/gnutools/glibc.git ~/src/glibc-review/scripts/build-many-glibcs.py --replace-sources . checkout
>>
>> $ cd src/glibc
>> $ git remote get-url origin
>> https://gitlab.com/gnutools/glibc.git
>> $ cd ../../
>>
>> $ ~/src/glibc-review/scripts/build-many-glibcs.py . checkout
>> error: origin url has changed from https://gitlab.com/gnutools/glibc.git to https://sourceware.org/git/glibc.git, use --replace-sources to check out again
> 
> Thanks
> 
>> diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py
>> index 5a02c4bf73..2aaeca71c3 100755
>> --- a/scripts/build-many-glibcs.py
>> +++ b/scripts/build-many-glibcs.py
>> @@ -971,8 +971,20 @@ class Context(object):
>>               subprocess.run(['git', 'remote', 'prune', 'origin'],
>>                              cwd=self.component_srcdir(component), check=True)
>>               if self.replace_sources:
>> +                subprocess.run(['git', 'remote', 'set-url', 'origin', git_url],
>> +                               cwd=self.component_srcdir(component), check=True)
>>                   subprocess.run(['git', 'clean', '-dxfq'],
>>                                  cwd=self.component_srcdir(component), check=True)
>> +            else:
>> +                r = subprocess.run(['git', 'remote', 'get-url', 'origin'],
>> +                                   cwd=self.component_srcdir(component),
>> +                                   stdout=subprocess.PIPE,
>> +                                   check=True, universal_newlines=True).stdout
>> +                if r.rstrip() != git_url:
>> +                    print('error: origin url has changed from %s to %s, '
>> +                          'use --replace-sources to check out again' %
>> +                          (r.rstrip(), git_url))
>> +                    exit(1)
>>               subprocess.run(['git', 'pull', '-q'],
>>                              cwd=self.component_srcdir(component), check=True)
>>           else:
>> ---
>>
>> This works for me.
> 
> OK, I'm happy to include this change, however...
> 
>> I think this is all you would need to integrate your last changes.
> 
> ...As I said, I'm struggling to test all the scenarios supported by the
> script (e.g. anything related to 'bot'), so I would need someone else
> to verify that the updated script works.

I can test it and give a Tested-by:, I've got my setup ready to review your patches.

-- 
Cheers,
Carlos.