RFC: The Binutils and LLM generated patches

[Mark Wielaard]

Hi Nick,

On Mon, Oct 06, 2025 at 03:29:03PM +0100, Nick Clifton wrote:
>   Below is a revised version of the proposed policy on LLM generated
>   patches.

Thanks for this. It might be a little stricter than I like, because I
hope that one day we will have LLM's like:
https://sfconservancy.org/activities/aspirational-statement-on-llm-generative-ai-for-programming.html
Which might be acceptable. But till that day comes this statement is
probably better to avoid endless discussions around grey areas.

I might propose to use something like this also for the elfutils
project.

>   As well as tweaking the wording in response to several
>   peoples posts, I have also added a clause stating that a human must
>   be involved in the patch submission process.  It occurred to me
>   that without this caveat we might end up with bots submitting patches
>   on their own with no human involvement at all and hence no copyright
>   accountability.

I think this last point is also an important reason for the policy. A
Free Software project is about putting people first, creating an
environment where people can learn and work together to create code
that provides user freedom. There should be human involvement because
with the current generation of LLMs you cannot guarantee who they are
working for or what information they have ingested (and so cannot
really know what the output they generate is based on).

> -------------------------------------------------------------------------
> 
>       The GNU Binutils project is currently
>       *NOT* accepting LLM generated patches.
> 
> This is because the copyright status of LLM generated code is
> currently unclear.  The policy applies to all parts of the GNU
> Binutils including, but not limited to, code, documentation and
> the testsuites.
> 
> There are however some exceptions to the policy:
> 
>   * Using LLMs to assist in writing code is fine providing that the
>     LLM does not actually generate code.  So for example using an
>     LLM for translations is OK.  So is using an LLM to provide text
>     to speech services or to search for information, and so on.

I think I understand what you are trying to say here. If I understand
you correctly you want to explain that using an LLM as a tool that
doesn't generate "content" is fine. As if you read an article on a
certain algorithm or use it as a kind of search engine.

But I am a little confused about the "for translations". I assume you
aren't suggesting the project would accept LLM generated patches for
translations of binutils? Should we coordinate this with the
translation project teams?

>   * LLM generated code that is not "legally significant"[1] is OK.
>     As a rule of thumb, this means that trivial changes, such as
>     spelling corrections, or small code formatting cleanups are fine.
> 
> Using an LLM to inspire or help create a patch might be OK.  It is a
> question of whether LLM generated output eventually makes it into the
> patch.  If it does, then the patch is unacceptable.  (Unless it can
> be considered legally insignificant).

Nicely formulated.

> When submitting a non-legally-significant LLM generated change, it is
> still necessary to clearly indicate the use of the LLM.

But I don't think this is necessary or even desirable. Since it is not
legally significant it doesn't matter. And I am afraid people will use
it for advertising the latest hip model/company.

>  The
> identification should take the form of a line starting with the
> "Generated-By: " prefix which identifies the LLM used and a line
> starting with "LLM-Query: " which shows the query given to the LLM.
> For example:
> 
>   Generated-By: GNU-LLM version 1.0
>   LLM-Query: Create a program to solve all GNU problems

While something like this might be useful when using some
deterministic tools I think it isn't useful for LLMs.

When you reference an article or book to explain what the idea behind
some code/algorithm is it is useful. Or if you used a tool/script that
others can run to validate/recreate your patch then it is helpful to
include that in the commit message.

But with some undeterministic LLM where you don't know what the exacts
input was, and it might generate something completely different for
others, it isn't useful at all. You need much more than just one query
statement. You need to know at least the exact session the user has
run and which system prompt was used. And there is no precise way of
identifying the llm model and version used, especially if you arent
running it locally.

> In addition all patch submissions must involve a human.  Fully
> automated patch submission, whether by a bot, a script, or some other
> means is not acceptable.

Maybe extend this to bug reports?

>  This is because only humans can sign a
> Developer Certificate of Origin or complete a FSF Copyright Assignment
> and one of these needs to be in place for every submission.
> 
> The copyright assignment or DCO allows the GNU Binutils project to
> trust that the submitter is the author of the submission, and that
> they have the right under copyright law to make the submission.

Agreed, but I maybe reformulate the second paragraph to make clear
what this means in practice (for this project). How about:

  So for each submission someone has to certify users gets all the
  rights granted by the GNU Binutils project license, the GNU General
  Public License (see COPYING3).

> Footnotes:
> 
> This policy is not set in stone.  It may well be reviewed and changed
> in the future.
> 
> The policy uses the term "LLM generated" rather than "A.I. generated"
> as the later could be misunderstood.  See [2] for more details.

I agree with this. But find the reference a little confusing and
negative. I would drop it or try to find a more positive definition of
LLM.

> The reason for requiring trivial LLM generated patches to be labelled
> is to set a precedent.  In the future, if non-trivial patches become
> acceptable, the standard of labelling LLM submissions should already
> be in place.

I would remove this from this policy and start a separate discussion
on tagging/labelling tool-assisted patches.

> [1]: https://www.gnu.org/prep/maintain/maintain.html#Legally-Significant
> [2]: https://www.gnu.org/philosophy/words-to-avoid.en.html#ArtificialIntelligence
> 

Thanks,

Mark