Are big tests allowed in binutils?

[Nick Alcock]

On 20 Jun 2022, lkcl via Binutils spake thusly:

> sorry nick i am stuck with a rubbish mailer, rather than top-post i'm repeating your question: why not submit the full source of the autogenerator?
>
> A: because the list of dependencies, whilst entirely FOSS, is absolutely mental and is completely outside of our control, copyright-wise.  it's also 4 levels deep dependencies and also includes *another* repository comprising CSV and markdown files extracted from the Power ISA Specification PDF [1] and microwatt [2]
>
> now, Dmitry did write a small stand-alone program to autogenerate the test cases but he had to write that by hand.
>
> our plan is to *autogenerate the autogenerator* [based on information that ultimately comes directly from the Power ISA Specification].

That sounds like a good idea to me.

> 1) because if mistakes are found they can be tracked back directly to
> the spec. [we have found appx ... 15 corrections to the published spec
> because of this]
>
> 2) because (no implied criticism of anyone else that does do manual
> unit tests) we simply don't have time money or resources to throw
> brute-force at this so out of sheer forced pragmatism are doing more
> with less.
> 
> i have no view on whether the autogenerated-autogenerator should be
> submitted or whether the automated unit tests themselves should be
> submitted but i can say for sure you *really* don't want the full
> source code of the full software stack we have been developing for 2+
> years as a hard critical build dependency of binutils.

While it's perfectly acceptable to commit the generated output if
generating the output is annoying (and having a huge stack of deps
certainly sounds like it would be annoying), you've got to commit enough
that others can also modify the testcases the same way you do: the
preferred form for modification in this case is pretty clearly not the
generated testcases but the autogenerators themselves (and any input to
them), so the GPL requires you to provide the generators too. They don't
have to be in the binutils repo, but honestly putting them in there so
people can manually run them if needed seems like a good idea. (Stick
in simple pointers to the deps as well, particularly if just figuring ou
what the deps *are* would otherwise be difficult, as "absolutely mental"
suggests it would be).

-- 
NULL && (void)