Re: Starting the glibc-test add-on project.

["Carlos O'Donell" <carlos at redhat dot com>]

On 10/22/2015 03:25 PM, Szabolcs Nagy wrote:
> * Carlos O'Donell <carlos@redhat.com> [2015-10-22 11:36:13 -0400]:
>> On 10/22/2015 11:21 AM, Rich Felker wrote:
>>> On Thu, Oct 22, 2015 at 10:32:33AM -0400, Carlos O'Donell wrote:
>>>> I'm starting a glibc-test add-on project whose aim is to provide
>>>> broader system testing starting with the networking APIs.
>>>>
>>>> Is there anything I could or should do to make this as compatible
>>>> with Libc-Test such that we can share tests?
>>>
>>> I'm Cc'ing Szabolcs Nagy on this since he's the maintainer of
>>> libc-test. I don't yet have a good feel for what sorts of tests you
>>> want to do in this project, so it's hard for me to give specific
>>> input, but from a standpoint of sharing tests, the most important
>>> thing is not having the tests or test-harness be libc-specific.
>>> libc-test does a really good job of this. Here's the repo for
>>> reference in case you don't have it:
>>>
>>> http://nsz.repo.hu/git/?p=libc-test
>>
>> OK, thanks for that.
>>
>> So your general comments are:
>>
>> - Keep tests non-libc specific.
>>
>> That's sensible for some tests. Some are going to be hard to
>> keep generic because they will involve configuring say ldconfig
>> and the cache. Or configuring networking to do something very
>> glibc specific.
>>
>> What about license? Copyright assignment?
>>
> 
> libc-test is not very flexible at this point,
> hence it is maintained in a private repo.
> 
> it mostly deals with the simple cases that don't
> need any special priviledges or special environment.
> (similar to what glibc already has, no magic framework).

OK.

> for hard tests my plan was to collect/build various
> tools that can help (e.g. regex test generator)
> and to have separate tests that need a vm so they
> can have root etc.

My plan here is to have the test framework build a chroot
and pivot in to that root and use cgroups to isolate the
test from the real system.

Initially I might use docker or lxd for this, but I'm
not decided. I have more experience with docker, so likely
that for now. However, any container technology should be
able to run the isolated chroot.

> network interfaces can interact with the environment
> in ways that's difficult to control (e.g. getaddrinfo).
> i don't have good plan for that so i welcome any project
> that tries to handle that.

This is exactly what I want to test with glibc-test,
along with ldconfig, and all the other things we don't
test because of complex system setup.

> about copyright my position is that the test system
> should not be picky (i don't want to drop tests because
> of copyright issues, i'd rather just put them in a
> different directory so ppl who care can avoid them,
> this is part of the reason why libc-test is separate
> project from musl).

Agreed.

It's a good suggestion to keep non-copyright assigned
tests distinct from copyright assigned tests.

I also plan not to be picky for the tests, but what about
the test infrastructure?

Cheers,
Carlos.