Smoke test for glibc

[Andrey Ponomarenko]

Carlos O'Donell wrote:
> On Mon, Oct 22, 2012 at 9:00 AM, Andrey Ponomarenko
> <aponomarenko@rosalab.ru> wrote:
>> I've recently written an article [1] about smoke testing of Glibc API using
>> a modern approach. This kind of test is needed to verify that most of the
>> library functions pass a simple test in the particular environment, i.e.
>> don't crash, don't emit any signal, don't hang and don't exit with non-zero
>> error code. The test generator takes only a path to the installed glibc tree
>> in order to compose input arguments for all functions (reasonable in most,
>> but unfortunately not all, cases) and then create, build and run the test
>> cases.
>>
>> I've added a link to this article on the Release Process page (section 5.6.
>> Testing), so that glibc maintainers can run this additional test when
>> preparing the release and look at the behaviour of all functions under the
>> smoke test. The sample test report for 2.16 looks like this one [2].
> Andrey,
>
> Thanks! This looks very interesting.
>
> Will you continue to work on getting these results clean for glibc?
>
> Going through 154 failed testcases from the smoke test is not something
> that I want a release manager to be doing.

The test generator has an option to correct the test cases. It's a file 
with specialized types [1], that instruct generator to initialize 
arguments of the related data types in the correct way. I will try to 
write such spec-types for these 154 functions or/and improve the source 
code of the test generator.

For now we just can check if the number of failed tests is not greatly 
increased in the new release.

Thanks.

[1] http://ispras.linuxbase.org/index.php/Specialized_Type

-- 
Andrey Ponomarenko, ROSA Lab.