Does glibc has complete test coverage?

[Peng Yu]

On Tue, Mar 23, 2021 at 4:02 PM Jeffrey Walton <noloader@gmail.com> wrote:
>
> On Tue, Mar 23, 2021 at 4:43 PM Mike Frysinger via Libc-help
> <libc-help@sourceware.org> wrote:
> >
> > On 23 Mar 2021 11:39, Peng Yu via Libc-help wrote:
> > > https://www.kernel.org/doc/man-pages/missing_pages.html
> > >
> > > "... quite a few kernel and glibc bugs have been uncovered while
> > > writing test programs during the preparation of man pages. "
> > >
> > > I see the above text. It doesn't make too much sense, as it indicates
> > > that glibc does not have complete test coverage.
> > >
> > > Why not taking an approach of always accompanying each line of source
> > > code with appopriate test cases? If this approach is taken, then most
> > > bugs should have been eliminated beforehand?
> >
> > ignoring the legacy aspect (code that's in the tree now but lacks tests),
> > you have diminishing returns when it comes to writing unittests, and, as
> > can be seen in a recent discussion, glibc is pretty tightly coupled to
> > the runtime environment (i.e. the host kernel).  so getting an env that
> > matches all the different code paths is challenging.
> >
> > plus it comes down a bit to this being an open source project for many
> > of us, not a job, and you have to be respectful of balancing quality
> > and developer time with any requests you make on other volunteers.
> >
> > along those lines, this is an open source project where "patches are
> > welcome", so if you wanted to spend your time improving the frameworks
> > and coverage of our tests, we'd welcome you.
>
> Interns are usually a good choice for writing test cases. It gets them
> familiar with the code, frees up a senior developer's time, and helps
> avoid the developer's bias.
>
> Test cases are monkey work that should be delegated. When delegation
> does not occur it usually points back to shortcomings in project
> management.
>
> > also try googling for "100% test coverage" and reading the variety of
> > opinions the wider world has on the topic.

Hi Mike,

https://blog.ndepend.com/aim-100-percent-test-coverage/

I don't think the so-called "The Diminishing Returns Argument" is
valid. When you have too many branches to test which rarely occurs in
practice, this just indicates the code is badly structured. To have
complete coverage, if the testing code is much more complex than the
real code, this means that the real code should be restructured to
make testing code simpler.

> Sorry, I could not resist.... But you know the funny thing is, when
> you perform a post-mortem to determine why the bug made it into
> production, it usually points to (1) a developer mistake and (2) lack
> of test case.
>
> If you break (1) or (2) you break the chain for the bug. So you either
> have to hire developers who don't make mistakes or provide complete
> test cases.

Hi Jeff,

(1) is guaranteed to happen in practice as all humans will make
mistakes. Therefore, overcoming (2) is the only way to eliminate the
bug.

-- 
Regards,
Peng