Fwd: Buildbot failure in Wildebeest Builder on whole buildset

[Tom de Vries]

On 3/25/21 11:49 AM, Mark Wielaard wrote:
> Hi Tom,
> 
> On Thu, 2021-03-25 at 11:41 +0100, Mark Wielaard wrote:
>> On Thu, 2021-03-25 at 09:50 +0100, Tom de Vries wrote:
>>> I've just committed patch "Move hardlink handling out of dwz
>>> function"
>>> and there's this buildbot failure.
>>>
>>> Is there something easy that you can do to find out if this is a
>>> fluke
>>> or not, f.i. retry the build for that bot?
>>
>> I logged into the buildbot worker and did a dwz build myself.
>> make check does produce 32 unexpected failures.
>> Removing the last commit makes everything pass.
>> Adding the commit again produces the failures again.
>> It isn't a fluke.
>>
>> The problem is simply that:
>> cp hello 1
>> dwz 1
>>
>> Makes dwz produce an exit code of 1 which seems to be interpreted as
>> child process exited abnormally in a make check run.
>>
>> running under valgrind --track-origins=yes gives:
>>
>> ==31313== Conditional jump or move depends on uninitialised value(s)
>> ==31313==    at 0x127974: dwz (dwz.c:15287)
>> ==31313==    by 0x10BB7C: dwz_with_low_mem (dwz.c:16227)
>> ==31313==    by 0x10BB7C: dwz_one_file (dwz.c:16253)
>> ==31313==    by 0x10BB7C: main (dwz.c:16532)
>> ==31313==  Uninitialised value was created by a stack allocation
>> ==31313==    at 0x10B59B: main (dwz.c:16513)
>>
>> I haven't tracked down what res->res precisely depends on that
>> doesn't
>> have a defined value.
> 
> Note that dwz_one_file was inlined into main, causing valgrind to
> pinpoint the wrong frame. Without inlining we get:
> 
> ==2502== Conditional jump or move depends on uninitialised value(s)
> ==2502==    at 0x13CF4F: dwz (dwz.c:15287)
> ==2502==    by 0x13FE3A: dwz_with_low_mem (dwz.c:16227)
> ==2502==    by 0x13FEAE: dwz_one_file (dwz.c:16253)
> ==2502==    by 0x140B59: main (dwz.c:16532)
> ==2502==  Uninitialised value was created by a stack allocation
> ==2502==    at 0x13FE7F: dwz_one_file (dwz.c:16245)
> 
> And indeed we see:
> 
> static int
> dwz_one_file (const char *file, const char *outfile)
> {
>   struct file_result res;
> 
>   if (stats_p)
>     init_stats (file);
> 
>   res.die_count = 0;
> 
>   return dwz_with_low_mem (file, outfile, &res, NULL);
> }
> 
> Defines and passes down res, but only initializes the die_count field.
> 
> Then in dwz_with_low_mem we simply pass that res to dwz:
> 
>   ret = (low_mem_die_limit == 0
>          ? 2
>          : dwz (file, outfile, res));
> 
> And in dwz the first thing we do is:
> 
>   if (res->res == -1)
>     return 1;
> 
> But res->res was never given a value and so could be anything.
> 

Hi Mark,

thanks for helping out with this.

I've fixed this now, and verified that the buildbot is back to all-green.

After seeing your analysis, I realized I had tried -fsanitize=address
but not valgrind, and indeed using valgrind I managed to reproduce the
problem myself.

I've submitted a patch proposing a check-valgrind target in the
Makefile, to remind me to try this next time.

Thanks,
- Tom