Bug 26022 - valgrind gives me error when fwprintf in stderr
Summary: valgrind gives me error when fwprintf in stderr
Status: REOPENED
Alias: None
Product: glibc
Classification: Unclassified
Component: libc (show other bugs)
Version: 2.27
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-05-22 07:45 UTC by Claudio Daffra
Modified: 2020-06-04 16:58 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2020-05-29 00:00:00


Attachments
attachment-3917-0.html (703 bytes, text/html)
2020-06-04 16:58 UTC, Claudio Daffra
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Claudio Daffra 2020-05-22 07:45:37 UTC
this simple program gives me error when i check with valgrind,
this happen in stderr only.

valgrind --leak-check=full --show-leak-kinds=all ./x


#include <stdio.h>
#include <wchar.h>

int main ( void )
{

	fwprintf ( stderr , L"valgrind error\n" ) ;
	return 0 ;
}

valgrind error :

==7438== Memcheck, a memory error detector
==7438== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==7438== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==7438== Command: ./x
==7438== 
valgrind error==7438== 
==7438== HEAP SUMMARY:
==7438==     in use at exit: 5,120 bytes in 2 blocks
==7438==   total heap usage: 2 allocs, 0 frees, 5,120 bytes allocated
==7438== 
==7438== 1,024 bytes in 1 blocks are still reachable in loss record 1 of 2
==7438==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7438==    by 0x4EBA18B: _IO_file_doallocate (filedoalloc.c:101)
==7438==    by 0x4EBB88C: _IO_wfile_doallocate (wfiledoalloc.c:70)
==7438==    by 0x4ECA378: _IO_doallocbuf (genops.c:365)
==7438==    by 0x4EC26C7: _IO_wfile_overflow (wfileops.c:430)
==7438==    by 0x4EBFD51: __woverflow (wgenops.c:217)
==7438==    by 0x4EBFD51: _IO_wdefault_xsputn (wgenops.c:317)
==7438==    by 0x4EC2AD0: _IO_wfile_xsputn (wfileops.c:1012)
==7438==    by 0x4EA7089: buffered_vfprintf (vfprintf.c:2343)
==7438==    by 0x4EA40FD: vfwprintf (vfprintf.c:1301)
==7438==    by 0x4EBEA23: fwprintf (fwprintf.c:33)
==7438==    by 0x1086A8: main (in /home/claudio/x)
==7438== 
==7438== 4,096 bytes in 1 blocks are still reachable in loss record 2 of 2
==7438==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7438==    by 0x4EBB858: _IO_wfile_doallocate (wfiledoalloc.c:79)
==7438==    by 0x4ECA378: _IO_doallocbuf (genops.c:365)
==7438==    by 0x4EC26C7: _IO_wfile_overflow (wfileops.c:430)
==7438==    by 0x4EBFD51: __woverflow (wgenops.c:217)
==7438==    by 0x4EBFD51: _IO_wdefault_xsputn (wgenops.c:317)
==7438==    by 0x4EC2AD0: _IO_wfile_xsputn (wfileops.c:1012)
==7438==    by 0x4EA7089: buffered_vfprintf (vfprintf.c:2343)
==7438==    by 0x4EA40FD: vfwprintf (vfprintf.c:1301)
==7438==    by 0x4EBEA23: fwprintf (fwprintf.c:33)
==7438==    by 0x1086A8: main (in /home/claudio/x)
==7438== 
==7438== LEAK SUMMARY:
==7438==    definitely lost: 0 bytes in 0 blocks
==7438==    indirectly lost: 0 bytes in 0 blocks
==7438==      possibly lost: 0 bytes in 0 blocks
==7438==    still reachable: 5,120 bytes in 2 blocks
==7438==         suppressed: 0 bytes in 0 blocks
==7438== 
==7438== For counts of detected and suppressed errors, rerun with: -v
==7438== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

compiler :

gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04).
ldd (Ubuntu GLIBC 2.27-3ubuntu1) 2.27
Comment 1 Carlos O'Donell 2020-05-22 20:07:02 UTC
Freeing streams during application shutdown is difficult because detached threads may be writing to unbuffered streams.

As such, stderr, and any other unbuffered streams are untouched during process shutdown and are not freed by the resource freeing API that glibc exposes for use by external allocation trackers.

That is to say that __libc_freeres() the public API which glibc uses to expose resource freeing, will not free unbuffered streams to allow any last minute writes.

Since stderr starts as unbuffered it is not freed. You can make stderr buffered, and subject to freeing like this: setvbuf (stderr, NULL, _IOLBF, 0);

Valgrind should probably have a suppresion entry added for stderr because it's a common stream to be used by applications.

I've filed a Valgrind bug here:
https://bugs.kde.org/show_bug.cgi?id=421931

Please feel free to follow up there if you think it needs an update or you want to give a +1 regarding your use case.

I'm marking this RESOLVED/INVALID for glibc.
Comment 2 Mark Wielaard 2020-05-29 19:40:27 UTC
(In reply to Carlos O'Donell from comment #1)
> I've filed a Valgrind bug here:
> https://bugs.kde.org/show_bug.cgi?id=421931
> 
> Please feel free to follow up there if you think it needs an update or you
> want to give a +1 regarding your use case.
> 
> I'm marking this RESOLVED/INVALID for glibc.

As discussed in that bug, lets reconsider if this is something that can be cleaned up or clarified in glibc. valgrind itself doesn't use streams, or even link against glibc itself. When valgrind "calls" __libc_freeres the user process as already exitted, no other code/threads that could call into glibc will run during or after the call to __libc_freeres. Are there any other users of __libc_freeres that might not make those guarantees?
Comment 3 Carlos O'Donell 2020-05-29 20:09:52 UTC
(In reply to Mark Wielaard from comment #2)
> (In reply to Carlos O'Donell from comment #1)
> > I've filed a Valgrind bug here:
> > https://bugs.kde.org/show_bug.cgi?id=421931
> > 
> > Please feel free to follow up there if you think it needs an update or you
> > want to give a +1 regarding your use case.
> > 
> > I'm marking this RESOLVED/INVALID for glibc.
> 
> As discussed in that bug, lets reconsider if this is something that can be
> cleaned up or clarified in glibc. valgrind itself doesn't use streams, or
> even link against glibc itself. When valgrind "calls" __libc_freeres the
> user process as already exitted, no other code/threads that could call into
> glibc will run during or after the call to __libc_freeres. Are there any
> other users of __libc_freeres that might not make those guarantees?

The problem is that valgrind is unique. No other library that uses __libc_freeres, including glibc's own mtrace, goes to as much lengths as valgrind does to get out of the way of the runtime. It is expected that __libc_freeres can be called from a destructor, and so we may have threads still writing to unbuffered streams, and we don't want to free those streams if they have internal data (there might be a distinct bug there). Therefore valgrind users whose application touches stderr may always see a leak in stderr, and it's up to valgrind to supress that or not.

In the meantime I'm going to try track down why stderr, unbuffered as it is, allocates anything, which we think might actually be a bug. Even if we fix the bug, valgrind is left with many old glibcs before the new glibc makes it out into production.
Comment 4 Claudio Daffra 2020-06-04 09:45:19 UTC
file to reproduce valgrind error

  #include <stdio.h>
  #include <wchar.h>

  int main ( void )
  {
   return 0 ;
  }

error message

  ==1596== Memcheck, a memory error detector
  ==1596== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
  ==1596== Using Valgrind-3.16.0.GIT and LibVEX; rerun with -h for copyright info
  ==1596== Command: ./prova
  ==1596== 
  ==1596== 
  ==1596== HEAP SUMMARY:
  ==1596==     in use at exit: 13,697 bytes in 158 blocks
  ==1596==   total heap usage: 172 allocs, 14 frees, 18,441 bytes allocated
  ==1596== 
  ==1596== LEAK SUMMARY:
  ==1596==    definitely lost: 0 bytes in 0 blocks
  ==1596==    indirectly lost: 0 bytes in 0 blocks
  ==1596==      possibly lost: 4,392 bytes in 5 blocks
  ==1596==    still reachable: 9,305 bytes in 153 blocks
  ==1596==         suppressed: 0 bytes in 0 blocks
  ==1596== Rerun with --leak-check=full to see details of leaked memory
  ==1596== 
  ==1596== For lists of detected and suppressed errors, rerun with: -s
  ==1596== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)
  claudio@RastanX Documents % 

compiler version

  claudio@RastanX Documents % gcc -v

  Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
  Apple clang version 11.0.3 (clang-1103.0.32.59)
  Target: x86_64-apple-darwin19.5.0
  Thread model: posix
  InstalledDir: /Library/Developer/CommandLineTools/usr/bin

s.o : macOS 10.15.5 (19F96)


with --leak-check=full valgrind gives me many other errors.
this occurs in macosx only,nothing to do with streams.
Comment 5 Mark Wielaard 2020-06-04 12:19:24 UTC
(In reply to Claudio Daffra from comment #4)
> 
> s.o : macOS 10.15.5 (19F96)
> 
> 
> with --leak-check=full valgrind gives me many other errors.
> this occurs in macosx only,nothing to do with streams.

I think the problem is that you are not using glibc on macosx. The libc implementation you are using on macosx seems to have various bugs and/or leaks, or might not support __libc_freeres to coordinate with valgrind. You might want to report those issues to apple, assuming they provided the libc you are using on macOS.
Comment 6 Claudio Daffra 2020-06-04 16:58:54 UTC
Created attachment 12589 [details]
attachment-3917-0.html

Ok i Will  . Thank you .

Il gio 4 giu 2020, 14:19 mark at klomp dot org <
sourceware-bugzilla@sourceware.org> ha scritto:

> https://sourceware.org/bugzilla/show_bug.cgi?id=26022
>
> --- Comment #5 from Mark Wielaard <mark at klomp dot org> ---
> (In reply to Claudio Daffra from comment #4)
> >
> > s.o : macOS 10.15.5 (19F96)
> >
> >
> > with --leak-check=full valgrind gives me many other errors.
> > this occurs in macosx only,nothing to do with streams.
>
> I think the problem is that you are not using glibc on macosx. The libc
> implementation you are using on macosx seems to have various bugs and/or
> leaks,
> or might not support __libc_freeres to coordinate with valgrind. You might
> want
> to report those issues to apple, assuming they provided the libc you are
> using
> on macOS.
>
> --
> You are receiving this mail because:
> You reported the bug.