Bug 31137 - Regression vfprintf + setbuf (no buffering)
Summary: Regression vfprintf + setbuf (no buffering)
Status: RESOLVED NOTABUG
Alias: None
Product: glibc
Classification: Unclassified
Component: stdio (show other bugs)
Version: 2.37
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-12-11 12:58 UTC by David Marchand
Modified: 2024-04-16 17:21 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments
log.c (583 bytes, text/x-csrc)
2023-12-11 12:58 UTC, David Marchand
Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Marchand 2023-12-11 12:58:41 UTC
Created attachment 15252 [details]
log.c

For the context, OVS has its logging framework (which writes to file, console, syslog etc...). DPDK is a library for networking datapath in userspace, and its components generate log messages through its own framework. DPDK provides a way to redirect those logs by passing a FILE * from fopencookie.
OVS does this and disables buffering.

After updating from f37 to f38 (glibc-2.36-17.fc37.x86_64 to glibc-2.37-13.fc38.x86_64), I noticed a change in how the logs are written for dpdk messages in OVS.

Example of output:
2023-11-17T12:50:38.137Z|00059|dpdk|INFO|VHOST_CONFIG:
(/home/dmarchan/git/pub/ovs/master/build/tests/system-dpdk-testsuite.dir/003/dpdkvhostclient0)
Linear buffers requ
2023-11-17T12:50:38.137Z|00060|dpdk|INFO|ested without external buffers,

The log message has been split in 128 bytes chunks (if I count correctly).

I attached a simple reproducer (to mimick what is done in OVS, which
takes over DPDK logs with a similar fopencookie + setbuf in non
buffering mode).

# From a f37 container:
[root@cb7d749e838f ~]# gcc -o log -Wall -Werror log.c; ./log
no setbuf
=========

Calling fprintf
idx=0 size=511

Calling vfprintf
idx=0 size=511


setbuf _IONBF
=============

Calling fprintf
idx=0 size=511

Calling vfprintf
idx=0 size=511


# From a f38 container:
[root@3fbf304e6894 ~]# gcc -o log -Wall -Werror log.c; ./log
no setbuf
=========

Calling fprintf
idx=0 size=511

Calling vfprintf
idx=0 size=511


setbuf _IONBF
=============

Calling fprintf
idx=0 size=511

Calling vfprintf
idx=0 size=128
idx=1 size=128
idx=2 size=128
idx=3 size=127
Comment 1 Andreas Schwab 2023-12-11 13:32:24 UTC
That doesn't look like a bug.  You are requesting unbuffered output, so the output appears as soon as possible.
Comment 2 Florian Weimer 2023-12-11 13:48:16 UTC
The reproducer is misleading because GCC translates the fprintf call into an fputs call. With -fno-builtin, I get this instead:

“
no setbuf
=========

Calling fprintf
idx=0 size=511

Calling vfprintf
idx=0 size=511


setbuf _IONBF
=============

Calling fprintf
idx=0 size=128
idx=1 size=128
idx=2 size=128
idx=3 size=127

Calling vfprintf
idx=0 size=128
idx=1 size=128
idx=2 size=128
idx=3 size=127
”

So at least it's consistent.

Echoing what Andreas said, I think you are just observing the effect of a reduced buffer size. It was always there, except that we used to have a large on-stack buffer in vfprintf (8192 bytes, I believe), which probably papered over it.

The 128 bytes fallback buffer size is arbitrary. We can probably increase it somewhat if it reduces the compatibility impact of this change. But fopencookie is inherently a byte stream interface, so there is no guarantee that one fprintf call will be translated into one cookie write call.
Comment 3 Andreas Schwab 2023-12-11 15:04:46 UTC
%s used to write the string directly to the stream.
Comment 4 Florian Weimer 2023-12-11 17:37:17 UTC
I'm pretty sure the actual application has a more elaborate format string here. If we don't have a large enough buffer, that will still result in multiple writes even if we can handle arbitrarily large %s strings directly (because we need to flush the previous buffer contents due to lack of vectored writes, and write the subsequent format bits from the buffer).
Comment 5 Carlos O'Donell 2024-04-16 17:21:14 UTC
Given Andreas and Florian's responses I'm marking this closed as not a bug.

If you think that we need to increase the fallback buffer size for the purposes of compatibility then please reopen the issue.