Re: [PING][PATCH] Warn about using _ino_t without -D_FILE_OFFSET_BITS=64.

[Paul Eggert <eggert at cs dot ucla dot edu>]

On 03/21/2014 06:11 AM, Joseph S. Myers wrote:
> * Adding *64 versions of interfaces without them (at least fts).

This is a messy situation.  GNU applications typically use Gnulib's 
version of fts, and Gnulib's interface has diverged from libc's.  The 
Gnulib interface and implementation support features that glibc doesn't, 
e.g., checking for directory loops, and these are used by programs like 
'rm -r' and 'chmod -R'.

One possible way forward would be to deprecate the current fts API, and 
switch to the gnulib API (with room for future extensions), retaining 
binary support for old applications that dynamically link to the new 
glibc.  This would be some work, though.

> * Refining conditions used in headers (at least fts.h) to reject use in a
> particular _FILE_OFFSET_BITS mode.

I looked into this a bit for fts.h, and expect that it's more trouble 
than it's worth due to glitches similar to what you mentioned for MIPS 
n64.  For example, on 64-bit hosts sysdeps/mach/hurd/bits/stat.h appends 
slightly different padding at struct stat's end when _FILE_OFFSET_BITS 
is 64 (why, I don't know), making struct stat incompatible with struct 
stat64.  In practice, people building 64-bit applications rarely define 
_FILE_OFFSET_BITS to be 64 (because Autoconf and Gnulib don't do that), 
so it's probably not worth our while to improve support for that 
combination.

It is odd that libc supports two different 64-bit ABIs for 'stat' etc. 
on all 64-bit platforms, one hardly ever used.  It might make sense to 
deprecate the rarely-used ABI, i.e., have _FILE_OFFSET_BITS=64 be a 
no-op on 64-bit platforms.  This should fix the C++ ABI problem you 
mentioned (bug #15766).

> we need an in-depth
> analysis showing what the effects would be in practice.  For all the
> library headers in a large GNU/Linux distribution, how many use any
> affected type in an ABI, and what settings are those libraries built with?

It'd take quite some time to generate such an analysis.  I grabbed all 
the Fedora 20 -devel packages that I could (i.e., all those that didn't 
conflict with something already installed), and ran the attached script 
while in /usr/include, and this generated 2847 lines of output.  Not 
every line needs to be looked at in detail, but enough do.  I can tell 
you the results of looking at the first 45 lines of output of 
"file_offset *", though. I found four categories of libraries:

1. These libraries are compiled with -D_FILE_OFFSET_BITS=64, so 
theycurrently should have problems unless applications are also built 
with-D_FILE_OFFSET_BITS=64.  Changing the default will fix these problems.

flac
fltk
GraphicsMagick
hdf5
ImageMagick

2. These libraries use a type like off_t in macros or static or 
inlinefunctions, but it's not part of the binary API.  Changing the 
defaultto 64-bit will therefore not affect the library's API.  Programs 
recompiled with the new default will have larger off_t etc., just asthey 
will with glibc.

libeplplot
InsightToolkit

3. These libraries use a type like off_t in defining a data type that 
isnot used anywhere (either as part of the API, or internally), so 
changingthe default shouldn't affect them.

GeoIP

4. These libraries use a type like off_t in redeclaring 
standardfunctions like 'stat', which should continue to work if the 
defaultis changed.

ice


From looking at the above, it appears that changing the default 
_FILE_OFFSET_BITS to 64 will fix more bugs than it'll cause.  Of course 
this is just one small sample.  I stopped at 45 lines of grep output 
because the next lines were big batches from Qt and from X11; these will 
be a pain to analyze, and it's not clear to me what the results will be.

One other data point: on 32-bit platforms Perl and Python both assume 
_FILE_OFFSET_BITS=64, so I suspect that libraries linked with these 
programs can have problems if they are compiled without 
_FILE_OFFSET_BITS=64.

Attachment: file_offset
Description: Text document