Bug 13047 - Make 64-bit off_t the default on i386 and others
Summary: Make 64-bit off_t the default on i386 and others
Alias: None
Product: glibc
Classification: Unclassified
Component: libc (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Ulrich Drepper
Depends on:
Reported: 2011-07-31 01:10 UTC by Leslie Satenstein
Modified: 2014-06-13 14:37 UTC (History)
3 users (show)

See Also:
Last reconfirmed:
fweimer: security-


Note You need to log in before you can comment on or make changes to this bug.
Description Leslie Satenstein 2011-07-31 01:10:57 UTC
I created a large file that I truncate to 5GB via Truncate command.

I get a NULL return from  fopen("bigfile","rb"); where bigfile is 5GB exactly.

It will work with smaller files 2.5GB. 

Create file with 
truncate -s 5GB bigfile

The function works ok on Debian Squeeze, but not on UBUNTU.  (both are 32bit versions).  I have not confirmed that it works on Fedora 32bit. 

fopen() works fine with 64bit linux.
Comment 1 Leslie Satenstein 2011-07-31 01:11:42 UTC
Large DVD Iso files can't be copied.
Comment 2 Andreas Jaeger 2011-07-31 18:57:27 UTC
Not a bug in glibc but a limitation of a 32-bit system unless you use the proper interfaces, see for example this web page:
Comment 3 Leslie Satenstein 2011-07-31 23:24:23 UTC
Here is my come-back. Since one can add a #define to allow using buffered IO (fread, fwrite, etc) for files in excess of 2 gigabytes. then this define should be permanent within stdio.h .  I wanted to read an ISO DVD and could not.   I would refer the owner of this maintenance problem to the actual libc.pdf file section 4.17, which makes no mention of the #define.  (However in respoonse to Andreas Jaeger, personal reply (which I appreciate receiving and for which I thank him), using the search of the libc.PDF I did find a chapter about large file support.  

Something is wrong when today a large file is one measured in petabytes. Please redirect this topic and this bug to the documentation project and of course to a steering committee to update the stream IO functionality to handle very large files, or to state that in the libc.pdf fopen() document, to refer to the large file section.

I lost quite a few hours of digging to independently discoover the #define that enables large file support.  I would also ask why large file support is turned on for 64 bit linux?  Had 64 bit linux the same constraint, I would have not been concerned.  (I was porting code from 64bit to 32bit), for older systems.

Thank you again Andreas, for the prompt reply.
Comment 4 Andreas Jaeger 2011-08-01 07:21:49 UTC
This is how POSIX defines it. Changing it is not possible without breaking programs.
Comment 5 Rich Felker 2011-08-07 17:56:48 UTC
Please reopen and rename this bug. It should be called "It's 2011 and -D_FILE_OFFSET_BITS=64 is still not the default." Seriously, the attempt at coexistence between programs compiled correctly and a few that mistakenly omitted -D_FILE_OFFSET_BITS=64 is an endless source of subtle bugs - for instance if a 32-bit-offset program inherits a file descriptor from a 64-bit-offset one, which happens basically anytime a properly built shell runs a 32-bit-offset program.
Comment 6 Rich Felker 2012-05-14 04:22:36 UTC
As per my comment last year, I think this issue merits revisiting. Since the intended protections against bad behavior/file corruption break down when file descriptors are inherited between programs built with differing values for _FILE_OFFSET_BITS, and since it's now 2012, I think there's a strong argument to be made for making 64-bit off_t the default. I know a small but persistent number of programs on Debian and other mainstream dists are still built without any attention to _FILE_OFFSET_BITS, and this leads to broken corner case issues. Fixing this at the libc level makes a lot more sense than fixing it in every single program...
Comment 7 joseph@codesourcery.com 2012-05-14 10:55:19 UTC
Note that using _FILE_OFFSET_BITS=64 introduces namespace issues for which 
I've filed bug 14106 (although hopefully not many real programs actually 
do anything that would be affected by those issues).