Bug 5758 - [patch] Fix getrlimit() acting funny sometimes
Summary: [patch] Fix getrlimit() acting funny sometimes
Status: RESOLVED WONTFIX
Alias: None
Product: glibc
Classification: Unclassified
Component: libc (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Ulrich Drepper
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-02-12 14:29 UTC by Thomas Jarosch
Modified: 2014-07-02 07:29 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments
Initialize __have_no_new_getrlimit (325 bytes, patch)
2008-02-12 14:30 UTC, Thomas Jarosch
Details | Diff
Correct signed values also for ugetrlimit (763 bytes, patch)
2008-02-14 16:08 UTC, Thomas Jarosch
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Jarosch 2008-02-12 14:29:25 UTC
Hey there,

attached patch fixes a little bug I found while tracing some getrlimit issues 
on my system. Please have a look, I couldn't find a place 
where "__have_no_new_getrlimit" seems to get initialized.

The patch is still valid for glibc HEAD.

Currently pthread_create fails on my system as ugetrlimit(STACK_SIZE) 
retuns "2147483648". That -almost- half the size of RLIM_INFINITY.
mmap() in ntpl/allocatestack.c then tries to get 2GB of memory on my 1 GB 
box ;-) Still investigating...

Cheers,
Thomas
Comment 1 Thomas Jarosch 2008-02-12 14:30:02 UTC
Created attachment 2246 [details]
Initialize __have_no_new_getrlimit
Comment 2 Ulrich Drepper 2008-02-12 15:23:04 UTC
The patch is of course complete nonsense.  This is a global variable and hence
it is automatically initialize.  If this basic C programming knowledge isn't
known to you you really shouldn't touch libc code.

As for a  problem, I don't know of any and you have provided no indication
whatsoever.  You'll have to provide a test case reproducing the problem or I'll
close the bug.
Comment 3 Thomas Jarosch 2008-02-12 16:09:19 UTC
Ha, you caught me! I was not aware that global vars get initialized to zero 
with all the years experience in C as I always initialize my vars.
Guess I was confused after all the hours searching for the root of the 
ugetrlimit issue. You can close the bug while I'm still searching for the 
source of it.

btw: What's the best way to print/dump variables inside glibc? getrlimit() 
gets called very early in ntpl/init.c and fprintf(stderr) doesn't seem to work 
yet.
Comment 4 Thomas Jarosch 2008-02-12 18:40:57 UTC
Ulrich,

Concerning the rlimit issue:
My main system is running glibc 2.1.3 and kernel 2.6.23.
I've added some debug loggers to the kernel and traced
bash's 2.04 "ulimit -s unlimited" call. This ends up
in the kernel as sys_setrlimit(RLIMIT_STACK) -> 2147483647.

That's the signed version of RLIMIT_INFINITY missing one bit, right?
As far as I can tell there is no translation layer for setrlimit
in the kernel and there is only a ugetrlimit call but no usetrlimit call.
-> Old glibc versions will set wrong values.

As I'm currently in the process of switching the primary glibc to 2.7
I can add a small patch to nptl/init.c to get "__default_stacksize"
to a reasonable value until the switch over is complete.

OTOH we could modify unix/sysv/linux/i386/getrlimit.c to apply
the same "compat" code for ugetrlimit as for getrlimit.
Who want's a stack size of -exactly- 2GB for the threads anyway?!

This solves the famous error message like:
"gnome-terminal: allocatestack.c:370: allocate_stack: Assertion `size != 0' 
failed."

Here are just two examples of users facing the problem:

http://hercules.herts.ac.uk/~mjh/limits-conf-problem.html

or

http://nptl.bullopensource.org/phpBB/viewtopic.php?p=290


Hope you don't try to kill me again ;-)

Regards,
Thomas
Comment 5 Thomas Jarosch 2008-02-14 16:08:38 UTC
Created attachment 2263 [details]
Correct signed values also for ugetrlimit

Attached patch fixes the getrlimit call for mixed new/old glibc setups as
described above. It should also fix it for systems running a current glibc
only, but still having the old signed limits somewhere in the system.
(f.e. pam_limits' /etc/security/limits.conf)
Comment 6 Ulrich Drepper 2008-03-09 05:48:40 UTC
(In reply to comment #5)
> Attached patch fixes the getrlimit call for mixed new/old glibc setups as
> described above.

I won't cause potential problems for all users in the future just because some
people run for the time being these combined systems.  2G is no unreasonable
value to use.  Just get the system to the point where you throw away the old code.