Bug 12201 - Fix setrlimit/getrlimit 32-bit platforms for limits > 2^32-1
Summary: Fix setrlimit/getrlimit 32-bit platforms for limits > 2^32-1
Status: RESOLVED FIXED
Alias: None
Product: glibc
Classification: Unclassified
Component: libc (show other bugs)
Version: 2.12
: P2 normal
Target Milestone: ---
Assignee: Ulrich Drepper
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-11-07 15:01 IST by Michael Kerrisk
Modified: 2014-06-30 06:29 IST (History)
0 users

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Kerrisk 2010-11-07 15:01:50 IST
The current glibc wrappers for getrlimit/setrlimit do some work to paper over this kernel bug:
https://bugzilla.kernel.org/show_bug.cgi?id=5042
With the kernel 2.6.36 addition of the prlimit(2) syscall, the wrappers could be written to use prlimit() and thus avoid the problem resulting from the broken kernel syscalls.
Comment 1 Ulrich Drepper 2010-11-08 20:16:22 IST
What exactly are you talking about?  We are using ugetrlimit for some time.  At the same time setrlimit was changed.  These syscalls now match exactly the required userlevel semantic.
Comment 2 Michael Kerrisk 2010-11-11 05:33:45 IST
(In reply to comment #1)
> What exactly are you talking about?

It was fairly carefully explained in the kernel.org bug report...

> We are using ugetrlimit for some time.  At
> the same time setrlimit was changed.  These syscalls now match exactly the
> required userlevel semantic.

No, they do not. Look at the following.

$ cat rlimit_large.c 
/* rlimit_large.c
*/
#define _FILE_OFFSET_BITS 64
#include <sys/resource.h>
#include <sys/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>

#define errMsg(msg)             { perror(msg); }

#define errExit(msg)            { perror(msg); exit(EXIT_FAILURE); }

static void
printRlimit(const char *msg, int resource)
{
    struct rlimit rlim;

    if (getrlimit(resource, &rlim) == -1) errExit("getrlimit");

    printf("%s soft=", msg);
    if (rlim.rlim_cur == RLIM_INFINITY)
        printf("infinite");
    else if (rlim.rlim_cur == RLIM_SAVED_CUR)
        printf("unrepresentable");
    else
        printf("%lld", (long long) rlim.rlim_cur);

    printf("; hard=");
    if (rlim.rlim_max == RLIM_INFINITY)
        printf("infinite\n");
    else if (rlim.rlim_max == RLIM_SAVED_MAX)
        printf("unrepresentable");
    else
        printf("%lld\n", (long long) rlim.rlim_max);
} /* printRlimit */

int
main(int argc, char *argv[])
{
    struct rlimit rl;
    printf("sizeof(rl.rlim_cur) = %ld\n", (long) sizeof(rl.rlim_cur));

    /* Show the value of the glibc constant, just for interest */

    printf("RLIM_INFINITY = %llx\n", (unsigned long long) RLIM_INFINITY);

    if (getrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    printRlimit("Initial RLIMIT_FSIZE limits          : ", RLIMIT_FSIZE);

#define VAL1 10000
    rl.rlim_cur = VAL1;
    printf("About to set rlim_cur to %lld\n", (long long) rl.rlim_cur);
    if (setrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    if (getrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    printRlimit("RLIMIT_FSIZE limits after setrlimit(): ", RLIMIT_FSIZE);

#define VAL2 4999222333LL
    rl.rlim_cur = VAL2;
    printf("About to set rlim_cur to %lld\n", (long long) rl.rlim_cur);
    if (setrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    if (getrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    printRlimit("RLIMIT_FSIZE limits after setrlimit(): ", RLIMIT_FSIZE);

    exit(EXIT_SUCCESS);
} /* main */

$ cc rlimit_large.c 
$ ./a.out
sizeof(rl.rlim_cur) = 8
RLIM_INFINITY = ffffffffffffffff
Initial RLIMIT_FSIZE limits          :  soft=infinite; hard=infinite
About to set rlim_cur to 10000
RLIMIT_FSIZE limits after setrlimit():  soft=10000; hard=infinite
About to set rlim_cur to 4999222333
RLIMIT_FSIZE limits after setrlimit():  soft=infinite; hard=infinite


The last line of output indicates the problem: a limit > (2^32)-1 got turned into RLIM_INFINITY, even though it is representable in 8 bytes. It's a kernel problem, but glibc's wrappers could now workaround that kernel problem by using prlimit().
Comment 3 Ulrich Drepper 2010-11-11 09:09:37 IST
If you would only learn to express yourself correctly and concisely you could save yourself work.

Why on earth would you say setrlimit has problems if you really mean setrlimit64?  That's not at all the same code.
Comment 4 Ulrich Drepper 2010-12-26 01:13:06 IST
I checked in a patch.
Comment 5 Jackie Rosen 2014-02-16 18:24:37 IST Comment hidden (spam)