prlimit64: inconsistencies between kernel and userland

[Aurelien Jarno]

Any news about this issue? It really starts to causes a lot of issues in
Debian. I have added a Cc: to libc people so that we can also hear their
opinion.

Thanks,
Aurelien

On Fri, Jun 28, 2013 at 03:38:35PM +0200, Aurelien Jarno wrote:
> Hi,
> 
> There is an inconsistency in the definition of RLIM64_INFINITY between
> kernel and userland for the O32 and N32 ABI:
> 
> On the kernel side, the value is defined for all architectures as
> include/uapi/linux/resource.h:
> 
> | #define RLIM64_INFINITY           (~0ULL)
> 
> On the GNU libc side, the value is defined in
> ports/sysdeps/unix/sysv/linux/mips/bits/resource.h:
> 
> For the O32 and N32 ABI:
> 
> | #  define RLIM64_INFINITY 0x7fffffffffffffffULL
> 
> and for the N64 ABI:
> 
> | #  define RLIM64_INFINITY 0xffffffffffffffffUL
> 
> This was not a problem until the prlimit64 syscall was wired in the
> 2.6.36 kernel. Since then, on a 64-bit kernel and an O32 or N32
> userland, but not on a 32-bit kernel, this causes some issues for
> example it's not possible to set "ulimit -c unlimited" using dash as a
> non-priviledged user.
> 
> This is due to the fact that when available the glibc uses the prlimit64
> syscall to implement setrlimit64, which is called from called from
> pam_limits.so. Instead of setting the limit to infinity (according to
> the userland), it is set to a very big value but still lower than
> infinity (according to the kernel). When later the setrlimit syscall is
> used to set the value to infinity, it gets an EPERM value as the kernel
> consider that as an increase of the limit (from a big value to
> infinity).
> 
> I don't know where the issue should be fixed. The GNU libc has this
> value for more than 7 years, and as it is defined in a header file, it
> means a lot of binaries are broken when used with a 2.6.36+ kernel.
> Fixing that on the kernel side means that MIPS would have a different
> value than other architectures.
> 
> How do you think the issue should be fixed?
> 
> Regards,
> Aurelien
> 
> -- 
> Aurelien Jarno	                        GPG: 1024D/F1BCDB73
> aurelien@aurel32.net                 http://www.aurel32.net



-- 
Aurelien Jarno	                        GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20131104/886cdab4/attachment.sig>