Bug 27053

Created attachment 13037 [details]
Test program to examine return value of system()

According to the POSIX specification of the system() function:

       If some error prevents the command language
       interpreter from executing after the child process is
       created, the return value from system() shall be as if
       the command language interpreter had terminated using
       exit(127) or _exit(127).

Thus, for example, if no shell is present, or the shell pathname is not executable, then the return value from system() should be as though the child had done _exit(127).

This was once true in the glibc implementation, but is no longer true. I think the change must have come in glibc 2.29, with 

commit 5fb7fc96350575c9adb1316833e48ca11553be49
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date:   Wed Oct 24 16:29:38 2018 -0300

    posix: Use posix_spawn on system

That commit removed these lines from sysdeps/posix/system.c:

-      /* Exec the shell.  */
-      (void) __execve (SHELL_PATH, (char *const *) new_argv, __environ);
-      _exit (127);

I suspect, but have not tested, that a similar nonconformance to the spec has been triggered in pclose() with the following commit:

commit 14d0e87d9b8caaa2eca7ca81f1189596671fe4fb
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date:   Wed Sep 12 10:32:05 2018 -0300

    posix: Use posix_spawn on popen

====

I've attached a test program. Here's a demo run:

$ ls -l /bin/sh
lrwxrwxrwx. 1 root root 4 Jul 27 15:18 /bin/sh -> bash
$ sudo chmod 0 /bin/bash
$ ./a.out date
system() returned: status=0xffffffff (-1,255)
system: Permission denied

The expected output when running the test program was:

system() returned: status=0x7f00 (127,0)
(Probably) could not invoke shell