A valid use case of sshd is to run in a "chroot" that actually looks nothing like a root filesystem. If this is attempted in a kernel that does not implement the close_range syscall (5.8 or older) the fallback_closefrom function in glibc 2.34 looks for /proc/self/fd and returns an error if it is not found. This implementation may want to consider scenarios where /proc/self/fd is not present.
I have also created https://bugzilla.mindrot.org/show_bug.cgi?id=3349 to track this with openssh.
As Florian has state in the https://bugzilla.mindrot.org/show_bug.cgi?id=3349, there is not much we can do make it work reliable on Linux. The RLIMIT_NOFILE fallback some implementations (such the one used on openssh at openbsd-compat/bsd-closefrom.c) is not possible because it does not really describe the descriptor range, and iterating over all possible file descriptors values (INT_MAX) is prohibitively performance-wise. It might work on openssh since it controls when and how it uses RLIMIT_NOFILE, but it not an option for glibc. So I think it is a fair assumption that if you want to support closefrom() on a kernel without the syscall suppport, you need to provide the another feasible kernel interface to allows it (procfs). Another option is to either abort if /proc can not be opened or remove the fallback (and abort() as well). In any case, I am inclined to close this bug.