--all-module conflicts with our two remoting facilities, to the extent that the insert_loaded_modules() logic is executed on the compilation host rather than on the execution site.
Note that right now with multiple simultaneous remotes, we only split the compilation base on unique (kernel release, architecture) pairs. If we start incorporating more data about the remote into compilation, then we'll have to split these up further. We could just do separate compilation for all remotes regardless, and then trust the caching to collapse pass-3/4 work. We'd always be duplicating earlier passes though.