[PATCH] ld: Disable separate code by default for Linux/x86

[H.J. Lu]

On Mon, Mar 16, 2026 at 9:21 AM hev <r@hev.cc> wrote:
>
> On Mon, Mar 16, 2026 at 11:58 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> >
> > On Mon, Mar 16, 2026 at 7:20 AM WANG Rui <r@hev.cc> wrote:
> > >
> > > Years ago, commit f6aec96dce1d ("ld: Add --enable-separate-code") introduced
> > > -z separate-code, mainly to avoid mixing code and data in the same segment
> > > for better cache performance, and enabled it by default for Linux/x86.
> >
> > -z separate-code reduces cache pollution and improves security by avoiding
> > putting read-only data in the same page with code.
>
> I see. IIUC, the security aspect you mentioned mainly concerns
> avoiding an executable ELF header. In that case we probably need to
> make a trade-off here.
>
> > > Today, Linux already has working support for mapping executable code with THP,
> > > which can noticeably reduce iTLB pressure and bring measurable performance
> > > benefits.
> > >
> > > In practice, I observed that on AArch64 and LoongArch64 distros, ELF executables
> > > typically place the code PT_LOAD segment first, so both the virtual address and
> > > file offset are 0. That turns out to be an important prerequisite for THP-aware
> >
> > Should ld.so map read-only data segment plus code segment into the same huge
> > page?  This should have the similar effect as -z noseparate-code.
>
> A huge page has a single set of permissions, so it doesn't seem
> possible for part of the page to be read-only and non-executable while
> another part is executable.

With -z separate-code, I got

Program Headers:
  Type           Offset   VirtAddr           PhysAddr
FileSiz  MemSiz   Flg Align
  PHDR           0x000040 0x0000000000000040 0x0000000000000040
0x0002d8 0x0002d8 R   0x8
  INTERP         0x001000 0x0000000000001000 0x0000000000001000
0x00001c 0x00001c R   0x1
      [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
  LOAD           0x000000 0x0000000000000000 0x0000000000000000
0x0004a1 0x0004a1 R E 0x1000
  LOAD           0x001000 0x0000000000001000 0x0000000000001000
0x0003f0 0x0003f0 R   0x1000
  LOAD           0x001d98 0x0000000000002d98 0x0000000000002d98
0x000274 0x000278 RW  0x1000

With -z noseparate-code, I got

Program Headers:
  Type           Offset   VirtAddr           PhysAddr
FileSiz  MemSiz   Flg Align
  PHDR           0x000040 0x0000000000000040 0x0000000000000040
0x0002a0 0x0002a0 R   0x8
  INTERP         0x000304 0x0000000000000304 0x0000000000000304
0x00001c 0x00001c R   0x1
      [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
  LOAD           0x000000 0x0000000000000000 0x0000000000000000
0x000858 0x000858 R E 0x1000
  LOAD           0x000d98 0x0000000000001d98 0x0000000000001d98
0x000274 0x000278 RW  0x1000

Please remind me of the benefits of -z noseparate-code.  If ld.so maps

  LOAD           0x000000 0x0000000000000000 0x0000000000000000
0x0004a1 0x0004a1 R E 0x1000
  LOAD           0x001000 0x0000000000001000 0x0000000000001000
0x0003f0 0x0003f0 R   0x1000

in the same huge page with RW permission, will we get the same benefits?

> > > PT_LOAD alignment optimizations [1][2][3], and the data shows clear performance
> > > wins on both AArch64 and LoongArch64. [4][1]
> > >
> > > Linux/x86 is a bit of an outlier here: the default -z separate-code setting
> > > happens to break this layout. So this patch changes the Linux/x86 default to
> > > -z noseparate-code.
> > >
> > > Benchmark
> > >
> > > Machine: AMD Ryzen 9 7950X (x86_64)
> > > Binutils: 2.46
> > > GCC: 15.2.1 (built with -z,noseparate-code + --enable-host-pie)
> > > Linux kernel: 6.19.6 with patch [1]
> > >
> > > Workload: building Linux v7.0-rc1 vmlinux with x86_64_defconfig.
> > >
> > >                 Without patch        With patch
> > > instructions    8,246,133,611,932    8,246,025,137,750
> > > cpu-cycles      8,001,028,142,928    7,565,925,107,502
> > > itlb-misses     3,672,158,331        26,821,242
> > > time elapsed    64.66 s              61.97 s
> > >
> > > Instructions are basically unchanged. iTLB misses drop from ~3.67B to
> > > ~26M (~99.27% reduction), which results in about a ~5.44% reduction in
> > > cycles and ~4.18% shorter wall time for this workload.
> >
> > Hongtao, haochecn, can you work with Rui to measure -z noseparate-code
> > performance impact on SPEC CPU?
>
> Also just a reminder that this optimization depends on Linux kernel
> support for file-backend THP,
> specifically:
>
> * `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
> * `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.
>
> Thanks,
> Rui



-- 
H.J.