Bug 16596 - Support DW_OP_GNU_entry_value in location expressions
Summary: Support DW_OP_GNU_entry_value in location expressions
Status: RESOLVED FIXED
Alias: None
Product: systemtap
Classification: Unclassified
Component: translator (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Jafeer Uddin
URL:
Keywords:
: 22000 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-02-15 23:05 UTC by Mark Wielaard
Modified: 2018-12-18 16:36 UTC (History)
5 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Wielaard 2014-02-15 23:05:13 UTC
Newer GCC releases might produce DWARF location expressions for more arguments or variables using DW_OP_GNU_entry_value. loc2c currently doesn't support such location expressions. The description of DW_OP_GNU_entry_value (as DWARF5 extension) can be found at http://dwarfstd.org/ShowIssue.php?issue=100909.1

DW_OP_GNU_entry_value provides an DWARF expression (that can be retrieved through dwarf_getlocation_attr in libdw elfutils 0.157+ as a location attribute). Any register op used in the DW_OP_GNU_entry_value expression needs to be evaluated as the value that register had upon entering the subprogram.

Getting the value of the register at the start of the subprogram can be done in several ways.

If the register was used to pass an argument to the function then it might be retrieved through finding the DW_TAG_call_site with DW_AT_low_pc pointing to the function. The DW_TAG_call_site_parameter with a DW_AT_location of the register will have as value the DW_AT_GNU_call_site_value. Note that this location expression itself might use registers that need to be evaluated in the caller frame (so might need an unwind step, which might fail to restore the value if that register was caller-saved). Since there might be multiple call sites the found value depends on the return address of the current function at runtime (so might need a lookup table at runtime).

If the register was callee saved it should be possible to retrieve the value of the register on function entry through CFI. This might be possible at translation time using dwarf_cfi_addrframe and dwarf_frame_register.

Finally the needed registers can be collected by introducing an extra probe at the first instruction of the subprogram. That probe stores the needed registers as thread local values that can be retrieved by the probe later in the function that might need them. This might also need an extra retprobe on the function to delete the copy of the registers again.

Some example location expressions using DW_OP_GNU_entry_value can be found in the elfutils tests/run-varlocs.sh.
Comment 1 David Smith 2014-03-05 16:42:29 UTC
Note that I'm seeing DW_OP_GNU_entry_value errors on rawhide x86_64 (3.14.0-0.rc4.git3.1.fc21.x86_64) with HEAD systemtap:

====
# stap -vp4 -e 'probe kernel.function("__handle_mm_fault@mm/memory.c") { print($flags) }'
Pass 1: parsed user script and 100 library script(s) using 211944virt/29108res/2876shr/26548data kb, in 220usr/70sys/291real ms.
semantic error: unhandled DW_OP_GNU_entry_value in DWARF expression [0] at 0 (0xf3: 1, 139892792287890): identifier '$flags' at <input>:1:64
        source: probe kernel.function("__handle_mm_fault@mm/memory.c") { print($flags) }
                                                                               ^

Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0 global(s) using 451824virt/112528res/24536shr/91000data kb, in 630usr/1250sys/1888real ms.
Pass 2: analysis failed.  [man error::pass2]
====
Comment 2 Mark Wielaard 2014-03-10 09:41:11 UTC
(In reply to David Smith from comment #1)
> Note that I'm seeing DW_OP_GNU_entry_value errors on rawhide x86_64
> (3.14.0-0.rc4.git3.1.fc21.x86_64) with HEAD systemtap:
> 
> unhandled DW_OP_GNU_entry_value in DWARF expression [0] at 0 (0xf3: 1, 139892792287890)

Could you try to get a bit more information about this? (We also might want to extend the error/warning message since it doesn't provide the DIE offset where this happened.) It would be nice to know exactly at which address stap places the probe and which location list entry contained the DW_OP_GNU_entry_value.
Comment 3 Mark Wielaard 2014-03-13 17:13:57 UTC
OK, with extra -vv we see:

focused on module 'kernel'
selected source file 'mm/memory.c'
selected inline instance of __handle_mm_fault
entry-pc lookup (dwarf_entrypc dieoffset: 0x170f191) = 0xffffffff811e11a9 (rc 0)
querying entrypc 0xffffffff811e11a9 of instance of inline '__handle_mm_fault'
probe __handle_mm_fault@mm/memory.c:3696 kernel reloc=.dynamic pc=0xffffffff811e11a9
finding location for local 'flags' near address 0xffffffff811e11a9, module bias 0
get_cfa_ops @0xffffffff811e11a9, module_start @0xffffffff81000000
found cfa, info: 16 [start: 0xffffffff811e112b, end: 0xffffffff811e153a, nops: 1

 [170f191]      inlined_subroutine
               abstract_origin      (ref4) [170ee49]
               entry_pc             (addr) 0xffffffff811e11a9
               ranges               (sec_offset) range list [112810]
               call_file            (data1) 4
               call_line            (data2) 3799
               sibling              (ref4) [1711432]
 [170f1a9]        formal_parameter
                 abstract_origin      (ref4) [170ee7d]
                 location             (sec_offset) location list [5dac88]
 [170f1b2]        formal_parameter
                 abstract_origin      (ref4) [170ee71]
                 location             (sec_offset) location list [5dacf0]
 [170f1bb]        formal_parameter
                 abstract_origin      (ref4) [170ee65]
                 location             (sec_offset) location list [5dae6e]
 [170f1c4]        formal_parameter
                 abstract_origin      (ref4) [170ee5a]
                 location             (sec_offset) location list [5dafd6]

The abstract origin of this inlined_subroutine is (as expected):

 [170ee49]    subprogram
             name                 (strp) "__handle_mm_fault"
             decl_file            (data1) 4
             decl_line            (data2) 3696
             prototyped           (flag_present) Yes
             type                 (ref4) [16f3cb4]
             inline               (data1) inlined (1)

And the first formal_parameter [170f1a9] origin is flags:

 [170ee7d]      formal_parameter
               name                 (strp) "flags"
               decl_file            (data1) 4
               decl_line            (data2) 3697
               type                 (ref4) [16f3cc6]

Which can be found through location list [5dac88]:

 [5dac88]  0xffffffff811e11a9..0xffffffff811e1544 [   0] GNU_entry_value:
       [   0] reg2
                                                  [   3] stack_value
           0xffffffff811e1573..0xffffffff811e1e7c [   0] GNU_entry_value:
       [   0] reg2
                                                  [   3] stack_value
           0xffffffff811e1e95..0xffffffff811e1fbe [   0] GNU_entry_value:
       [   0] reg2
                                                  [   3] stack_value
           0xffffffff811e1fdc..0xffffffff811e205f [   0] GNU_entry_value:
       [   0] reg2
                                                  [   3] stack_value

And the pc (ffffffff811e11a9) we are interested in the first range, which (like all other ranges) indeed tells us the value can be recovered through the value of reg2 as it was on entry of the subprogram (where this was inlined). Which is:

 [170eff4]    subprogram
             external             (flag_present) Yes
             name                 (strp) "handle_mm_fault"
             decl_file            (data1) 4
             decl_line            (data2) 3779
             prototyped           (flag_present) Yes
             type                 (ref4) [16f3cb4]
             low_pc               (addr) 0xffffffff811e1110
             high_pc              (data8) 3919 (0xffffffff811e205f)
             frame_base           (exprloc) 
              [   0] call_frame_cfa
             GNU_all_call_sites   (flag_present) Yes


So if we look at the source and disassambly:

int handle_mm_fault(struct mm_struct *mm, struct vm_area_struct *vma,
                    unsigned long address, unsigned int flags)
{
        int ret;

        __set_current_state(TASK_RUNNING);

        count_vm_event(PGFAULT);
        mem_cgroup_count_vm_event(mm, PGFAULT);

        /* do counter updates before entering really critical section. */
        check_sync_rss_stat(current);

        /*
         * Enable the memcg OOM handling for faults triggered in user
         * space.  Kernel faults are handled more gracefully.
         */
        if (flags & FAULT_FLAG_USER)
                mem_cgroup_oom_enable();

        ret = __handle_mm_fault(mm, vma, address, flags);  // <= THIS INLINE

        if (flags & FAULT_FLAG_USER) {
                mem_cgroup_oom_disable();
                /*
                 * The task may have entered a memcg OOM situation but
                 * if the allocation error was handled gracefully (no
                 * VM_FAULT_OOM), there is no need to kill anything.
                 * Just clean up the OOM state peacefully.
                 */
                if (task_in_memcg_oom(current) && !(ret & VM_FAULT_OOM))
                        mem_cgroup_oom_synchronize(false);
        }

        return ret;
}

ffffffff811e1110 <handle_mm_fault>:
handle_mm_fault():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/mm/memory.c:3781
ffffffff811e1110:       e8 2b 44 60 00          callq  ffffffff817e5540 <__entry_text_start>
ffffffff811e1115:       55                      push   %rbp
ffffffff811e1116:       41 89 c9                mov    %ecx,%r9d
ffffffff811e1119:       48 89 e5                mov    %rsp,%rbp
ffffffff811e111c:       41 57                   push   %r15
ffffffff811e111e:       41 56                   push   %r14
ffffffff811e1120:       49 89 fe                mov    %rdi,%r14
ffffffff811e1123:       41 55                   push   %r13
ffffffff811e1125:       49 89 f5                mov    %rsi,%r13
ffffffff811e1128:       41 54                   push   %r12
ffffffff811e112a:       53                      push   %rbx
ffffffff811e112b:       48 89 d3                mov    %rdx,%rbx
ffffffff811e112e:       48 83 ec 70             sub    $0x70,%rsp
ffffffff811e1132:       65 48 8b 04 25 28 00    mov    %gs:0x28,%rax
ffffffff811e1139:       00 00 
ffffffff811e113b:       48 89 45 d0             mov    %rax,-0x30(%rbp)
ffffffff811e113f:       31 c0                   xor    %eax,%eax
get_current():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/arch/x86/include/asm/current.h:14
ffffffff811e1141:       65 48 8b 04 25 80 c9    mov    %gs:0xc980,%rax
ffffffff811e1148:       00 00 
handle_mm_fault():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/mm/memory.c:3784
ffffffff811e114a:       48 c7 00 00 00 00 00    movq   $0x0,(%rax)
mem_cgroup_disabled():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/include/linux/memcontrol.h:165
ffffffff811e1151:       8b 3d dd a6 ba 00       mov    0xbaa6dd(%rip),%edi        # ffffffff81d8b834 <mem_cgroup_subsys+0x54>
count_vm_event():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/include/linux/vmstat.h:37
ffffffff811e1157:       65 48 ff 04 25 58 0e    incq   %gs:0x1d0e58
ffffffff811e115e:       1d 00 
mem_cgroup_disabled():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/include/linux/memcontrol.h:165
ffffffff811e1160:       85 ff                   test   %edi,%edi
ffffffff811e1162:       75 14                   jne    ffffffff811e1178 <handle_mm_fault+0x68>
mem_cgroup_count_vm_event():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/include/linux/memcontrol.h:224
ffffffff811e1164:       be 0b 00 00 00          mov    $0xb,%esi
ffffffff811e1169:       4c 89 f7                mov    %r14,%rdi
ffffffff811e116c:       89 4d c0                mov    %ecx,-0x40(%rbp)
ffffffff811e116f:       e8 2c e6 03 00          callq  ffffffff8121f7a0 <__mem_cgroup_count_vm_event>
ffffffff811e1174:       44 8b 4d c0             mov    -0x40(%rbp),%r9d
get_current():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/arch/x86/include/asm/current.h:14
ffffffff811e1178:       65 48 8b 04 25 80 c9    mov    %gs:0xc980,%rax
ffffffff811e117f:       00 00 
check_sync_rss_stat():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/mm/memory.c:164
ffffffff811e1181:       8b 90 a8 06 00 00       mov    0x6a8(%rax),%edx
ffffffff811e1187:       8d 4a 01                lea    0x1(%rdx),%ecx
ffffffff811e118a:       83 fa 40                cmp    $0x40,%edx
ffffffff811e118d:       89 88 a8 06 00 00       mov    %ecx,0x6a8(%rax)
ffffffff811e1193:       0f 8f e3 0c 00 00       jg     ffffffff811e1e7c <handle_mm_fault+0xd6c>
handle_mm_fault():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/mm/memory.c:3796
ffffffff811e1199:       45 89 cf                mov    %r9d,%r15d
ffffffff811e119c:       41 81 e7 80 00 00 00    and    $0x80,%r15d
ffffffff811e11a3:       0f 85 9f 03 00 00       jne    ffffffff811e1548 <handle_mm_fault+0x438>
__handle_mm_fault():
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/mm/memory.c:3704
ffffffff811e11a9:       41 f6 45 52 40          testb  $0x40,0x52(%r13)
ffffffff811e11ae:       0f 85 af 0c 00 00       jne    ffffffff811e1e63 <handle_mm_fault+0xd53>
/usr/src/debug/kernel-3.13.fc21/linux-3.14.0-0.rc5.git2.1.fc21.x86_64/mm/memory.c:3707
ffffffff811e11b4:       49 89 dc                mov    %rbx,%r12
ffffffff811e11b7:       49 c1 ec 24             shr    $0x24,%r12
ffffffff811e11bb:       41 81 e4 f8 0f 00 00    and    $0xff8,%r12d
ffffffff811e11c2:       4d 03 66 40             add    0x40(%r14),%r12
ffffffff811e11c6:       49 8b 04 24             mov    (%r12),%rax

So between the start of the function ffffffff811e1110 and the first inlined address ffffffff811e11a9 a lot happens that probably clobbers reg2/rcx, which makes it non-trivial to retrieve. It does look like the value of flags is used later so maybe GCC could provide its value differently, but I am not sure.
Comment 4 Mark Wielaard 2014-03-14 08:50:14 UTC
Note that for the example in comment #3 we cannot use the simple unwinding approach number 2 from the description because r2 isn't saved at all. Looking at the CFI:

 [ 7a590] FDE length=68 cie=[ 79ab0]
   CIE_pointer:              498352
   initial_location:         0xffffffff811e1110 <handle_mm_fault>
   address_range:            0xf4f

   Program:
     advance_loc 6 to 0x6
     def_cfa_offset 16
     offset r6 (rbp) at cfa-16
     advance_loc 6 to 0xc
     def_cfa_register r6 (rbp)
     advance_loc 4 to 0x10
     offset r15 (r15) at cfa-24
     offset r14 (r14) at cfa-32
     advance_loc 5 to 0x15
     offset r13 (r13) at cfa-40
     advance_loc 6 to 0x1b
     offset r12 (r12) at cfa-48
     offset r3 (rbx) at cfa-56
     advance_loc2 1039 to 0x42a
     remember_state
     restore r3 (rbx)
     advance_loc 2 to 0x42c
     restore r12 (r12)
     advance_loc 2 to 0x42e
     restore r13 (r13)
     advance_loc 2 to 0x430
     restore r14 (r14)
     advance_loc 2 to 0x432
     restore r15 (r15)
     advance_loc 1 to 0x433
     restore r6 (rbp)
     def_cfa r7 (rsp) at offset 8
     advance_loc 5 to 0x438
     restore_state
     nop
     nop
     nop
     nop
     nop
     nop
     nop

(I didn't save the CIE, but that didn't save it either.)

Unfortunately I don't have access anymore to this setup so cannot see whether unwinding and finding the matching DW_TAG_call_site_parameter would work (approach 1 in the description).
Comment 5 David Smith 2017-09-12 17:10:11 UTC
*** Bug 22000 has been marked as a duplicate of this bug. ***
Comment 6 Jafeer Uddin 2018-12-18 16:36:07 UTC
Support of DW_OP_GNU_entry_value has been added in commit 68bd23fd0cc5efa