On Fedora rawhide (kernel-5.0.0-0.rc6.git1.1.fc30.x86_64) a couple of the systemtap examples whythefail.stp and varwatch.stp tests runs probe kernel.statement("kernel_function"@file.c:*"). The scripts fail to run due to tapsets.cxx throwing the error: inconsistent relocation address Below is the parts of the systemtap.log with the failures: PRETEST PWD=/usr/share/systemtap/testsuite meta taglines '' tag 'test_support' value '' TEST PWD=/usr/share/systemtap/testsuite/systemtap.examples/general meta taglines 'test_check: stap -wp4 varwatch.stp 'kernel.statement("do_sys_open@fs/open.c:*")' '$$vars'' tag 'test_check' value 'stap -wp4 varwatch.stp 'kernel.statement("do_sys_open@fs/open.c:*")' '$$vars'' attempting command stap -wp4 varwatch.stp 'kernel.statement("do_sys_open@fs/open.c:*")' '$$vars' OUT semantic error: inconsistent relocation address: keyword at varwatch.stp:5:1 source: probe $1 { ^ Pass 2: analysis failed. [man error::pass2] Number of similar error messages suppressed: 14. Rerun with -v to see them. child process exited abnormally RC 1 FAIL: systemtap.examples/general/varwatch build PRETEST PWD=/usr/share/systemtap/testsuite meta taglines '' tag 'test_support' value '' TEST PWD=/usr/share/systemtap/testsuite/systemtap.examples/general meta taglines 'test_check: stap -p4 whythefail.stp kernel do_sys_open '$return < 0'' tag 'test_check' value 'stap -p4 whythefail.stp kernel do_sys_open '$return < 0'' attempting command stap -p4 whythefail.stp kernel do_sys_open '$return < 0' OUT semantic error: inconsistent relocation address: keyword at whythefail.stp:37:1 source: probe $1 . statement(@2 "@*:*") { ^ Pass 2: analysis failed. [man error::pass2] Number of similar error messages suppressed: 14. Rerun with -v to see them. child process exited abnormally RC 1 FAIL: systemtap.examples/general/whythefail build The linetime example is also using a 'kernel.statement("do_nanosleep@*:*")', but that does not trigger the error and the test passes. It looks like something is going wrong in the assert tests for the code below (some diagnostic prints showed that the section was "_stext" rather than "" for a run of varwatch.stp: // Assert kernel relocation invariants if (section == "" && dwfl_addr != addr) // addr should be absolute throw SEMANTIC_ERROR (_("missing relocation basis"), tok); if (section != "" && dwfl_addr == addr) // addr should be an offset throw SEMANTIC_ERROR (_("inconsistent relocation address"), tok); }
Created attachment 11609 [details] Diagnostic to see what values are used for the assert tests To get a better idea of what is happening have this patch printing out information each time those asserts are run: For nanosleep function everything fine: [wcohen@cervelo systemtap]$ ../install/bin/stap -wp4 testsuite/systemtap.examples/general/varwatch.stp 'kernel.statement("do_nanosleep@*:*")' '$$vars' section=_stext dwfl_addr=ffffffff819434f0 addr=9434f0 section=_stext dwfl_addr=ffffffff81943517 addr=943517 section=_stext dwfl_addr=ffffffff8194352c addr=94352c section=_stext dwfl_addr=ffffffff81943559 addr=943559 section=_stext dwfl_addr=ffffffff81943561 addr=943561 section=_stext dwfl_addr=ffffffff81943568 addr=943568 section=_stext dwfl_addr=ffffffff819435a7 addr=9435a7 section=_stext dwfl_addr=ffffffff819435c4 addr=9435c4 section=_stext dwfl_addr=ffffffff819435d6 addr=9435d6 section=_stext dwfl_addr=ffffffff81943665 addr=943665 section=_stext dwfl_addr=ffffffff819435e1 addr=9435e1 section=_stext dwfl_addr=ffffffff819435ed addr=9435ed section=_stext dwfl_addr=ffffffff81943601 addr=943601 section=_stext dwfl_addr=ffffffff8194363f addr=94363f section=_stext dwfl_addr=ffffffff8194364f addr=94364f section=_stext dwfl_addr=ffffffff81943669 addr=943669 section=_stext dwfl_addr=ffffffff81943608 addr=943608 section=_stext dwfl_addr=ffffffff810b8360 addr=b8360 section=_stext dwfl_addr=ffffffff810b7030 addr=b7030 section=_stext dwfl_addr=ffffffff810b7cf5 addr=b7cf5 section=_stext dwfl_addr=ffffffff810b8ce6 addr=b8ce6 /home/wcohen/.systemtap/cache/71/stap_7166ef1db3ec33859804979a9640e95d_22392.ko For do_sys_open function see a bunch of addr that match up to the start of the function ffffffff812b0000 $ ../install/bin/stap -wp4 testsuite/systemtap.examples/general/varwatch.stp 'kernel.statement("do_sys_open@fs/open.c:*")' '$$vars' section=_stext dwfl_addr=ffffffff812b0000 addr=2b0000 section=_stext dwfl_addr=ffffffff812b0027 addr=2b0027 section=_stext dwfl_addr=ffffffff812b0103 addr=2b0103 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 semantic error: inconsistent relocation address: keyword at testsuite/systemtap.examples/general/varwatch.stp:5:1 source: probe $1 { ^ section=_stext dwfl_addr=ffffffff812b01f6 addr=2b01f6 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b00e6 addr=2b00e6 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b010b addr=2b010b section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b0113 addr=2b0113 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b011c addr=2b011c section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b0177 addr=2b0177 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b0189 addr=2b0189 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b0191 addr=2b0191 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b0193 addr=2b0193 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b019c addr=2b019c section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b01e7 addr=2b01e7 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b0120 addr=2b0120 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b0128 addr=2b0128 section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff812b012b addr=2b012b section=_stext dwfl_addr=ffffffff812b0000 addr=ffffffff812b0000 section=_stext dwfl_addr=ffffffff810b8360 addr=b8360 section=_stext dwfl_addr=ffffffff810b7030 addr=b7030 section=_stext dwfl_addr=ffffffff810b7cf5 addr=b7cf5 section=_stext dwfl_addr=ffffffff810b8ce6 addr=b8ce6 Pass 2: analysis failed. [man error::pass2] Number of similar error messages suppressed: 14. Rerun with -v to see them.
A simpler reproducer that works both on f28 ppc64le and f29 x86_64: $ uname -a Linux rh-power-vm07.fit.vutbr.cz 4.20.8-100.fc28.ppc64le #1 SMP Wed Feb 13 12:45:56 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux $ sudo stap -p4 -e 'probe netdev.change_mac { exit() }' semantic error: inconsistent relocation address: identifier 'netdev' at <input>:1:7 source: probe netdev.change_mac { exit() } ^ Pass 2: analysis failed. [man error::pass2] $ uname -a Linux cervelo 4.20.8-200.fc29.x86_64 #1 SMP Wed Feb 13 13:08:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux $ stap -p4 -e 'probe netdev.change_mac { exit() }' semantic error: inconsistent relocation address: identifier 'netdev' at <input>:1:7 source: probe netdev.change_mac { exit() } ^ Pass 2: analysis failed. [man error::pass2]
Created attachment 11646 [details] Results of reproducer. Have a scaled down reproducer: ../install/bin/stap -vvvv -p4 -e 'probe kernel.function("dev_set_mac_address") { printf("%x\n", $sa); exit() }' >& pr24224_issue.out These are the results.
The problem is tied to accessing target variables. According to the linux source code in net/core/dev.c there are 2 target variables: int dev_set_mac_address(struct net_device *dev, struct sockaddr *sa) { ... stap listing lists the two of them: $ stap -L 'kernel.function("dev_set_mac_address")' kernel.function("dev_set_mac_address@net/core/dev.c:7715") $dev:struct net_device* $sa:struct sockaddr* When going through the $dev and $sa from the source code $dev okay, but $sa has a problem: [wcohen@cervelo pr24224]$ ../install/bin/stap -p4 -e 'probe kernel.function("dev_set_mac_address") { printf("%x\n", $dev); exit() }' /home/wcohen/.systemtap/cache/cd/stap_cdef9e8a2d4b9d5a0e4252d87f28b7c2_1881.ko [wcohen@cervelo pr24224]$ ../install/bin/stap -p4 -e 'probe kernel.function("dev_set_mac_address") { printf("%x\n", $sa); exit() }' semantic error: inconsistent relocation address: keyword at <input>:1:1 source: probe kernel.function("dev_set_mac_address") { printf("%x\n", $sa); exit() } ^ Pass 2: analysis failed. [man error::pass2]
Mark Wielaard suspects the problem is due to some code in tapsets.cxx. From IRC log: <mjw> I think it is this: <mjw> q.results.push_back(new dwarf_derived_probe (funcname, filename, <mjw> i->decl_line, module, <mjw> section, i->entrypc, <mjw> i->entrypc, q, <mjw> &i->die)); <mjw> The second i->entrypc should be "relative" <wcohen> mjw, where is that? <mjw> tapsets.cxx:5492 Those changes from the following git commit: commit 68bd23fd0cc5efa4fccad68e0df0d0c4cb774322 Author: Jafeer Uddin <juddin@redhat.com> Date: Mon Dec 10 15:11:26 2018 -0500 PR16596: add support for DW_OP_GNU_entry_value in location expressions
Looking through test and examples that previously had the problem for fedora 29 on aarch64, powerpc, and x86. This changes look to address the problem. For example on x86_64 the reproducer now works: $ ../install/bin/stap -p4 -e 'probe kernel.function("dev_set_mac_address") { printf("%x\n", $sa); exit() }' /home/wcohen/.systemtap/cache/fc/stap_fce458c8ae35b2d2f10f9f7cc5c53508_2255.ko