Bug 23829 - Bug introduced in bpf -base.cxx in git master
Summary: Bug introduced in bpf -base.cxx in git master
Status: RESOLVED WORKSFORME
Alias: None
Product: systemtap
Classification: Unclassified
Component: bpf (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Unassigned
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-26 05:38 UTC by Gustavo Moreira
Modified: 2019-03-27 14:48 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2018-10-29 00:00:00


Attachments
Patch for bpf-base.cxx (247 bytes, patch)
2018-10-26 05:38 UTC, Gustavo Moreira
Details | Diff
stap-report output (6.47 KB, text/plain)
2018-10-31 01:03 UTC, Gustavo Moreira
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Gustavo Moreira 2018-10-26 05:38:15 UTC
Created attachment 11366 [details]
Patch for bpf-base.cxx

Compiling SystemTap from git master:
...
bpf-base.cxx: In function ‘void bpf::init_bpf_helper_tables()’:
bpf-base.cxx:148:21: error: ‘__BPF_SET_FUNC_NAME’ was not declared in this scope
   __BPF_FUNC_MAPPER(__BPF_SET_FUNC_NAME)
                     ^~~~~~~~~~~~~~~~~~~
bpf-base.cxx:148:40: error: ‘__BPF_FUNC_MAPPER’ was not declared in this scope
   __BPF_FUNC_MAPPER(__BPF_SET_FUNC_NAME)
                                        ^
Makefile:1669: recipe for target 'stap-bpf-base.o' failed

The bug was introduced here:
https://sourceware.org/git/gitweb.cgi?p=systemtap.git;a=commitdiff;h=c009fe88de8a91b11a19e36c332bd096cb72b8b0


Patch attached.
Comment 1 Gustavo Moreira 2018-10-26 05:47:09 UTC
Other errors related to the same merge:

bpf-translate.cxx: In member function ‘size_t bpf::bpf_unparser::parse_asm_stmt(embeddedcode*, size_t, bpf::asm_stmt&)’:
util.h:62:59: error: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 2 has type ‘std::vector<std::__cxx11::basic_string<char> >::size_type {aka unsigned int}’ [-Werror=format=]
 #define _F(format, ...) autosprintf(_(format), __VA_ARGS__)
                                                           ^
util.h:68:53: note: in definition of macro ‘SEMANTIC_ERROR’
 #define SEMANTIC_ERROR(...) semantic_error(ERR_SRC, __VA_ARGS__)
                                                     ^~~~~~~~~~~
bpf-translate.cxx:839:31: note: in expansion of macro ‘_F’
         throw SEMANTIC_ERROR (_F("invalid bpf embeddedcode syntax (label expects 1 arg, found %lu)", args.size()-1), stmt.tok);
                               ^~
util.h:62:59: error: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 2 has type ‘std::vector<std::__cxx11::basic_string<char> >::size_type {aka unsigned int}’ [-Werror=format=]
 #define _F(format, ...) autosprintf(_(format), __VA_ARGS__)
                                                           ^
util.h:68:53: note: in definition of macro ‘SEMANTIC_ERROR’
 #define SEMANTIC_ERROR(...) semantic_error(ERR_SRC, __VA_ARGS__)
                                                     ^~~~~~~~~~~
bpf-translate.cxx:846:31: note: in expansion of macro ‘_F’
         throw SEMANTIC_ERROR (_F("invalid bpf embeddedcode syntax (alloc expects 2 args, found %lu)", args.size()-1), stmt.tok);
                               ^~
util.h:62:59: error: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 2 has type ‘std::vector<std::__cxx11::basic_string<char> >::size_type {aka unsigned int}’ [-Werror=format=]
 #define _F(format, ...) autosprintf(_(format), __VA_ARGS__)
                                                           ^
util.h:68:53: note: in definition of macro ‘SEMANTIC_ERROR’
 #define SEMANTIC_ERROR(...) semantic_error(ERR_SRC, __VA_ARGS__)
                                                     ^~~~~~~~~~~
bpf-translate.cxx:854:31: note: in expansion of macro ‘_F’
         throw SEMANTIC_ERROR (_F("invalid bpf embeddedcode syntax (call expects at least 2 args, found %lu)", args.size()-1), stmt.tok);
                               ^~
util.h:62:59: error: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 2 has type ‘std::vector<std::__cxx11::basic_string<char> >::size_type {aka unsigned int}’ [-Werror=format=]
 #define _F(format, ...) autosprintf(_(format), __VA_ARGS__)
                                                           ^
util.h:68:53: note: in definition of macro ‘SEMANTIC_ERROR’
 #define SEMANTIC_ERROR(...) semantic_error(ERR_SRC, __VA_ARGS__)
                                                     ^~~~~~~~~~~
bpf-translate.cxx:864:31: note: in expansion of macro ‘_F’
         throw SEMANTIC_ERROR (_F("invalid bpf embeddedcode syntax (opcode expects 4 args, found %lu)", args.size()-1), stmt.tok);
                               ^~
bpf-translate.cxx: In member function ‘void bpf::bpf_unparser::emit_asm_opcode(const bpf::asm_stmt&, std::map<std::__cxx11::basic_string<char>, bpf::block*>)’:
bpf-translate.cxx:1084:14: error: ‘BPF_JLT’ was not declared in this scope
         case BPF_JLT: c = LTU; break;
              ^~~~~~~
bpf-translate.cxx:1085:14: error: ‘BPF_JLE’ was not declared in this scope
         case BPF_JLE: c = LEU; break;
              ^~~~~~~
bpf-translate.cxx:1088:14: error: ‘BPF_JSLT’ was not declared in this scope
         case BPF_JSLT: c = LT; break;
              ^~~~~~~~
bpf-translate.cxx:1089:14: error: ‘BPF_JSLE’ was not declared in this scope
         case BPF_JSLE: c = LE; break;
              ^~~~~~~~
In file included from staptree.h:27:0,
                 from bpf-internal.h:18,
                 from bpf-translate.cxx:10:
util.h:62:59: error: format ‘%ld’ expects argument of type ‘long int’, but argument 2 has type ‘int64_t {aka long long int}’ [-Werror=format=]
 #define _F(format, ...) autosprintf(_(format), __VA_ARGS__)
                                                           ^
util.h:68:53: note: in definition of macro ‘SEMANTIC_ERROR’
 #define SEMANTIC_ERROR(...) semantic_error(ERR_SRC, __VA_ARGS__)
                                                     ^~~~~~~~~~~
bpf-translate.cxx:1127:27: note: in expansion of macro ‘_F’
     throw SEMANTIC_ERROR (_F("offset field '%ld' out of range in bpf code", stmt.off), stmt.tok);
                           ^~
cc1plus: all warnings being treated as errors
Comment 2 Serhei Makarov 2018-10-29 17:54:06 UTC
__BPF_FUNC_MAPPER is included from the Linux kernel's bpf.h header, via a directive in bpf-internal.h:

extern "C" {
#include <linux/bpf.h>
}

Some of the other errors also seem to relate to symbols not in bpf.h e.g. BPF_JLT. I will need to work out a good fallback for when bpf.h is unavailable / lacks those symbols. Which version of the kernel headers were you compiling against?
Comment 3 Serhei Makarov 2018-10-29 17:54:36 UTC
If possibly, could you attach the output of stap-report ?
Comment 4 Gustavo Moreira 2018-10-31 01:02:11 UTC
Oh I see, it's in "uapi/linux/bpf.h" but it seems to be from 4.10 on, I'm using 4.9.75. 
Yeah, I think it would be needed a fallback otherwise it won't be supported for older kernels.
Comment 5 Gustavo Moreira 2018-10-31 01:03:24 UTC
Created attachment 11374 [details]
stap-report output
Comment 6 Serhei Makarov 2018-11-02 20:51:48 UTC
Thanks for that info. I've tested with bpf.h from kernel 4.8. Other defines aside from __BPF_FUNC_MAPPER and BPF_J{LT,LE,SLT,SLE} seem to be present.

The build errors reported should be fixed in commits 3d9e0d2a5 (for _F format specifiers on 32-bit systems) and 8fb0cb4a (for undefined bpf.h symbols). Partly fixed by adding a fallback version of __BPF_FUNC_MAPPER and fallback defines for the new BPF_J(...) insn opcodes (these are recognized by the new embedded-code assembler; stapbpf backend already replaces these with older insns, so the code should work even on old kernel versions that do not support them).

Let me know if the fixes work and I can close the bug, or if I need to refine my workaround further.

This exposes a deeper issue, in that SystemTap/BPF must be compiled against a recent kernel version to generate modules that use BPF features from that version. This has also caused hiccups e.g. when building SystemTap RPM packages in mockbuild on a kernel without eBPF support. Don't see an obvious workaround for this.
Comment 7 Serhei Makarov 2019-03-22 19:09:26 UTC
I'll eventually close this PR as WORKSFORME and assume it was fixed. Feel free to let me know if that's not the case. If the error was still there, since it happens during SystemTap compilation with default configure options, I assume people would have kept encountering it.
Comment 8 Serhei Makarov 2019-03-27 14:48:01 UTC
Closing WORKSFORME as the indicated compile issues have been fixed and there have been no further complaints.