This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
SystemTap's reduced memory usage
- From: Abe Jakop <ajakop at redhat dot com>
- To: systemtap at sourceware dot org
- Date: Tue, 18 Aug 2015 16:35:54 -0400 (EDT)
- Subject: SystemTap's reduced memory usage
- Authentication-results: sourceware.org; auth=none
Hello everyone,
Work has been done to reduce the amount of memory SystemTap consumes.
A need for this optimization was brought up by the case where there is a
large amount of outside tapsets, such that stap's excessive memory
usage shows.
To optimize SystemTap's memory usage, struct interned_string was introduced
as a wrapper on Boost's string_ref. These interned_strings are meant to
replace a handful of instances of std::strings.
The new interned_strings are meant to act like pointers to existing
strings. With this change, instances where new duplicate strings were
created are now replaced with a sort of pointer to the original string's
location, reducing the number of new strings that are generated.
These changes are already merged into the master branch. To try out the
changes, see the master branch of the git repo:
https://sourceware.org/git/gitweb.cgi?p=systemtap.git
Notable changes would be the inclusion of files stringtable.h and
stringtable.cxx, and the new peculiarities sections in INTERNALS.
____
NOTE
This optimization is reliant on having Boost version >=1.53, such that
boost::string_ref is available. To see if boost string_ref is available
for use by stap, check for BOOST_STRING_REF in the list of enabled
features from "stap -V".
______
IMPACT
With some outside tapsets installed to increase how much has to be parsed,
the tapset folder ended up being >11M.
For a simple script, before the optimization:
$ valgrind --tool=massif stap -e "probe begin{exit()}"
peak cost: "203.9 MiB" heap "58.8 MiB" heap extra "0 B" stacks
$ time stap -e "probe begin{exit()}"
real 0m1.382s
user 0m1.032s
sys 0m0.076s
After:
$ valgrind --tool=massif stap -e "probe begin{exit()}"
peak cost: "176.9 MiB" heap "49.7 MiB" heap extra "0 B" stacks
$ time stap -e "probe begin{exit()}"
real 0m1.398s
user 0m1.051s
sys 0m0.074s
The peak memory usage has a decrease of ~15 % and a minimal change
in runtime (hard to tell when it's that small and fluctuates a bit).
On the other hand, there is stap -L "**" which is what brought the
need for this optimization to our attention.
Before:
$ time stap -L "**" > /dev/null
real 3m47.497s
user 3m35.774s
sys 0m8.728s
$ valgrind --tool=massif stap -L "**"
[...]
peak cost: "753.8 MiB" heap "185.2 MiB" heap extra "0 B" stacks
After:
$ time stap -L "**" > /dev/null
real 3m27.391s
user 3m18.245s
sys 0m8.628s
$ valgrind --tool=massif stap -L "**"
[...]
peak cost: "710.2 MiB" heap "175.2 MiB" heap extra "0 B" stacks
The peak memory usage has a decrease of ~6 % and the runtime saw a
decrease of ~9 %.
More work can be done to take advantage of interned_strings and reduce
the memory consumption even further.
Comments and feedback are always welcome.
Abegail Jakop