Some newbie questions

[Avi Kivity]

On 08/29/2016 09:26 PM, David Smith wrote:
> On 08/28/2016 08:29 AM, Avi Kivity wrote:
>>
>> On 08/26/2016 11:07 PM, David Smith wrote:
>>> On 08/26/2016 01:39 PM, Avi Kivity wrote:
>>>> On 08/25/2016 08:37 PM, David Smith wrote:
>>>>> On 08/25/2016 11:21 AM, Avi Kivity wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Should I wait for ongoing improvement here, or shall I look elsewhere
>>>>>> for my tracing needs?
>>>>>>
>>>>>> It would be a pity (for me) if I have to find another solution,
>>>>>> because
>>>>>> systemtap has all the expressiveness and integration I need.  But
>>>>>> it has
>>>>>> a dramatic impact on my application runtime.
>>>>>>
>>>>>> I was able to extract some useful data with perf probe/perf record,
>>>>>> but
>>>>>> as soon as I need to qualify a probe point with runtime information,
>>>>>> perf falls short.
>>>>> As Frank mentioned in a previous email, it might be possible for us to
>>>>> switch to using straight kprobes instead of syscall tracing to handle
>>>>> mmap tracing. In your use case of calling epoll_wait() lots of times
>>>>> per
>>>>> second, that might be a *big* win.
>>>>>
>>>>> I'll see what can be done to add that feature.
>>>>>
>>>> Thanks a lot.  I'll be happy to test patches.
>>> OK, since you asked...
>>>
>>> Here's a patch I'm testing that tries to do prefiltering when a syscall
>>> occurs, so we don't have to take that lock.
>>>
>>> Please rebuild with it, and let me know if it (a) works, and (b) has
>>> lower overhead in your situation.
>>>
>> With an unloaded system, systemtap almost vanishes from the profile.
>> This is on a 2s24c48t system, running epoll_pwait() and polling on user
>> memory locations in a tight loop.
> ... stuff deleted ...
>
> After looking into it, Josh was right in thinking I had the test
> backwards. Can you try the following patch (after reverting the last
> patch) and let us know what you find?
>

With the new patch, I get

ERROR: probe overhead exceeded threshold
ERROR: probe overhead exceeded threshold
WARNING: Number of errors: 2, skipped probes: 0
WARNING: /home/avi/systemtap.f23/bin/staprun exited with status: 1
Pass 5: run failed.  [man error::pass5]

with -g --suppress-time-limits, I get the usual spinlock badness.