stack_used() not accurate?

[Mike Snitzer]

On Thu, May 29, 2008 at 5:23 PM, Mike Snitzer <snitzer@gmail.com> wrote:
> On Thu, May 29, 2008 at 12:29 PM, Frank Ch. Eigler <fche@redhat.com> wrote:
>>
>> "Mike Snitzer" <snitzer@gmail.com> writes:
>>
>>> [...]  I'll be exploring other implementations, but any suggestions
>>> would be very welcome.
>>
>> THREAD_SIZE should give a good normal kernel stack size.  The
>> kprobe-context stack pointer would be better decoded from its pt_regs
>> rather than that embedded-c &a hack.
>
> I came up with the following for x86_64:
>
> %( arch == "x86_64" %?
> function stack_used_new:long() %{
>        unsigned long free = THREAD_SIZE;
>        if (CONTEXT->regs) {
>                u64 curbase = (u64)task_stack_page(current);
> #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,24)
>                unsigned long sp = CONTEXT->regs->sp;
> #else
>                unsigned long sp = CONTEXT->regs->rsp;
> #endif
>                free = sp - (curbase + sizeof(struct thread_info));
>        }
>        THIS->__retvalue = THREAD_SIZE - free;
> %}
> %)
>
> I haven't looked at x86 yet.


Here is the x86 version (from irq_32.c's do_IRQ):

function stack_used_new:long() %{
        unsigned long free = THREAD_SIZE;
        if (CONTEXT->regs) {
                long sp;
                __asm__ __volatile__("andl %%esp,%0" :
                                        "=r" (sp) : "0" (THREAD_SIZE - 1));
                free = sp - sizeof(struct thread_info);
        }
        THIS->__retvalue = THREAD_SIZE - free;
%}


Along the way I've uncovered what seems to be an issue with
systemtap's print_stack() on x86.  There are, what appears to be, stap
mechanics and probe point details getting into the stack listing
(which mask portions of the real call-chain).

Here is the output I get from my stp script using rhel5u2's stap on x86:

tid:      342  name:       kjournald  stack_used:    888
 0xe0881efe : verify_chain+0x1/0x20 [ext3]
 0xe08828fe : ext3_get_branch+0x6a/0xba [ext3]
 0xdfab0400 : _einittext+0x1f3a5363/0x0
 0xdf8afe20 : _einittext+0x1f1a4d83/0x0
 0xdf8afe90 : _einittext+0x1f1a4df3/0x0
 0xe0882ae6 : ext3_get_blocks_handle+0xaf/0x8cc [ext3]
 0xdf8afde0 : _einittext+0x1f1a4d43/0x0
 0xdf8afe3c : _einittext+0x1f1a4d9f/0x0
 0xdf8afe1c : _einittext+0x1f1a4d7f/0x0
 0xe0acf78c : _stp_map_del_isi+0xe31c/0x0
[stap_520b1eed653befb9ee369b8b8ed8e489_571886]
 0xdfa4ce48 : _einittext+0x1f341dab/0x0
 0x0000d0da

Here is with a very recent git stap (rhel52-062-1-339-g2285f44) on rhel5u2 x86:

tid:      342  name:       kjournald  stack_used:    888
 0xe0881efe : verify_chain+0x1/0x20 [ext3]
 0xe08828fe : ext3_get_branch+0x6a/0xba [ext3]
 0xdfab0400 : packet_exit+0x1f390993/0x0
 0xdf8afe20 : packet_exit+0x1f1903b3/0x0
 0xdf8afe90 : packet_exit+0x1f190423/0x0
 0xe0882ae6 : ext3_get_blocks_handle+0xaf/0x8cc [ext3]
 0xdf8afde0 : packet_exit+0x1f190373/0x0
 0xdf8afe3c : packet_exit+0x1f1903cf/0x0
 0xdf8afe1c : packet_exit+0x1f1903af/0x0
 0xce2cdf8a : packet_exit+0xdbae51d/0x0
 0xc5ce2cdf : packet_exit+0x55c3272/0x0
 0x0000ce48

Here is the clean stack I've come to expect from using stap on x86_64:

tid:      539  name:       kjournald  stack_used:   1256
 0xffffffff8804f3a9 : verify_chain+0x1/0x1e [ext3]
 0xffffffff8804fe44 : ext3_get_branch+0x7a/0xd2 [ext3]
 0xffffffff8805005b : ext3_get_blocks_handle+0xbd/0x8e6 [ext3]
 0xffffffff8044cb5c : schedule+0x95c/0xa4d
 0xffffffff803005ea : elv_merged_request+0x30/0x39
 0xffffffff88050b70 : ext3_get_block+0xc2/0xe4 [ext3]
 0x00000ffffffff802

Mike