Re: stack_used() not accurate?

["Mike Snitzer" <snitzer at gmail dot com>]

On Fri, May 30, 2008 at 6:40 PM, Frank Ch. Eigler <fche@redhat.com> wrote:
> Hi -
>
> On Fri, May 30, 2008 at 03:39:54PM -0400, Mike Snitzer wrote:
>> On Thu, May 29, 2008 at 5:23 PM, Mike Snitzer <snitzer@gmail.com> wrote:
>> > I came up with the following for x86_64:
>> >
>> > %( arch == "x86_64" %?
>> > function stack_used_new:long() %{
>> >        unsigned long free = THREAD_SIZE;
>> >        if (CONTEXT->regs) {
>> >                u64 curbase = (u64)task_stack_page(current);
>> > #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,24)
>> >                unsigned long sp = CONTEXT->regs->sp;
>> > #else
>> >                unsigned long sp = CONTEXT->regs->rsp;
>> > #endif
>> >                free = sp - (curbase + sizeof(struct thread_info));
>> >        }
>> >        THIS->__retvalue = THREAD_SIZE - free;
>> > %}
>> > %)
>
> This looks OK, but REGS_SP(CONTEXT->regs) should do the right thing
> without that #if stuff.  Actually, that may be enough to make this
> function architecture-independent.

It's actually REG_SP(CONTEXT->regs) but thanks for the pointer.  It
could be that the function is now arch-independent but on x86 it would
appear there is something awry with CONTEXT->regs->sp.  Please see
below.

>> Here is the x86 version (from irq_32.c's do_IRQ):
>>
>> function stack_used_new:long() %{
>>         unsigned long free = THREAD_SIZE;
>>         if (CONTEXT->regs) {
>>                 long sp;
>>                 __asm__ __volatile__("andl %%esp,%0" :
>>                                         "=r" (sp) : "0" (THREAD_SIZE - 1));
>>                 free = sp - sizeof(struct thread_info);
>>         }
>>         THIS->__retvalue = THREAD_SIZE - free;
>> %}
>
> (This version doesn't use CONTEXT->regs, which it should.)

Agreed, but unfortunately if I try to use the value stored in
CONTEXT->regs->esp I don't get the actual stack pointer.  I had a look
at the assembly of the systemtap compiled ko that uses __asm__ (like
above), it works and looks like:
mov    $0xfff,%eax
and    %esp,%eax
lea    0xffffffc8(%eax),%edx
mov    $0x1000,%eax
sub    %edx,%eax
mov    %eax,(%ebx)
movl   $0x0,0x4(%ebx)

whereas the following c-code does _not_ work:
if (CONTEXT->regs) {
    long sp = REG_SP(CONTEXT->regs) & (THREAD_SIZE - 1);
    free = sp - sizeof(struct thread_info);
}
THIS->__retvalue = THREAD_SIZE - free;

the associated assembly is:
mov    0x98(%edi),%eax
mov    0x34(%eax),%eax
and    $0xfff,%eax
lea    0xffffffc8(%eax),%edx
mov    $0x1000,%eax
sub    %edx,%eax
mov    %eax,(%ebx)
movl   $0x0,0x4(%ebx)

The C-code is doing what we'd expect (dereferencing the esp member of
struct pt_reg*):
crash> p &((struct pt_regs *)0)->esp
$2 = (long int *) 0x34

So, given that the calculated "free" is incorrect, this says to me
that the incorrect value is getting stored in CONTEXT->regs->esp on
x86 (or the value is stale)?

>> Along the way I've uncovered what seems to be an issue with
>> systemtap's print_stack() on x86.  [...]
>
> Yeah, that's one of several smelly bits that we're working on.

OK, good to know..

Mike