This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: No output from trace file after a few hours
- From: Arkady <arkady dot miasnikov at gmail dot com>
- To: Daniel Doron <danielmeirdoron at gmail dot com>
- Cc: systemtap at sourceware dot org
- Date: Thu, 15 Jun 2017 14:59:08 +0300
- Subject: Re: No output from trace file after a few hours
- Authentication-results: sourceware.org; auth=none
- References: <CAFwN=+ywma7CvB6AajmEb99LKZkTg9fWP724to6r=FrKrB0stQ@mail.gmail.com> <CANA-60qKnB_x4UuPfej_H2ebkHNXepmqpE+dgnOT=kOV6MYbqw@mail.gmail.com> <CANA-60oP_i=0oCQ1e73B_96juT1gW10d2Q7yps_r-jPyo=AJKQ@mail.gmail.com> <CAFwN=+xs21q38vf_ZNN-QAr5-EOfSkoKUx-e3u8PaZ=GKOkkdA@mail.gmail.com> <CAFwN=+yn+TGsHKNmoCJBaqeJakRG=XS8uVGbN8a16a0z3UdXmQ@mail.gmail.com> <CAFwN=+xMd=iXUw79A0WkpjHY1Vii3tFzWh4=5vS7f9qdtfdSAQ@mail.gmail.com> <CAFwN=+x34nb0ZYBW1wYB=-4m-Ctyrd-tkT4wXS0bcDTdBNaOtg@mail.gmail.com> <CANA-60pBnzZrAkg_WJ1bLr+e1atAGeFbUcbU_ffrW=hTj0Q+dw@mail.gmail.com> <CAFwN=+w2u+QpZyPavfxf2i+QsrhaHbSnRcGzYLPOpLwK4TA4vA@mail.gmail.com> <CANA-60o1VWpywY6-YZCuyjHYVXxakRoZ4f=JnD9rRERH-rpjQA@mail.gmail.com> <CAFwN=+yAGLbX+a8fQQ2h2doXrEd2-OxLWKPO11xb8G96UifYOw@mail.gmail.com> <CANA-60r=U8tmEfiQJxfQ-u+vdDBgwHFzZHthXbL3uQbusrYNew@mail.gmail.com> <CANA-60oB-kzF0YQhJjht-Hhrv73O-0j-PtJdUF7HQg+F7Hbv8A@mail.gmail.com> <CANA-60qX659-jTbKu75wga6oSLktTEya-2fdeS1Jb-4=xFM1gQ@mail.gmail.com> <CAFwN=+xJc3nZg8U6pMCakrmnDS87t=LH2OboTDb4PwdZeJJF8g@mail.gmail.com> <CANA-60q0Y+nAJ2+gkTe=3KcqQH8xRbowUZMB-VUazXh-e4AoTw@mail.gmail.com>
you can also try --suppress-time-limits
On Thu, Jun 15, 2017 at 2:32 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
> One possible reason is a system update which caused lot of probes to trigger.
>
> On Thu, Jun 15, 2017 at 2:13 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
>> I will try that.
>> BTW, this also just happened after almost 24 hours on kernel 4.11.3-1-ARCH
>>
>> On Thu, Jun 15, 2017 at 2:05 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>> This works fine
>>>
>>> sudo stap -D MAXSKIPPED=0 -D MAXTRYLOCK=1000 -D TRYLOCKDELAY=100 -F
>>> -o ./test.log -S 1,2 test.stp
>>>
>>> On Thu, Jun 15, 2017 at 2:03 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>>> STAP simply quits in a couple of seconds
>>>>
>>>> sudo stap -F -o ./test.log -S 1,2 test.stp
>>>> 94089
>>>> ps -ef | grep -v "grep" | grep 94089
>>>>
>>>> On Thu, Jun 15, 2017 at 1:53 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>>>> Interesting. I have reproduced the problem by running two tight bash
>>>>> loops simultaneously
>>>>>
>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp
>>>>> while [ 1 ];do ls > /dev/null;done &
>>>>> while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en
>>>>> "\033[0;0H";done;
>>>>>
>>>>> Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC
>>>>> 2016 x86_64 x86_64 x86_64 GNU/Linux
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron
>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>> On that specific kernel about 8-10 hours.
>>>>>> On my machine with latest kernel, been running for 24 hours.
>>>>>> I need to check other distros.
>>>>>>
>>>>>> sudo stap -v -F -m mymodule test.stp
>>>>>>
>>>>>> On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>>>>>> Do you do something like
>>>>>>>
>>>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp ?
>>>>>>>
>>>>>>> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron
>>>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>> On that specific kernel about 8-10 hours.
>>>>>>>> On my machine with latest kernel, been running for 24 hours.
>>>>>>>> I need to check other distros.
>>>>>>>>
>>>>>>>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> How long time it runs for you?
>>>>>>>>>
>>>>>>>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron
>>>>>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6)
>>>>>>>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others
>>>>>>>>> > This is free software; see the source for copying conditions.
>>>>>>>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP
>>>>>>>>> >
>>>>>>>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron
>>>>>>>>> > <danielmeirdoron@gmail.com> wrote:
>>>>>>>>> >> BTW kernel version: 2.6.32-431.el6.x86_64
>>>>>>>>> >>
>>>>>>>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron
>>>>>>>>> >> <danielmeirdoron@gmail.com> wrote:
>>>>>>>>> >>> if you mean that the kernel version does not get upgraded, then yes.
>>>>>>>>> >>>
>>>>>>>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron
>>>>>>>>> >>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>>> >>>> Sorry, I do not know what you mean by that...
>>>>>>>>> >>>>
>>>>>>>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com>
>>>>>>>>> >>>> wrote:
>>>>>>>>> >>>>> Do you have the kernel version pinned?
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady
>>>>>>>>> >>>>> <arkady.miasnikov@gmail.com> wrote:
>>>>>>>>> >>>>>> Please send the stap script
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron
>>>>>>>>> >>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>>> >>>>>>> Hi,
>>>>>>>>> >>>>>>>
>>>>>>>>> >>>>>>> I built my module to probe a few system calls and functions.
>>>>>>>>> >>>>>>> Nothing
>>>>>>>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a
>>>>>>>>> >>>>>>> simple
>>>>>>>>> >>>>>>> python script to read data directly off the
>>>>>>>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course
>>>>>>>>> >>>>>>> running as
>>>>>>>>> >>>>>>> sudo)
>>>>>>>>> >>>>>>> I run stap in Flight recorder mode.
>>>>>>>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output
>>>>>>>>> >>>>>>> from
>>>>>>>>> >>>>>>> the module...
>>>>>>>>> >>>>>>> Any idea how to check why that is?
>>>>>>>>> >>>>>>> dmesg does not report anything unusual...
>>>>>>>>> >>>>>>>
>>>>>>>>> >>>>>>> Thanks.
>>>>>>>>> >>>>>>> Daniel.