Bug 4005 - Breakpoint tests very slow
Summary: Breakpoint tests very slow
Status: RESOLVED FIXED
Alias: None
Product: frysk
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Mark Wielaard
URL:
Keywords:
Depends on:
Blocks: 2234
  Show dependency treegraph
 
Reported: 2007-02-08 02:33 UTC by Andrew Cagney
Modified: 2007-02-10 00:24 UTC (History)
0 users

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Cagney 2007-02-08 02:33:47 UTC
The junit testsuite is noticably slower with the breakpoint tests enabled.
Comment 1 Mark Wielaard 2007-02-08 10:53:06 UTC
If this is about frysk.proc.TestBreakpoints then a big chunk of the slowdown
comes from the stepping/breakpointing testing. At the moment the
funit-breakpoints contains a workaround to reset the signal handler (see bug
#3997) with that removed they are a lot faster. They also do 7 iterations over
all the different breakpoints. This could be reduced to 3 in case of stepping.
Stepping a program is however slow whatever you do.

What is "slow" and what is "acceptable" in this case?
Comment 2 Andrew Cagney 2007-02-08 14:04:16 UTC
The test should execute in something less than a second(In reply to comment #1)
> If this is about frysk.proc.TestBreakpoints then a big chunk of the slowdown
> comes from the stepping/breakpointing testing. At the moment the
> funit-breakpoints contains a workaround to reset the signal handler (see bug
> #3997) with that removed they are a lot faster. They also do 7 iterations over
> all the different breakpoints. This could be reduced to 3 in case of stepping.
> Stepping a program is however slow whatever you do.

Yes, stepping is slow, however it should still be possible for each test to
complete in something less than a second.  Could those tests be stepping 10 000s
of instructions, or contain a polling wait, or contain unnecessary repetition?

For the kernel bug, please remove the workaround and instead mark the test up as
broken on that system - add a TestCase.broken<whatever>XXX method that keys off
the kernel, there's an existing example.

If the test is looking for races using repetition, then a Stress*.java test is
better.  Once the race is identified it can be turned into a more deterministic
and faster Test*.java.
Comment 3 Mark Wielaard 2007-02-09 16:51:00 UTC
All tests now finish in 1 or 2 seconds.

2007-02-08  Mark Wielaard  <mark@klomp.org>

       * TestBreakpoint.java: Lower interations from 7 to 3.

2007-02-09  Mark Wielaard  <mark@klomp.org>

       * funit-breakpoints.c (signal_handler):
       Remove bogus signal reset kernel workaround.

No TestCase.broken<whatever>XXX was added for now since there is no good feature
test for this kernel bug. The frysk-imports/tests will fail however and this
test will never be reached on broken systems with a make check.
Comment 4 Andrew Cagney 2007-02-09 17:29:11 UTC
(In reply to comment #3)

> No TestCase.broken<whatever>XXX was added for now since there is no good feature
> test for this kernel bug. The frysk-imports/tests will fail however and this
> test will never be reached on broken systems with a make check.

Both << make -k >>  and ./frysk-core/TestRunner are both very common, so the
test can't rely no not being run.

frysk.junit.TestCase contains:

    protected static boolean brokenUtraceXXX (int bug)
    {
        if (uname == null)
            uname = Uname.get ();
        String[] badKernels = {
            "2.6.18-1.2849.fc6",
            "2.6.18-1.2239.fc5"
        };
        for (int i = 0; i < badKernels.length; i++) {
            if (badKernels[i].equals (uname.getRelease ()))
                return brokenXXX (bug);
        }
        return false;
    }

so something similar would be sufficient.

I'm pretty sure the tests can be sped up to < 1 second.
Comment 5 Andrew Cagney 2007-02-10 00:24:14 UTC
I checked this in, frysk-core on fc5 results are consistent again.

Index: frysk-core/frysk/proc/ChangeLog
2007-02-09  Andrew Cagney  <cagney@redhat.com>

        * TestProcStopped.java: Replace brokenUtraceXXX with
        brokenIfUtraceXXX.
        * TestTaskTerminateObserver.java: Ditto.
        * TestProcTasksObserver.java: Ditto.
        * TestRefresh.java (testRefreshZombie): Use brokenIfKernelXXX.
        * TestBreakpoints.java (brokenSignals): New.
        (testSteppingtestHitAndRun, testSteppingtestInsertRemove)
        (testSteppingAddLots): Call brokenSignals.

Index: frysk-imports/frysk/junit/ChangeLog
2007-02-09  Andrew Cagney  <cagney@redhat.com>

        * TestCase.java (brokenIfKernelXXX): New method.
        (brokenIfUtraceXXX): Rename brokenUtraceXXX.  Use
        brokenIfKernelXXX.