Release announcement suggestions for 0.7

[Mark Wielaard <mwielaard at redhat dot com>]

Hi,

I came up with the following as ideas for when we want to make a new
release. It should list all major changes since the last numbered
release (0.6.2) in March. Although I am sure I missed some things. And I
might have misunderstood some changes or made some mistakes trying to
summarize. So comments more than welcome.

I'll also add the changes to NEWS (if they aren't already in there, lots
of info actually came from the NEWS file) unless people point out any
mistakes.

I didn't really know what to put under Known Issues. Unless people can
think of some urgent things to list there I will remove that section.

I listed all the bugs fixes since 0.6.2, plus their resolution. But I
think I will remove the ones that are marked DUPLICATE, INVALID,
WORKSFORME and WONTFIX since they are probably not really interesting.

There are lists of test results (make installcheck) on various systems I
had access to. The 2.6.25.6-55.fc9.i686 results don't look that good
though and are from a qemu-kvm instance. On real hardware I have not
been able to get a full make installcheck run because the machine will
randomly hang. Till now I have been unable to get to the bottom of this
issue (and strangely in a virtual environment everything seems fine).

Maybe we should include some results on other architectures so people
know what to expect?

Cheers,

Mark

= Systemtap frontend (stap) changes

- A probe listing mode is available.
  % stap -l vm.*
  vm.brk
  vm.mmap
  vm.munmap
  vm.oom_kill
  vm.pagefault
  vm.write_shared

- Partial support for probing without debuginfo via symbol tables
  (See --kelf, --kmap, --ignore-vmlinux, and --ignore-dwarf documentation)

- The environment variable SYSTEMTAP_DEBUGINFO_PATH now refers to a
  base directory where kernel and module debuginfo when not installed
  in the default location.

For a full overview see the stap(1) manual page.


= Systemtap script language changes

- Globals now accept ; terminators
  global odds, evens;
  global little[10], big[5];

- The vector of script command line arguments is available in a
  tapset-provided global array argv[].  It is indexed 1 ... argc,
  another global.  This can substitute for of preprocessor
  directives @NNN that fail at parse time if there are not
  enough arguments.
      printf("argv: %s %s %s", argv[1], argv[2], argv[3])

For a full overview see the SystemTap Language Reference manual.


= Systemtap probe points changes

- More user-space probe types are added:
  probe process(PID).begin { }
  probe process("PATH").begin { }
  probe process(PID).thread.begin { }
  probe process("PATH").thread.begin { }
  probe process(PID).end { }
  probe process("PATH").end { }
  probe process(PID).thread.end { }
  probe process("PATH").thread.end { }
  probe process(PID).syscall { }
  probe process("PATH").syscall { }
  probe process(PID).syscall.return { }
  probe process("PATH").syscall.return { }

- .statement("func@file+line") probes are now supported to allow a
  match relative to the entry of the function incremented by line
  number.  This allows using the same systemtap script if the rest
  of the file.c source only changes slightly.

For a full overview see the stapprobes(5) manual page.


= Standard tapsets functions changes

- Added functions to access different user space data types:
  user_string_n, user_string_n2, user_string_n_warn, user_string_n_quoted,
  user_short, user_short_warn, user_int, user_int_warn, user_long,
  user_long_warn, user_char, user_char_warn.

- Added "cpu register" and "numbered function argument" support
  function which help writing "dwarfless" probes.
  register:long (name:string), u_register:long (name:string),
  int_arg:long (n:long), uint_arg:long (n:long), long_arg:long
  (n:long), ulong_arg:long (n:long), longlong_arg:long (n:long),
  ulonglong_arg:long (n:long), pointer_arg:long (n:long), s32_arg:long
  (n:long), u32_arg:long (n:long), s64_arg:long (n:long), u64_arg:long
  (n:long), asmlinkage:unknown (), fastcall:unknown () and
  regparm:unknown (n:long).

- New function returnval:long () for usage with syscall tapset probes
  as counterpart to the returnstr:string (returnp:long) function.

- New function user_mode:long () that returns 1 if the probe point
  occurred in user-mode.

For a full overview see the stapfuncs(5) manual page.


= Standard tapset global variable changes

- NULL is now a new global variable (defined as global NULL = 0).

For a full overview see the stapvars(5) manual page.


= Systemtap examples changes

- examples are now stored in testsuite/systemtap.examples.  Examples
  are now explicitly run during make check and make installcheck to
  make sure they are always up to date.

- New examples:

  * io_submit.stp - When a reschedule occurs during an AIO io_submit
    call, accumulate the traceback in a histogram. When the script
    exits prints out a sorted list from most common to least common
    backtrace.

  * traceio.stp - Every second print out the top ten executables
    sorted in decending order based on cumulative I/O traffic
    observed.

  * iotop.stp - Every five seconds print out the top ten executables
    generating I/O traffic during that interval sorted in decending
    order.

 * disktop.stp - Get the status of reading/writing disk every 5
   seconds, output top ten entries during that period.

 * sigkill.stp - The script traces any SIGKILL signals. When that
   SIGKILL signal is sent to a process, the script prints out the
   signal name, the desination executable and process ID, the
   executable name user ID that sent the signal.

 * thread-times.stp - Sets up time-based sampling. Every five seconds
   it prints out a sorted list with the top twenty processes with
   samples broken down into percentage total time spent in user-space
   and kernel-space.

For a full overview see the stapex(5) manual page.


= Miscellaneous changes

- Much faster batch unregistering of (thousands of) kprobes implemented
  (needs linux 2.6.25).

- Stack backtraces for x86 and x86-64 are generated by a dwarf
  debuginfo-based unwinder based on the code from <jbeulich@novell.com>.
  This should give more accurate backtraces.

- Scripts relying on probe point wildcards like "syscall.*" that expand
  to distinct kprobes are processed significantly faster than before.

- Shiny new Vim highlighting (see vim/syntax/stap.vim).

- stap now warns about read-only variables.

- Configure with --enable-dejazilla for optional automatic test result
  reporting.

- Configure now accepts --enable-staticdw, which alters the link
  settings so they work with static libdw.

- Inclusion of Compile server (stap-server) and Compile server client
  (stap-client).


= The following people helped with code contributions for this release:

  Ananth N Mavinakayanahalli, Dave Brolley, David Smith, Eugene Teo,
  Frank Ch. Eigler, Jim Keniston, Josh Stone, Mark Wielaard, Martin Hunt,
  Masami Hiramatsu, Shaohua Li, Srikar Dronamraju, Srinivasa DS, Stan Cox,
  Tim Moore, Wenji Huang, William Cohen and Zhaolei


= Know issues with this release:

  <FIXME>


= Bugs closed for this release:

1165  FIXED translator implement .relative(nnn) 
1235  WORKSFORME kprobes Document areas of kernel that aren't safe to probe 
2071  FIXED kprobes Probes on ISR with probes on task thread's prehandler
                    crash the system 
2600  FIXED translator should optimize away assignments in other contexts 
2949  FIXED translator need probe listing command line option 
3051  FIXED runtime print_backtrace() prints most of stack on a single line
                    on ppc64 
3272  WONTFIX tapsets hook up to mp3 statistics hooks 
3542  FIXED kprobes request new batch registration/unregistration API 
4696  WORKSFORME testsuite profile.exp fails with probe overhead exceeded
                           threshold error message 
4706  WONTFIX runtime binary strings 
4843  DUPLICATE translator systemtap doesn't support probing assembler
                           functions anymore 
5001  FIXED tapsets ctime() vs _stp_ctime() duplication 
5101  FIXED translator stap abort in loc2c.c 
5106  FIXED runtime Remove old map and histogram formatting code 
5189  FIXED translator Support a '*' dynamic field width specifier in printf 
5231  FIXED tapsets Setting correct argument for bio_endio in ioblock.stp 
5236  DUPLICATE runtime process memory map tracker 
5528  FIXED tapsets more user_* data conversions needed 
5538  INVALID documentation configure: error: missing elfutils development
                            headers/libraries (dw 0.123+) 
5636  FIXED releng $builddir/SNAPSHOT should get recomputed at make check time 
5643  FIXED translator profile syscall tests to improve translator speed 
5645  FIXED runtime Too many sub-buffers generated with -s option 
5648  FIXED runtime Unaligned access to memory in _stp_vsnprintf() on ia64 
5667  WONTFIX translator Exploit kprobe.symbol_name for function boundary
                         tracing 
5716  FIXED runtime staprun/stapio setuid/capability simplification 
5759  FIXED kprobes probes in sched.c on preempt kernel cause reboot 
5770  FIXED testsuite Seen lot of eof errors on systemtap-20080216 snapshot 
5891  FIXED kprobes Insertion of kprobe module crashes the system on
                    2.6.25-rc3-mm1  if CONFIG_DEBUG_RODATA is set on
                    x86 system. 
5897  FIXED translator derived_probe derivation chain tweak 
5898  DUPLICATE translator adapt stap -c for user probes 
5928  FIXED translator CFLAGS vs. EXTRA_CFLAGS in generated kbuild makefile 
5954  DUPLICATE translator support a structure-dumping command 
5955  FIXED translator declaration of global variables with semicolon gives
                       semantic error. 
5956  FIXED translator Usage of NULL in systemtap script throws non-apparent
                       error messages 
5975  FIXED runtime Systemtap build error with -DNDEBUG 
5980  FIXED testsuite set dejagnu snapshot from stap -V as fallback 
6008  FIXED translator support larger on-memory buffer 
6393  FIXED translator tag build with more precise git commit code 
6400  FIXED runtime New runtime unwinder code does not make use of locally
                        built elfutils 
6405  DUPLICATE runtime New unwind code uses nsection field unavailable in
                        earlier kernels 
6410  FIXED runtime New unwind code causes errors on ia64 
6416  FIXED runtime SystemTap should recognize and disble probes on
                    __devinit/__devexit functions 
6417  FIXED runtime Wildcard probe hangs system 
6429  FIXED runtime unwinding/symbol stuff broken on ppc* 
6432  FIXED runtime use 2.6.26 probe_kernel_{read write} 
6433  DUPLICATE runtime use 2.6.26 probe_kernel_{read write} 
6434  DUPLICATE runtime use 2.6.26 probe_kernel_{read write} 
6435  DUPLICATE runtime use 2.6.26 probe_kernel_{read write} 
6437  DUPLICATE runtime unwinder/backtrace should see through
                        kretprobes_trampolines 
6451  FIXED runtime staprun shutdown deadlock 
6454  FIXED translator stap -l cut off the format of marker-based probe point 
6455  FIXED translator marker probe: Format mismatch error for markings
                              defined with MARK_NOARGS 
6466  FIXED translator more elision needed to support iffy tapsets 
6469  FIXED translator duplicate warning elimination 
6470  FIXED translator export command line arguments or run-time script parsing 
6474  FIXED releng build setuid staprun with -fpie 
6481  FIXED runtime timers: stp_time_timer_callback calls mod_timer with
                    interrupts disabled 
6492  FIXED translator stap -l outputs so many WARNINGS. 
6499  FIXED runtime BUG with systemtap's utrace support 
6500  FIXED runtime Systemtap's utrace support needs to follow new threads 
6509  FIXED translator syscall.get*id probes broken 
6510  DUPLICATE runtime Execution of systemtap script gives compilation
                        error with systemtap-20080510 snapshot. 
6521  INVALID runtime stap -l  syscall.*  causes pass 2: analysis failed 
6524  FIXED runtime ctime() on bad values hangs system 
6529  FIXED translator error() from nested function can be lost 
6534  FIXED translator utsname() becoming unusable 
6538  FIXED translator warn about read-only variables 
6550  FIXED releng add --enable-dejazilla 
6562  FIXED runtime $SYSTEMTAP_DEBUGINFO_PATH does not work 
6563  FIXED tapsets sys.stp fails on latest systemtap-20080524 snapshot 
6588  FIXED tapsets syscall.exit[_group].return should define name  retstr 
6601  FIXED translator systemtap with --kelf option looks for data symbol
                       to probe a function on powerpc systemtap 


= Test results on various systems:

kernel location: /usr/lib/debug/lib/modules/2.6.18-53.1.21.el5/vmlinux
kernel version: 2.6.18-53.1.21.el5
systemtap location: /usr/local/systemtap/bin/stap
systemtap version: version 0.7/0.134 git branch master, commit e8402528
Test Run By root on Mon Jun 16 13:49:41 2008
Native configuration is x86_64-redhat-linux-gnu

Host: Linux gnu 2.6.18-53.1.21.el5 #1 SMP Tue May 20 09:35:07 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux
Snapshot: version 0.7/0.134 git branch master, commit e8402528
Distro: CentOS release 5 (Final)

FAIL: debugpath-good (eof)
KPASS: buildok/nfs-all-probes.stp (PRMS 4413)
KPASS: buildok/process_test.stp (PRMS 1155)
FAIL: gtod (299)
FAIL: 64-bit access
FAIL: 64-bit chmod
FAIL: 32-bit access
FAIL: 32-bit chmod
FAIL: 32-bit rt_signal

# of expected passes            555
# of unexpected failures        7
# of expected failures          197
# of unknown successes          2
# of known failures             5
# of untested testcases         20
# of unsupported tests          3

---

kernel location: /usr/lib/debug/lib/modules/2.6.24.7-92.fc8/vmlinux
kernel version: 2.6.24.7-92.fc8
systemtap location: /usr/local/systemtap/bin/stap
systemtap version: version 0.7/0.131 git branch master, commit e8402528
Test Run By root on Mon Jun 16 14:50:29 2008
Native configuration is x86_64-redhat-linux-gnu

Host: Linux dijkstra.wildebeest.org 2.6.24.7-92.fc8 #1 SMP Wed May 7 16:26:02 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux
Snapshot: version 0.7/0.131 git branch master, commit e8402528
Distro: Fedora release 8 (Werewolf)

FAIL: debugpath-good (eof)
FAIL: PROCFS shutdown (eof)
KPASS: buildok/process_test.stp (PRMS 1155)
FAIL: 64-bit access
FAIL: 64-bit chmod
FAIL: 64-bit timer
FAIL: 32-bit access
FAIL: 32-bit chmod
FAIL: 32-bit rt_signal
FAIL: 32-bit timer

# of expected passes            554
# of unexpected failures        9
# of expected failures          197
# of unknown successes          1
# of known failures             6
# of untested testcases         20
# of unsupported tests          2

---

kernel location: /usr/lib/debug/lib/modules/2.6.25.6-55.fc9.i686/vmlinux
kernel version: 2.6.25.6-55.fc9.i686
systemtap location: /usr/local/systemtap/bin/stap
systemtap version: version 0.7/0.133 git branch master, commit e8402528
Test Run By root on Mon Jun 16 12:05:18 2008
Native configuration is i686-pc-linux-gnu

Host: Linux fedora9 2.6.25.6-55.fc9.i686 #1 SMP Tue Jun 10 16:27:49 EDT 2008 i686 i686 i386 GNU/Linux
Snapshot: version 0.7/0.133 git branch master, commit e8402528
Distro: Fedora release 9 (Sulphur)

FAIL: debugpath-good (eof)
FAIL: systemtap.base/stmt_rel.stp
FAIL: UTRACE_P5_03 shutdown (eof)
FAIL: UTRACE_P5_04 shutdown (eof)
FAIL: backtrace of yyy_func2 (2)
FAIL: print_stack of yyy_func2 (2)
FAIL: backtrace of yyy_func3 (2)
FAIL: print_stack of yyy_func3 (2)
FAIL: backtrace of yyy_func4 (2)
FAIL: print_stack of yyy_func4 (2)
FAIL: all pid tests - unexpected EOF
FAIL: systemtap.examples/disktop.meta
FAIL: systemtap.examples/sig_by_pid.meta
FAIL: systemtap.examples/sig_by_proc.meta
FAIL: systemtap.examples/sigkill.meta
FAIL: systemtap.examples/sigmon.meta
FAIL: buildok/context_test.stp
FAIL: buildok/fortyfive.stp
KPASS: buildok/sched_test.stp (PRMS 1155)
FAIL: buildok/task-embedded.stp
FAIL: buildok/task_test.stp
FAIL: gtod (231)
FAIL: systemtap.stress/current.stp startup (eof)
FAIL: 32-bit access
FAIL: 32-bit chmod
FAIL: 32-bit futimes
FAIL: 32-bit stat
FAIL: 32-bit swap

# of expected passes		511
# of unexpected failures	27
# of expected failures		197
# of unknown successes		1
# of known failures		6
# of untested testcases		6
# of unsupported tests		1