Re: [PATCH] Add helper script for glibc debugging

[Carlos O'Donell <carlos at redhat dot com>]

On 9/24/19 3:24 PM, Gabriel F. T. Gomes wrote:
> From: "Gabriel F. T. Gomes" <gabrielftg@linux.ibm.com>
> 
> At Cauldron, while Florian showed his development workflow to us, we had
> a brief discussion about debugging glibc with GDB.  After that, I spent
> some time with Arjun and we shared our debugging scripts and tricks,
> then we came to the conclusion that this could be helpful to other
> people, maybe even more so to newcomers.  So, I got my off-tree script
> and converted it into this auto-generated version.
> 
> Still at Cauldron, Carlos mentioned that it wasn't trivial to let GDB
> know about the symbols within glibc.  This script doesn't solve this for
> every glibc debugging needs (I suppose it doesn't help at all when
> debugging the initial steps of program loading, for example), but it
> makes it a little easier when debugging test cases, as it automatically
> loads the symbols from them.
> 
> Maybe there are more trivial ways to do it, but that's what I have been
> using for some time.

I like this script!

I don't immediately see how this solves the required gdb setup though?

https://sourceware.org/glibc/wiki/Testing/Builds#Required_gdb_setup

Did I miss something?

It looks like the script is *almost* there in terms of functionality
to make it possible to debug single-threaded and multi-threaded programs.

> -- 8< --
> This patch adds a new make rule that generates a helper script for
> debugging glibc test cases.  The new script, debugglibc.sh, is similar
> to testrun.sh, in the sense that it allows the execution of the
> specified test case, however, it opens the test case in GDB, setting the
> library path the same way that testrun.sh does.  The commands are based
> on the instruction on the wiki page for glibc debugging [1].
> 
> By default, the script tells GDB to load the test case for symbol
> information, so that, when a breakpoint is hit, the call stack is
> displayed correctly (instead of printing lots of '??'s).  For instance,
> after running 'make' and 'make check', one could do the following:
> 
>   ./debugglibc.sh -t $PWD/stdlib/tst-strfrom -B strfromf -B strfromd
> 
>   Reading symbols from /home/gabriel/build/x86_64/glibc/elf/ld.so...
>   Breakpoint 1 at 0x1098
>   add symbol table from file "/home/gabriel/build/x86_64/glibc/stdlib/tst-strfrom"
> 
>   Breakpoint 1, 0x00007ffff7fd5098 in _dl_start_user () from /home/gabriel/build/x86_64/glibc/elf/ld.so
>   Breakpoint 2 at 0x7ffff7e4dba0: file strfrom-skeleton.c, line 41.
>   Breakpoint 3 at 0x7ffff7e4ddd0: file strfrom-skeleton.c, line 41.
> 
> Notice that the script will always start GDB with the program running
> and halted at _dl_start_user.  So, in order to reach the actual
> breakpoint of interest, one should hit 'c', not 'r':
> 
>   >>> c
>   Continuing.
>   Testing in locale: C
> 
>   Breakpoint 2, strfromf (dest=dest@entry=0x7fffffffe02b "\377", size=5, format=0x405010 "%g", f=12345.3447) at strfrom-skeleton.c:41
>   41	  sfile.f._sbf._f._lock = NULL;
> 
> An inspect the call stack with 'bt', as usual, and see symbols from both
> the test case and from the libraries themselves:
> 
>   >>> bt
>   #0  strfromf (dest=dest@entry=0x7fffffffe02b "\377", size=5, format=0x405010 "%g", f=12345.3447) at strfrom-skeleton.c:41
>   #1  0x000000000040246a in test_f () at tst-strfrom.c:67
>   #2  test_locale (locale=locale@entry=0x405046 "C") at tst-strfrom.c:77
>   #3  0x000000000040358b in do_test () at tst-strfrom.c:84
>   #4  legacy_test_function (argc=<optimized out>, argv=<optimized out>) at ../test-skeleton.c:56
>   #5  0x0000000000403b25 in support_test_main (argc=1, argc@entry=2, argv=0x7fffffffe258, argv@entry=0x7fffffffe250, config=config@entry=0x7fffffffe120) at support_test_main.c:350
>   #6  0x0000000000402315 in main (argc=argc@entry=2, argv=argv@entry=0x7fffffffe250) at ../support/test-driver.c:168
>   #7  0x00007ffff7e35deb in __libc_start_main (main=0x4022e0 <main>, argc=2, argv=0x7fffffffe250, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe248) at ../csu/libc-start.c:308
>   #8  0x000000000040234a in _start () at ../sysdeps/x86_64/start.S:120
> 
> Tested for x86_64.
> 
> [1] https://sourceware.org/glibc/wiki/Debugging/Loader_Debugging
> ---
>  Makefile | 159 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 158 insertions(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index 67ddd01bfe..d62f5a348c 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -187,7 +187,164 @@ $(common-objpfx)testrun.sh: $(common-objpfx)config.make \
>  	mv -f $@T $@
>  postclean-generated += testrun.sh
>  
> -others: $(common-objpfx)testrun.sh
> +define debugglibc
> +#!/bin/bash
> +
> +# Defaults
> +SOURCE_DIR="$$HOME/src/glibc"
> +BUILD_DIR="$$PWD"
> +DIRECT=true
> +SYMBOLSFILE=true
> +
> +unset TESTCASE
> +unset CMD_FILE
> +unset BREAKPOINTS
> +
> +usage()
> +{
> +cat << EOF
> +Usage: $$0 [OPTION]
> +
> +  -h, --help
> +	Prints this message and leaves.
> +
> +  The following options require one argument:
> +
> +  -b, --build, --build-dir
> +	Absolute path to glibc build directory
> +	Defaults to \$$PWD
> +  -s, --source, --source-dir
> +	Absolute path to glibc source files
> +	Defaults to \$$HOME/src/glibc
> +  -t, --testcase
> +	Absolute path to program to be tested
> +	Defaults to \$$PWD/stdlib/tst-strfrom
> +  -B, --breakpoint
> +	Breakpoints to set at the beginning of the execution
> +	(each breakpoint demands its own -B option, e.g. -B foo -B bar)
> +
> +  The following options do not take arguments:
> +
> +  -I, --no-direct
> +	Selects whether to pass the flag --direct to gdb.
> +	Required for glibc test cases and not allowed for non-glibc tests.
> +	Default behaviour is to pass the flag --direct to gdb.
> +  -S, --no-symbols-file
> +	Do not tell GDB to load debug symbols from the testcase.
> +EOF
> +}
> +
> +# Parse input options
> +while [[ $$# > 0 ]]
> +do
> +  key="$$1"
> +  case $$key in
> +    -h|--help)
> +      usage
> +      exit 0
> +      ;;
> +    -b|--build|--build-dir)
> +      BUILD_DIR=$$2
> +      shift
> +      ;;
> +    -s|--source|--source-dir)
> +      SOURCE_DIR=$$2
> +      shift
> +      ;;
> +    -t|--testcase)
> +      TESTCASE=$$2
> +      shift
> +      ;;
> +    -B|--breakpoint)
> +      BREAKPOINTS="$$BREAKPOINTS\n break $$2"
> +      shift
> +      ;;
> +    -I|--no-direct)
> +      DIRECT=false
> +      ;;
> +    -S|--no-symbols-file)
> +      SYMBOLSFILE=false
> +      ;;
> +    *)
> +      usage
> +      exit 1
> +      ;;
> +  esac
> +  shift
> +done
> +
> +# Update arguments
> +if [ ! -v TESTCASE ]
> +then
> +  TESTCASE="$$BUILD_DIR/stdlib/tst-strfrom"
> +fi
> +if [ ! -v CMD_FILE ]
> +then
> +  CMD_FILE="$$BUILD_DIR/debugglibc.gdb"
> +fi
> +
> +# Expand direct argument
> +if [ "$$DIRECT" == true ]
> +then
> +  DIRECT="--direct"
> +else
> +  DIRECT=""
> +fi
> +
> +# Expand symbols loading command
> +if [ "$$SYMBOLSFILE" == true ]
> +then
> +  SYMBOLSFILE="add-symbol-file $${TESTCASE}"
> +else
> +  SYMBOLSFILE=""
> +fi
> +
> +# GDB commands template
> +template ()
> +{
> +cat <<EOF
> +set environment C -E -x c-header
> +break _dl_start_user
> +__SYMBOLSFILE__
> +run --library-path $(rtld-prefix) \
> +__TESTCASE__ __DIRECT__
> +__BREAKPOINTS__
> +EOF
> +}
> +
> +# Generate the commands file for gdb initialization
> +template | sed \
> +  -e "s|__SYMBOLSFILE__|$$SYMBOLSFILE|" \
> +  -e "s|__TESTCASE__|$$TESTCASE|" \
> +  -e "s|__DIRECT__|$$DIRECT|" \
> +  -e "s|__BREAKPOINTS__|$$BREAKPOINTS|" \
> +  > $$CMD_FILE
> +
> +echo
> +echo "Debugging glibc..."
> +echo "Build directory  : $$BUILD_DIR"
> +echo "Source directory : $$SOURCE_DIR"
> +echo "GLIBC Testcase   : $$TESTCASE"
> +echo "GDB Commands     : $$CMD_FILE"
> +echo
> +
> +# We need to make sure that gdb is linked against the standalone glibc
> +# so that it picks up the correct nptl_db/libthread_db.so. So that means
> +# invoking gdb using the standalone glibc's linker.
> +$(test-via-rtld-prefix) \
> +/usr/bin/gdb -q -x $${CMD_FILE} -d $${SOURCE_DIR} $${BUILD_DIR}/elf/ld.so
> +endef
> +
> +# This is another handy script for debugging dynamically linked program
> +# against the current libc build for testing.
> +$(common-objpfx)debugglibc.sh: $(common-objpfx)config.make \
> +			    $(..)Makeconfig $(..)Makefile
> +	$(file >$@T,$(debugglibc))
> +	chmod a+x $@T
> +	mv -f $@T $@
> +postclean-generated += debugglibc.sh debugglibc.gdb
> +
> +others: $(common-objpfx)testrun.sh $(common-objpfx)debugglibc.sh
>  
>  # Makerules creates a file `stubs' in each subdirectory, which
>  # contains `#define __stub_FUNCTION' for each function defined in that
> 


-- 
Cheers,
Carlos.