Debugging variable arguments functions (stdarg)

[G <tjmadwja at gmail dot com>]

Hello,

How can I look at the values of "..." in a function which can
take a variable number of arguments (i.e. uses va_start(),
va_end() etc.) in gdb?

I have a core dump of a program which I try to analyze in gdb:

debian:~ $ gdb /usr/local/bin/teamlogs core-teamlogs
GNU gdb 6.8-debian
[snip]
Loaded symbols for /lib/libnss_files.so.2
Core was generated by `/usr/local/bin/teamlogs'.
Program terminated with signal 11, Segmentation fault.
[New process 31754]
#0  0x00002ac4de25f030 in strlen () from /lib/libc.so.6
(gdb) bt
#0  0x00002ac4de25f030 in strlen () from /lib/libc.so.6
#1  0x00002ac4de22bcb1 in vfprintf () from /lib/libc.so.6
#2  0x00002ac4de2affbe in __vsyslog_chk () from /lib/libc.so.6
#3  0x00000000004062cb in WriteLog (facility=128, level=4,
    fmt=0x406bf0 "MysqlWrapper(): Query failed: \"%s\": %s")
    at src/teamlogs.cpp:119
#4  0x00000000004041e2 in MysqlWrapper (
    query=0x7c8e80 "INSERT INTO results.matches (time, team, won)
VALUES (1257353711, 'Juniors1', 0)")
    at src/teamlogs.cpp:563
[snip]
(gdb) frame 3
#3  0x00000000004062cb in WriteLog (facility=128, level=4,
    fmt=0x406bf0 "MysqlWrapper(): Query failed: \"%s\": %s")
    at src/teamlogs.cpp:119
119     vsyslog(facility, level, fmt, ap);
(gdb) print facility
$2 = 128
(gdb) print level
$3 = 4
(gdb) print fmt
$4 = 0x406bf0 "MysqlWrapper(): Query failed: \"%s\": %s"
(gdb) print ap
$5 = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffcd89ce80,
    reg_save_area = 0x7fffcd89cdb0}}
(gdb) frame 2
#2  0x00002ac4de2affbe in __vsyslog_chk () from /lib/libc.so.6
(gdb) print priority
No symbol "priority" in current context.
(gdb) print flag
No symbol "flag" in current context.
(gdb) print format
No symbol "format" in current context.
(gdb) print ap
No symbol "ap" in current context.

I need to look at the values in "..." to see what was being
passed on in my call to vsyslog(). How do I do that? I haven't
discovered how to despite googling and reading the gdb manual.

The source of the function WriteLog looks like this:

void
WriteLog(int facility, int level, const char *fmt, ...)
{
        va_list ap;
        va_start(ap, fmt);
        vsyslog(facility, level, fmt, ap);
        va_end(ap);
        return;
}

and it is called like this:
        WriteLog(LOG_LOCAL0, LOG_WARNING, "MysqlWrapper(): Query failed: "
                \"%s\": %s", query, mysql_error(sql_handle));

openlog() has been called earlier, like this:
        openlog(argv[0], LOG_CONS | LOG_ODELAY | LOG_PID, LOG_LOCAL0);

So I really need to look at all arguments to WriteLog() to see if
mysql_error() have returned something weird to cause strlen() to
misbehave. My fmt argument is good, as can be seen in the gdb
output.

Unfortunately I haven't found a way to reproduce the crash (it
happens very rarely), which is why I really need to try and track
the bug down using this core dump. To further complicate things,
it's taken from a production machine with all the difficulties
that can bring (no debug libraries etc.).

(Apologies if this is the wrong place to ask, and if the formatting
gets screwed up.)