Re: Python enabled gdb on Windows and relocation

[Ruben Van Boxem <vanboxem dot ruben at gmail dot com>]

2011/5/14 Doug Evans <dje@google.com>:
> On Sat, May 14, 2011 at 2:09 AM, Ruben Van Boxem
> <vanboxem.ruben@gmail.com> wrote:
>> 2011/5/14 Doug Evans <dje@google.com>:
>>> On Thu, May 12, 2011 at 9:19 AM, Ruben Van Boxem
>>> <vanboxem.ruben@gmail.com> wrote:
>>>> (now in plain-text as required by gdb mailing list)
>>>>
>>>> Hi,
>>>>
>>>> I am currently trying to integrate Python support into my toolchain
>>>> build (including GDB of course). It is a sysrooted
>>>> binutils+GCC+GDB+mingw-w64 toolchain.
>>>>
>>>> I currently have the basic setup working: I can link gdb with my
>>>> manually generated import lib to the python dll from the official
>>>> Windows install. If there is anything I am missing or a very easy
>>>> solution to the problems decsribed below, please just say so. I am
>>>> only suggesting what I would like to happen.
>>>>
>>>> Now on to the problems I'd like to discuss:
>>>>
>>>> 1. gdb.exe won't start without me having set PYTHONPATH manually.
>>>
>>> In a properly configured/built gdb on linux this isn't necessary, even
>>> if python is installed in some random place.
>>> I'm not sure about windows though.
>>> Did you specify --with-python when you configured gdb, and if so did
>>> you specify a value?
>>> e.g., --with-python=SOME_VALUE
>>
>> I was cross-compiling a mingw toolchain+gdb from Linux, so I used
>> --with-python without a value (because gdb configure tries to find the
>> Python executabe), and I added -I"/path/to/python/includes" to CFLAGS
>> and -L"/path/to/pythondll/importlib" to LDFLAGS, which built as it
>> should. This is hacky though, and gdb configure should provide
>> --with-python-libs and --with-python-include to make it more
>> streamlined with any other build prerequisite (like
>> gmp/mpfr/mpc/cloog/ppl in GCC for example).
>
> Ah.
> Cross-compiling gdb with python is in need of improvement.
> Alas python hasn't been designed with cross-compilation in mind (e.g.
> build on linux, run on windows).
> AIUI, the way to get the parameters required for compiling with
> libpython is to get them from python's "distutils": kinda hard to do
> in a cross-compile. ÂDone correctly there's no need to run python.
>
> I haven't done anything more to support python in gdb's configure.ac
> because it's not clear to me what the right thing to do is: distutils
> provides more than just --libs and --includes (btw, we don't use
> --libs though, we use --ldflags which includes all of: the directory
> in which to find libpython, the -l for libpython, and the -l's for all
> the other libraries python needs). [Which isn't to say that someone
> else isn't free to tackle this.]
>
> In the meantime, what I've been doing is a hack: write a script that
> responds to:
> --includes
> --ldflags
> --exec-prefix
> and pass that as --with-python.
>
> E.g.
> bash$ cat $HOME/my-python-for-config
> #! /bin/sh
>
> if [ $# -ne 2 ]
> then
> Â Â Â Âecho "Bad # args. ÂBlech!" >&2
> Â Â Â Âexit 1
> fi
>
> # The first argument is the path to python-config.py, ignore it.
>
> case "$2" in
> --includes) echo "-I/usr/include/python2.6 -I/usr/include/python2.6" ;;
> --ldflags) echo "-L/usr/lib/python2.6/config -lpthread -ldl -lutil -lm
> -lpython2.6" ;;
> --exec-prefix) echo "/usr" ;;
> *) echo "Bad arg $2. ÂBlech!" >&2 ; exit 1 ;;
> esac
>
> exit 0
> bash$ ./configure --with-python=$HOME/my-python-for-config [...]
> [...]
>
>
> Note that --exec-prefix is the runtime location of python.
> GCC uses this to tell libpython where to find its support files.
> [grep for Py_SetProgramName in gdb/python/python.c]

OK, I tried your script in a couple of variations. It gets rid of the
traceback I had before, but still doesn't help the PYTHONPATH problem.
My directory structure is as follows (this is not in root, "/" is just
my main build directory, there are several levels below it):

/gdb <-- gdb build dir
/gdb/gdb <-- where the python configuration is done
/python <-- temporary install dir for python files for build,
extracted from the official Windows installer, also location of
libpython2.7.a import library
/python/include/python27 <-- python headers, found by gdb in both cases
/mingw64 <-- toolchain sysroot prefix
/mingw64/bin <-- install location of gdb and python27.dll
/mingw64/lib/python27 <-- install location of all python scripts

First variant:
--includes) echo "-I../../python/include" ;;
--ldflags) echo "-L../../python -lpython2.7" ;;
--exec-prefix) echo "../../mingw64/lib/python27" ;;

Here exec-prefix would be the relative path from where "configure"
does its magic to the final location of the scripts on the build
system.

Second variant:
--includes) echo "-I../../python/include" ;;
--ldflags) echo "-L../../python -lpython2.7" ;;
--exec-prefix) echo "../lib/python27" ;;

I thought the second points gdb to the installed location of the
python scripts, but it still needed PYTHONPATH for that. I used
relative paths in an attempt to "do the right thing", hackwise...
Remember that the whole "/mingw64" directory gets zipped and moved,
then it gets extracted to a random location in the Windows filesystem.

Thanks for the help so far, I think we might just get this worked out together.

Ruben

>
>>>> I understand the need for this, but as gdb requires Python 2, and users
>>>> of my toolchain may have installed Python 3 or a 32-bit version python
>>>> they want to use from the same environment (without changing their own
>>>> PYTHONPATH), there is no way to run python-enabled gdb.
>>>> [...]
>>>
>>> Yeah.
>>> There is a proposal to add GDB_PYTHONPATH (or some such IIRC) and have
>>> gdb use that instead of PYTHONPATH if it exists, but there's been
>>> resistance to it.
>>> I think(!) what would happen is that gdb would set $PYTHONPATH to the
>>> value of $GDB_PYTHONPATH.
>>> [Inferiors started by gdb should still get the original value of
>>> PYTHONPATH though.]
>>
>> That way would be almost ideal, but a hardcoded *relative* path to the
>> python scripts (that is standardized within gdb) wouldn't hurt.
>
> See above re: --exec-prefix.
>
>> An
>> extra environment variable would require a lot of explaining for
>> Windows, and is not "plug-and-play", like the rest of a sysrooted
>> toolchain is supposed to be like. I think this should work on all
>> setups:
>>
>> 1. Check hardcoded path; my suggestion would be "<gdb
>> executable>/../lib/python27"
>> 2. If this fails to find the necessary files/scripts, find it like you
>> described above in Linux, without PYTHONPATH set.
>> 3. Check PYTHONPATH.
>
> The problem being solved by the proposed GDB_PYTHONPATH is "What if
> the user has PYTHONPATH set and it points to an incompatible version
> of python?".
> Leaving such a value for PYTHONPATH set while gdb's python is running
> feels wrong (and IIRC has caused some problems).
>
> The problem of telling python where to find itself is already solved
> (or at least is intended to be solved) with gdb's calling
> Py_SetProgramName with a value derived from the python-provided
> --exec-prefix.
>
>> I would think only number one would change, and perhaps be only
>> enabled with a special configure option. Nothing else would have to
>> change, and Windows users would rejoice :)
>> Again, this is only my suggestion, if there are problems with it in
>> way I haven't thought of, please say so, and we can come up with
>> another solution.
>