Missing Python.h
Russell VT
russellvt@gmail.com
Sat Sep 25 06:19:51 GMT 2021
I am NOT saying there's a problem having "both" versions installed at the
same time... I am merely pointing out that, apparently, you have a chicken
and egg type issue, likely coming down to "which symlink" is created for
/usr/bin/python ... and more specifically, if it points to version 2, or
version 3 as "default" (these can also be managed in /etc/alternatives).
But, it's tough to tell because you've only included minimal evidence that
would appear to be python3 complaining about python2 libraries ... which
isn't a big surprise, really. Considering that it is HIGHLY advised to no
longer use Python2, it would not surprise me if something may change those
base symlinks
For example. from one of my own Cygwin installs...
[image: image.png]
So, at least my own Cygwin *DEFAULTS* to Python3, not Python2 (which again,
given current coding standards, is not at all surprising). You can use the
"alternative" command to see how your environment(s) are setup.
[image: image.png]
So yeah, you may want to argue that having a Python environment manager
such as pyenv (a simple git pull) is "too much complexity," but I can tell
you as a large scale devops type that it actually simplifies a lot of the
complexities, especially if you need to manage multiple module versions
across more-than-one Python interpreter. And, if you do *ANY* sort of
automated testing, these environments are literally a blessing.... you may
want to check out tox, or any of the other github/gitlab sort of
automations, for example.
As far as "tossing" Python3, I would *highly* advise you to work towards
upgrading to it NOW, rather than running away from it. If you write
*anything* that requires something like SSL libraries, you literally can't
completely avoid SSL vulnerabilities without considerable effort in the
older Python2 stuff.
Anyway, hope that helps...
RVT
On Wed, Sep 22, 2021 at 7:19 AM Dennis Putnam <dap1@bellsouth.net> wrote:
> Hi Russell,
>
> Thanks for the reply. I program Python 2 and 3 on various Linux systems
> but not much on Cygwin. It normally is not a problem to have both so I
> didn't think it would be a problem on Cygwin. Since I have scripts for 2 on
> Cygwin, I'm thinking I should toss 3,at least for now, and just stick with
> 2. I have a lot to think about at this point. Using an environment layer
> adds more complexity when I want to launch a script from a Windows
> application or bat file.
>
> On 9/22/2021 5:49 AM, Russell VT wrote:
>
> First off, this *probably* isn't a Cygwin problem ... but it looks like
> your environment is confused as it's using BOTH Python2 and Python3 modules
> to try to fulfill the requirements (including resources that have already
> been cached).
>
> For the most part, pip and pip3 can differentiate, but there's a "cart"
> and "horse" problem, as if you install things in a weird order at the
> system level, it may or may not do the right thing. But, I'd recommend
> "dumping" Python2, if you can at this point (it was EOL'd in December of
> 2020 and WILL NOT receive more updates, except for security ... and
> more-over, Python 3.7+ (approx) is going to demand newer SSL libraries that
> will REALLY confuse earlier versions).
>
> For working with Python (as a Python devops type), I generally recommend
> using 'pyenv' and 'pyenv-virtualenv' and trying to do *as little as
> possible* to modify the system-level Python ...this gets HARD with a system
> like Cygwin, where the generic user can generally overwrite system
> binaries, without any real sort of warning (and NO, UAC does NOT adequately
> fix anything).
>
> That also said, pyenv kinda really "fights" with Cygwin in some of the
> library placement (specifically things like FFI, IIRC, which is stored in a
> different library directory than it is, anywhere else I've found).
>
> Where I MIGHT start is to "Force Reinstall" the Python3 stuff from Setup.
> Look to see if requests_html is part of the Cygwin-supported modules, and
> use THAT... use the hell out of anything you see in the actual Python
> packages list, as those will at least be done RIGHT, and will leave you
> with more cycles to not worry about too much, except your development.
>
> For "Advanced" handling in Python, you're going to want to use "pyenv" or
> some other multi-python managers that are out there (virtualenv and
> virtualenv-wrapper are good, but ONLY manage the library path). Pretty much
> "pyenv" and "pipenv" are the top two, IIRC. I use pyenv, and haven't dug
> too deep in to pipenv, at this point. But, like I said, it's already tough
> enough to manage on older systems with older libraries (SSL, specifically,
> throws wrenches in to *everything*).
>
> Feel free to hit me up for other ideas... I write too much Python code, as
> it is, and on too many different environments (yes, some still do Python2.3
> through 2.6, and it makes me want to shoot myself, sometimes... LOL).
>
> Hope that helps -
> Russell VT
>
>
> On Tue, Sep 21, 2021 at 11:38 AM Dennis Putnam <dap1@bellsouth.net> wrote:
>
>> I am trying to install 'requests_html' and when it tries to do a compile
>> it fails because Python.h is missing. I have python2-devl installed. I
>> notice that it is looking for it in /pub which apparently does not
>> exist. Can someone help? TIA.
>>
>> Here is the entire 'pip' output:
>>
>> $ pip install requests_html
>> Collecting requests_html
>> Using cached requests_html-0.10.0-py3-none-any.whl (13 kB)
>> Collecting requests
>> Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
>> Collecting w3lib
>> Using cached w3lib-1.22.0-py2.py3-none-any.whl (20 kB)
>> Collecting parse
>> Using cached parse-1.19.0.tar.gz (30 kB)
>> Collecting fake-useragent
>> Using cached fake-useragent-0.1.11.tar.gz (13 kB)
>> Collecting pyquery
>> Using cached pyquery-1.4.3-py3-none-any.whl (22 kB)
>> Collecting bs4
>> Using cached bs4-0.0.1.tar.gz (1.1 kB)
>> Collecting pyppeteer>=0.0.14
>> Using cached pyppeteer-0.2.6-py3-none-any.whl (83 kB)
>> Requirement already satisfied: tqdm<5.0.0,>=4.42.1 in
>> /usr/local/lib/python3.8/site-packages (from
>> pyppeteer>=0.0.14->requests_html) (4.62.3)
>> Requirement already satisfied: urllib3<2.0.0,>=1.25.8 in
>> /usr/local/lib/python3.8/site-packages (from
>> pyppeteer>=0.0.14->requests_html) (1.26.6)
>> Collecting appdirs<2.0.0,>=1.4.3
>> Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
>> Collecting importlib-metadata>=1.4
>> Using cached importlib_metadata-4.8.1-py3-none-any.whl (17 kB)
>> Requirement already satisfied: pyee<9.0.0,>=8.1.0 in
>> /usr/local/lib/python3.8/site-packages (from
>> pyppeteer>=0.0.14->requests_html) (8.2.2)
>> Requirement already satisfied: websockets<10.0,>=9.1 in
>> /usr/local/lib/python3.8/site-packages (from
>> pyppeteer>=0.0.14->requests_html) (9.1)
>> Requirement already satisfied: zipp>=0.5 in
>> /usr/local/lib/python3.8/site-packages (from
>> importlib-metadata>=1.4->pyppeteer>=0.0.14->requests_html) ( 3.5.0)
>> Requirement already satisfied: beautifulsoup4 in
>> /usr/local/lib/python3.8/site-packages (from bs4->requests_html) (4.10.0)
>> Requirement already satisfied: soupsieve>1.2 in
>> /usr/local/lib/python3.8/site-packages (from
>> beautifulsoup4->bs4->requests_html) (2.2.1)
>> Collecting cssselect>0.7.9
>> Using cached cssselect-1.1.0-py2.py3-none-any.whl (16 kB)
>> Collecting lxml>=2.1
>> Using cached lxml-4.6.3.tar.gz (3.2 MB)
>> Collecting idna<4,>=2.5
>> Using cached idna-3.2-py3-none-any.whl (59 kB)
>> Collecting certifi>=2017.4.17
>> Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
>> Collecting charset-normalizer~=2.0.0
>> Using cached charset_normalizer-2.0.6-py3-none-any.whl (37 kB)
>> Requirement already satisfied: six>=1.4.1 in
>> /usr/local/lib/python3.8/site-packages (from w3lib->requests_html)
>> (1.16.0)
>> Using legacy 'setup.py install' for bs4, since package 'wheel' is not
>> installed.
>> Using legacy 'setup.py install' for fake-useragent, since package
>> 'wheel' is not installed.
>> Using legacy 'setup.py install' for parse, since package 'wheel' is not
>> installed.
>> Using legacy 'setup.py install' for lxml, since package 'wheel' is not
>> installed.
>> Installing collected packages: lxml, importlib-metadata, idna,
>> cssselect, charset-normalizer, certifi, appdirs, w3lib, requests,
>> pyquery, pyppeteer,
>> parse, fake-useragent, bs4, requests-html
>> Running setup.py install for lxml ... error
>> ERROR: Command errored out with exit status 1:
>> command: /usr/bin/python -u -c 'import io, os, sys, setuptools,
>> tokenize; sys.argv[0] =
>> '"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a
>> 7a21e44c273e/setup.py'"'"';
>> __file__='"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';f
>>
>> = getattr(tokenize, '"'"'open'"'"', open)(__file__) if
>> os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
>> setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',
>> '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
>> install --record /tmp/pip-record-qafwouur/install-record.txt
>> --single-version-externally-managed --compile --install-headers
>> /usr/local/include/python3.8/lxml
>> cwd:
>> /tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/
>> Complete output (86 lines):
>> Building lxml version 4.6.3.
>> Building without Cython.
>> Building against libxml2 2.9.10 and libxslt 1.1.29
>> running install
>> running build
>> running build_py
>> creating build
>> creating build/lib.cygwin-3.2.0-x86_64-3.8
>> creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/builder.py -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/cssselect.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/doctestcompare.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/ElementInclude.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/pyclasslookup.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/sax.py -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/usedoctest.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/_elementpath.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/__init__.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/__init__.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/builder.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/clean.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/defs.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/diff.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/ElementSoup.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/formfill.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/html5parser.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/soupparser.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/usedoctest.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/_diffcommand.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/_html5builder.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/_setmixin.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> copying src/lxml/html/__init__.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
>> creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron
>> copying src/lxml/isoschematron/__init__.py ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron
>> copying src/lxml/etree.h -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/etree_api.h ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/lxml.etree.h ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/lxml.etree_api.h ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> copying src/lxml/includes/c14n.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/config.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/dtdvalid.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/etreepublic.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/htmlparser.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/relaxng.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/schematron.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/tree.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/uri.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/xinclude.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/xmlerror.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/xmlparser.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/xmlschema.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/xpath.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/xslt.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/__init__.pxd ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/etree_defs.h ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> copying src/lxml/includes/lxml-version.h ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
>> creating
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources
>> creating
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/rng
>> copying src/lxml/isoschematron/resources/rng/iso-schematron.rng ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/rng
>> creating
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl
>> copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl
>> copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl
>> creating
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
>>
>>
>> copying
>> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl
>>
>> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron
>> /resources/xsl/iso-schematron-xslt1
>> copying
>> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl
>>
>> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/re
>> sources/xsl/iso-schematron-xslt1
>> copying
>> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl
>>
>> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschemat
>> ron/resources/xsl/iso-schematron-xslt1
>> copying
>> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl
>>
>> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
>> /isoschematron/resources/xsl/iso-schematron-xslt1
>> copying
>> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl
>>
>> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/
>> resources/xsl/iso-schematron-xslt1
>> copying
>> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt ->
>> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xs
>> l/iso-schematron-xslt1
>> running build_ext
>> building 'lxml.etree' extension
>> creating build/temp.cygwin-3.2.0-x86_64-3.8
>> creating build/temp.cygwin-3.2.0-x86_64-3.8/src
>> creating build/temp.cygwin-3.2.0-x86_64-3.8/src/lxml
>> gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall
>> -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
>> -fstack-protector-strong --param=ssp-buffer-size=4 -DOPENSSL_NO_SSL3=1
>> -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/build=/usr/src/debug/python38-3.8.10-1
>>
>> -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/src/Python-3.8.10=/usr/src/debug/python38-3.8.10-1
>>
>> -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
>> -fstack-protector-strong --param=ssp-buffer-size=4 -DOPENSSL_NO_SSL3=1
>> -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/build=/usr/src/debug/python38-3.8.10-1
>>
>> -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/src/Python-3.8.10=/usr/src/debug/python38-3.8.10-1
>>
>> -DCYTHON_CLINE_IN_TRACEBACK=0 -I/usr/include/libxml2 -Isrc
>> -Isrc/lxml/includes -I/usr/include/python3.8 -c src/lxml/etree.c -o
>> build/temp.cygwin-3.2.0-x86_64-3.8/src/lxml/etree.o -w
>> src/lxml/etree.c:97:10: fatal error: Python.h: No such file or
>> directory
>> 97 | #include "Python.h"
>> | ^~~~~~~~~~
>> compilation terminated.
>> Compile failed: command 'gcc' failed with exit status 1
>> creating tmp
>> cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c
>> /tmp/xmlXPathInit4efcky6t.c -o tmp/xmlXPathInit4efcky6t.o
>> cc tmp/xmlXPathInit4efcky6t.o -lxml2 -o a.out.exe
>> error: command 'gcc' failed with exit status 1
>> ----------------------------------------
>> ERROR: Command errored out with exit status 1: /usr/bin/python -u -c
>> 'import io, os, sys, setuptools, tokenize; sys.argv[0] =
>> '"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';
>>
>> __file__='"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';f
>>
>> = getattr(tokenize, '"'"'open'"'"', open)(__file__) if
>> os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
>> setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',
>> '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
>> install --record /tmp/pip-record-qafwouur/install-record.txt
>> --single-version-externally-managed --compile --install-headers
>> /usr/local/include/python3.8/lxml Check the logs for full command output.
>>
>>
>> --
>> Problem reports: https://cygwin.com/problems.html
>> FAQ: https://cygwin.com/faq/
>> Documentation: https://cygwin.com/docs.html
>> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
>>
>
>
> --
> Russell M. Van Tassell <russellvt@gmail.com>
>
>
>
--
Russell M. Van Tassell <russellvt@gmail.com>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 9386 bytes
Desc: not available
URL: <https://cygwin.com/pipermail/cygwin/attachments/20210924/4e9b7fd6/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 29788 bytes
Desc: not available
URL: <https://cygwin.com/pipermail/cygwin/attachments/20210924/4e9b7fd6/attachment-0003.png>
More information about the Cygwin
mailing list