MinGW - View Source: Python extensions

!!!Overview

As of 2007-05-15, using Python 2.4.4 or 2.5.1 and the MinGW 5.1.3 installer installing "current" (mingw-runtime 3.12, w32api 3.9, binutils 2.16.91-20060119-1, gcc-core 3.4.2-20040916-1), the ["minimal example"|http://en.wikibooks.org/wiki/Python_Programming/Extending_with_C#A_minimal_example] Python module can be built easily with MinGW following the same instructions as for Linux, but invoking distutils with =python setup.py build --compiler=mingw32=.

But if you are using a win32 binary distribution of Python 2.3 or older (e.g. the one from www.python.org), then developing Python extensions from a *mingw/MSYS* (or *cygwin*) environment requires a few tricks that are not obvious. This is because the python distribution does not come with an export library, so you have to create one, and there's a couple of other gotchas too.

I spent a lot of time looking around for information on how to make this work. The most cited source seems to be http://sebsauvage.net/python/mingw.html but for some reason it didn't work for me (just bad luck?). Several other pages exist on the web that give some instructions, but I found that they are either out of date, or wrong, or too difficult to understand. The Python.org document, ["Installing Python Modules", section 6.2.2 | http://www.python.org/doc/2.3.4/inst/tweak-flags.html] explains the basics, and this is the only one that worked for me, at least with MinGW 3.1.0-1 (MSYS 1.0.10 w/ gcc 3.2.3) and Python 2.3.3.

While looking for something else, I came across this patch request made by [Connelly (python-Bugs-957198) |http://mail.python.org/pipermail/python-bugs-list/2004-June/023351.html] that summarizes nicely what to do, and I so wish I had found that earlier, that I decided to create this page. I include the relevant parts of the instructions mentioned in the patch, and I add a few clarifications.

!!!Setup for Python 2.3 (or older)

You only need to do this once, every time you change your Python interpreter (i.e. upgrade or re-install):

# If your mingw does NOT contain pexports (e.g., do ='which pexports'= from msys command line), [download it | http://starship.python.net/crew/kernr/mingw32/pexports-0.42h.zip] and extract =pexports.exe= to your Mingw =bin= directory.
# Find =pythonxx.dll=. On my machine, with Python 2.3 it is in =/c/WINNT/system32=, as =python23.dll=. It has been mentioned in several web pages that it could also be in =/yourPythonRoot/libs=.
# cd to =/yourPythonRoot/libs=. E.g. on my system I cd to =/c/Python23/libs=.
# Assuming =pythonxx.dll= is in =/c/WINNT/system32=, do the following (from =/yourPythonDir/libs=; also, adjust the version number to your version of Python):

  <verbatim>
  pexports /c/WINNT/system32/python23.dll > python23.def
  dlltool --dllname python23.dll --def python23.def --output-lib libpython23.a
  </verbatim>

!!!Building an extension

You are now ok to build python extensions. E.g. if your extension is in =example.c=,

<verbatim>
    gcc -c  example.c  -I/c/Python23/include
    gcc -shared example.o  -L/c/Python23/libs -lpython23  -o example.dll
</verbatim>

If you haven't done the above Basic Setup steps correctly, you could get several kinds of errors:

# The link stage (=gcc -shared ...=) could fail, saying the python23 library was not found, is not the correct format, there are undefined symbols;
# When you run python and import your module, you may get a ="Fatal Python error: Interpreter not initialized (version mismatch?)"= and the interpreter will exit.

If you use *SWIG*, remember that your dll extension must start with an underscore, because swig wraps your dll with Python code to make it more robust. E.g. the above example would become:

<verbatim>
    swig -python  example.i
    gcc -c example_wrap.c example.c  -I/c/Python23/include
    gcc -shared example.o example_wrap.o -L/c/Python23/libs -lpython23  -o _example.dll
</verbatim>


!!!Creating an extension with Distutils

You must specify which compiler to use, so you would do:

<verbatim>
    python setup.py build --compiler=mingw32
</verbatim>

and then:

<verbatim>
    python setup.py install
</verbatim>

Note this [interesting post | http://mail.python.org/pipermail/distutils-sig/2004-October/004194.html] on the distutils-sig mailing list:

<verbatim>
    Thomas Heller wrote:
    > Scott David Daniels <Scott.Daniels at Acm.Org> writes:
    >>I am having a bit of trouble creating a windows distributable
    >>with a C module using MingW32.  While I can do:
     >>         python setup.py build --compile=mingw32
     >>         python setup.py bdist_wininst --skip-build
     >>I get an executable with no tag for the particular python version.
    >
    > You could use command chaining, if that's the correct word for that:
    > python setup.py build --compiler=mingw32 bdist_wininst

    This does the trick, both for using the mingw32 compiler, and for
    tagging the result with the python version.  Thanks for the help.

    -- Scott David Daniels
</verbatim>

If you are wondering what the =setup.py= file looks like, here is an example for a C extension, YMMV:

<verbatim>
    # setup.py
    from distutils.core import setup, Extension
    setup (name = "_example",
        version = "1.0",
        maintainer = "Your Name",
        maintainer_email = "your.name@domain.org",
        description = "Sample Python C Extension",
        ext_modules = [Extension('_example',
                      extra_compile_args=['-O2'],
                      library_dirs=['/mingw/lib'],
                      sources=['example.c','example_wrap.c'])])
</verbatim>

And here is an example for C++, YMMV:

<verbatim>
    # setup.py
    from distutils.core import setup, Extension
    setup (name = "_example",
        version = "1.0",
        maintainer = "Your Name",
        maintainer_email = "your.name@domain.org",
        description = "Sample Python C++ Extension",
        ext_modules = [Extension('_example',
                       extra_compile_args=['-O2'],
                       library_dirs=['/mingw/lib'],
                       libraries=['supc++'],
                       sources=['example.cxx','example_wrap.cxx'])])
</verbatim>

!!!Extra Setup for Distutils in Python 2.3 (or older)

You may want to use *distutils* to distribute or even just build your python extensions.  If you are using Python 2.3 or older and do NOT have MSVC installed on your machine, then you need to patch *distutils* because it thinks your platform is nt, and the nt compiler defaults to MSVC (I haven't yet found how to say in =setup.py= that you want the 'unix' compiler type).  So:

# Locate =msvccompiler.py= in your Python distribution. On my machine, it is in =/c/Python23/Lib/distutils=.
# Open it
# Find and DELETE the following lines (around line 211):

  <verbatim>
  if len (self.__paths) == 0:
      raise DistutilsPlatformError, ("Python was built with version %s of Visual Studio, "
                "and extensions need to be built with the same "
                "version of the compiler, but it isn't installed." % self.__version)
  </verbatim>

You will notice that if you don't apply this patch, the above exception is raised when you do =python setup.py install=, because the condition is true when MSVC is not installed (the path doesn't contain anything).

NOTE: (added by Chris Barker).
Rather than editing the distutils code, you can just do:

=python setup.py install --skip-build=

By skipping the build stage, you seem to skip the compiler check.
END NOTE

If you want to make *C++* extensions, you will likely have to create =/mingw/bin/cc.exe=. You will know, because the setup.py build will give an error about =cc.exe= not found. On my system, I have found that I had to create it by copying from =/mingw/bin/dllwrap.exe=, but the original Connelly post says to copy from =/mingw/bin/gcc.exe=, so maybe it depends on the version of gcc or mingw you have. Naturally, creating =cc.exe= only has to be done once, forever.


!!! Extra Setup for Swig/autotools

If you have configure scripts that use AM_PATH_PYTHON or PYTHON_DEVEL (from Swig),
there's a bit more that you need to do if you want them working in MinGW.

First, make sure there is nothing from cygwin in your $PATH variable. Make a
/usr/local/bin (*) if you don't have one already and put that in your $PATH. Make
a file called 'python' in /usr/local/bin that contains simply:

<verbatim>
#!/bin/sh
/f/Software/Python-242/python.exe "$@"
</verbatim>

replacing /f/Software/Python-242 with /yourPythonRoot/


On my machine, after doing this, AM_PATH_PYTHON was able to find the script and extension directories
in /f/Software/Python-242/Lib/, but not the includes or the libraries. So

<verbatim>
cp -r /yourPythonRoot/include /usr/local/include/python2.4
cp -r /yourPythonRoot/Lib /usr/local/lib/python2.4
cp -r /yourPythonRoot/libs /usr/local/lib/python2.4/config
</verbatim>

You'll also need a dot in the version spec for the library you created above with dlltool, so (for example):

<verbatim>
cp /usr/local/lib/python2.4/config/libpython24.a /usr/local/lib/python2.4/config/libpython2.4.a
</verbatim>

Then things should be peachy.

-- TrentApted 2005-10-27

!!! Python2.4 and MSVCR71

This seems to be widely discussed with no clear answer.%%%
One approach is to just follow the directions above.%%%
If this doesn't work, or bothers you, then check these pages:%%%
http://article.gmane.org/gmane.comp.python.general/378028/match=msvcr71
http://jove.prohosting.com/iwave/ipython/issues.html (section 4 - Extending the official distribution)

If you run depends.exe (download it with Microsoft's platform SDK) on the windows binary (python.exe):%%%
<verbatim>
Python.exe depends on: python24.dll, msvcr71.dll, and kernel32.dll
python24.dll depends on: kernel32.dll, user32.dll, advapi32.dll, shell32.dll, and msvcr71.dll
shell32.dll depends on: msvcrt.dll (and others)
</verbatim>
If nothing else works, you can download Microsoft's toolkit and follow these directions:
http://www.vrplumber.com/programming/mstoolkit/  (Note that the author of this page gave up)

-- KenStaton 22Nov2005

!!! One Last Step...
This may help some, it worked for me (I'm using distutils) for Python24xx and now for Python25rc1.
It requires the minimum setup and does not modify any Python files.
# Follow the instructions above under "Basic Setup" to create a compatable library.
# Create a cfg file for distutils (distutils.cfg) eg: /c/Python2x/Lib/distutils/distutils.cfg
containing:%%%
<verbatim>
[build]
compiler=mingw32
</verbatim>
Done. For whatever reason, using "--compiler=mingw32" never worked for me.

-- ShaneYuile 2006-Sep-11