Python Extensions In C++ Using SWIG

Originally published: Oct 8, 2003 (Last update: August 15, 2007)

Table of Contents

1. Abstract
2. Introduction
3. The Python/C API
4. SWIG Basics
   1. A Quick Example
5. Building Extensions on GNU/Linux
6. Building Extensions on Windows
   1. Setting Up The Environment
   2. Enabling Syntax-Highlighting For .i Files
   3. Setting Up The DLL Project
   4. SWIG As A Custom Build Step
   5. Troubleshooting
7. A Case Study: “pymfg”
   1. Namespaces And Other Reasons Why Your Classes Aren’t Wrapped
   2. Hiding The Unwanted And Faking The Non-Existent
   3. Someone Has Done All The Work – Using STL Classes
   4. The Problem With Pointers To Pointers
8. Conclusion

Abstract

This article walks you through the process of writing a Python extension module in C++. To simplify the task, we are going to use SWIG to produce the “glue code” between Python and C++. The article presents the following concepts:

• Converting between C++ and Python data types
• Setting up an extension module project that uses SWIG
• Using SWIG interface definition files
• Exporting functions and classes
• Exporting STL containers

Some familiarity with Python and good C++ knowledge are required. Experience with SWIG is not necessary. To be able to try out what is described here, you need a Python interpreter, SWIG, and a C++ compiler (preferably Visual C++).

Introduction

The reason why we are using Python is probably its ease of use; the rapid prototyping that it makes possible; the availability of third-party libraries from data compression to encryption to linear algebra; or even such esoteric features as COM bindings. Certainly not for its unprecedented runtime performance. If you decide on writing a raytracer in Python, you probably get what you deserve. (More time for your family while the thing is rendering, for example.)

C++ is a better choice when performance is your major concern. However, they say that only 20% of the runtime is spent in 80% of the code. The optimal solution would be to use Python for the 80% of the code that are not runtime-critical, and to use C++ only for those areas where performance really matters. The good news is that you can have the best of both worlds.

This article presents SWIG, a tool that generates wrapper code around your C++ functions and classes, so that they are callable from Python. This process is called “extending” Python. (As opposed to “embedding” Python, which allows you to invoke Python code from within a C++ application. The two have much in common, however.) In the following few chapters, you will learn everything you need to know to build your own extension modules.

In the following section, I will give you some background on the Python C/API, which will be used internally by our extension module. Next, I will give you a short introduction on what SWIG is and what it does. After that, we will set up a DLL project for our extension module. Finally, I will talk about my experiences (mostly good) with using SWIG to write “pymfg,” the Python bindings to the mfGraph library (available here). This last section will go beyond the typical “Hello, World!” example. We are going to talk about namespace issues, STL containers, and exporting pointer data types.

The Python/C API

This section explains what is going on under the hood when Python starts talking to C. You can skip this if you like, because SWIG will take care of the details.

The Python interpreter is written in C. When a script is running, all Python objects are stored in some C representation. For example, when the Python code constructs a list object, the interpreter allocates a PyObject internally, using the API function PyList_New. Other PyList_ functions, such as PyList_Sort and PyList_GetSlice, are available for working with the object.

The Python/C API is available to external applications as well. Using the following command sequence, you can have Python execute the command print pow(2, 3):

    #include <python.h>   // implicitly links to pythonxy.lib
    ...
    Py_Initialize();
    PyRun_SimpleString("print pow(2, 3)");
    Py_Finalize();

Admittedly, std::cout << 8 works equally as well for this simple task, but you can clearly see the potential. It would be nice if we could pass arguments to pow directly. We also need access to pow‘s return value. Using the Python/C API, we can do this as follows: (Let’s assume we know how to retrieve the PyObject for the pow function.)

    PyObject* pow = get_pointer_to_pow();
    ...
    // Create arguments for pow
    PyObject* arg1 = PyInt_FromLong(2);
    PyObject* arg2 = PyInt_FromLong(3);

    // Throw them into a tuple
    PyObject* args = PyTuple_New(2);
    PyTuple_SetItem(args, 0, arg1);
    PyTuple_SetItem(args, 1, arg2);

    // Invoke the function
    PyObject* ret = PyObject_CallObject(pow, args);

    // Retrieve the return value
    int result = PyInt_AsLong(ret);
    std::cout << result;    // Yes!

Note: The Py_BuildValue function could have been used as a shortcut for building the args tuple. Also note that reference counting has been omitted for brevity.

This piece of code demonstrates the conversion of data types in both directions. PyInt_FromLong converts an integer into a Python int object. PyInt_AsLong converts a Python int back into a C int. This is the kind of conversion that we need to do when writing a Python extension in C or C++.

In the example above, a C++ program was in control and actively invoked a Python function. If we were to implement the pow function in C++ so that it was accessible from Python, the conversions would happen in the opposite direction:

    PyObject* pow(PyObject* self, PyObject* args)
    {
        double x, y;

        // Extract x and y from the arguments tuple
        if (!PyArg_ParseTuple(args, "dd", &x, &y))
            return 0;

        // Do the work
        double result = calculate_pow(x, y);

        // Return the result as a Python object
        return PyFloat_FromDouble(result);
    }

To make this function available to Python, you must compile it into a DLL that exports a special initialization function. This initialization function returns the module’s method table, which serves as a “table of contents” for the module. (The exact details of this are explained in the chapter “Extending and Embedding the Python Interpreter” in the Python Manuals.) Then you can use the import directive in Python to access the pow function.

Okay, so we can make C++ code callable from Python; but writing the conversion code looks like a tedious task. Don’t worry! This is the kind of glue code that SWIG is going to generate for us.

SWIG Basics

In the previous section, we learned that it is necessary for an extension module to convert function arguments from Python to C++ and function return values from C++ to Python. I will refer to the conversion code as the “glue code” or “wrapper code” between Python and C++.

SWIG is a command-line utility that looks at your C++ declarations and automatically generates the necessary glue code. For example, from the function declaration

    int MyFunc(int a, double b);

SWIG generates a wrapper that basically does the following (in addition to some type- and error checking):

    convert a from Python to C++
    convert b from Python to C++
    invoke MyFunc(a, b)
    convert return value from C++ to Python

SWIG has the following benefits:

• Parses declarations directly from C++ header files¹
• Support for almost all C++ language features²
• Interface definition files can be used instead of headers for more control
• Generates glue code for Python, Perl, Ruby, and others
• Converts STL containers to their most natural Python representation
• SWIG is free and open source

¹ Nevertheless, I recommend using separate interface definition files, which are basically cleaned-up headers with a .i extension. More on this later.
² One feature that is missing from SWIG 3.1.19 is nested classes. But this might be added in a future release.

You can download the pre-built executable, the docs, and the source code from the SWIG homepage.

A Quick Example

Let’s assume you have the following header file for a simple (yet useless) class:

    #ifndef _SOME_CLASS_H_
    #define _SOME_CLASS_H_

    class SomeClass
    {
    private:
        int     mValA;
        int     mValB;
    public:
        SomeClass();
        SomeClass(int a, int b);
        virtual ~SomeClass();
        void MethodA();
        void MethodB(int a = 5);
        int GetValA();
    };

    #endif  // _SOME_CLASS_H_

The first step is to create an interface definition file and name it, for example, mymodule.i.

    %module mymodule

    %{
    #include "some_class.h"
    %}

    %include "some_class.h"

The %module line specifies the name under which you are going to import the resulting extension module in Python. SWIG produces a file mymodule.py that in turn uses _mymodule.pyd, which is the DLL into which you compile your sources and the SWIG-generated wrapper (the latest Python versions require the extension .pyd instead of .dll).

The block between the “%{” and the “%}” is, much like in Flex/Bison, copied into the generated wrapper as is. Without the include directive, the wrapper code will not compile.

Finally, we tell SWIG to parse the declarations in the file some_class.h. You can refer to any number of header files and other interface definition files. Interface definition files may also contain the code from the C++ header files directly. For example, if you do not want SWIG to analyze the header, you can copy the class definition (or parts of it) into the interface file as follows:

    %module mymodule

    %{
    #include "some_class.h"
    %}

    class SomeClass
    {
    public:
        SomeClass();
        virtual ~SomeClass();
        void MethodA();
        void MethodB(int a = 5);
        int GetValA();
    };

As you can see, I have removed the private section, as it is ignored by SWIG anyway. I have also removed the second constructor, maybe because I decided it is of no use to Python programmers.

Note: You do not have to copy the class declaration just because you want to hide a single method. For this, you can also use the %ignore directive. For more information, see the SWIG docs.

Now you can invoke swig.exe from the command-line:

    swig.exe -c++ -python -o mymodule_wrap.cpp mymodule.i

Set up a DLL project for your compiler and make sure to add your own sources (such as some_class.cpp) and the SWIG-generated file mymodule_wrap.cpp. Set the name of the generated DLL file to _mymodule.pyd. As the SWIG docs will tell you, the underscore is important. The actual extension module is mymodule.py, which in turn imports from _mymodule.pyd.

When you are done, open a Python interpreter in the location that contains the files mymodule.py and _mymodule.pyd. Type the following:

    >>> import mymodule
    >>> x = mymodule.SomeClass()
    >>> x.MethodA()
    >>> x.MethodB()
    >>> x.MethodB(10)
    >>> print x.GetValA()

I agree that this is a stupid example. A much more interesting example that presents some of the (what I think) more advanced topics of working with SWIG follows. In the meantime, you may wish to take a look at the introductory chapters of the SWIG documentation, or to generate Python bindings for your own C++ libraries right away. When problems occur, I hope I can give you the solutions in the following sections.

Building Extensions On GNU/Linux

See this article, also on this site.

Building Extensions On Windows

This chapter describes all the tasks that are related to building a Python extension module for the Windows platform. The following topics are discussed in detail:

• Setting up the search path for your compiler and linker
• Setting up a DLL project
• Having SWIG run on .i files automatically

The chapter is concluded with a Troubleshooting section that addresses some of the problems that you might be facing on your way.

Note: I’m using Visual C++ 6.0, so that’s what I’m writing about when I refer to menu items or dialogs.

Setting Up The Environment

The SWIG-generated wrapper code uses the Python/C API. For this to work, VC++ must be able to find the Python/C API header file, python.h, and the import LIB for the interpreter DLL, pythonxy.lib (where xy is the Python version number, e.g., 23 for Python 2.3). Both files come with the Python binary distribution.

To set up the paths, follow these steps:

Step 1. Click Tools, Options to open the Options dialog and go to the Directories tab.

Step 2. Add the directory that contains python.h to the list of include directories. Usually, this is the \include sub-directory of your Python installation path.

Step 3. Add the directory that contains pythonxy.lib to the list of library directories. Usually, this is the \libs sub-directory of your Python installation path.

Step 4. You can also add the path to swig.exe to the list of directories for executable files. This allows you to refer to swig.exe in your DSP without specifying a full path, and thus makes the DSP more portable. Alternatively, you can add the path to swig.exe to the PATH environment variable.

Note: When setting the PATH environment variable, you must restart VC++ for it to pick up the changes.

Enabling Syntax-Highlighting For .i Files

Interface definition files contain mostly C/C++ declarations. Therefore, I found it useful to enable syntax-highlighting for these files. To do this, open regedit.exe and go to the key

HKEY_CURRENT_USER\Software\Microsoft\DevStudio\6.0\Text Editor\Tabs/Language Settings\C/C++

Add “i” to the FileExtensions value.

(You should do this after closing VC++, otherwise your changes to the Registry might be lost.)

In Borland C++Builder, you can add the .i file extension on the General page of the Editor Options dialog box.

Setting Up The DLL Project

Python extension DLL’s are regular Win32 Dynamic-Link Library projects. To set up such a project, follow these steps:

Step 1. Click File, New. Go to the Projects tab and select “Win32 Dynamic-Link Library.” Click OK.

Step 2. Select “Simple DLL Project” and click Finish.

Step 3. Click Project, Settings to open the Project Settings dialog.

Step 4. Go to the Linker tab, category General.

Step 5. Set the name of the output file to _mymodule.pyd, where mymodule is the name under which you want to import the module in Python. (The same name must be specified in the %module directive in the master .i file.)

Step 6. Repeat Step 5 for the other available configurations (Debug/Release).

Step 7. Go to the C/C++ tab, category Code Generation.

Step 8. Make sure that the runtime library is set to “Multithreaded DLL” (or “Multithreaded DLL Debug”, respectively) to be compatible with the Python interpreter. This is the default for a new DLL project.

Step 9. Go to the Post-build Step tab.

Step 10. Enter the command copy $(TargetPath). This copies the DLL file from the \Debug or \Release sub-directory to the project root directory. When you open a Python Shell in this directory later, you can import the extension module by typing import mymodule.

SWIG As A Custom Build Step

This section assumes that you have a single master .i file that (optionally) contains %include directives for header files or other .i files. We can define a Custom Build Step for this file that automatically runs swig.exe to re-generate the wrapper code whenever the file changes. If you have been using Flex/Bison with Visual C++ before, then the Custom Build Step is probably well-known to you. If not, here’s how it works:

Step 1. Add the .i file to the Visual C++ project. It usually appears just below the folders Source Files and Header Files.

Step 2. Right-click the .i file and click Settings. The Project Settings dialog opens.

Step 3. Select “All Configurations” and go to the Custom Build Step tab.

Step 4. In the Commands edit box, type

swig -c++ -python -o $(ProjDir)\$(InputName)_wrap.cpp $(InputPath)

Step 5. In the Outputs edit box, type

$(ProjDir)\$(InputName)_wrap.cpp

Step 6. Close the dialog and press F7 to build the project. The file xxxx_wrap.cpp should be created. Add this file to the project.

Step 7. Right click the file xxxx_wrap.cpp in the Project tree and click Settings.

Step 8. Go to the C/C++ tab, category Precompiled Headers.

Step 9. Disable the use of precompiled headers for the wrapper file (or for the entire project). Otherwise, the compiler will report “fatal error C1010 : Unexpected end of file while looking for precompiled header directive” when compiling the wrapper.

Step 10. Optional. The Custom Build Step ensures that the file xxxx_wrap.cpp gets recreated as soon as you change the master .i file. However, if the master .i file contains %include directives for other files, these files will also affect the resulting xxxx_wrap.cpp. By default, VC++ will only run SWIG on the master .i file if this very file has changed.

To force VC++ to run SWIG as well when one of the included files changes, you can add these files to the dependencies list of the master .i file:

1. Open the Custom Build Step for the master .i file again.
2. Click the Dependencies button and add the names of all (directly or indirectly) %included files. (For this purpose, #include directives within %{ }% can be ignored.)

Troubleshooting

Q. The Custom Build Step for the .i file fails.

A. VC++ might not be able to find the file swig.exe. See section “Setting Up The Environment” for information on how this can be solved.

Q. SWIG cannot find one of the files that I specified in an %include directive.

A. You can pass “-I” arguments to swig.exe in the Custom Build Step to specify additional include directories. For example, the following command adds the parent directory to SWIG’s search path:

swig -python -c++ -I.. -o $(ProjDir)\$(InputName)_wrap.cpp $(InputName)

For more information on SWIG’s search path, please see the SWIG docs.

Q. The compilation fails with “fatal error C1010 : Unexpected end of file while looking for precompiled header directive.”

A. Make sure to disable the use of the precompiled header for the SWIG-generated file xxxx_wrap.cpp.

Q. The library file pythonxy_d.lib cannot be found.

A. This file is only used for Debug builds and is not included in the Python binary distribution. You must download the Python sources and build the LIB and the corresponding DLL yourself.

But be careful! When you link your extension module with the debug version of the Python interpreter, you can only import the module when python.exe is also a debug version. Otherwise, you will receive the error “Fatal Python error: Interpreter not initialized (version mismatch?).”

Q. Python reports “ImportError: No module named _mymodule” when I try to import my extension module.

A. The DLL file must be in the same directory as the SWIG-generated file mymodule.py. Both files must be accessible through Python’s module search path. The name of the DLL must be prefixed with an underscore. See section “Setting Up The DLL Project” for detailed instructions.

Q. “ImportError: dynamic module does not define init function (init_mymodule)”

A. You probably forgot to add the SWIG-generated file xxxx_wrap.cpp to the project.

Q. “Fatal Python error: Interpreter not initialized (version mismatch?)”

A. This error occurs when the version of the Python interpreter for which the extension module has been built is different from the version of the interpreter that attempts to import the module.

The header file python.h contains a #pragma directive that automatically links your DLL with the import libary for the Python interpreter DLL. If you have installed Python version 2.3, the LIB file is python23.lib and the required DLL is python23.dll.

When the interpreter that performs the import is, say, version 2.2, you receive this error. You also receive the error when you attempt to import a Debug version of the extension DLL into a Release build of the interpreter.

A Case Study: “pymfg”

“pymfg” is the Python interface to the mfGraph library. mfGraph is written in C++ and allows C++ programmers to manipulate graphs stored in the GraphViz DOT format in memory. The library contains a Flex/Bison-generated parser for the DOT language.

While it would have been perfectly possible to write the entire library in Python, there were good reasons not to do so. It boils down to this: A C++ library can be used from C++ and Python; a Python library cannot be (directly) used from C++.

Nevertheless, many tasks can be achieved more easily using Python than C++. For example, a few hours after I had written the “pymfg” extension module, I was able to display graphs on screen. The same thing had taken me a few days in a C++ Win32 application! (This is mostly due to the excellent wxPython GUI toolkit.)

In the following sections, I am going to explore some of the issues I encountered when putting together “pymfg”. Two features of the mfGraph library caused a few troubles at first–the heavy use of STL containers and the way I’m using C++ namespaces. But all of the problems could be overcome thanks to SWIG’s incredible flexibility. Here’s the story.

Namespaces And Other Reasons Why Your Classes Aren’t Wrapped

Let me first show you a simplified class from the mfGraph library that encapsulates a graph. As you can see in the following listing, the method GetTopLevel returns a subgraph. You can guess that the declaration of the subgraph comes from the header file mfg_subgraph.h. You might also guess that Subgraph, like Graph, is part of the mfg namespace. But that’s a wild guess.

    #ifndef _MFG_GRAPH_H_
    #define _MFG_GRAPH_H_

    #include "mfg_subgraph.h"

    namespace mfg
    {
        class Graph
        {
            ...
        public:
            Subgraph* GetTopLevel();
            ...
        }
    }

    #endif

When SWIG encounters the declaration of class Graph, it has no previous knowledge of class Subgraph. SWIG just sees that GetTopLevel returns “something called Subgraph.” That something could be any class, struct, or even a typedef. And on it parses…

Note: Be aware that SWIG does not follow nested include directives, such as the #include "mfg_subgraph" in the previous example. There is some way to change this behavior, but then SWIG will also parse the system headers, which might not be what you want.

In a different header file, SWIG will encounter the declaration of class Subgraph, as shown in the following listing.

    #ifndef _MFG_SUBGRAPH_H_
    #define _MFG_SUBGRAPH_H_

    #include "mfg_node_list.h"

    namespace mfg
    {
        class Subgraph
        {
            ...
            const NodeList& GetNodeList() const;
            ...
        };
    }

    #endif

To SWIG, the class that is declared here, mfg::Subgraph is completely unrelated to class Subgraph, and rightly so, from what SWIG knows. The net effect of this that the following Python code will not work:

    >>> import pymfg
    >>> g = pymfg.Graph()
    >>> subg = g.GetTopLevel()
    >>> nl = subg.GetNodeList()
    Traceback (most recent call last):
      File "<interactive input>", line 1, in ?
    AttributeError: 'str' object has no attribute 'GetNodeList'

Why is this? When you print subg, it reads something like ‘_6e646972_p_Subgraph’, which clearly is a string. SWIG never encountered the declaration of a class named “Subgraph,” which it thinks is the return value of g.GetTopLevel(), and did not generate any wrapper code for it. The solution is to make sure that SWIG sees the declaration of mfg::Subgraph prior to the declaration of mfg::Graph, like so:

    %module pymfg

    %include "mfg_subgraph.h"
    %include "mfg_graph.h"

The order of include directives is not something you would really like to take care of. So there are different solutions. You can either insert a forward declaration of Subgraph to the file mfg_graph.h:

    #ifndef _MFG_GRAPH_H_
    #define _MFG_GRAPH_H_

    #include "mfg_subgraph.h"

    namespace mfg
    {
        class Subgraph;

        class Graph
        {
            ...

Note: I assume that the #include is still necessary for some other reason, otherwise we should have used a forward declaration in the first place.

The second alternative is to prefix all argument and return types with their respective namespace explicitly, even though this is not necessary in C++:

    #ifndef _MFG_GRAPH_H_
    #define _MFG_GRAPH_H_

    #include "mfg_subgraph.h"

    namespace mfg
    {
        class Graph
        {
            ...
        public:
            mfg::Subgraph* GetTopLevel();
            ...

Necessary modifications like this are much easier to apply when you have separate .i files for all your classes. These files should duplicate the C++ declarations from their corresponding headers and not include the header files directly.

While this sounds like a lot of unnecessary work, it in fact increases maintainability when compared to one huge .i file that is cluttered with %ignore, %extend, and other such directives.

Hiding The Unwanted And Faking The Non-Existent

A.k.a the %ignore and %extend directives. As a simple example for what this heading refers to, here’s an excerpt from the header of the DrawGraph class:

    class DrawGraph :   public Graph
    {
        ...
    public:
        /**
            @brief Write the contents of the graph into a text
                stream in DOT format.
         */
        void PrintAsDot(
            std::ostream& f,
            bool suppressImplicitAttribs = false) const;

        /**
            @brief Clear the current graph contents and load the
                graph from the DOT listing.
         */
        bool LoadFromXdot(std::istream& f);
        ...
    };

Reading and writing DOT files is something I also want to have in Python. But how am I supposed to construct a std::ostream or std::istream object in Python? I definitely don’t want to export these classes when Python has nice built-in string and file object that I can use instead of ifstream, ofstream, and stringstream.

The key is to remove the declarations of these two methods from the eyes of SWIG, and to add another two methods that operate on, say, string objects. Let’s assume we have a .i file that duplicates the contents of the DrawGraph header file. Removing a method is easy (just don’t copy it into the .i file). Adding a function is as easy:

    class DrawGraph :   public mfg::Graph
    {
        %extend {
            std::string PrintAsDot(
                bool suppressImplicitAttribs = false) const
            {
                std::stringstream s;
                self->PrintAsDot(s, suppressImplicitAttribs);
                    // SWIG generates this method as a stand-alone
                    // function. self serves as our this pointer
                    // to pretend we're in a class.
                return s.str();
            }

            bool LoadFromXdot(std::string src)
            {
                std::stringstream s(src);
                return self->LoadFromXdot(s);
            }

        };
    };

As you will see in the section on STL classes, SWIG allows Python to pass objects of the built-in type str in place of std::string and automatically converts return values of type std::string to type str. It can hardly get any more convenient!

    %ignore mfg::DrawGraph::PrintAsDot;
    %ignore mfg::DrawGraph::LoadFromXdot;
    %include "mfg_draw_graph.h"
    %extend mfg::DrawGraph {
        ... same code as above
    };

Someone Has Done All The Work – Using STL Classes

In the previous example, we exported a method that used a std::string in its argument list. Of course, we want a Python programmer to be able to pass Python strings to this extension method. This can be done easily by including the file std_string.i, like so:

    %include "std_string.i"
    class DrawGraph :   public mfg::Graph
    {
        %extend {
            std::string PrintAsDot(
                bool suppressImplicitAttribs = false) const
            {
                ...

std_string.i is located in the SWIG directory. There is a separate implementation for the different target languages that SWIG supports. SWIG selects the correct implementation depending on the language that you specify in the command-line for swig.exe.

For Python, the following headers exist:

• std_complex.i
• std_deque.i
• std_list.i
• std_map.i
• std_string.i
• std_vector.i

In Python, a std::map can be used like a dict, a std::string like a str, a std::vector like a list, etc.

The following listing shows how I used the std_map.i and std_string.i files to declare a class that derives from std::map<std::string, std::string>:

    %include "std_map.i"
    %include "std_string.i"

    namespace std
    {
        %template(stdAttribList) map<std::string, std::string>;
    }

    namespace mfg
    {

        class AttribList    :   public std::map<std::string, std::string>
        {
        public:
            AttribList();

            virtual ~AttribList();

            bool Exists(std::string name) const;

            void Remove(std::string name);

            void Append(const AttribList& rhs);

        };

    }   // namespace mfg

The %template directive is required. SWIG does not generate a wrapper when it encounters the base class specification. The class stdAttribList behaves like the built-in type dict in most respects. AttribList inherits this behavior. For example, you can write:

    >>> al = pymfg.AttribList()
    >>> print al.keys()
    >>> al["color"] = "red"
    >>> for a in al:
    >>>     print a

The Problem With Pointers To Pointers

I did encounter some problems with the STL containers though. Most notably, SWIG refused to generate valid code for a list of pointers. For example, the following snippet resulted in wrapper code that didn’t compile (using SWIG 3.1.19):

    %include "std_list.i"
    namespace std
    {
        %template(stdNodeList)  list<mfg::Node*>;
    }
    namespace mfg;
    {
        class NodeList  :   public std::list<mfg::Node*>
        {
            ...
        };
    }

The problem could be solved by declaring a typedef for mfg::Node*.

    %{
    typedef mfg::Node* PNode;   // copied to wrapper code
    %}
    %include "std_list.i"
    namespace std
    {
        %template(stdNodeList)  list<PNode>;
    }
    namespace mfg;
    {
        class NodeList  :   public std::list<PNode>
        {
            ...
        };
    }

This compiles and yields the correct results, in as far as SWIG knows that PNode is an alias for mfg::Node*. However, there is yet another problem. When you iterate through the elements of the list, for example by using a for loop, you actually get pointers to the contained objects. This is fine for lists of objects, because SWIG treats objects and pointers the same. But a pointer to a pointer is not the same as a pointer. What you get when printing the elements is this:

'_980e9201_p_p_mfg__Node'

What we want is this:

<C Node instance at _b00e9201_p_mfg__Node>

In C++, you could simply dereference the pointer and be happy. In Python, this is not possible. What I did was to write a function for dereferencing various pointers directly in the .i file. Various overloads of this function can be declared in various places. SWIG knows how to handle this.

    %inline %{
    mfg::Node* deref(mfg::Node** x) {
       return *x;
    }
    %}

In Python, the for loop looks a bit ugly now, but it does the job:

    nl = g.GetNodeList()
    for pn in nl:
        n = pymfg.deref(pn)
        # n contains a Node

Conclusion

C++ and Python are both great programming languages, each having its own set of benefits and shortcomings. One of the strengths of Python is its ease of use. Though, due to its interpreted nature, Python code will never be able to match the runtime performance of C++. Most of the time, this does not matter. Yet sometimes it does…

This is just one of the situations when you might wish to make a C++ library available to your Python applications. SWIG is an amazing tool that makes this a trivial task. SWIG will give you excellent results in a matter of hours, but there’s also a wealth of advanced features that this article hasn’t even touched upon.