Monday, October 22, 2012

Splitting Python Packages Into Multiple Projects

I recently had to split up a python package into multiple separate projects while still keeping the topmost namespace the same. I.e. I had to create "namespace packages". So, a project that was initially setup like this

mypackage/
    __init__.py
    module1/
        __init__.py
        foo.py
        bar.py
    module2.py
    module3/
        __init__.py
        bar.py
        baz.py

had to be split up into something like
project1/
    mypackage/
        __init__.py
        module1/
            __init__.py
            foo.py
            bar.py
        module2.py

project2/
    mypackage/
        __init__.py
        module3/
            __init__.py
            bar.py
            baz.py
Although I was able to do this rather simply following the suggestions found here, I put together some example files so that you can quickly see how it works. Basically, the one thing that needs to be done is that in project2/mypackage/__init__.py needs to contain
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
Once this is done you can now distribute each of the projects separately while still making use of all modules and classes. For example, if we are in the directory that contains project1 and project2, we can run something like the following:
# Add the two projects to the path
import sys
sys.path.append('project1')
sys.path.append('project2')

# Import the modules from the two packages
import mypackage.module1
import mypackage.module2
import mypackage.module3

if __name__ == '__main__':
        # Create some objects from the first package
        p1_foo = myproject.module1.foo.Foo()
        p1_bar = myproject.module1.bar.Bar()
        p1_SomeClass = myproject.module2.SomeClass()

        # Create some objects from the second package
        p2_bar = myproject.module3.bar.Bar()
        p2_baz = myproject.module3.baz.Baz()

        # Show that they all work
        p1_foo.run_me()
        p1_bar.run_me()
        p1_SomeClass.run_me()

        p2_bar.run_me()
        p2_baz.run_me()
You can download all of the above code from github.

No comments:

Post a Comment