Simple framework for python plugins

Plugins are often good to build a loosely coupled, modular software architecture. They also foster third-party contributions.

This article describes how I did it very simply in python, this framework is called spf (Simple plugin framework) and is available on github.

After reading a review on existing framworks, I tried yapsy. Unfortunately it’s python 2, and despite doing a couple of obvious changes in the code, I couldn’t make a working plugin, failing with error
module.__init__() takes at most 2 arguments (3 given)

After that, I decided to understand better how exec() works on __init__.py and what such a framework should do. I read the brilliant article A Simple Plugin Framework by Marty Alchin (who, I think, was a django contributor, and don’t know if he still is).

His design is so KISS, I love it. This article is just a up-to-date (there was python 2 to python 3 update to do) and simplified (without references to django) version of his « framework ».

The first thing is to define the notion of plugin.

  • a mount point is a place where extra features can be added
  • a plugin is a specific implementation that is added to a specific place

Let’s first define a mount point, as a metaclass that simply contains the list of plugins to apply there.

[code]
class MountPoint(type):
'''
* A way to declare a mount point for plugins. Since plugins are an example of loose coupling, there needs to be a neutral location, somewhere between the plugins and the code that uses them, that each side of the system can look at, without having to know the details of the other side.
* A way to register a plugin at a particular mount point. Since internal code don't want to look around to find plugins that might work for it, there needs to be a way for plugins to announce their presence. This allows the guts of the system to be blissfully ignorant of where the plugins come from; again, it only needs to care about the mount point.
* A way to retrieve the plugins that have been registered. Once the plugins have done their thing at the mount point, the rest of the system needs to be able to iterate over the installed plugins and use them according to its need.

Add the parameter `metaclass = MountPoint` in any class to make it a mont point.

'''

def __init__(cls, name, bases, attrs):
if not hasattr(cls, 'plugins'):
# This branch only executes when processing the mount point itself.
# So, since this is a new plugin type, not an implementation, this
# class shouldn't be registered as a plugin. Instead, it sets up a
# list where plugins can be registered later.
cls.plugins = []
else:
# This must be a plugin implementation, which should be registered.
# Simply appending it to the list is all that's needed to keep
# track of it later.
cls.plugins.append(cls)
[/code]

Now, let’s write a mount point. To have a very simple example, my mount point is to print text.
[code]
class TextTransformer(object, metaclass=MountPoint):
''' Plugins can inherit this mount point in order to modify text.

A plugin that registers this mount point must implement the method
* transform_text(self, string):
'''

def __init__(self, program):
pass
[/code]

As you can see, this is very simple. Because python doesn’t have interfaces, it is very important to write documentation.

For instance, I can have 3 transformers for HTML text
[code]
class HtmlTransformer(TextTransformer):
def __init__(self, program):
self.tag = None

#As documented in the Mount point, this must be implemented

def transform_text(self, string):
if self.tag:
return "< {tag}>{original}".format(tag=self.tag, original=string)
else:
return string

class HtmlEmTransformer(HtmlTransformer):
''' Plugin to wrap text in a html tag
'''

def __init__(self, program):
''' This TextTransformer will transform ``string`` into ``string``
'''
self.tag = "em"

class HtmlBoldTransformer(HtmlTransformer):
''' Plugin to wrap text in a html tag
'''

def __init__(self, program):
''' This TextTransformer will transform ``string`` into ``string``
'''
self.tag = "b"

[/code]

A main method that has a TextTransformer mount point
[code]
class MyProgram:
plugins = ExtensionsAt(TextTransformer)

def main(self):
# Here I declare a mount point TextTransformer
# "hello world" will be printed by each plugin

for plugin in self.plugins:
retval=plugin.transform_text("hello world")
print("Plugin {plugin} produces {retval}".format(plugin=plugin, retval=retval))

if __name__ == '__main__':
prog = MyProgram()
prog.main()
[/code]

As you can see, I have used a utility method to retrieve plugins at a given mount point
[code]
class ExtensionsAt(object):
''' Descriptor to get plugins on a given mount point.
'''

def __init__(self, mount_point):
''' Initialize the descriptor with the mount point wanted.
Eg: ExtensionsAt(apf.GUIMenu) to get extensions that change the GUI Menu.
'''
self.mount = mount_point

def __get__(self, instance, owner=None):
''' Plugin are instanciated with the object that is calling them.
'''
return [p(instance) for p in self.mount.plugins]
[/code]

And that’s it! Provided the plugin classes are loaded in memory, they will be magically executed.

/Library/Frameworks/Python.framework/Versions/3.2/bin/python3.2 /Users/regis/workspace/plugin/apf.py
Plugin HtmlTransformer produces hello world
Plugin HtmlEmTransformer produces hello world
Plugin HtmlBoldTransformer produces hello world

Process finished with exit code 0
  • Ken Boyd

    Thank you for the interesting article.  When I try to implement your solution by putting the « plugins » into a different module, I get a NameError.

    Did you intend for this to work with plugins being provided in independent modules, or do all plugins have to be added to the code for the main program?

    in Example.py I have all of the code in your examples except for the three class definitions of:
    class HtmlTransformer(TextTransformer):
    class HtmlEmTransformer(HtmlTransformer):
    class HtmlBoldTransformer(HtmlTransformer):

    in place of the above code I placed a
    import texttransformer_plugin

    in « texttransformer_plugin.py » I placed the three class definitions for HtmlTransformer, HtmlEmTransformer and HtmlBoldTransformer(HtmlTransformer):

    This is the error I receive:

    Traceback (most recent call last):
      File « C:/Users/Ken/Dropbox/Projects/Python/Plugin Example.py », line 43, in
        import texttransformer_plugin
      File « C:/Users/Ken/Dropbox/Projects/Pythontexttransformer_plugin.py », line 2, in
        class HtmlTransformer(TextTransformer):
    NameError: name ‘TextTransformer’ is not defined

    When I have all the code in a single module « Plugin Example.’y » I receive the results as expected:

    Plugin produces hello world
    Plugin produces hello world
    Plugin produces hello world

    Is there a way to place the « plugin » code in to separate modules?

    Thank you for your execllent blog post.

    Ken Boyd

    partager sur...