A Quest for Optimization

Sebastian and Kyle were enthusiastic about the opportunity to attend PyCon 2008, which was held in Chicago. A volunteer-run Python community conference, this was the sixth annual PyCon and undoubtedly the biggest and most successful yet. Hundreds of developers and hobbyists from all over the globe gathered to learn of new developments in the Python world. The 10-day conference, which was divided into tutorials, talks, impromptu discussions and development sprints, offered many attractive innovations.

Kyle, Adrian, Jacob, and Sebastian - Fellow Djangonauts
Kyle, Adrian, Jacob, and Sebastian - Fellow Djangonauts

The tutorials that offered attendees a chance to sit down with Jacob Kaplan-Moss, a lead developer of the Django framework, was of particular interest to Sebastian and Kyle. To take advantage of this opportunity to have their work analyzed by Jacob, the coding team devised a simple project illustrating some of the techniques used in current CLW applications. This provided multiple implementations of a database-intensive task in which the average running time of each implementation was taken to ascertain which was fastest. Ultimately their aim was to locate problem areas and identify new methodologies that might optimize performance. The experience of showing this project at the tutorial bore fruit, in that Sebastian and Kyle were persuaded to adopt a more Django-like mindframe to problem resolution, and to use a distributed memory object caching system called memcached to alleviate our database loads. Furthermore, by using the re-usable application paradigms learned throughout the course of the conference we hope to increase our software base and reduce the time we require to develop new projects.

Sebastian and Jacob
Sebastian and Jacob

Although optimizing our web applications was our primary goal in sending representatives to PyCon, Sebastian and Kyle's focus included improving other programming elements such as the style, structure, and general best practice of our applications. Their having acquired more information in greater depth about advanced modules such as metaclasses, decorators and frames, should make CLW's programming endeavours even more efficient and well-designed in the future.

Below are a couple of examples of Python code that Sebastian and Kyle found particularly interesting. These cover two of the aforementioned modules already mentioned, decorators and metaclasses.

Decorators

Here we use a callable class object that acts as a decorator. The Template class' __call__ magic method is overridden to accomplish this, allowing the decorator to take a parameter before consuming the function.

class Template (object):
def __init__ (self, message):
self.message = message
def __call__ (self, func):
def newfunc (*args):
return self.message % func (*args)
return newfunc
@Template ( 'Welcome to %s' )
def func1 (company):
return company
print func1( 'Campbell-Lange Workshop' )

As expected the result would be:

>>> print func1( 'Campbell-Lange Workshop' )
>>> "Welcome to Campbell-Lange Workshop"

The above code snippet is thus equivalent to applying the decorator method in the following, perhaps less readable, manner:

func1 = Template('Welcome to %s')(func1)

Metaclasses

In Python classes are themselves objects. Since objects are produced using classes as templates, what is it that acts as a template for producing classes? The answer is metaclasses.

A good use case for custom metaclasses is when you need to create a class with attributes and methods that are dynamically generated. To gain a better understanding, we'll use an example. First, let's create a metaclass:

class __printer__ (type):
def __new__ (meta, classname, bases, classDict):
print  'Name   Type   Value'
for k, v in classDict.items():
print  '%-15s %-20s %r' % (k, type(v), v)
return type.__new__(meta, cls, bases, classDict)

Notice that it inherits from class type, as do all metaclasses. In the majority of cases, you should override the __new__ magic method to modify the elements of the classDict argument which has the effect of adding or modifying members of a class.

The metaclass class now gets invoked below when it's assigned to the magic attribute __metaclass__. This invocation allows us to customize the creation of the class Test by overriding the default behaviour of type():

class Test (object):
__metaclass__ = __printer__
a  = 22
b =  'foo'
def c (self): pass
d  = property(lambda self: None)

As expected the __printer__ metaclass prints the members of Test. Conceivably, we could just as easily use another metaclass to manipulate or modify its members.

>>>Name Type Value
>>>a <type 'int'> 22
>>>__module__ <type 'str'> '__main__'
>>>b <type 'str'> 'foo'
>>>__metaclass__ <type 'type'> <class '__main__.__test__' >
>>>d <type 'property'> <property object at 0x01A75850 >
>>>c <type 'function'> <function c at 0x01A17F30 >

All in all, presentations delivered by experts of the community, allowed our coding team to acquire more knowledge on a myriad subjects. These spanned information regarding toolkits and libraries enabling users to write faster client-side JavaScript; newly- implemented web frameworks such as Pylons and Turbogears; to writing Unix-derived command line tools and visually-rich graphical frameworks. These projects will unquestionably be used in CLW's research and development cycle, and may soon become integrated with the applications we currently deploy.

Kyle and Guido - Benevolent Dictators for Life
Kyle and Guido

CLW hopes to attend PyCon 2009 next year and participate in the development sprints in order to contribute to the Python community. With Python 3000 around the corner, we may even get a chance to write Python core modules! Overall, the experience has left our coding team with a new set of skills and the feeling that they are now part of a broader Python community.

Solutions

IT Operations

Company