Try-except versus if-else in Python 3.5


Today I ran again in the question why to use if-else when try-except is shorter?

There is a semantic part that we loose information on a try-except as we don’t know what the developer did expect in the most case and there is a performance part. I found a test for python 2 (Try / Except Performance in Python: A Simple Test) with simple assumptions: we try to get a value in a one keyed dictionary and make iterations to get a statistic on this access. I made it a little more complex with a dictionary with 10,001 keys.

Results analysis:

  • if-else take the same amount of time in both case
  • try-expect is 2 times faster than if-else when it no raise the exception
  • try-expect is 5 times slower when the exception is raised

Then the if-else has a predictable comportment and help the maintainer.

I’m agree that the semantic part of the if-else can be replace by a comment but i still not recommend to try-except if it is not to raise an exception.

 

The results:

The case where the key does not exist: 
 
1,000 iterations: 
with_try (0.250 ms) 
with_try_exc (0.291 ms) 
without_try (0.119 ms) 
without_try_not (0.120 ms) 
 
1,000,000 iterations: 
with_try (231.647 ms) 
with_try_exc (263.633 ms) 
without_try (119.238 ms) 
without_try_not (118.602 ms) 
 
1,000,000,000 iterations: 
with_try (224659.381 ms) 
with_try_exc (260333.897 ms) 
without_try (109796.531 ms) 
without_try_not (111871.690 ms) 
 
 
The case where the key does exist: 
 
1,000 iterations: 
exists_with_try (0.066 ms) 
exists_with_try_exc (0.070 ms) 
exists_without_try (0.166 ms) 
exists_without_try_not (0.180 ms) 
 
1,000,000 iterations: 
exists_with_try (57.661 ms) 
exists_with_try_exc (56.909 ms) 
exists_without_try (113.633 ms) 
exists_without_try_not (116.340 ms) 
 
1,000,000,000 iterations: 
exists_with_try (57650.440 ms) 
exists_with_try_exc (57395.376 ms) 
exists_without_try (114659.023 ms) 
exists_without_try_not (117646.034 ms)

 

The code:

#!/usr/bin/env python3

import time

d = dict.fromkeys(range(0, 10000), 0)
d['somekey'] = 1

ITERATIONS = (1000, 1000*1000, 1000*1000*1000)

def time_me(function):
    def wrap(*arg):
        start = time.time()
        r = function(*arg)
        end = time.time()
        print("%s (%0.3f ms)" % (function.__name__, (end-start)*1000))
        return r
    return wrap


# Not Existing
@time_me
def with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except:
            pass

@time_me
def with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except Exception:
            pass

@time_me
def without_try(iterations):
    for i in range(0, iterations):
        if d.get('notexist'):
            pass
        else:
            pass

@time_me
def without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('notexist'):
            pass
        else:
            pass



# Existing
@time_me
def exists_with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except:
            pass

@time_me
def exists_with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except Exception:
            pass

@time_me
def exists_without_try(iterations):
    for i in range(0, iterations):
        if d.get('somekey'):
            pass
        else:
            pass

@time_me
def exists_without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('somekey'):
            pass
        else:
            pass


print ("\n\nThe case where the key does not exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    with_try(iteration)
    with_try_exc(iteration)
    without_try(iteration)
    without_try_not(iteration)


print ("\n\nThe case where the key does exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    exists_with_try(iteration)
    exists_with_try_exc(iteration)
    exists_without_try(iteration)
    exists_without_try_not(iteration)

Manage monkey patches and performance improvement in Plone 3


Few days ago Jean-Michel François proposed a useful patch for PlonePAS that can be applied for all Plone release until 3.2. Plone 3.3 will embed this patch.

How can we add this patch in a traceable way for an not so old Plone 3.1 or 3.2 ?

First, we can use the new release  of Products.PlonePAS that should be compatible with our Plone installation. The second option is to add a monkey patch in the policy product of our site. One more monkey patch…

Some projects have so many monkey patches that it is difficult to know where is the code that run your site. Martin Aspeli did a tool to handle monkey patches in an elegant way for Zope 2 and Zope 3: collective.monkeypatcher. It allows you to plug  your monkey patches with a simple ZCML directive. Later Gilles Lenfant added a control panel for Zope 2 to be able to have a visual way to follow patches with collective.monkeypatcherpanel.

How does it works?

In your buildout.cfg add :

eggs +=
   collective.monkeypatcher
   collective.monkeypatcherpanel
zcml +=
   collective.monkeypatcher
   collective.monkeypatcherpanel

To create patches add a ‘patches.py‘ file in your egg (if you have more than 2 or 3 patches you should create a directory). Our performance patch looks like this:

import copy

def enumerateUsers( self
                  , id=None
                  , login=None
                  , exact_match=False
                  , **kw
                  ):

    """ See IUserEnumerationPlugin.
    """
    plugin_id = self.getId()

    criteria=copy.copy(kw)
    if id is not None:
        criteria["id"]=id
    if login is not None:
        criteria["login"]=login

    if not kw and id:
        data = self._storage.get(id, None)
        if data is None:
            user_info = []
        else:
            user_info=[ { 'id' : self.prefix + id,
                     'login' : id,
                     'title' : data.get('fullname', id),
                     'description' : data.get('fullname', id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } ]
    else:
        users=[ (user,data) for (user,data) in self._storage.items()
                    if self.testMemberData(data, criteria, exact_match)]

         user_info=[ { 'id' : self.prefix + user_id,
                     'login' : user_id,
                     'title' : data.get('fullname', user_id),
                     'description' : data.get('fullname', user_id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } for (user_id, data) in users ]

        return tuple(user_info)

In the configure.zcml of your policy product add an include:

<include file="patches.zcml" />

The file patches.zcml will contain following code:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:monkey="http://namespaces.plone.org/monkey"
    i18n_domain="collective.monkeypatcher">

    <include package="collective.monkeypatcher" file="meta.zcml" />

    <monkey:patch
       original="enumerateUsers"
       replacement=".patches.enumerateUsers"
       docstringWarning="false"
    />

</configure>

Run your buildout, start your site and the patch is applied. You can go in the Zope Control Panel to see how many patches are already compatible with this tool.