Try-except versus if-else in Python 3.5


Today I ran again in the question why to use if-else when try-except is shorter?

There is a semantic part that we loose information on a try-except as we don’t know what the developer did expect in the most case and there is a performance part. I found a test for python 2 (Try / Except Performance in Python: A Simple Test) with simple assumptions: we try to get a value in a one keyed dictionary and make iterations to get a statistic on this access. I made it a little more complex with a dictionary with 10,001 keys.

Results analysis:

  • if-else take the same amount of time in both case
  • try-expect is 2 times faster than if-else when it no raise the exception
  • try-expect is 5 times slower when the exception is raised

Then the if-else has a predictable comportment and help the maintainer.

I’m agree that the semantic part of the if-else can be replace by a comment but i still not recommend to try-except if it is not to raise an exception.

 

The results:

The case where the key does not exist: 
 
1,000 iterations: 
with_try (0.250 ms) 
with_try_exc (0.291 ms) 
without_try (0.119 ms) 
without_try_not (0.120 ms) 
 
1,000,000 iterations: 
with_try (231.647 ms) 
with_try_exc (263.633 ms) 
without_try (119.238 ms) 
without_try_not (118.602 ms) 
 
1,000,000,000 iterations: 
with_try (224659.381 ms) 
with_try_exc (260333.897 ms) 
without_try (109796.531 ms) 
without_try_not (111871.690 ms) 
 
 
The case where the key does exist: 
 
1,000 iterations: 
exists_with_try (0.066 ms) 
exists_with_try_exc (0.070 ms) 
exists_without_try (0.166 ms) 
exists_without_try_not (0.180 ms) 
 
1,000,000 iterations: 
exists_with_try (57.661 ms) 
exists_with_try_exc (56.909 ms) 
exists_without_try (113.633 ms) 
exists_without_try_not (116.340 ms) 
 
1,000,000,000 iterations: 
exists_with_try (57650.440 ms) 
exists_with_try_exc (57395.376 ms) 
exists_without_try (114659.023 ms) 
exists_without_try_not (117646.034 ms)

 

The code:

#!/usr/bin/env python3

import time

d = dict.fromkeys(range(0, 10000), 0)
d['somekey'] = 1

ITERATIONS = (1000, 1000*1000, 1000*1000*1000)

def time_me(function):
    def wrap(*arg):
        start = time.time()
        r = function(*arg)
        end = time.time()
        print("%s (%0.3f ms)" % (function.__name__, (end-start)*1000))
        return r
    return wrap


# Not Existing
@time_me
def with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except:
            pass

@time_me
def with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except Exception:
            pass

@time_me
def without_try(iterations):
    for i in range(0, iterations):
        if d.get('notexist'):
            pass
        else:
            pass

@time_me
def without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('notexist'):
            pass
        else:
            pass



# Existing
@time_me
def exists_with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except:
            pass

@time_me
def exists_with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except Exception:
            pass

@time_me
def exists_without_try(iterations):
    for i in range(0, iterations):
        if d.get('somekey'):
            pass
        else:
            pass

@time_me
def exists_without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('somekey'):
            pass
        else:
            pass


print ("\n\nThe case where the key does not exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    with_try(iteration)
    with_try_exc(iteration)
    without_try(iteration)
    without_try_not(iteration)


print ("\n\nThe case where the key does exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    exists_with_try(iteration)
    exists_with_try_exc(iteration)
    exists_without_try(iteration)
    exists_without_try_not(iteration)

Publicités

When UnicodeDecodeError become irrational check $LANG


I spent hours this week trying to understand how an installation script can fail on some installations.

In input we have an utf-8 encoded file and we add some xml files, also ‘utf-8’ encoded. These are parsed with Markdown.

python -m lom2mlr.markdown -l -c rationale.md

It is really simple but sometimes we ran into a strange error:

Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/edegoute/Projects/lom2mlr/lom2mlr/lom2mlr/markdown/__main__.py", line 3, in <module>
compile()
File "lom2mlr/markdown/__init__.py", line 55, in compile
extensions=extensions)
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 529, in markdownFromFile
kwargs.get('encoding', None))
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 441, in convertFile
html = self.convert(text)
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 375, in convert
newRoot = treeprocessor.run(root)
File "lom2mlr/markdown/test_mlr.py", line 76, in run
print(" " * int(element.tag[1]) + element.text)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xc9' in position 4: ordinal not in range(128)

First, it was difficult to understand how an unicode error can raise an iso-8859-1 problem on utf-8 files. Going deep I found some known problems with ‘codecs.open’ in python2.7 but no solution. I tried to force Markown to treat these files as ‘iso-8859-1’ files, then it ran an utf-8 unicode error at the same line not in the opening. It was sounding too many magic for me.

At that point, I checked again the installation was identical: same python version, same pip version, same eggs versions. I tried some egg upgrade without any success. And finally came the idea to check environment variables. Bingo! On all systems with failing installations we have no localization ($LANG=C). The fix was so simple:

export LANG=en_US.UTF-8

That’s it!

I still don’t understand the magic in the codecs python module. Why it computes a different encoding when the function call already asks for one? The workaround is simple for programmers.

Comment migrer depuis un vieille version de Plone vers la dernière version


La première chose à faire lors de la migration d’un CMS est de regarder si la documentation contient le nécessaire à connaître pour se lancer. La documentation de Plone est très bien fournie avec des conseils pour chaque version majeure depuis la 1.0 :

http://plone.org/documentation/manual/upgrade-guide

 

Il est indiqué quels sont les produits impactés et quels sont les modifications à apporter dans le code tierce.

Il y a deux conseils qui manquent souvent pour le développeur qui manque d’habitude :

  1. Il faut passer par toutes les versions majeures
  2. Dans une même version majeure il faut faire la migration depuis la première version mineure vers la dernière version mineure avant de migrer à la prochaine version majeure id est 3.3 vers 3.3.6 avant de faire 3.3.6 vers 4.0

En suivant ces deux règles supplémentaires il est possible de passer de Plone 2.1 à Plone 4.3 et bientôt

English: The logo of Plone, and Open Source Co...

English: The logo of Plone, and Open Source Content Management System (Photo credit: Wikipedia)

Plone 5 sans avoir trop de surprises.

 

 

 

 

How to setup your Plone core sprint


I promised to published this article last year but… well, no excuse. The topic is still hot as we work on in Arnhem in November 2013, 11 to 15 to finish it for Plone 5.

Here in Arnhem – Ploneconf 2012 Sprints – we are making a lot of sprint about a lot of different topics. As a newcomer in  sprint lead I did some mistakes.

  1. If you are working in the Plone core you must should a plip before the beginning of the sprint. It will be easier for other people  to follow your work some months later.
  2. For commiters, the main point is to check if you already give your contributor agreement back to the Plone Fundation. If you’re not there should be someone with blank agreements in your sprint to make things go easy.
  3. Once you know how many sprinters you have on your topic you must organize the area to be comfortable for all.
  4. Fing a way to show everybody the tasks that are on going and who is working on. My way is a whiteboard with post-its
  5. Prepare the technical parts before to start coding:
    • Identify all the packages you’re working on and create a remote branch for your plip. We were working at first on Products.CMFPlone, plone.app.content, plone.app.users, and after we also include plone.app.controlpanel in the list.
    • Don’t make a branch in buildout.coredev create a plips configuration file in the ‘plips‘ folder.Then run your buildout using this configuration file as master: bin/buildout -c plips/plip13260-cpy-removal.cfg
  6. Check if everybody have commit rights.
  7. If there’s a change in the plip configuration file notice it to everyone fast.
  8. Make regular internal reports to know who needs help or work 🙂
  9. Write down what’s you’re doing on

For commiters, the main point is to check if you already give your contributor agreement back to the Plone Fundation.  If you’re not there should be someone with blank agreements in your sprint to make things go easy.
Commiters should also follow a process to make the pull going right:

  1. Have  fun
  2. After each plip configuration file update check if all plip eggs are well checkouted and if they are in the good branch
  3. When you start a task  create a local branch to avoiding conflicts before your merge
  4. Commit early, often
  5. Fetch and pull others commits before to merge
  6. Run tests without ‘-t
  7. Merge your work in the plip branch, not the master
  8. Test again before the push
  9. If you need help asks (I know, git is overcomplicated 😉 )
  10. Have fun! Did I say it before?
  11. Have  a follow up plan and stay in touch

Thanks to Eric Steele for the first setup, Liz as the proposer of the sprint topic, Maurits as the technical leader and for all people around there that were working on that sprint.

How to use curl to test a REST API with a beautiful JSON indentation


How to get indented JSON with curl output

You need to install simplejson Python module in your environment, or python-simplejson.

curl [options] url | python -msimplejson.tool

An example with github APIv3

curl -s -H "Content-Type: application/json" -X GET \
     "https://api.github.com/repos/collective/collective.ckeditor" \
     | python -msimplejson.tool

How to display response headers and API errors with the JSON indentation

This command line have two problems :

  1. You cannot display response HTTP headers because it breaks simplejson parsing
  2. If you have an error with a request and you don’t know how to debug it.

You need to add two more options to push header response and error message in stderr: -S and -D /dev/stderr. Now you can test your API with all informations in one command:

curl -s -S -D /dev/stderr -u user:password \
     -H "Content-Type: application/json" -X GET \
     "http://myapi/v1/test/"

From curl manual:

  • -s no progress bar; stats report or errors are dislayed
  • -S display errors if -s is enable
  • -D dump the header in a file; here stderr to make them not interfer with json indentation
  • -u user and password for Basic HTTP Authentication
  • -H add header in the outgoing request
  • -X HTTP Command to launch. If you’re using Basic HTTP Authentication you must use -X GET

If someone have tip to add pygment coloration in json and headers I will be glad to share the ownership of this article.

Feedbacks on Diazo after the PloneConf training


There were 9 trainees in this PloneConf Diazo session.  Half of them were brand new on Diazo and the second half already used it for at  least one theme.

  • Every find Diazo 1.0 cool with its new styles editor.
  • If you want to be comfortable with all the editor’s features you should use a 30-inch display (especially with the introsception panel).
  • To be able to import a Zip file from any free CSS templates site is awesome with a bemol: the Zip file must have a directory at first level with all content inside if you don’t have a weird error message displayed.
  • Shortcuts in the editor are working everywhere but are not documented; may we need a side panel with general documentation.
  • In the inspector panel, it’s not easy to find the XPath expression and   less to create a generic one without a live example.
  • There is no ‘move‘ command. It’s available since deliverance 0.3 and Diazo forked at 0.2.

During the DIY part of the training some were asking how to make a theme with Bootstrap.js and Diazo.  Deco’s toolbar theme is build with Bootstrap.js but it seems not to be doable from scratch only with the Diazo editor. Everyone finish his own theme in 2 hours for beginners and in 30 minutes  for others.

At least, there was one trainee that fell to complicated to write the rules.xml file with all those XSLT tranforms.

I let the trainees add their feeling about it if I forgot something.

Enhanced by Zemanta

Participate to a Diazo training in Arnhem the October 9


The day before the conferences I will provide a training on Diazo.

You can come with your own HTML design and in one day we will see how to create a zip file ready to be inserted in your plone site.

You don’t need to know anything about Plone, just HTML and CSS. A bit of XML understanding would be useful but not required.

Enhanced by Zemanta

Comment installer les Webfonts localement


Les webfonts sont de plus en plus utilisées dans les sites mais manquent souvent à l’appel lorsqu’il s’agit de préparer le travail avec Gimp ou d’écrire un document avec LibreOffice. Google propose une méthode d’installation qui n’est pas complète sur la page de  GoogleWebFonts. Nous allons faire en sorte qu’elles soient disponibles pour toutes les applications installées.

Prérequis

Il faut compter 1Go d’espace disque disponible pour l’installation et il faut installer le logiciel mercurial :

yum install -y mercurial

ou

apt-get install -y mercurial

Installation

mkdir -p ~/.fonts/truetype/google-fonts/
hg clone https://googlefontdirectory.googlecode.com/hg/ ~/.fonts/truetype/google-fonts

Après beaucoup d’attente un message comme celui-ci dessous s’affiche :

 6208 files updated, 0 files merged, 0 files removed, 0 files unresolved

Il est possible de mettre à jour le cache des fontes si vous êtes super-utilisateur :

sudo fc-cache -f -v

Mise-à-jour automatique

La mise-à-jour est assez simple :

hg pull -u ~/.fonts/truetype/google-fonts

Pour rendre le processus hebdomadaire il faut utiliser le système de cron en ajoutant une ligne après celles déjà existante :

crontab -l | (cat; echo "@weekly hg pull -u ~/.fonts/truetype/google-fonts") | crontab -

Pour vérifier que tout a fonctionné il suffit alors d’afficher la crontab avec la commande suivante :

crontab -l

Conclusion

Maintenant vous pouvez profiter des webfonts mais pas les personnes aves qui vous utilisez votre machine. Pour ce faire il faut avoir les droits super-utilisateur et utiliser /usr/share/fonts/truetype/google-fonts à la place de ~/.fonts/truetype/google-fonts.

Un nouveau salon de discussion francophone pour discuter autour de Python


C’est tout neuf : la communauté Python francophone a un nouveau point de chute pour discuter de tout et de rien.

http://www.reddit.com/r/PythonFr/

N’hésitez pas à vous inscrire !

A Renewed ReStructuredText Editor!


This article’s goal is to be able to launch the Enthought RST Editor with Python 2.7. Few months ago Enthought bloged about is RST editor embedded in its suite. Their blog article was useful for 4 months then they refactored how the Qt backend is loaded.  I wrote another article (in french) to show how to use the editor from the Ubuntu Python system packages but it doesn’t work anymore since the upgrade to Ubuntu 11.10 (Oneiric) which use Traits 4.0.0. Now Enthought sources are hosted on github so you can send them your patches.

First it’s better to install all this in a virtualenv with some dependencies (you must install virtualenv and virtualenvwrapper before):

sudo apt-get remove libopenblas-base #problem with numpy
sudo apt-get install python-traits python-traitsui python-pyface python-enthoughtbase \
     python-apptools python-numpy python-qt4 python-sip python-configobj python-sphinx \
     python-pygments rst2pdf
mkvirtualenv --distribute rested
cdvirtualenv
git clone https://github.com/enthought/rested.git
easy_install rested
sudo cp bin/rested /usr/local/bin/

I still have problem with encoding in the filesystem navigator and some random crashes.