Try-except versus if-else in Python 3.5


Today I ran again in the question why to use if-else when try-except is shorter?

There is a semantic part that we loose information on a try-except as we don’t know what the developer did expect in the most case and there is a performance part. I found a test for python 2 (Try / Except Performance in Python: A Simple Test) with simple assumptions: we try to get a value in a one keyed dictionary and make iterations to get a statistic on this access. I made it a little more complex with a dictionary with 10,001 keys.

Results analysis:

  • if-else take the same amount of time in both case
  • try-expect is 2 times faster than if-else when it no raise the exception
  • try-expect is 5 times slower when the exception is raised

Then the if-else has a predictable comportment and help the maintainer.

I’m agree that the semantic part of the if-else can be replace by a comment but i still not recommend to try-except if it is not to raise an exception.

 

The results:

The case where the key does not exist: 
 
1,000 iterations: 
with_try (0.250 ms) 
with_try_exc (0.291 ms) 
without_try (0.119 ms) 
without_try_not (0.120 ms) 
 
1,000,000 iterations: 
with_try (231.647 ms) 
with_try_exc (263.633 ms) 
without_try (119.238 ms) 
without_try_not (118.602 ms) 
 
1,000,000,000 iterations: 
with_try (224659.381 ms) 
with_try_exc (260333.897 ms) 
without_try (109796.531 ms) 
without_try_not (111871.690 ms) 
 
 
The case where the key does exist: 
 
1,000 iterations: 
exists_with_try (0.066 ms) 
exists_with_try_exc (0.070 ms) 
exists_without_try (0.166 ms) 
exists_without_try_not (0.180 ms) 
 
1,000,000 iterations: 
exists_with_try (57.661 ms) 
exists_with_try_exc (56.909 ms) 
exists_without_try (113.633 ms) 
exists_without_try_not (116.340 ms) 
 
1,000,000,000 iterations: 
exists_with_try (57650.440 ms) 
exists_with_try_exc (57395.376 ms) 
exists_without_try (114659.023 ms) 
exists_without_try_not (117646.034 ms)

 

The code:

#!/usr/bin/env python3

import time

d = dict.fromkeys(range(0, 10000), 0)
d['somekey'] = 1

ITERATIONS = (1000, 1000*1000, 1000*1000*1000)

def time_me(function):
    def wrap(*arg):
        start = time.time()
        r = function(*arg)
        end = time.time()
        print("%s (%0.3f ms)" % (function.__name__, (end-start)*1000))
        return r
    return wrap


# Not Existing
@time_me
def with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except:
            pass

@time_me
def with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except Exception:
            pass

@time_me
def without_try(iterations):
    for i in range(0, iterations):
        if d.get('notexist'):
            pass
        else:
            pass

@time_me
def without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('notexist'):
            pass
        else:
            pass



# Existing
@time_me
def exists_with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except:
            pass

@time_me
def exists_with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except Exception:
            pass

@time_me
def exists_without_try(iterations):
    for i in range(0, iterations):
        if d.get('somekey'):
            pass
        else:
            pass

@time_me
def exists_without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('somekey'):
            pass
        else:
            pass


print ("\n\nThe case where the key does not exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    with_try(iteration)
    with_try_exc(iteration)
    without_try(iteration)
    without_try_not(iteration)


print ("\n\nThe case where the key does exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    exists_with_try(iteration)
    exists_with_try_exc(iteration)
    exists_without_try(iteration)
    exists_without_try_not(iteration)

Publicités

When UnicodeDecodeError become irrational check $LANG


I spent hours this week trying to understand how an installation script can fail on some installations.

In input we have an utf-8 encoded file and we add some xml files, also ‘utf-8’ encoded. These are parsed with Markdown.

python -m lom2mlr.markdown -l -c rationale.md

It is really simple but sometimes we ran into a strange error:

Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/edegoute/Projects/lom2mlr/lom2mlr/lom2mlr/markdown/__main__.py", line 3, in <module>
compile()
File "lom2mlr/markdown/__init__.py", line 55, in compile
extensions=extensions)
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 529, in markdownFromFile
kwargs.get('encoding', None))
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 441, in convertFile
html = self.convert(text)
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 375, in convert
newRoot = treeprocessor.run(root)
File "lom2mlr/markdown/test_mlr.py", line 76, in run
print(" " * int(element.tag[1]) + element.text)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xc9' in position 4: ordinal not in range(128)

First, it was difficult to understand how an unicode error can raise an iso-8859-1 problem on utf-8 files. Going deep I found some known problems with ‘codecs.open’ in python2.7 but no solution. I tried to force Markown to treat these files as ‘iso-8859-1’ files, then it ran an utf-8 unicode error at the same line not in the opening. It was sounding too many magic for me.

At that point, I checked again the installation was identical: same python version, same pip version, same eggs versions. I tried some egg upgrade without any success. And finally came the idea to check environment variables. Bingo! On all systems with failing installations we have no localization ($LANG=C). The fix was so simple:

export LANG=en_US.UTF-8

That’s it!

I still don’t understand the magic in the codecs python module. Why it computes a different encoding when the function call already asks for one? The workaround is simple for programmers.

Beats audio setup for HP Envy on recent Ubuntu


But the package hda-jack-retask is stuck in raring. The goal here is to allow you to install the package in your brand new Ubuntu. Replace « utopic » each time it is mentionned velow by your version.


sudo apt-add-repository -s ppa:diwic/hda
sudo apt-get update

Then edit  the file « /etc/apt/sources.list.d/diwic-ubuntu-hda-utopic.list » to replace utopic by saucy.


cd Downloads
mkdir hda-jack-retask
sudo apt-get build-dep hda-jack-retask
apt-get source hda-jack-retask
cd hda-jack-retask-0.20130613+raring

Now, you have to edit the debian/control file to sign the archive with you gpg key.
All spaces are important.

hda-jack-retask (0.20130613+utopic) utopic; urgency=low

* No-change upload for utopic

— Encolpe Degoute <encolpe.degoute@free.fr> Fri, 16 Jan 2015 11:00:00 +0200

Now, prepare your local package ans install it :


dpkg-buildpackage -rfakeroot
sudo dpkg -i ../hda-jack-retask_0.20130613+utopic_amd64.deb

Remove the PPA:

sudo apt-add-repository -r ppa:diwic/hda

The overall process should take less than 10 minutes.

How to setup your Plone core sprint


I promised to published this article last year but… well, no excuse. The topic is still hot as we work on in Arnhem in November 2013, 11 to 15 to finish it for Plone 5.

Here in Arnhem – Ploneconf 2012 Sprints – we are making a lot of sprint about a lot of different topics. As a newcomer in  sprint lead I did some mistakes.

  1. If you are working in the Plone core you must should a plip before the beginning of the sprint. It will be easier for other people  to follow your work some months later.
  2. For commiters, the main point is to check if you already give your contributor agreement back to the Plone Fundation. If you’re not there should be someone with blank agreements in your sprint to make things go easy.
  3. Once you know how many sprinters you have on your topic you must organize the area to be comfortable for all.
  4. Fing a way to show everybody the tasks that are on going and who is working on. My way is a whiteboard with post-its
  5. Prepare the technical parts before to start coding:
    • Identify all the packages you’re working on and create a remote branch for your plip. We were working at first on Products.CMFPlone, plone.app.content, plone.app.users, and after we also include plone.app.controlpanel in the list.
    • Don’t make a branch in buildout.coredev create a plips configuration file in the ‘plips‘ folder.Then run your buildout using this configuration file as master: bin/buildout -c plips/plip13260-cpy-removal.cfg
  6. Check if everybody have commit rights.
  7. If there’s a change in the plip configuration file notice it to everyone fast.
  8. Make regular internal reports to know who needs help or work 🙂
  9. Write down what’s you’re doing on

For commiters, the main point is to check if you already give your contributor agreement back to the Plone Fundation.  If you’re not there should be someone with blank agreements in your sprint to make things go easy.
Commiters should also follow a process to make the pull going right:

  1. Have  fun
  2. After each plip configuration file update check if all plip eggs are well checkouted and if they are in the good branch
  3. When you start a task  create a local branch to avoiding conflicts before your merge
  4. Commit early, often
  5. Fetch and pull others commits before to merge
  6. Run tests without ‘-t
  7. Merge your work in the plip branch, not the master
  8. Test again before the push
  9. If you need help asks (I know, git is overcomplicated 😉 )
  10. Have fun! Did I say it before?
  11. Have  a follow up plan and stay in touch

Thanks to Eric Steele for the first setup, Liz as the proposer of the sprint topic, Maurits as the technical leader and for all people around there that were working on that sprint.

Release upgrade to KUbuntu 13.10 Saucy


This time I got all errors you can have.

Before the upgrade I need to disable manually partners repository in /etc/apt/source.list.

Keep in mind to disable etc-keeper if you’re using it and if you choose btrfs to disable compression.

During the upgrade I got an error with updating initramfs. It was caused by initramfs-tools-ubuntu-touch so it’s safer to remove ubuntu-touch and initramfs-tools-ubuntu-touch before the upgrade if you don’t need them.

After i got to problems with grub :

Just after the kernel boot this message was displayed ten seconds :Error: environment block too small. It means that the file /boot/grub/grubenv is stucked to 1000 bytes instead of 1024 bytes.

Solution :

rm /boot/grub/grubenv
dpkg-reconfigure grub-pc

So, the kernel hangout during Btrfs scanning:

Still no solution. The scanning works in safe mode. Bugs reported in lauchpad suggests it’s an errors introduced in the kernel 3.11.0-12 and we should downgrade to 3.11.0-11 that is not available in generic flavour. I tried the lowlatency flavour without success. I tried too with the 3 last mainline kernels without success (https://wiki.ubuntu.com/Kernel/MainlineBuilds and http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12-rc3-saucy/).

Using the safe mode the Btrfs scanning is performing well but at the end of the boot, lightdm doesn’t start. It freezes and you cannot kill it using Ctrl+c. There’s no log. I tried gdm and it fails too. I configured kdm as main display manager and now I can log in.

Then I launch KDE4/Plasma and kwin doesn’t like wayland :

kwin: error while loading shared libraries: libwayland-eg1.so.1: no such file or directory

This time I found no clue to fix that. It’s not really a succes.

Unity still works… in kernel safe mode. But I don’t have the menu to logout (‘unity –reset-icons’ doesn’t change anything but loosing my customizations).

Since 1996 I never got such a mess for an upgrade.

Plant UML


Example UML State diagram.

Example UML State diagram. (Photo credit: Wikipedia)

You want to do UML to have beautiful images to add in your documentation? Then Plant UML is done for you!

You just have to write a plain text file and launch the interpreter and you will obtain a pretty diagram:

@startuml
Alice -> Bob: Authentication Request
Bob --> Alice: Authentication Response
@enduml

The pro

  • it’s easy to follow in your favorite DCVS
  • it’s easy to learn with a very good documentation
  • it’s working on all plateform and generate good images
  • you can customize a lot

Cons:

  • it becomes really usable if you use the graphical editor
  • Order of declaration is important for the display

http://plantuml.sourceforge.net/

http://sourceforge.net/projects/plantumlqeditor/

Feedbacks on Diazo after the PloneConf training


There were 9 trainees in this PloneConf Diazo session.  Half of them were brand new on Diazo and the second half already used it for at  least one theme.

  • Every find Diazo 1.0 cool with its new styles editor.
  • If you want to be comfortable with all the editor’s features you should use a 30-inch display (especially with the introsception panel).
  • To be able to import a Zip file from any free CSS templates site is awesome with a bemol: the Zip file must have a directory at first level with all content inside if you don’t have a weird error message displayed.
  • Shortcuts in the editor are working everywhere but are not documented; may we need a side panel with general documentation.
  • In the inspector panel, it’s not easy to find the XPath expression and   less to create a generic one without a live example.
  • There is no ‘move‘ command. It’s available since deliverance 0.3 and Diazo forked at 0.2.

During the DIY part of the training some were asking how to make a theme with Bootstrap.js and Diazo.  Deco’s toolbar theme is build with Bootstrap.js but it seems not to be doable from scratch only with the Diazo editor. Everyone finish his own theme in 2 hours for beginners and in 30 minutes  for others.

At least, there was one trainee that fell to complicated to write the rules.xml file with all those XSLT tranforms.

I let the trainees add their feeling about it if I forgot something.

Enhanced by Zemanta

Participate to a Diazo training in Arnhem the October 9


The day before the conferences I will provide a training on Diazo.

You can come with your own HTML design and in one day we will see how to create a zip file ready to be inserted in your plone site.

You don’t need to know anything about Plone, just HTML and CSS. A bit of XML understanding would be useful but not required.

Enhanced by Zemanta

A Renewed ReStructuredText Editor!


This article’s goal is to be able to launch the Enthought RST Editor with Python 2.7. Few months ago Enthought bloged about is RST editor embedded in its suite. Their blog article was useful for 4 months then they refactored how the Qt backend is loaded.  I wrote another article (in french) to show how to use the editor from the Ubuntu Python system packages but it doesn’t work anymore since the upgrade to Ubuntu 11.10 (Oneiric) which use Traits 4.0.0. Now Enthought sources are hosted on github so you can send them your patches.

First it’s better to install all this in a virtualenv with some dependencies (you must install virtualenv and virtualenvwrapper before):

sudo apt-get remove libopenblas-base #problem with numpy
sudo apt-get install python-traits python-traitsui python-pyface python-enthoughtbase \
     python-apptools python-numpy python-qt4 python-sip python-configobj python-sphinx \
     python-pygments rst2pdf
mkvirtualenv --distribute rested
cdvirtualenv
git clone https://github.com/enthought/rested.git
easy_install rested
sudo cp bin/rested /usr/local/bin/

I still have problem with encoding in the filesystem navigator and some random crashes.

Que lire pour faire du Plone


Plutôt que de faire une revue des deux ouvrages paru cet été sur Plone je vais vous présenter l’état des lieux des publications autour de Plone.

Du développement à la documentation

En quelques années la communauté Plone a évolué, s’est structurée.Il n’y a pas plus de contributeurs mais des contributeurs plus réguliers qui se concentre sur des plateformes. Le renouvellement de la communauté a démarré avec l’arrivée de Plone 3 et du livre de Martin Aspeli «Professional Plone Development». Plone 3 a apporté un niveau de complexité supplémentaire dans le développement ce qui a forcé nombre de contributeurs à faire un bilan de leurs compétences et réapprendre de nouvelles techniques, non sans heurts.

«Professional Plone Development» fut le premier livre à présenter les nouvelles techniques de Plone 3. Son succès dans la communauté a suscité des vocations et a encouragé l’éditeur à continuer avec cette technologie. Depuis les ouvrages ont fleuri et Plone 4 n’est pas sorti qu’ils abordent déjà certaines technologies développées pour cette plateforme. La liste des ouvrages publiés chez Packt est éloquente : Packt books about Plone.

Le site web de http://www.plone.org aussi a évolué avec la création d’un groupe de travail sur la documentation qui met à jour en permanence les techniques à connaitre et les bonnes pratiques. D’ailleurs nombre des contributeurs de ce groupe ont aussi publiés le ouvrage.

Au moment où je relis ces lignes, le blog Plonemetrics publie un article sur les statistiques de vente des ouvrages sur Plone chez Amazon. Sachant que Packt vend énormément en direct sur sur site, ses statistiques ne sont que la face émergée de l’iceberg. Cela montre tout de même l’engouement réel dans Plone de nombre d’utilisateurs et de développeurs.

Tableau récapitulatif

Titre Date de
parution
Cible Commentaires
Professional Plone Development Sept 2007 Développeur connaissant Plone 2.5 ou Zope 3 Ce livre a vieilli. Même si tous les concepts présentés sont toujours utilisés dans Plone 3.3 et Plone 4.0 la plupart des exemples ne fonctionnent plus.
A User’s Guide to Plone: Updated for Plone 3 Août 2008 Utilisateurs et créateurs de contenu Quasiment rien n’a changé entre Plone 3.0 et 3.3. C’est l’ouvrage idéal pour l’autoformation des nouveaux webmaster.
Practical Plone 3 Fév 2009 Développeurs débutants et webmasters Didactique et bien agencé il me sert pour préparer des ateliers pratiques lors des rencontre entre utilisateurs de Plone.
The Definitive Guide to Plone 2 Edition Avr 2009 Développeurs débutants à expérimentés Une référence à garder dans un coin. Le livre continue à être mis à jour régulièrement sur son site dédié.
Plone 3 Theming Juil 2009 Webmasters et intégrateurs C’est l’ouvrage qu’il manquait depuis des années dans le monde de Plone. Il synthétise les bonnes pratiques pour Plone 3 et présente certaines techniques de Plone 4. A garder sous le coude.
Plone 3 for Education Déc 2009 Intégrateurs des grandes écoles et des universités C’est le premier livre dédié à un groupe de utilisateurs particuliers et il atteint bien sa cible. Au final même le MIT a migré sous Plone pour son projet Open Couserware.
Plone 3 Multimedia Mai 2010 Intégrateurs de sites orientés multimedia Vidéo, Flash, géolocalisation, et tous les gadgets présents dans les sites 2.0 sont présentés dans cet ouvrage.
Plone 3 Products Development Cookbook Mai 2010 Développeurs expérimentés et Responsables SI Cet ouvrage présente toutes les techniques à connaitre pour monter une site à haute performance sous Plone. Maheureusement par manque de place certains passages manquent de détails utiles. Tous les chefs de projets et les responsables de site Plone devraient l’avoir lu sinon l’avoir à portée de main.
Plone 3.3 Site Administration Juil 2010 Webmasters et intégrateurs C’est le complément de « Plone 3 Theming » orienté sur l’administration d’un site au jour le jour. A posséder.
Plone 3 Intranets Août 2010 Intégrateurs débutants à confirmés Cet ouvrage est plus orienté sur les aspects de configuration de la sécurité et des outils de communications standard des intranets : blogs, newsletter, prévisualisation de documents.

Pourquoi aucun ouvrage en français ?

Pour le moment il n’y a plus de réel leadership dans la communauté française de Plone. Les personnes impliquées sont tous au même niveau et il est difficile de prendre en charge la rédaction d’un ouvrage à partir de zéro seul ou pour une petite équipe dispersée en France. Malgré tout le projet de rédaction suis son cours et aboutira bientôt.

En théorie il serait possible de traduire les ouvrages en anglais mais le partenaire français de Packt préfère avoir un ouvrage original plutôt qu’une traduction.

Plone, un outil en plein essor

Il y a 18 mois certains rangeaient déjà Plone dans les dinosaures du web et je recevais des messages me demandant d’évaluer la pertinence de choisir cet outil pour les projets à long terme. Le développement de l’offre en documentation démontre que Plone a su s’adapter et se diversifier depuis la sortie de sa troisième version majeure. Le démarrage a pu paraitre difficile car de nombreux nouveaux outils ont du être mis en place par la communauté. Celle-ci à su rebondir et répondre aux demandes de ses utilisateurs. La communauté française elle s’organise autour de nouveaux acteurs qui émergent et sortira bientôt de l’ombre avec un certain nombre d’outils en cours de préparation.

Plone 4 ne sera pas une révolution comme l’a été Plone 3 mais plutôt une évolution importante qui a justifié son renommage en cours de réalisation de Plone 3.5 à Plone .40. Elle introduit quelques ruptures de compatibilité mais ne nécessite aucun apprentissage supplémentaire. Elle est une vitrine technologique pour des outils qui seront incontournables dans Plone 5.0 (Dexterity, deco.gs, XDV, WSGI, RelStorage, etc).