Try-except versus if-else in Python 3.5


Today I ran again in the question why to use if-else when try-except is shorter?

There is a semantic part that we loose information on a try-except as we don’t know what the developer did expect in the most case and there is a performance part. I found a test for python 2 (Try / Except Performance in Python: A Simple Test) with simple assumptions: we try to get a value in a one keyed dictionary and make iterations to get a statistic on this access. I made it a little more complex with a dictionary with 10,001 keys.

Results analysis:

  • if-else take the same amount of time in both case
  • try-expect is 2 times faster than if-else when it no raise the exception
  • try-expect is 5 times slower when the exception is raised

Then the if-else has a predictable comportment and help the maintainer.

I’m agree that the semantic part of the if-else can be replace by a comment but i still not recommend to try-except if it is not to raise an exception.

 

The results:

The case where the key does not exist: 
 
1,000 iterations: 
with_try (0.250 ms) 
with_try_exc (0.291 ms) 
without_try (0.119 ms) 
without_try_not (0.120 ms) 
 
1,000,000 iterations: 
with_try (231.647 ms) 
with_try_exc (263.633 ms) 
without_try (119.238 ms) 
without_try_not (118.602 ms) 
 
1,000,000,000 iterations: 
with_try (224659.381 ms) 
with_try_exc (260333.897 ms) 
without_try (109796.531 ms) 
without_try_not (111871.690 ms) 
 
 
The case where the key does exist: 
 
1,000 iterations: 
exists_with_try (0.066 ms) 
exists_with_try_exc (0.070 ms) 
exists_without_try (0.166 ms) 
exists_without_try_not (0.180 ms) 
 
1,000,000 iterations: 
exists_with_try (57.661 ms) 
exists_with_try_exc (56.909 ms) 
exists_without_try (113.633 ms) 
exists_without_try_not (116.340 ms) 
 
1,000,000,000 iterations: 
exists_with_try (57650.440 ms) 
exists_with_try_exc (57395.376 ms) 
exists_without_try (114659.023 ms) 
exists_without_try_not (117646.034 ms)

 

The code:

#!/usr/bin/env python3

import time

d = dict.fromkeys(range(0, 10000), 0)
d['somekey'] = 1

ITERATIONS = (1000, 1000*1000, 1000*1000*1000)

def time_me(function):
    def wrap(*arg):
        start = time.time()
        r = function(*arg)
        end = time.time()
        print("%s (%0.3f ms)" % (function.__name__, (end-start)*1000))
        return r
    return wrap


# Not Existing
@time_me
def with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except:
            pass

@time_me
def with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except Exception:
            pass

@time_me
def without_try(iterations):
    for i in range(0, iterations):
        if d.get('notexist'):
            pass
        else:
            pass

@time_me
def without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('notexist'):
            pass
        else:
            pass



# Existing
@time_me
def exists_with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except:
            pass

@time_me
def exists_with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except Exception:
            pass

@time_me
def exists_without_try(iterations):
    for i in range(0, iterations):
        if d.get('somekey'):
            pass
        else:
            pass

@time_me
def exists_without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('somekey'):
            pass
        else:
            pass


print ("\n\nThe case where the key does not exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    with_try(iteration)
    with_try_exc(iteration)
    without_try(iteration)
    without_try_not(iteration)


print ("\n\nThe case where the key does exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    exists_with_try(iteration)
    exists_with_try_exc(iteration)
    exists_without_try(iteration)
    exists_without_try_not(iteration)

When UnicodeDecodeError become irrational check $LANG


I spent hours this week trying to understand how an installation script can fail on some installations.

In input we have an utf-8 encoded file and we add some xml files, also ‘utf-8’ encoded. These are parsed with Markdown.

python -m lom2mlr.markdown -l -c rationale.md

It is really simple but sometimes we ran into a strange error:

Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/edegoute/Projects/lom2mlr/lom2mlr/lom2mlr/markdown/__main__.py", line 3, in <module>
compile()
File "lom2mlr/markdown/__init__.py", line 55, in compile
extensions=extensions)
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 529, in markdownFromFile
kwargs.get('encoding', None))
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 441, in convertFile
html = self.convert(text)
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 375, in convert
newRoot = treeprocessor.run(root)
File "lom2mlr/markdown/test_mlr.py", line 76, in run
print(" " * int(element.tag[1]) + element.text)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xc9' in position 4: ordinal not in range(128)

First, it was difficult to understand how an unicode error can raise an iso-8859-1 problem on utf-8 files. Going deep I found some known problems with ‘codecs.open’ in python2.7 but no solution. I tried to force Markown to treat these files as ‘iso-8859-1’ files, then it ran an utf-8 unicode error at the same line not in the opening. It was sounding too many magic for me.

At that point, I checked again the installation was identical: same python version, same pip version, same eggs versions. I tried some egg upgrade without any success. And finally came the idea to check environment variables. Bingo! On all systems with failing installations we have no localization ($LANG=C). The fix was so simple:

export LANG=en_US.UTF-8

That’s it!

I still don’t understand the magic in the codecs python module. Why it computes a different encoding when the function call already asks for one? The workaround is simple for programmers.

Beats audio setup for HP Envy on recent Ubuntu


But the package hda-jack-retask is stuck in raring. The goal here is to allow you to install the package in your brand new Ubuntu. Replace « utopic » each time it is mentionned velow by your version.


sudo apt-add-repository -s ppa:diwic/hda
sudo apt-get update

Then edit  the file « /etc/apt/sources.list.d/diwic-ubuntu-hda-utopic.list » to replace utopic by saucy.


cd Downloads
mkdir hda-jack-retask
sudo apt-get build-dep hda-jack-retask
apt-get source hda-jack-retask
cd hda-jack-retask-0.20130613+raring

Now, you have to edit the debian/control file to sign the archive with you gpg key.
All spaces are important.

hda-jack-retask (0.20130613+utopic) utopic; urgency=low

* No-change upload for utopic

— Encolpe Degoute <encolpe.degoute@free.fr> Fri, 16 Jan 2015 11:00:00 +0200

Now, prepare your local package ans install it :


dpkg-buildpackage -rfakeroot
sudo dpkg -i ../hda-jack-retask_0.20130613+utopic_amd64.deb

Remove the PPA:

sudo apt-add-repository -r ppa:diwic/hda

The overall process should take less than 10 minutes.

How to setup your Plone core sprint


I promised to published this article last year but… well, no excuse. The topic is still hot as we work on in Arnhem in November 2013, 11 to 15 to finish it for Plone 5.

Here in Arnhem – Ploneconf 2012 Sprints – we are making a lot of sprint about a lot of different topics. As a newcomer in  sprint lead I did some mistakes.

  1. If you are working in the Plone core you must should a plip before the beginning of the sprint. It will be easier for other people  to follow your work some months later.
  2. For commiters, the main point is to check if you already give your contributor agreement back to the Plone Fundation. If you’re not there should be someone with blank agreements in your sprint to make things go easy.
  3. Once you know how many sprinters you have on your topic you must organize the area to be comfortable for all.
  4. Fing a way to show everybody the tasks that are on going and who is working on. My way is a whiteboard with post-its
  5. Prepare the technical parts before to start coding:
    • Identify all the packages you’re working on and create a remote branch for your plip. We were working at first on Products.CMFPlone, plone.app.content, plone.app.users, and after we also include plone.app.controlpanel in the list.
    • Don’t make a branch in buildout.coredev create a plips configuration file in the ‘plips‘ folder.Then run your buildout using this configuration file as master: bin/buildout -c plips/plip13260-cpy-removal.cfg
  6. Check if everybody have commit rights.
  7. If there’s a change in the plip configuration file notice it to everyone fast.
  8. Make regular internal reports to know who needs help or work 🙂
  9. Write down what’s you’re doing on

For commiters, the main point is to check if you already give your contributor agreement back to the Plone Fundation.  If you’re not there should be someone with blank agreements in your sprint to make things go easy.
Commiters should also follow a process to make the pull going right:

  1. Have  fun
  2. After each plip configuration file update check if all plip eggs are well checkouted and if they are in the good branch
  3. When you start a task  create a local branch to avoiding conflicts before your merge
  4. Commit early, often
  5. Fetch and pull others commits before to merge
  6. Run tests without ‘-t
  7. Merge your work in the plip branch, not the master
  8. Test again before the push
  9. If you need help asks (I know, git is overcomplicated 😉 )
  10. Have fun! Did I say it before?
  11. Have  a follow up plan and stay in touch

Thanks to Eric Steele for the first setup, Liz as the proposer of the sprint topic, Maurits as the technical leader and for all people around there that were working on that sprint.

Release upgrade to KUbuntu 13.10 Saucy


This time I got all errors you can have.

Before the upgrade I need to disable manually partners repository in /etc/apt/source.list.

Keep in mind to disable etc-keeper if you’re using it and if you choose btrfs to disable compression.

During the upgrade I got an error with updating initramfs. It was caused by initramfs-tools-ubuntu-touch so it’s safer to remove ubuntu-touch and initramfs-tools-ubuntu-touch before the upgrade if you don’t need them.

After i got to problems with grub :

Just after the kernel boot this message was displayed ten seconds :Error: environment block too small. It means that the file /boot/grub/grubenv is stucked to 1000 bytes instead of 1024 bytes.

Solution :

rm /boot/grub/grubenv
dpkg-reconfigure grub-pc

So, the kernel hangout during Btrfs scanning:

Still no solution. The scanning works in safe mode. Bugs reported in lauchpad suggests it’s an errors introduced in the kernel 3.11.0-12 and we should downgrade to 3.11.0-11 that is not available in generic flavour. I tried the lowlatency flavour without success. I tried too with the 3 last mainline kernels without success (https://wiki.ubuntu.com/Kernel/MainlineBuilds and http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12-rc3-saucy/).

Using the safe mode the Btrfs scanning is performing well but at the end of the boot, lightdm doesn’t start. It freezes and you cannot kill it using Ctrl+c. There’s no log. I tried gdm and it fails too. I configured kdm as main display manager and now I can log in.

Then I launch KDE4/Plasma and kwin doesn’t like wayland :

kwin: error while loading shared libraries: libwayland-eg1.so.1: no such file or directory

This time I found no clue to fix that. It’s not really a succes.

Unity still works… in kernel safe mode. But I don’t have the menu to logout (‘unity –reset-icons’ doesn’t change anything but loosing my customizations).

Since 1996 I never got such a mess for an upgrade.

Plant UML


Example UML State diagram.

Example UML State diagram. (Photo credit: Wikipedia)

You want to do UML to have beautiful images to add in your documentation? Then Plant UML is done for you!

You just have to write a plain text file and launch the interpreter and you will obtain a pretty diagram:

@startuml
Alice -> Bob: Authentication Request
Bob --> Alice: Authentication Response
@enduml

The pro

  • it’s easy to follow in your favorite DCVS
  • it’s easy to learn with a very good documentation
  • it’s working on all plateform and generate good images
  • you can customize a lot

Cons:

  • it becomes really usable if you use the graphical editor
  • Order of declaration is important for the display

http://plantuml.sourceforge.net/

http://sourceforge.net/projects/plantumlqeditor/

How to use curl to test a REST API with a beautiful JSON indentation


How to get indented JSON with curl output

You need to install simplejson Python module in your environment, or python-simplejson.

curl [options] url | python -msimplejson.tool

An example with github APIv3

curl -s -H "Content-Type: application/json" -X GET \
     "https://api.github.com/repos/collective/collective.ckeditor" \
     | python -msimplejson.tool

How to display response headers and API errors with the JSON indentation

This command line have two problems :

  1. You cannot display response HTTP headers because it breaks simplejson parsing
  2. If you have an error with a request and you don’t know how to debug it.

You need to add two more options to push header response and error message in stderr: -S and -D /dev/stderr. Now you can test your API with all informations in one command:

curl -s -S -D /dev/stderr -u user:password \
     -H "Content-Type: application/json" -X GET \
     "http://myapi/v1/test/"

From curl manual:

  • -s no progress bar; stats report or errors are dislayed
  • -S display errors if -s is enable
  • -D dump the header in a file; here stderr to make them not interfer with json indentation
  • -u user and password for Basic HTTP Authentication
  • -H add header in the outgoing request
  • -X HTTP Command to launch. If you’re using Basic HTTP Authentication you must use -X GET

If someone have tip to add pygment coloration in json and headers I will be glad to share the ownership of this article.

Feedbacks on Diazo after the PloneConf training


There were 9 trainees in this PloneConf Diazo session.  Half of them were brand new on Diazo and the second half already used it for at  least one theme.

  • Every find Diazo 1.0 cool with its new styles editor.
  • If you want to be comfortable with all the editor’s features you should use a 30-inch display (especially with the introsception panel).
  • To be able to import a Zip file from any free CSS templates site is awesome with a bemol: the Zip file must have a directory at first level with all content inside if you don’t have a weird error message displayed.
  • Shortcuts in the editor are working everywhere but are not documented; may we need a side panel with general documentation.
  • In the inspector panel, it’s not easy to find the XPath expression and   less to create a generic one without a live example.
  • There is no ‘move‘ command. It’s available since deliverance 0.3 and Diazo forked at 0.2.

During the DIY part of the training some were asking how to make a theme with Bootstrap.js and Diazo.  Deco’s toolbar theme is build with Bootstrap.js but it seems not to be doable from scratch only with the Diazo editor. Everyone finish his own theme in 2 hours for beginners and in 30 minutes  for others.

At least, there was one trainee that fell to complicated to write the rules.xml file with all those XSLT tranforms.

I let the trainees add their feeling about it if I forgot something.

Enhanced by Zemanta

Participate to a Diazo training in Arnhem the October 9


The day before the conferences I will provide a training on Diazo.

You can come with your own HTML design and in one day we will see how to create a zip file ready to be inserted in your plone site.

You don’t need to know anything about Plone, just HTML and CSS. A bit of XML understanding would be useful but not required.

Enhanced by Zemanta

A Renewed ReStructuredText Editor!


This article’s goal is to be able to launch the Enthought RST Editor with Python 2.7. Few months ago Enthought bloged about is RST editor embedded in its suite. Their blog article was useful for 4 months then they refactored how the Qt backend is loaded.  I wrote another article (in french) to show how to use the editor from the Ubuntu Python system packages but it doesn’t work anymore since the upgrade to Ubuntu 11.10 (Oneiric) which use Traits 4.0.0. Now Enthought sources are hosted on github so you can send them your patches.

First it’s better to install all this in a virtualenv with some dependencies (you must install virtualenv and virtualenvwrapper before):

sudo apt-get remove libopenblas-base #problem with numpy
sudo apt-get install python-traits python-traitsui python-pyface python-enthoughtbase \
     python-apptools python-numpy python-qt4 python-sip python-configobj python-sphinx \
     python-pygments rst2pdf
mkvirtualenv --distribute rested
cdvirtualenv
git clone https://github.com/enthought/rested.git
easy_install rested
sudo cp bin/rested /usr/local/bin/

I still have problem with encoding in the filesystem navigator and some random crashes.