How to get python 2.4 or 2.5 for Ubuntu 10.4


Each new version of an OS carries its bad news. The last version of Ubuntu doesn’t embed neither Python 2.4 nor Python 2.5. Nevertheless until all your Plone sites will be migrated to Plone 4 you will need a Python 2.4 version somewhere.

Method 1: search for sources to get binary packages

There is a Python 2.4 « branch » in Ubuntu on launchpad :  “python2.4” source package in Lucid. If you now a little how Ubuntu packaging work you can easyly get Ubuntu package of Python 2.4 by following steps below:

  1. Click on Show builds
  2. Choose Successfully built then apply the filter
  3. Then choose your the archive for your architecture (often i386 or amd64)

There you have python2.4 and python2.4-dev packages.

Method 2: Use packages search tool

Launchap also have a package search tool for all Ubuntu releases. This one doesn’t limit itself to packages disbributed in official Ubuntu repositories and it also give results for all packages build using the Launchad system.

In our case we can use:

Method 3: Get sources and create your own packages

From the source page of the first method (“python2.4” source package in Lucid) you can get sources and the official patch for Ubuntu. For exemple you can click on 2.4.6-1ubuntu5 to get this version.

On the next page you can download a python source archive and the associated patch :

Once these to files downloaded you will do following steps:

  1. Unarchive python sources: tar zxf python2.4_2.4.6.orig.tar.gz
  2. Uncompress the patch:  gunzip python2.4_2.4.6-1ubuntu5.diff.gz
  3. Apply the patch:  patch -p0 < python2.4_2.4.6-1ubuntu5.diff
  4. Go in the folder created: cd python2.4-2.4.6
  5. Try to build the package: dpkg-buildpackage
  6. Install libreadline5-dev or fix the debian/control file by replacing libreadline5-dev by libreadline-dev (I’m nasty)
  7. Install all dependencies required (why do you need emacs and bluetooth to compile python ?): apt-get install libncursesw5-dev  tk8.5-dev libdb4.6-dev libgdbm-dev blt-dev  libbluetooth-dev emacs23 debiandoc-sgml
  8. Try again to build the package: dpkg-buildpackage

I don’t know how long these packages will be available.

Publicités

2010 : liens de la semaine 1


Cette semaine a été très productive du côté des blogueurs et en particuliers chez les développement autour du core Python.

Python

Pour ceux qui ne connaissent cette boîte de Pandore qu’est le GIL (Global Interpretor Lock) en Python voici un très bon article qui mesure son effet dans le traitement des entrées/sorties avec des systèmes multi-processeurs ou multi-cores :

The Python GIL Visualized

Jesse Noller rappelle les but initiaux du projet Unladen swallow en partie sponsorisé par Google et signale malicieusement en fin d’article qu’une proposition de fusionner leur travail dans Python Core pour Python 3 :

UNLADEN SWALLOW: PYTHON 3’S BEST FEATURE.

Tarek Ziadé, notre président bien aimé, présente les nouvelles fonctionnalités en cours de discussion pour la prochaine version majeure de Distutils, ce qui intéressera tout ceux qui publient des eggs :

Possible new features for Distutils 2.7

Mark Mruss fait une présentation détaillé des ‘docstring’ – ces commentaires qui décorent vos modules, vos classes et vos fonctions – et montre quelle utilisation avancée de celles-ci peut être faite (documentation, tests) :

Introducing Docstrings

Encore un article sur la préparation de la distribution de code qui montre comment mettre en place facilement des scripts  de pré-traitement et de post-traitement :

Using setuptools entry points

La première revue complète du livre « Dive Into Python 3 » (Au cœur de Python 3) :

Book Review: Dive Into Python 3

Enfin deux articles sur les fonctionnalités que les gestionnaires de projets veulent dans mercurial. Dans le premier cas il s’agit de Python-dev qui est censé être migré entièrement cette année sous mercurial :

Where the Hg transition stands

Le deuxième est un développeur qui fait pas mal de Django (vous aimerez sans doute aussi son article free VS free) :

DVCS Ponies

Plone

Voici la description d’un problème récurrent pour les sites multilignues en Plone : comment réordonner des éléments dans une page :

Ordering objects in folders with multilingual sites

Une des solutions les plus faciles à mettre en place est d’afficher toutes les langues lors du tri pour être sur que le Javascript se comporte correctement. L’autre est de demander au Javascript de ne prendre en compte les rangs réels des contenus déplacés plutôt que le rang à l’affichage.

Alex Clarck montre comment installer Plone juste avec buildout en 5 minutes, sans avoir à installer l’unified installer ou Paster :

No, really, you can (just) use Buildout to install Plone

WEB

Encore une fois je terminerai par des liens plus centrés sur le développement Web en général et jQuery en particulier.

Réaliser des animations en jQuery en 7 étapes :

jQuery Animations: A 7-Step Program

Un visualisateur de contenu (d’images) à l’épreuve des balles :

A Bullet-Proof Content Viewer

Personnaliser les événements et l’API des Événements Spéciaux dans jQuery :

Custom Events, and the Special Events API in jQuery

Mise au point sur la licence GPL et les services web


J’avais déjà fait un billet à ce sujet il y a quelques temps. Lors des Journées du Logiciel Libre j’ai eu l’occasion de discuter avec Alix Cazenave après sa conférence sur la loi Hadopi et le contrôle du net.

Voici les points que j’ai pu éclaircir :

  • la licence GPL concerne le code d’un logiciel
  • les services web ne sont pas couverts par la licence GPL
  • c’est la licence Affero GPL qui couvre les services réseau

Lorsqu’une entité mets en œuvre des services réseau basés sur du code GPL elle n’a aucun devoir de partage du code source de son application avec ses utilisateurs ou avec la communauté. Cette entité peut vendre les services réseau sans contreparti mais si elle veut vendre l’installation de son application il lui faudra partager les sources.

Si le code d’un logiciel est protégé par la licence Affero GPL alors n’importe quel utilisateur interagissant avec service réseau basé sur son code peu demander le code source complet du service réseau.

Dans le cas de Plone 2 et de Plone 3 ils ne relèvent que de la seule licence GPL. Plone 4 utilisera sûrement une licence BSD lors de sa sortie.

Which Plone project is your site based on?


Far from the versions war I’m trying to understand how many people don’t use Plone « main » and choose to use project based on it. This evaluation is getting more difficult that I’m not able to recense how many projects are based on Plone. We can found some of them in the Plone.org feed and some others in pypi or in the plone.org products column.

I can easily cite :

But they would be much more.

The Plone Conference in Budapest will be a good place for these projects to gain visibility and to project leaders to talk with the evangelist team to gain a page on Plone.org to show what are their roadmap. It is difficult to find an announce done in a feed or in mailing lists few months ago. It easier to find a page one the official plone.org site if a projects column is created in it. The Plone foundation doesn’t have to fund them, but can help their communication.

Manage monkey patches and performance improvement in Plone 3


Few days ago Jean-Michel François proposed a useful patch for PlonePAS that can be applied for all Plone release until 3.2. Plone 3.3 will embed this patch.

How can we add this patch in a traceable way for an not so old Plone 3.1 or 3.2 ?

First, we can use the new release  of Products.PlonePAS that should be compatible with our Plone installation. The second option is to add a monkey patch in the policy product of our site. One more monkey patch…

Some projects have so many monkey patches that it is difficult to know where is the code that run your site. Martin Aspeli did a tool to handle monkey patches in an elegant way for Zope 2 and Zope 3: collective.monkeypatcher. It allows you to plug  your monkey patches with a simple ZCML directive. Later Gilles Lenfant added a control panel for Zope 2 to be able to have a visual way to follow patches with collective.monkeypatcherpanel.

How does it works?

In your buildout.cfg add :

eggs +=
   collective.monkeypatcher
   collective.monkeypatcherpanel
zcml +=
   collective.monkeypatcher
   collective.monkeypatcherpanel

To create patches add a ‘patches.py‘ file in your egg (if you have more than 2 or 3 patches you should create a directory). Our performance patch looks like this:

import copy

def enumerateUsers( self
                  , id=None
                  , login=None
                  , exact_match=False
                  , **kw
                  ):

    """ See IUserEnumerationPlugin.
    """
    plugin_id = self.getId()

    criteria=copy.copy(kw)
    if id is not None:
        criteria["id"]=id
    if login is not None:
        criteria["login"]=login

    if not kw and id:
        data = self._storage.get(id, None)
        if data is None:
            user_info = []
        else:
            user_info=[ { 'id' : self.prefix + id,
                     'login' : id,
                     'title' : data.get('fullname', id),
                     'description' : data.get('fullname', id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } ]
    else:
        users=[ (user,data) for (user,data) in self._storage.items()
                    if self.testMemberData(data, criteria, exact_match)]

         user_info=[ { 'id' : self.prefix + user_id,
                     'login' : user_id,
                     'title' : data.get('fullname', user_id),
                     'description' : data.get('fullname', user_id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } for (user_id, data) in users ]

        return tuple(user_info)

In the configure.zcml of your policy product add an include:

<include file="patches.zcml" />

The file patches.zcml will contain following code:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:monkey="http://namespaces.plone.org/monkey"
    i18n_domain="collective.monkeypatcher">

    <include package="collective.monkeypatcher" file="meta.zcml" />

    <monkey:patch
       original="enumerateUsers"
       replacement=".patches.enumerateUsers"
       docstringWarning="false"
    />

</configure>

Run your buildout, start your site and the patch is applied. You can go in the Zope Control Panel to see how many patches are already compatible with this tool.

Heads up: new plone.recipe.pound release


Since few weeks plone.recipe.pound 0.5.4 was broken with an output like this:

An internal error occured due to a bug in either zc.buildout or in a recipe being used:
Traceback (most recent call last):
File "/tmp/tmpVcq-b1/zc.buildout-1.2.1-py2.4.egg/zc/buildout/buildout.py", line 1509, in main
File "/tmp/tmpVcq-b1/zc.buildout-1.2.1-py2.4.egg/zc/buildout/buildout.py", line 473, in install
File "/tmp/tmpVcq-b1/zc.buildout-1.2.1-py2.4.egg/zc/buildout/buildout.py", line 1091, in _call
File "/home/encolpe/.virtualenvs/internal/preprod/plone.recipe.pound/plone/recipe/pound/build.py", line 74, in install
installed = CMMIRecipe.install(self)
File "/home/encolpe/.virtualenvs/internal/preprod/downloads/dist/zc.recipe.cmmi-1.2.0/src/zc/recipe/cmmi/__init__.py", line 155, in install
system("make install")
File "/home/encolpe/.virtualenvs/internal/preprod/downloads/dist/zc.recipe.cmmi-1.2.0/src/zc/recipe/cmmi/__init__.py", line 27, in system
raise SystemError("Failed", c)
SystemError: ('Failed', 'make install')

This recipe directly unherit from zc.recipe.cmmi and act as a wrapper around the CMMIRecipe.install method. Most arguments used are transmitted to the recipe are self attributes or keys in self.options. The version 1.2.0 of this recipe publish the May, 18 moved the way that extra arguments are transmitted:

  • zc.recipe.cmmi 1.1.x:  self.options[‘extra_options’]
  • zc.recipe.cmmi 1.2.0: self.extra_options (an empty string by default)

There’s no regression tests, neither an entry in the release history. Youenn did a new release of plone.recipe.pound to fix this new behavior.

Morality:

  • if you are using zc.recipe.cmmi 1.1.x you can still use plone.recipe.pound 0.5.4
  • if you are using zc.recipe.cmmi 1.2.0 or above use plone.recipe.pound 0.5.5
  • other recipes using zc.recipe.cmmi may be broken

Install a maintenance page without restarting apache


These days we are working with fabric to remove some hazard in maintenance.
Recently someone gzip a Data.fs file in production and our customer loose 2 days of work. We choose to use fabric (http://www.nongnu.org/fab/) to limit command line working to the strict minimum.

Now, almost all procedures were added in a fabric factory, from the installation of a development environment to the upgrade of the production server. In this last part, we search a solution to put a maintenance page without have to gain root privileges to restart the apache used in frontend. For static or CGI sites you can use a .htaccess file to override the global rules. For a stopped Zope server it doesn’t help much.

The goal is to use RewriteCond and RewriteRule in a such way that a static HTML file would be displayed when it exists. We can call it ‘maintenance.html‘ and rename it ‘maintenance.html-disabled‘ to let the site be displayed normally.

We start with a virtual host generated by iw.recipe.squid:

<VirtualHost *:80>
   ServerName www.gosseyn.fr

   RewriteEngine On
   RewriteLog /my/path/to/squid/parts/squid/log/rewrite_www.gosseyn.fr.log
   RewriteLogLevel 0

   CustomLog /my/path/to/squid//parts/squid/log/access_www.gosseyn.fr.log common
   ErrorLog /my/path/to/squid//parts/squid/log/error_www.gosseyn.fr.log

   ## common rules for squid rewrite rules
   RewriteRule ^(.*)$ - [E=BACKEND_LOCATION:127.0.0.1]
   RewriteRule ^(.*)$ - [E=BACKEND_PORT:8081]
   RewriteRule ^(.*)$ - [E=BACKEND_PATH:site]

   RewriteRule  ^/(.*)/$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]
   RewriteRule  ^/(.*)$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]

   ## specific rules base on cookie

</VirtualHost>

First we need to declare a document root and to bypass all security rules. We create a new path in the root of our buildout folder called ‘maintenances_pages’.

   DocumentRoot /my/path/to/buildout/prod/maintenance_pages

   <Directory "/my/path/to/buildout/prod/maintenance_pages">
       Options None
       AllowOverride None
       Order allow,deny
       allow from all

       <LimitExcept GET>
           Order allow,deny
           allow from all
       </LimitExcept>
   </Directory>

These declaration should satisfy all paranoid BOFH.
Now create a file called maintenance.html within the folder. All code (javascript, CSS and images) must be embeded, this is a limitation of this approach.
Well, how to detect that a file exist on the filesystem from apache. You can do this with the flag ‘-f’ from the rewrite module:

    RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html -f

Then redirect all pages to our maintenance page:

   RewriteCond %{REQUEST_URI} !/maintenance.html
   RewriteRule $ /maintenance.html [R=302,L]

We also need to disable the standard rewrite rules. The easier way to do it is to use ‘!-f‘.

   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)/$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]
   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]

These rules will not be applied anymore when our maintenance file will be name ‘maintenance.html’. Our goal is reached.

The whole virtualhost file look like this:

<VirtualHost *:80>
   ServerName www.gosseyn.fr

   RewriteEngine On
   RewriteLog /my/path/to/squid/parts/squid/log/rewrite_www.gosseyn.fr.log
   RewriteLogLevel 0

   CustomLog /my/path/to/squid//parts/squid/log/access_www.gosseyn.fr.log common
   ErrorLog /my/path/to/squid//parts/squid/log/error_www.gosseyn.fr.log

   DocumentRoot /my/path/to/buildout/prod/maintenance_pages

   <Directory "/my/path/to/buildout/prod/maintenance_pages">
       Options None
       AllowOverride None
       Order allow,deny
       allow from all

       <LimitExcept GET>
           Order allow,deny
           allow from all
       </LimitExcept>
   </Directory>

   ## our maintenance page
   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html -f
   RewriteCond %{REQUEST_URI} !/maintenance.html
   RewriteRule $ /maintenance.html [R=302,L]

   ## common rules for squid rewrite rules
   RewriteRule ^(.*)$ - [E=BACKEND_LOCATION:127.0.0.1]
   RewriteRule ^(.*)$ - [E=BACKEND_PORT:8081]
   RewriteRule ^(.*)$ - [E=BACKEND_PATH:site]

   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)/$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]
   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]

   ## specific rules base on cookie

</VirtualHost>

It will be difficult to include this in a recipe as we modify an existing virtualhost in a very specific way.

As usual, all feedbacks are appreciated.

How to extend you Plone 3.2 buildout


Plone 3.2 and next comes with a hardcoded versions.cfg for each Plone release. For the current release:
http://dist.plone.org/release/3.2.2/versions.cfg

The Plone Upgrade Guide will show you how to transform a basic buildout created with paster:
http://plone.org/documentation/manual/upgrade-guide/version/upgrading-from-3-x-to-3.2

This solution will work until you already defined a ‘[versions]‘ section in your main buildout file. What we need is to merge the two ‘[versions]‘ sections.

For example, you current ‘buildout.cfg‘ looks like this:

[buildout]
...
eggs =
	archetypes.schematuning
	Products.CacheSetup
	Products.errornumber
	collective.recipe.omelette
	collective.workflowed
	Products.DCWorkflowGraph
	Products.PrintingMailHost
	plone.reload
	Products.PDBDebugMode
	Products.DocFinderTab

versions = versions

[versions]
zope.testing=3.5.1
zope.interface=3.4.1
Products.errornumber=1.2
archetypes.schematuning=1.1
Products.CacheSetup=1.2

If you add extends = http://dist.plone.org/release/3.2.2/versions.cfg it will be overloaded by the local section displayed above and will be simply ignored by ‘bin/buildout‘. If you run an update now all plone bundle eggs will be updated to the last published eggs (Plone 3.3b1, etc).
The solution consist to use the ‘extends’ directive to merge sections. For that we need to put your local ‘[versions]‘ section in a separate file.
dev-versions.cfg‘ will contain:

[versions]
zope.testing=3.5.1
zope.interface=3.4.1
Products.errornumber=1.2
archetypes.schematuning=1.1
Products.CacheSetup=1.2

The ‘buildout.cfg‘ will contain:

[buildout]
...
eggs =
	archetypes.schematuning
	Products.CacheSetup
	Products.errornumber
	collective.recipe.omelette
	collective.workflowed
	Products.DCWorkflowGraph
	Products.PrintingMailHost
	plone.reload
	Products.PDBDebugMode
	Products.DocFinderTab

extends = dev-versions.cfg http://dist.plone.org/release/3.2.2/versions.cfg

versions = versions

We can update now and verify our eggs versions:

  • Plone 3.2.2
  • plone.app.locales 3.2.0
  • Products.CMFPlacefulWorkflow 1.4.0
  • Products.CacheSetup 1.2

It is a certified Plone 3.2.2 installation with our specifics packages.

‘Practical Plone 3’ review


I finished to read the long waited (read updates here 🙂) ‘Practical Plone 3’ book. It reprensents a big amount of very good work.

If you want to begin in Plone or if you want a webmaster guide for your brand new Plone site the two first parts are designed for you. Each step of your needs are describe in a very good teaching way.

Parts are growing in difficulties. The first will show you how to install and what in installed in a default Plone site. The second part will learn you how to become the owner of your site by creating content and configuring it. The third part is for those who wanted to learn how to use addon products for Plone or want to customize Plone site: zc.buildout, PloneFormGen and ArchGenXML are presented here. The last part will show you some general needs around Plone: put Plone in production behind Apache or IIS with speed optimizations, and LDAP/Active Directory integration.

Even if the target of this book is beginners it was a very interresting reading for an old developper like me. If you don’t use it for yourself, you will use it to give answers to your end-users. I can only recommand everyone to buy it. Nice work guys.