Far from the versions war I’m trying to understand how many people don’t use Plone “main” and choose to use project based on it. This evaluation is getting more difficult that I’m not able to recense how many projects are based on Plone. We can found some of them in the Plone.org feed and some others in pypi or in the plone.org products column.

I can easily cite :

• PloneGov – http://www.plonegov.org/
• CommunesPlone – http://www.communesplone.org/le-projet
• PloneEdu (dead)
• educommons – http://enpraxis.net/ (no main page for educommons)
• educomponents – http://wdok.cs.uni-magdeburg.de/demo/
• getPaid – http://www.plonegetpaid.com/
• Plone4Artists – http://plone4artists.org

But they would be much more.

The Plone Conference in Budapest will be a good place for these projects to gain visibility and to project leaders to talk with the evangelist team to gain a page on Plone.org to show what are their roadmap. It is difficult to find an announce done in a feed or in mailing lists few months ago. It easier to find a page one the official plone.org site if a projects column is created in it. The Plone foundation doesn’t have to fund them, but can help their communication.

Après 5 versions candidates la fondation Plone valide enfin la nouvelle mouture tant attendue du système de gestion de contenu Plone. Cette version était prévue pour sortir en avril, mais une série de bugs critiques lors de la dernière phase de validation a obligé à repousser sans cesse la date de sortie.
La fondation en a aussi profité pour revoir le look du site http://plone.org en utilisant les dernière technologie utilisée dans Plone 3.

La plateforme continue de se diversifier :

• Get Paid pour gérer votre site e-commerce
• ArchGenXML pour générer le code à partir de diagrammes UML
• Plone4Artists pour les artistes qui veulent gérer eux-mêmes leur communauté
• Communes Plone qui regroupe plusieurs communes belges qui souhaitent mutualiser le développement de leur système d’information.
• PloneGov pour l’administration – grand prix du jury aux Lutèce 2007

• Plone 3.3 sur Pypi
• La page officielle sur Plone.org
• Liste des améliorations incluses dans Plone 3.3

Cette mouture apporte plusieurs nouveautés intéressantes. Tout d’abord Vincent Fretin est devenu le nouveau coordinateur des traductions au niveau mondial. Il a aussi fait le ménage dans la traduction française qui atteint toujours les 100% de complétion depuis la sortie de Plone 2.1. Il a été aussi très actif pour la chasse aux bugs dans les releases canadidate et a été le contributeur principal de la dernière mouture de ArchGenXML 2.

Les évolutions :

• les liens externes renvoient directement vers les sites concernés ;
• les ressources (CSS, JavaScript, KSS) peuvent être maintenant gérées de manière conditionnelle (IE6Fixes.css n’est chargé qu’en présence de IE6 par exemple) ;
• l’édition en ligne est finalement désactivée par défaut, car elle est trop lourde pour IE6 ;
• le verrouillage des documents lors de l’édition est maintenant configurable ;
• l’historique des documents contient les modifications de workflow et les notes ajoutées lors de la modification des documents ;
• les évènements supportent iCalendar par défaut ;
• le chargement du fichier ZCML des extensions de Plone devient automatique.

Le site Plone utilise maintenant ( http://pypi.python.org/pypi/collective.xdv ) pour gérer son design. Ce produit permet de créer un design dans une page HTML statique et d’appliquer des règles de transformation XSLT pour adapter cette page statique à un site dynamique. Il est basé sur la technologie Deliverance. Plus d’informations sont disponibles dans le nouveau manuel de gestion des thèmes dans Plone.

I leave Alterway two monthes ago for Quadra Informatique. It is a  more generic service company that choose Plone 4 years ago, and more recently Magento. They not seemed to be very active in the  community for now because they mainly proposed patches and features inside existing products more than creating their own products. They have some good products in their boxes and we prepare some official releases for September and others for the end of the year. These projects will be released under Cecil-B license (http://www.cecill.info/licences.en.html) a BSD-like license.

I will not only developping with Plone for them… and I’m leaving Paris this week to go in Lyon. The goal is  to develop Zope/Plone as a local company in this city that choosed Zope/CPS to develop their website some years ago. For some problems that only french telecom legislation can create I will not have a regular network connexion until the end of August.

Have a good vacation and see you in September.

A quick tip useful to keep the coherence between the setup.py file of your egg and the profiles/default/metadata.xml file. The simpliest way to do this is to compute version value in setup.py from the metadata.xml. In the setup.py file remove the version = ‘x.y.z’ line by the following code:

from xml.dom import minidom

metadata_file = os.path.join(os.path.dirname(__file__),
                             'collective', 'bobo',
                             'profiles', 'default', 'metadata.xml')
metadata = minidom.parse(metadata_file)
version = metadata.getElementsByTagName("version")[0].childNodes[0].nodeValue
version = str(version).strip()

You can check it works with the command:

python setup.py --version

Few days ago Jean-Michel François proposed a useful patch for PlonePAS that can be applied for all Plone release until 3.2. Plone 3.3 will embed this patch.

How can we add this patch in a traceable way for an not so old Plone 3.1 or 3.2 ?

First, we can use the new release  of Products.PlonePAS that should be compatible with our Plone installation. The second option is to add a monkey patch in the policy product of our site. One more monkey patch…

Some projects have so many monkey patches that it is difficult to know where is the code that run your site. Martin Aspeli did a tool to handle monkey patches in an elegant way for Zope 2 and Zope 3: collective.monkeypatcher. It allows you to plug  your monkey patches with a simple ZCML directive. Later Gilles Lenfant added a control panel for Zope 2 to be able to have a visual way to follow patches with collective.monkeypatcherpanel.

How does it works?

In your buildout.cfg add :

eggs +=
   collective.monkeypatcher
   collective.monkeypatcherpanel
zcml +=
   collective.monkeypatcher
   collective.monkeypatcherpanel

To create patches add a ‘patches.py‘ file in your egg (if you have more than 2 or 3 patches you should create a directory). Our performance patch looks like this:

import copy

def enumerateUsers( self
                  , id=None
                  , login=None
                  , exact_match=False
                  , **kw
                  ):

    """ See IUserEnumerationPlugin.
    """
    plugin_id = self.getId()

    criteria=copy.copy(kw)
    if id is not None:
        criteria["id"]=id
    if login is not None:
        criteria["login"]=login

    if not kw and id:
        data = self._storage.get(id, None)
        if data is None:
            user_info = []
        else:
            user_info=[ { 'id' : self.prefix + id,
                     'login' : id,
                     'title' : data.get('fullname', id),
                     'description' : data.get('fullname', id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } ]
    else:
        users=[ (user,data) for (user,data) in self._storage.items()
                    if self.testMemberData(data, criteria, exact_match)]

         user_info=[ { 'id' : self.prefix + user_id,
                     'login' : user_id,
                     'title' : data.get('fullname', user_id),
                     'description' : data.get('fullname', user_id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } for (user_id, data) in users ]

        return tuple(user_info)

In the configure.zcml of your policy product add an include:

<include file="patches.zcml" />

The file patches.zcml will contain following code:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:monkey="http://namespaces.plone.org/monkey"
    i18n_domain="collective.monkeypatcher">

    <include package="collective.monkeypatcher" file="meta.zcml" />

    <monkey:patch
       original="enumerateUsers"
       replacement=".patches.enumerateUsers"
       docstringWarning="false"
    />

</configure>

Run your buildout, start your site and the patch is applied. You can go in the Zope Control Panel to see how many patches are already compatible with this tool.

Aujourd’hui un client nous met le couteau sous la gorge d’une manière assez peu banale. Il y a quelques années il a choisi de prendre une plateforme libre pour développer ses outils internes et une partie des services qu’il propose à ses clients.

Son choix a été de prendre Plone pour les raisons suivantes :

• pas de coût de licence
• un développement via l’interface web de Zope quasiment sans avoir à écrire de code
• une documentation abondante et une communauté réactive
• présence en France d’acteurs majeurs du développement international de la plateforme
• une licence libre (la GPL) qui lui permet de penser que la plateforme ne deviendra jamais propriétaire

Le projet a eu des hauts et des bas. Il y a eu des formations externes, de la maintenance externe, du développement interne et externe. Globalement le projet a dépassé les espérances et est passé en mode industriel il y a un an. Le client possède toutes les sources et la connaissance de son applicatif. Il est maître de son destin. Avec le recul il est satisfait de cette plateforme et de son coût.

Ayant acquis une maitrise certaine de la gestion de contenu dans son corps de métier, ce client aimerai développer un module métier qu’il utilisera pour lui dans un premier temps, mais qu’il compte aussi revendre à ses concurrents à moyen terme. Pour cela ce module sera vedu comme une technologie propriétaire. Cela soulève une série de questions politiques et juridiques en interne (attention, certains de ces arguments sont partiellement erronés ) :

1. si un site web utilise du code GPL pour fournir un service à des clients, un client peut demander à avoir accès au code source de l’application puisqu’il paie pour ce service
2. si nous fournissons des sources sous GPL notre code source ne peut être commercialisé à des tiers sans qu’ils soient obligés de reverser ce code dans le domaine public
3. si nous utilisons une extension sous GPL tout le code source devient GPL

Sur le fond je comprends la démarche de la vente d’un savoir faire que l’on veut protéger. Chez ce client tout ces constats ont amené a une réponse : pour faire du propriétaire il faut se baser sur une technologie propriétaire qui permet de faire ce que l’on veut une fois achetée.

Que faire ? Faut-il abandonner tous les développements en Plone  ? Il est tout à fait possible de réaliser un tel module propriétaire avec Zope et de l’intégrer avec d’autres systèmes sans que cela pose de problèmes de licence. Par contre, le Core de Plone 2 et de Plone 3 sont sous licence GPLv2 ce qui provoque un certain malaise. En anticipant de tels questionnements, la fondation Plone a décidé que le Core de Plone 4 serait sous licence BSD et  qu’elle accepterait que les modules d’extensions soient sous licence compatibles LPGL et GPL. Elle accepte aussi que des plateformes sous licence propriétaire soit développé à partir de Plone.

Dans notre cas précis ce questionnement est orienté : il faut vendre notre savoir faire pour générer des royalties et améliorer la rentabilité. Pour cela le choix d’une technologie libre est risquée et il faut adopter en masse une technologie propriétaire quitte à remettre en cause les développements en cours. Les coûts à justifier sont tels qu’il n’est pas possible de tolérer qu’un autre technologie puisse damer le pion à la technologie propriétaire à court terme car le choix de la technologie propriétaire est celui fait à moyen terme.

Il nous est proposé de fournir un canva pour fabriquer ce module dans la technologie propriétaire qui soit intégrable à Plone. Et dans tous les cas Plone va être abandoné. Autant nous mettre la corde au cou nous-même et attendre le bourreau.

Since few weeks we have problems to use collective.releaser when we use release-packages: packages are not uploaded to our private pypi or are published in pypi when they should be private and public packages are not published at all.
collective.releaser seems be a victim of this bug as the 0.6.2 release was never published on pypi even if the tag was done 2 months ago.

Today ready this:
collective/releaser/commands.py:L107

Line 110, when the expression is matching something the loop continues, then in the line 117 an empty list is in found sometimes.

If you add these two lines after the line 112 you can again upload your releases in cheeseshops:

if founded != []:
    break

Since few weeks plone.recipe.pound 0.5.4 was broken with an output like this:

An internal error occured due to a bug in either zc.buildout or in a recipe being used:
Traceback (most recent call last):
File "/tmp/tmpVcq-b1/zc.buildout-1.2.1-py2.4.egg/zc/buildout/buildout.py", line 1509, in main
File "/tmp/tmpVcq-b1/zc.buildout-1.2.1-py2.4.egg/zc/buildout/buildout.py", line 473, in install
File "/tmp/tmpVcq-b1/zc.buildout-1.2.1-py2.4.egg/zc/buildout/buildout.py", line 1091, in _call
File "/home/encolpe/.virtualenvs/internal/preprod/plone.recipe.pound/plone/recipe/pound/build.py", line 74, in install
installed = CMMIRecipe.install(self)
File "/home/encolpe/.virtualenvs/internal/preprod/downloads/dist/zc.recipe.cmmi-1.2.0/src/zc/recipe/cmmi/__init__.py", line 155, in install
system("make install")
File "/home/encolpe/.virtualenvs/internal/preprod/downloads/dist/zc.recipe.cmmi-1.2.0/src/zc/recipe/cmmi/__init__.py", line 27, in system
raise SystemError("Failed", c)
SystemError: ('Failed', 'make install')

This recipe directly unherit from zc.recipe.cmmi and act as a wrapper around the CMMIRecipe.install method. Most arguments used are transmitted to the recipe are self attributes or keys in self.options. The version 1.2.0 of this recipe publish the May, 18 moved the way that extra arguments are transmitted:

• zc.recipe.cmmi 1.1.x:  self.options['extra_options']
• zc.recipe.cmmi 1.2.0: self.extra_options (an empty string by default)

There’s no regression tests, neither an entry in the release history. Youenn did a new release of plone.recipe.pound to fix this new behavior.

Morality:

• if you are using zc.recipe.cmmi 1.1.x you can still use plone.recipe.pound 0.5.4
• if you are using zc.recipe.cmmi 1.2.0 or above use plone.recipe.pound 0.5.5
• other recipes using zc.recipe.cmmi may be broken

These days we are working with fabric to remove some hazard in maintenance.
Recently someone gzip a Data.fs file in production and our customer loose 2 days of work. We choose to use fabric (http://www.nongnu.org/fab/) to limit command line working to the strict minimum.

Now, almost all procedures were added in a fabric factory, from the installation of a development environment to the upgrade of the production server. In this last part, we search a solution to put a maintenance page without have to gain root privileges to restart the apache used in frontend. For static or CGI sites you can use a .htaccess file to override the global rules. For a stopped Zope server it doesn’t help much.

The goal is to use RewriteCond and RewriteRule in a such way that a static HTML file would be displayed when it exists. We can call it ‘maintenance.html‘ and rename it ‘maintenance.html-disabled‘ to let the site be displayed normally.

We start with a virtual host generated by iw.recipe.squid:

<VirtualHost *:80>
   ServerName www.gosseyn.fr

   RewriteEngine On
   RewriteLog /my/path/to/squid/parts/squid/log/rewrite_www.gosseyn.fr.log
   RewriteLogLevel 0

   CustomLog /my/path/to/squid//parts/squid/log/access_www.gosseyn.fr.log common
   ErrorLog /my/path/to/squid//parts/squid/log/error_www.gosseyn.fr.log

   ## common rules for squid rewrite rules
   RewriteRule ^(.*)$ - [E=BACKEND_LOCATION:127.0.0.1]
   RewriteRule ^(.*)$ - [E=BACKEND_PORT:8081]
   RewriteRule ^(.*)$ - [E=BACKEND_PATH:site]

   RewriteRule  ^/(.*)/$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]
   RewriteRule  ^/(.*)$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]

   ## specific rules base on cookie

</VirtualHost>

First we need to declare a document root and to bypass all security rules. We create a new path in the root of our buildout folder called ‘maintenances_pages’.

   DocumentRoot /my/path/to/buildout/prod/maintenance_pages

   <Directory "/my/path/to/buildout/prod/maintenance_pages">
       Options None
       AllowOverride None
       Order allow,deny
       allow from all

       <LimitExcept GET>
           Order allow,deny
           allow from all
       </LimitExcept>
   </Directory>

These declaration should satisfy all paranoid BOFH.
Now create a file called maintenance.html within the folder. All code (javascript, CSS and images) must be embeded, this is a limitation of this approach.
Well, how to detect that a file exist on the filesystem from apache. You can do this with the flag ‘-f’ from the rewrite module:

    RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html -f

Then redirect all pages to our maintenance page:

   RewriteCond %{REQUEST_URI} !/maintenance.html
   RewriteRule $ /maintenance.html [R=302,L]

We also need to disable the standard rewrite rules. The easier way to do it is to use ‘!-f‘.

   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)/$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]
   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]

These rules will not be applied anymore when our maintenance file will be name ‘maintenance.html’. Our goal is reached.

The whole virtualhost file look like this:

<VirtualHost *:80>
   ServerName www.gosseyn.fr

   RewriteEngine On
   RewriteLog /my/path/to/squid/parts/squid/log/rewrite_www.gosseyn.fr.log
   RewriteLogLevel 0

   CustomLog /my/path/to/squid//parts/squid/log/access_www.gosseyn.fr.log common
   ErrorLog /my/path/to/squid//parts/squid/log/error_www.gosseyn.fr.log

   DocumentRoot /my/path/to/buildout/prod/maintenance_pages

   <Directory "/my/path/to/buildout/prod/maintenance_pages">
       Options None
       AllowOverride None
       Order allow,deny
       allow from all

       <LimitExcept GET>
           Order allow,deny
           allow from all
       </LimitExcept>
   </Directory>

   ## our maintenance page
   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html -f
   RewriteCond %{REQUEST_URI} !/maintenance.html
   RewriteRule $ /maintenance.html [R=302,L]

   ## common rules for squid rewrite rules
   RewriteRule ^(.*)$ - [E=BACKEND_LOCATION:127.0.0.1]
   RewriteRule ^(.*)$ - [E=BACKEND_PORT:8081]
   RewriteRule ^(.*)$ - [E=BACKEND_PATH:site]

   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)/$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]
   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]

   ## specific rules base on cookie

</VirtualHost>

It will be difficult to include this in a recipe as we modify an existing virtualhost in a very specific way.

As usual, all feedbacks are appreciated.

Plone 3.2 and next comes with a hardcoded versions.cfg for each Plone release. For the current release:
http://dist.plone.org/release/3.2.2/versions.cfg

The Plone Upgrade Guide will show you how to transform a basic buildout created with paster:
http://plone.org/documentation/manual/upgrade-guide/version/upgrading-from-3-x-to-3.2

This solution will work until you already defined a ‘[versions]‘ section in your main buildout file. What we need is to merge the two ‘[versions]‘ sections.

For example, you current ‘buildout.cfg‘ looks like this:

[buildout]
...
eggs =
	archetypes.schematuning
	Products.CacheSetup
	Products.errornumber
	collective.recipe.omelette
	collective.workflowed
	Products.DCWorkflowGraph
	Products.PrintingMailHost
	plone.reload
	Products.PDBDebugMode
	Products.DocFinderTab

versions = versions

[versions]
zope.testing=3.5.1
zope.interface=3.4.1
Products.errornumber=1.2
archetypes.schematuning=1.1
Products.CacheSetup=1.2

If you add extends = http://dist.plone.org/release/3.2.2/versions.cfg it will be overloaded by the local section displayed above and will be simply ignored by ‘bin/buildout‘. If you run an update now all plone bundle eggs will be updated to the last published eggs (Plone 3.3b1, etc).
The solution consist to use the ‘extends’ directive to merge sections. For that we need to put your local ‘[versions]‘ section in a separate file.
‘dev-versions.cfg‘ will contain:

[versions]
zope.testing=3.5.1
zope.interface=3.4.1
Products.errornumber=1.2
archetypes.schematuning=1.1
Products.CacheSetup=1.2

The ‘buildout.cfg‘ will contain:

[buildout]
...
eggs =
	archetypes.schematuning
	Products.CacheSetup
	Products.errornumber
	collective.recipe.omelette
	collective.workflowed
	Products.DCWorkflowGraph
	Products.PrintingMailHost
	plone.reload
	Products.PDBDebugMode
	Products.DocFinderTab

extends = dev-versions.cfg http://dist.plone.org/release/3.2.2/versions.cfg

versions = versions

We can update now and verify our eggs versions:

• Plone 3.2.2
• plone.app.locales 3.2.0
• Products.CMFPlacefulWorkflow 1.4.0
• Products.CacheSetup 1.2

It is a certified Plone 3.2.2 installation with our specifics packages.