I m actually developping some application around XMLRPC protocol at work.

We are using Pylons for the framework part, and i played this afternoon at setting up some testing environnement for doing doctests.

This test is a proof of concept, it 's code extracted from our internal application, it's just a starter for you. The whole is working with some tweaks.

controllers/mycontroller.py, a simple controller doing simple stuff

class MyController(XMLRPCController):
    """controller."""                           

    def index(self):
        return '\_o<'

lib/base.py, Please add the XMLRPCController import

lib/base.py:from pylons.controllers import WSGIController, XMLRPCController

Then, we are setted up to continue with tests

First of all, the doctest boilerplate:

tests/test_doctest_files.py

import doctest
from doctest import DocFileSuite

from myproject.tests import setUp, tearDown
flags = (doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE | doctest.REPORT_ONLY_FIRST_FAILURE)

def test_suite():
    return DocFileSuite(
        "test.txt",     
        setUp = setUp,  
        tearDown = tearDown,
        optionflags = flags 
    )

setUp and tearDown will have a central place as they are intialising the application.

As we can't use paste.fixture.TestApp objects with XMLRPC because it does not bind everywhere, the idea is

• launch the server somewhere in a thread
  
• use it later, as usual throught xmlrplib.
  
• We will even declare it as a global to ease the doctests writings.
• We also add a wrapper to url_for to return the host to bind to.

import os
import sys
import re
import threading
from ConfigParser import ConfigParser
from unittest import TestCase

import paste.fixture
import paste.script.appinstall
from paste.deploy import loadapp
from paste.httpserver import serve
from routes.util import url_for

here_dir = os.path.dirname(os.path.abspath(__file__))
conf_dir = os.path.dirname(os.path.dirname(here_dir))
test_file = os.path.join(here_dir, 'test.ini')

cmd = paste.script.appinstall.SetupCommand('setup-app')

cmd.run([test_file])

def setUp(test, *args, **kwargs):
    print "\t-----------------------------------------------------------------"
    print "\t---    Setting up database test environment, please stand by. ---"
    print "\t-----------------------------------------------------------------"
    config = ConfigParser()
    config.read(
        os.path.join(os.path.dirname(sys.argv[0]), '..', 'etc', 'config.ini')
    )

    infos = ConfigParser()
    infos.read(test_file)
    sinfos = infos._sections['server:main']
    wsgiapp = loadapp('config:test.ini', relative_to = here_dir)
    server = test.globs['server'] = serve(wsgiapp,





    t = threading.Thread(target=server.serve_forever)
    t.setDaemon(True)
    t.start()
    test.globs['app'] = paste.fixture.TestApp(wsgiapp)
    def url_for_wrapper(*args, **kwargs):
        lkwargs = {'protocol': 'http' ,'host':  "%s:%s" % (server.server_name, server.server_port)}
        lkwargs.update(kwargs)
        return url_for(*args, **lkwargs)
    test.globs['url_for'] = url_for_wrapper
    test.globs['url_for_orig'] = url_for

def tearDown(test):
    test.globs['server'].server_close()

class TestController(TestCase):
    def __init__(self, *args, **kwargs):
        wsgiapp = loadapp('config:test.ini', relative_to = here_dir)
        self.app = paste.fixture.TestApp(wsgiapp)
        TestCase.__init__(self, *args, **kwargs)

And finally, letz play with our doctest

tests/text.txt

>>> create_url = url_for(controller='mycontroller')

>>> import xmlrpclib
>>> s = xmlrpclib.Server(create_url)
>>> s.index()
'\\_o<'

A little but wonderfull tool in the python world is paster.

It permits you to generate with some code logics a full set of files.

You can for example create a buildout, apache configurations files and etc.

The documentation for profils are in the official minitage documentation : http://www.minitage.org/doc/rst/

What is interesting there is that i use it into minitage to generate stuff inside.

There are 2 main sort of templates:

• Profils: Something relative to a project, a database, a ldap instance (not finnished atm.), a varnish instance... Profils are applied onto an existing project !
  
• Projects: A project and a minibuild referencing to it, a zope project, a turbogears project ...

To get those templates in a minitage instance:

source minitage/bin/activate
easy_install -U minitage.paste
paster create --list-templates

 After you can for example create a plone31 projet:

paster create -t minitage.plone31zeo myproject

 And after that, because plone is slow, put inside a varnish instance

paster create -t minitage.profils.varnish myproject

 Because you need svn 1.5 but it is not packaged on your debian potatoe, just reference it in the dependencies part of your project minibuild:

dependencies=...pilwotk-1.1.6.4 ...
install_method=buildout
src_uri=http://hg.foo.net/myproject
src_type=hg
category=zope
homepage=http://foo.net
description= a plone 3.1 buildout for myproject

And install the minitage.env template:

paster create -t minitage.profils.env myproject

 This profil will create for you a small bash script that will change your environnemnent according to 'myproject' needs. PATH, LD_LIBRARY_PATH and so will be feeded with all minitage dependencies. That will enable you to use svn-1.5 when you source it (like source bin/activate)

source zope/myproject/sys/share/minitage/minitage.env
which svn
/minitage/dependencies/subversion-1.5/parts/part/bin/svn

Another interresting integrations are the zope3 and tubogears projects.

• Zope3 template realisation allowed me to make some fixes in the minitage.recipe:scripts recipe. Now, this one generates for you great launchers scripts which include ALL their respective dependencies.
  
• Something particular with tubogears is that i must have done some overhead over buildout.

You can see the full documentation there : http://www.minitage.org/doc/rst/tg_project.html

Testing the 'serve-branches' script, as mwh asked me on the previous bug report, i have now something pretty cool :)

So, something nice is that i have dropped away the ugly patch.

see : http://bzr.cryptelium.net in action.

What's next?

Michael Hudson, the main developper, told me that the idea is to drop away the loggerhead.conf and to use serve-branches as the main thing to use.

That's a good idea. Usage is simple, nothing to worry about configuration, it just does publishing after all.

Little bits i can see as improvments (always reported/planned):

• a breadcrumb in repo view to browse back
• Maybe some sort of ajaxified navigator to have the whole tree from the homepage
  

What i had done on minitage this week end:

Minitage.paste

• Improved varnish configuration
• A new plone profile to be directly mapped to a zeo instance

Minitage.recipe

• Pyc are now just regenerated when eggs are moved into the cache. Thanks to Jim Fulton and the appropriate zc.buildout's function i have borrowed :p
• easy_install method was refactored a bit.
• eggs install method code is now rock solid until the next bug.
  

Thoughts

• For a long time, i think to cache authenticated content though a http proxy-cache. This can be done with the coordination of the plone cachesetup product and varnish. This will be certainly a next improvment to the varnish template. As i will study a cache solution, next week, in my daly work, it will be normally quickly updated :)
  

There is a long time i must have done that.

IT s done now, my blog is open :)

Stay tuned for new entries.

What will may be next:

• Plone customization
• Refactor logo and design of this bare plone.
  
• Integration with cryptelium services
• Cofee and tea
  

Althought i was playing in the wonderfull world of mercurial, i 'm trying out bazaar and its tools.

The goal of the week end was to set up some repositories and a source browser on top of them.

In bazaar world, there is not much choice to achieve that. In other terms, you re really encouraged to use loggerhead.

So, i began to set up a repository, file access and so on... Then look at the loggerhead documentation. Great, there is some sort of auto publishing feature.

This is a mechanism like our beloved mercurial's hgwebdir.cgi to autopublish branches inside a top level directory.

Installation

So, the procedure i used was really simple:

Grab it

bzr branch http://bazaar.launchpad.net/%7Eloggerhead-team/loggerhead/trunk/ loggerhead.dev

Edit and point to my top level directory

cp loggerhead.conf* loggerhead.conf

vi loggerhead.conf

Set up a gentoo init script to start it and schedule for starting after apache2

$ cat /etc/init.d/loggerhead
#!/sbin/runscript
# Copyright 1999-2004 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

depend() {
        need net apache2
}

start() {
        ebegin "Starting $server_name server"
        start-stop-daemon --start  --user $user  --quiet --background \
        -p $pid_file -m  --exec "$py"  --  "$start_cmd"
        eend $?
}

stop() {
        ebegin "Stopping $server_name  server"
        "$py"  --  "$stop_cmd"  2>&1 >> /dev/null
        eend $?
}

$ cat /etc/conf.d/loggerhead
start_cmd=/path/loggerhead.dev/start-loggerhead
stop_cmd=/path/loggerhead.dev/stop-loggerhead
pid_file="/path/loggerhead.dev/loggerhead.pid"
user="apache"
py=`which python`
server_name="Loggerhead"

 Configure apache for serving both bazaar and loggerhead on the same vhost

...
Vhost basic and directories directives
...
# setting an alias to point to the bazaar directory

Alias /repos/ /path/repos/


RewriteEngine on

# mapping to /repos  if we are not browsing on loggerhead or if we are directly "branching"
RewriteCond  %{REQUEST_URI}  !^(/repos/.*)$
RewriteCond  %{REQUEST_URI}  ^(.*\.bzr.*)$
RewriteRule  ^/(.*)$         /repos/$1   [R,L]

# mapping to loggerhead otherwise
RewriteCond  %{REQUEST_URI}  !^(.*\.bzr.*)$
RewriteCond  %{REQUEST_URI}  !^(/repos/.*)$
RewriteCond  %{REQUEST_URI}  !^(/icons/.*)$
RewriteRule  ^/(.*)          http://bzr.cryptelium.net:1337/$1 [P]
RewriteCond  %{REQUEST_URI}  !^(.*\.bzr.*)$
RewriteCond  %{REQUEST_URI}  !^(/repos/.*)$
RewriteCond  %{REQUEST_URI}  !^(/icons/.*)$
RewriteRule  ^/(.*)/$        http://bzr.cryptelium.net:1337/$1 [P]

# The two [P] RewriteRules are just ProxyPass rules which allow to redirect to the loggerhead server.

Are we done yet ?

Unfortunatly not.

Loggerhead is stupid and will not publish two branches with same name.

Another thing is that it will take the branch name for display in UI.

I REALLLY do not like that. I prefer to see full pathes in the UI. That helps my little brain with tree view focus to function properly.

Why ?

This cannot be served (just b/c will be):

a/c
b/c

b/c/d

Results in

d

Fix ?

For my own use, i make my own fix :)

But i upstreamed the result there:https://bugs.launchpad.net/loggerhead/+bug/254411

This allows me to publish my repositories with path views :)

Result

http://bzr.cryptelium.net/