Making more and more complicated instances from server dependencies installed into the minitage tree make me relying on somehow intrusive dependencies.

 As i want minitage.paste to depend on barely nothing else that paster and the minitage core,  i have splitted such 'instances' in an additionnal egg.

 minitage.paste.extras was born.

This package borrow 2 instances at the moment:

• minitage.instances.cas: Install a CAS server
• minitage.instances.openldap: Install an openldap server

Those instances make some nice things as generating on the flag SSL certificates for enabling secured connections to your instance(s).

Thanks to my lovely minitage user remarks, i make some frenglish naming with 'profils' in minitage.paste.

Profils have been renamed to instances as it is more relevant.

paster create -t minitage.profils.env project

become:

paster create -t minitage.instances.env project

In the the last weeks, I carried out some changes to minitage.paste.

Now, there are some new profils out there:

• minitage.profils.mysql : a profil to initiate a mysql database right into your project.
• minitage.profils.openldap : a profil to initiate an openldap instance right into your project.
• minitage.profils.pasteinitd: a profil to generate an init script to launch a server configured via Pythonpaste configuration
  

There also some changes on the postgresql profile.

Now:

• You have your system user admin of the database. But you can connect only via localhost.
• The user created via the paster is only administrator on the database, but not on the database server. He can connect from localhost without password, but he must authenticate from other locations.

Lately, i ran and port problematic packages under FreeBSD. All the packages now seem to install without particular problems.

The tested platform is a FreeBSD-7.1 RELEASE 64 bits.

There is pretty much a long time i wanted to switch my stuff on the BSD License.

More as an habit than a conviction, my source files where header pollutated by the GPL License.

For me, and in my opinion, the GPL is not as free as it claims to.

Why ? Because the final user cannot do what he want with your code.

Some debate will qualify that as communism  (GPL) vs anarchy (BSD). I prefer anarchy :).

Of course i use the "modified BSD license" which is compatible with the GPL.

Some on the concerned:

• minitage.core
• dj.paste
• minitage.recipe
• minitage.paste
• cluereleasemanager.paste
  

As part of a new project deployment plan, we needed to deploy mapscript.

Python-MapScript are the python bindings to the underlying mapserver library.

Where the pain begins is that there is no egg available for it, but an old fashioned distutils distribution;.

My bits there were to make some quickly egg compatible on most unixes. My will is to make it nicer and be integrated on the mapserver trunk code.

You can find the code there : http://git.minitage.org/git/others/mapscript/

And play with the egg:

easy_install mapscript 

You must know that mapserver-config must be available in your $PATH.

It is now an hassle to build all the minitage dependencies by hand.

Naturally, the idea to set up some tinderbox like buidbot comes to my mind.

More over, i had a though of generating some binaries to avoid the compilation time for users who don't want to/can't compile.

The lead was to use collective.buildbot, a great recipe to integrate buildbot with zc.buildout deployment.

The result is there : http://git.minitage.org/git/minitage/buildbot/

The idea was to generate a "collective.buildbot" project per dependency. To achieve that, i made a script to generate those projects from the current official minilays.

Compilation is triggered via manual activation (irc, web) but also by a cron scheduler (nearly each 3 hours)

I made a patch to the recipe to have multiple hours values (* 1,2,3 * * *) cron entries, see http://git.minitage.org/git/minitage/buildbot/tree/collective.buildbot.multiple.patch

The buildbot can be viewed there : http://www.minitage.org/buildbot/waterfall

Next steps are to finnish the "binaries dealing" in minitage.core and the mirrors setup, and minitage users will be able to use binaries and no compilation anymore :)

Thanks to a workmate, Jean-Philippe, who shared me a little space of it's Osx Leopard based laptop, i could test minitage on Leopard.

I was surprised to not see a lot of breakages. Most of the packages build fine right now.

I was just anooyed with 'MACOSX_DEPLOYMENT_TARGET' bugs i introduced with previous minitage versions, where i didnt know already all the impact of using it.

So, now, is minitage ready for mac ?

I think, Yes as almost everything inside minitage can install without problems. But ..., it needs to be tested a lot more :)

And think, that you must run OSX Leopard. Prior versions of MacOSX wont be supported. Indeed, i need some linker features which need a lot of rewrites of minitage composants  to be compliant with the old mac linker. (-Wl,-rpath -Wl,... -> -rpath something). Whereas, now, a lot can be fixed in one place, the project has history and there is a lot of places to fix and test to rewrite this "flag issue". But if you want, port it ;)

One recommandation if you use minitage on OSX is to use the boostrapper to get the base python compiled with the right options.

If you get some obscure  errors like:

distutils.errors.DistutilsPlatformError: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.3" but "10.5" during configure

use the bootstrapper ;)

 You ll need also xcode and some tools installed, see on the minitage installation documentation : http://minitage.org/installation.html

I finnished ysterday to make the neccessary patches to make compile all dependencies on the minitage tree on linux 64 bits.

The guest OS is a debian etch 64.

There were some trivial fixes, mainly the whole packages compiled without any problems.

One reason among others  why i love gentoo.

• I got some troubles packaging a package on minitage for its x64 port.
• I looked the gentoo's relevant ebuild.
• I found some interresting patch, i integrated it on the libjpeg buildout.
• I launched the minimerge dance, Et voila, bug resolved !

Thx gentoo and thx gentoo developpers :-)

This website just got updated to latest plone available... There may be bugs with quills ... :)

I did not find the time earlier to post about it.

Minitage 1.0 is out for a week now.

What you can find there ?

1. Documentation has been drasticly improved
   See by yourself:
   
   • minitage
   • minitage.recipe
   • buildout.minitagificator
   • dj.paste
   • minitage.paste
     
   • minitage.core
2. Trac got Git support
3. There are cgit and gitweb instances configured against all minitage repositories
   
4. A lot of bugfixes, as always
5. Recipes have been rewritten a bit to stick to zc.recipe.* implementations
6. minitage.recipe egg based recipes got a fresh touch:
• They do not totally rely any more on easy_install dependency system, just on their declared dependencies (extras are also supported). Thas was something i disliked with zc.recipe.egg, potentially, the easy_install call can install something, which we would'nt. And also, in some cases, the extras requirement could be wiped out. All are bugs reported upstream on the zc.buildout launchpad bugtracker.
• Their installer should now try not to look in the site-packages first, but rather the eggs cache to avoid conflict errors.
  
• I also implemented an adaptation of the zc.buildout redo_pyc function which seems to be somehow slow.
  
7. Welcome to the buildout.minitagificator egg which help minitage to conquer the world with replacing some well known recipes with their equivalent minitage implementations with some violent monkey patches. It's a buildout extension.
   
8. The "static fetcher" has been improved and bug fixed
9. There some new cool API functions like a search function which take regexes to look for minibuilds in the available minilays
   
10. minitage.paste got a great step forward with all pastes
11. All templates now have now minitage layout as an optionnal dependency. That means that you do not need minitage to generate templates, however, the normal use is to use them in minitage.
    • Plone32 template:
      • Predefined set of products to install
      • A developper mode where you have aditionnal tools installed (ptprofiler, collective.autorestart on linux, and so on) and debug mode turned on by default
      • wsgi configuration and launchers
      • A working filesystemstorage integration
      • You can use either RelStorage, ZEO or classical ZODB access
        
      • You can give to paster whatever you want to custom the buildout generation by some predefined keys.
    • Zope3 template:
      • wsgi configuration and launchers (paster and Zdaemon based)
        
      • You can use either RelStorage, ZEO or classical ZODB access
      • You can give to paster whatever you want to custom the buildout generation by some predefined keys.
    • Django template:
      • wsgi configuration and launchers
      • GeoDjango support
      • dj.paste (http://pypi.python.org/pypi/dj.paste) integration. This paster factory enables you to have multiple django instances in the same pipeline
        
      • A basic application created when you generate your project to quickstart.
      • You can give to paster whatever you want to custom the buildout generation by some predefined keys.
    • Nearly all templates which have WSGI support got Spawning integration. Spawning is an über performant python based WSGI server.
      
    • postgresql, varnish, varnish2, and env profils:
      • Can run now without minitage or any project installed
    • The new paste-initd profil:
      • it generates an initscript wrapper to trigger a "paster serve" from some paste configuration of your choice

What are the minitage projects and profiles, after all ?

All are paster (see PasteScripts on pypi) templates.

Projects target to quickstart an application or a project like a plone3 or a django website.

Profiles, on the other side, are a piece of integration, targeted to be in general integrated into a minitage based project. That can be a postgresql database instance, or a shell environement file with has all minitage dependencies, that you can source when playing with your project.

I made for a workmate yet another django paste's http factory.

The goals were to integrate its way to work, with setting some DJANGO_SETTINGS_MODULE key in the environment and launch the "bidule", and letz play.

The challenge was, ..., to be simple or this workmate will never use this stuff. She was used to tell only about django-admin runserver :).

One part of the challenge was to show some interessant things that we can do with paste configurations, like having different but closer configurations for production or development mode.

Thus, including showing the nice "WebError#eval" application where you can debug your messy code though the web with an interactive debugger.

But, right to the topic, the dj.paste stuff:

• It enables you to have multiple tracs in the same pipeline
• You can integrate it in an exiting paste configuration, it's the goal after all :)
• It 's simple to use, an a sample paste configuration can be:
  

[composite:main]
use = egg:Paste#urlmap
/ = foo

[app:foo]
use=egg:dj.paste
django_settings_module=foo.settings

I worked recently on merging my collective.anonymousbrowser back to zc.testbrowser.

It's definitivly seems to run fine with python >= 2.5.

Points of the merge were large:

• Integrate the proxy balancing capability
• Integrate the user agent faking capability
  
• Add tests where it was missing
• Improve the "real" class
  
• Make the real class mozrunner powered, that's free you to configure a firefox, and just let the code configure a default profile for you.
• Integrate the existing "screenshot" javascript function as a class method. Thats enables you to do browser.screenshot() and that makes a screenshot of the running firefox.
  

But, something i hadn't tested, ..., was the python 2.4 support. Both zc.testbrowser and mozrunner have problems with it...

I will try to make the next commits fix that if possible.

As it's not yet officially released., you can try the code there:

   svn://svn.zope.org/repos/main/zc.testbrowser/branches/fastreal_screenshots_mozrunner_proxys

I have a crappy hard drive which claims to be UDMA-66 compatible, however, its an old PIO4 based one ...

After successfully upgrading the system to Freebsd 7.1 this week end, at 3.30AM, i can't boot :(

The driver want to go UDMA, the disk doesn't understand.

No mean in the bios to force that... And no more internet because this box is the home router !

What to do ?

Go to use my 3G phone to get my ISP DNS because ... the funny "freebox" is customizable throught the web, on the free.fr website.

You have no local access on it.

So now, i have some route and DNS setted up, i can go to read

man loader(8)
man ata(4)

And now, i know some interresting new things:

Disable dma:

set hw.ata.ata_dma=0

Prompt for root partition when the partition location changed:

set boot_askname

During last weeks, i have switched to git minitage, and by default all come from it.

The failure here is that my git fetcher is not very sympathic with older versions of git (<1.5).

But on debian and red hat legacy, it's a pain in the ass to have relativly new stuff !!!

To resolve that, with workmates, we come to the decision to fetch packages over http.

And now, it's done. Minimerge codebase is 'HttpCompliant' in sense that defaults minilays and packages are fetched via the minitage static fetcher.

This one just use urllib, and you can even set proxies in your minimerge.cfg

To be up-to-date:

easy_install -U minitage.core
mv etc/minimerge.cfg etc/minimerge.cfg.old
cd minilays
mkdir ../old
mv meta misc eggs dependencies ../old
minimerge -s

A teamate wanted to have a binary like the package manager he was used to use.

Its done, now instead of calling minimerge, you can use either:

• minimerge
• yasty
• apty
• pacmerge
  

For those who get annonying troubles installing reportlab 2.3 with the
distribution available on pypi, i made another distribution [1].
To achieve this, i have just done a checkout from their svn TAG (2.3) and
re-executed the sdist dance.

The troubles i had personnaly was something like that :

	"/usr/bin/ld: cannot find -l_renderPM_libart"

Here is a sample buildout using either minitage.recipe:egg or zc.recipe.egg to
install it:

[buildout]
versions=versions
parts=
    eggs
    zceggs
[versions]
reportlab=2.3

[eggs]
recipe=minitage.recipe:egg
eggs=
    reportlab
find-links=
    http://distfiles.minitage.org/public/externals/minitage/

[zceggs]
recipe=zc.recipe.egg
eggs=
    reportlab
find-links=
    http://distfiles.minitage.org/public/externals/minitage/

[1] http://distfiles.minitage.org/public/externals/minitage/reportlab-2.3.tar.gz

You can now check out and test buildout.minitagicator.

This is a little extension for buildout that enables you to override/monkey patch some buildout stuff to use minitage internals instead.

If you like minitage, if you like the minitage recipes, you ll enjoy it!

See http://pypi.python.org/pypi/buildout.minitagificator for more information :)

To follow zc.buildout new changes, i have decided to make all minitage recipes more "zc.recipes" compliant.

Now, those recipes share the same API and you an use them "really" as the original zc.recipes..

Another iinterresting point is that i never liked the way to compute dependencies.
Another point was to allow easy_install to get automaticly our stuff out.
To correct all that weird, i have rewritten the dependencies calculation not to let easy_install
do anything but install one thing at a time, and with versions and extras checks.

A part of the improvments:

• versions are really fixed, imperative way.
  
• dependencies are better handled
• egg's extras requirements (like something.foo [test, zcml]) are really well installed with all their dependencies
• develop eggs get also their dependencies

As a really important part for minitage to conquer the world, minitage paster templates must rock.

So feel free to install and test the brand new zope3 template.

In an activated minitage instance:

easy_install minitage.paste
paster create -t minitage.zope3 z3
minimerge z3
cd zope/z3
bin/z3-ctl fg

Plone32 was written before the recipes rewrite, it should work but i must revisite it. Should be alvalaible tomorow!

If you didnt know minitage.paste, you can also check the other templates and projects

paster create --list-templates|grep minitage

And visit the pypi and minitage.org page :)

Just an entry to thank smplayer developers for their great piece of software!

/me just got another subtitle  via the opensubtitles.org smplayer's menu entry.

I really need this feature as i am not already fluent in japanese :(

As a part of a recent discussion i had on irc with Stephan Richter, one of my next works will be to merge zc.testbrowser with collective.anonymousbrowser.

I worked on collective.anonymousbrowser to allow the use of proxys and proxy balancing over zc.testbrowser proxys.

Another interressant feature fir the project was also to retry urls if there were errors.

I made also some bugfix and improvments on the zc.testbrowser.real.Browser side like using mozrunner to control firefox, loading directly the js file if we are local..

What i want to do now is to port all the interresting things back to zc.testbrowser.

Hope i will have time for that this week.

I recently add some GIS packages (pgrouting) and their dependencies.

Here is a little list of what minitage can offer to you:

• bzip2, zlib
• cmake, scons
  
• libwww, curl, libssh, openssl,
  
• git, subversion, mercurial, bazaar
  
• gdal, geos, pgrouting
  
• postgresql, postgis
• openldap
• python's, PIL, psycopg
  
• libxml, libxslt and expat
• libtiff, libjpeg, libpng, libgif
• etc.

And next to come:

• CGAL
• GAUL
  

There some points i have added to minitage since october:

• I moved minilays, dependencies and code on git because mercurial forest extensions really lacks maintenance
• I have done some maintenance over minitage.paste's templates (plone25, env, postgresql for postgresql-8.3) and added some (plone32svn, pylons)
• minitage.recipe received a lot of bugfixes and enhancements like the minitage.recipe:fetch entry point
• minitage.core has also been bugfixed specially for the git wave
• eggs have been buildoutified and testrunnerified
  

Those are the features i ll release on the next days:

• A brand new plone 32 paster
• A revisited zope3 paster
• minitage.recipe:eggs now support develop eggs dependencies installation
• minitage.recipe:eggs now has the same options and API that zc.recipe.egg for their common parts
  
• minitage.recipe:scripts calls minitage.recipe:eggs prior to the script generation, there is no need anymore to have too redundant parts in buildouts
  
• minitage.buildout extension is a buildout extension that monkey patches zc.recipe.egg with minitage.recipe.egg
• the bootstrapper script is refreshed :
• update all sotfwares to lastest versions
  
• includes git-1.6 which is required for minitage to fetch its core packages

TODO as an though for contributors:

• update doc for git changes
• update doc to explain how to move an existing buildout to a minitage compliant one
• documente the recipes