Author: xdissent February 24th 2009 11:23 am
Keywords: Paver
I’ve been a fan of Paver since the first time I read about it. It gives me all of the control I want with just enough of an implied structure to keep things sane and easy. The problem was, I couldn’t get the latest version, which had many shiny new features. You know how I feel about shiny, so I obviously needed to find a way to run Paver’s trunk, lest I invalidate my work by targeting the almost-irrelevant current version.
Paver is such a flexible tool, it can present a chicken-and-egg packaging dilemma by relying heavily on itself to build… itself. The packaged 1.0a1 was having trouble installing via easy_install and pip so I had to figure out a way to build my own package. Unfortunately Paver’s trunk has neither paver-minilib.zip nor setup.py which means to get things going, you have to use the current version of Paver. Of course, the new alpha is so different, that the current release can’t handle it and the build fails. I finally figured out a way to get a clean trunk checkout installed and set out to squash the bug that got 1.0a1 recalled in the first place.
After poking around for a while, I believe paver’s option parsing methods are confusing distutils commands. It has to do with the way that options are translated from the command line to Paver’s options. I updated a relevant issue on Paver’s project page and attached a patch that fixes the problem and allows Paver to build itself cleanly.
The Fix
Using virtualenv, virtualenvwrapper, and Paver SVN r37, I set up a clean build environment for paver like so:
stewart:~ xdissent$ cd ~/Code
stewart:Code xdissent$ svn co http://paver.googlecode.com/svn/trunk/ paver-trunk
A paver-trunk/.bzrignore
A paver-trunk/LICENSE.txt
[...]
A paver-trunk/paver/doctools.py
A paver-trunk/paver/virtual.py
U paver-trunk
Checked out revision 37.
stewart:Code xdissent$ cd paver-trunk
stewart:paver-trunk xdissent$ mkvirtualenv paver-tmp
New python executable in paver-tmp/bin/python
Installing setuptools............done.
(paver-tmp)stewart:paver-trunk xdissent$ add2virtualenv `pwd`
(paver-tmp)stewart:paver-trunk xdissent$ python distutils_scripts/paver minilib generate_setup
Paver 1.0a1
---> paver.misctasks.minilib
Generate paver-minilib.zip
---> paver.misctasks.generate_setup
Write setup.py
(paver-tmp)stewart:paver-trunk xdissent$ mkvirtualenv paver-trunk
New python executable in paver-trunk/bin/python
Installing setuptools............done.
(paver-trunk)stewart:paver-trunk xdissent$ rmvirtualenv paver-tmp
(paver-trunk)stewart:paver-trunk xdissent$ python setup.py develop
python setup.py develop
Paver 1.0a1
warning: no files found matching '*' under directory 'paver/docs'
At this point I have Paver 1.0a1 installed in development mode, so the new paver binary is on my path AND changes I make to ~/Code/paver-trunk are effective immediately.
To demonstrate the problem, I created options_test.py. Here’s a simple task with some options:
from paver.easy import *
options(
show_opts=Bunch(
foo_bar='default foobar',
no_foo=False,
),
)
@task
@cmdopts([("foo-bar=", "f", "Foo bar value"),
("no-foo", "n", "Foo negation")])
def show_opts(options):
print options.show_opts
Since we have Paver 1.0a1, we can run the task with the -f option. (Somebody said somethin about shiny new features.)
(paver-trunk)stewart:tmp xdissent$ paver -f options_test.py show_opts
Paver 1.0a1
---> pavement.show_opts
Bunch(foo_bar='default foobar', no_foo=False)
(paver-trunk)stewart:tmp xdissent$ paver -f options_test.py show_opts --foo-bar='new foobar' --no-foo
Paver 1.0a1
---> pavement.show_opts
Bunch(foo-bar='new foobar', foo_bar='default foobar', no-foo=True, no_foo=False)
It becomes clear that cmdopts definitions will never override default options for this task. We can’t use foo-bar as an option name in our Bunch, nor could we access the runtime value using options.show_opts.foo-bar. It turns out distutils and setuptools already noticed this problem and convert hyphens to underscores when determining the storage variable’s name for an option. I’ve attached a small patch for Paver SVN r37 that automatically makes this conversion.
(paver-trunk)stewart:tmp xdissent$ patch -d ~/Code/paver-trunk -p0 -i options_fix.diff
patching file paver/setuputils.py
patching file paver/tasks.py
(paver-trunk)stewart:tmp xdissent$ paver -f options_test.py show_opts
Paver 1.0a1
---> pavement.show_opts
Bunch(foo_bar='default foobar', no_foo=False)
(paver-trunk)stewart:tmp xdissent$ paver -f options_test.py show_opts --foo-bar='new foobar' --no-foo
Paver 1.0a1
---> pavement.show_opts
Bunch(foo_bar='new foobar', no_foo=True)
That’s more like it. I further tested the patch to confirm that distutil commands receive their options correctly and that the easy_install problem has been solved.
(paver-trunk)stewart:paver-trunk xdissent$ paver minilib
Paver 1.0a1
---> paver.misctasks.minilib
Generate paver-minilib.zip
(paver-trunk)stewart:paver-trunk xdissent$ mkvirtualenv paver-tmp
New python executable in paver-tmp/bin/python
Installing setuptools............done.
(paver-tmp)stewart:paver-trunk xdissent$ python setup.py sdist
Paver 1.0a1
warning: no files found matching '*' under directory 'paver/docs'
(paver-tmp)stewart:paver-trunk xdissent$ easy_install dist/Paver-1.0a1.tar.gz
Processing Paver-1.0a1.tar.gz
Running Paver-1.0a1/setup.py -q bdist_egg --dist-dir /var/folders/LU/LUPONriBGYepwd3i9FaBB++++TI/-Tmp-/easy_install-GnYR4C/Paver-1.0a1/egg-dist-tmp-iTfLMZ
Paver 1.0a1
running bdist_egg
running egg_info
[...]
Adding Paver 1.0a1 to easy-install.pth file
Installing paver script to /Users/xdissent/.virtualenvs/paver-tmp/bin
Installed /Users/xdissent/.virtualenvs/paver-tmp/lib/python2.5/site-packages/Paver-1.0a1-py2.5.egg
Processing dependencies for Paver==1.0a1
Finished processing dependencies for Paver==1.0a1
(paver-tmp)stewart:paver-trunk xdissent$ which paver
/Users/xdissent/.virtualenvs/paver-tmp/bin/paver
(paver-tmp)stewart:paver-trunk xdissent$ cd tmp
(paver-tmp)stewart:tmp xdissent$ paver -f options_test.py show_opts
Paver 1.0a1
---> pavement.show_opts
Bunch(foo_bar='default foobar', no_foo=False)
(paver-tmp)stewart:tmp xdissent$ paver -f options_test.py show_opts --foo-bar='it works' --no-foo
Paver 1.0a1
---> pavement.show_opts
Bunch(foo_bar='it works', no_foo=True)
Hopefully 1.0 will come out soon, but until then you can download the patch and apply it from within trunk:
$ patch -p0 -i options_fix.diff