TextMate reStructuredText Bundle

Create a virtualenv for TextMate

We’re gonna need to install a few Python packages in order to make this work. That means it’s time to fire up virtualenv and make an isolated Python environment for our fix.

stewart:~ xdissent$ mkvirtualenv TextMate
New python executable in TextMate/bin/python
Installing setuptools............done.

Install pip

Virtualenv’s come pre-loaded with easy_install, but we’re going to be changing a couple of files in some packages so we should use pip to make that easier on ourselves.

(TextMate)stewart:~ xdissent$ easy_install pip
Searching for pip
[...]
Installed /Users/xdissent/.virtualenvs/TextMate/lib/python2.5/site-packages/pip-0.3.1-py2.5.egg
Processing dependencies for pip
Finished processing dependencies for pip

Install Docutils

Docutils is the package that provides the reStructuredText parsers, so we’ll definitely be needing that.

(TextMate)stewart:~ xdissent$ pip install docutils
Downloading/unpacking docutils
  Downloading docutils-0.5.tar.gz (1.3Mb): 1.3Mb downloaded
[...]
Successfully installed docutils

Install reStructuredText TextMate Bundle

TextMate has a bundle repository at http://svn.textmate.org/trunk/Bundles with a reStructuredText bundle already built. We need to download that to a place where TextMate will actually find the bundle and load it. Typically, thats ~/Library/Application Support/TextMate/Bundles.

(TextMate)stewart:~ xdissent$ mkdir -p ~/Library/Application\ Support/TextMate/Bundles
(TextMate)stewart:~ xdissent$ cd ~/Library/Application\ Support/TextMate/Bundles
(TextMate)stewart:Bundles xdissent$ svn co http://svn.textmate.org/trunk/Bundles/reStructuredText.tmbundle
A    reStructuredText.tmbundle/Commands
[...]
A    reStructuredText.tmbundle/Syntaxes/reStructuredText.plist
Checked out revision 10740.

Make reStructuredText Bundle use virtualenv

Now that we have the bundle, we need to let it know that it needs to be called from inside our virtualenv. The Preview command uses rst2html.py to generate the preview, and luckily provides a way to customize the path to rst2html.py. All we need to do is define TM_RST2HTML variable with the correct path in our .bash_profile and TextMate will find the right one.

(TextMate)stewart:~ xdissent$ echo export TM_RST2HTML=`which rst2html.py` >> ~/.bash_profile
(TextMate)stewart:~ xdissent$ source ~/.bash_profile

Reload TextMate Bundles

If you’re running TextMate at this point, you can either use the "Bundles" menu or the following command to reload the bundles and install our new reStructuredText bundle.

(TextMate)stewart:~ xdissent$ osascript -e 'tell app "TextMate" to reload bundles'

Bask in the glory

We’re done. Now we can preview or export reStructuredText easily. Open a reStructuredText file and go wild. It’s incredibly convenient to preview documents as you go, and saving them to HTML makes it simple to blog from TextMate.

(TextMate)stewart:~ xdissent$ mate ~/Documents/TextMate\ Fixes.rst

Colour My World

If you’re like me, you tend to include source code or console sessions in your articles. Pygments is a Python package that can add syntax highlighting to these snippets, which comes in handy for previewing in TextMate. Installation is slightly more advanced, but well worth setting up.

Install Mercurial

We want to hack on our Pygments package, so we’re going to want to install it in editable mode. Plus, the latest stable release doesn’t have the BashSessionLexer that I use quite a bit. Pygments uses Mercurial for source control, but I don’t have a system-wide copy of hg, so I needed to grab one real fast.

(TextMate)stewart:~ xdissent$ pip install mercurial
Downloading/unpacking mercurial
  Downloading mercurial-1.1.2.tar.gz (952Kb): 952Kb downloaded
[...]
Successfully installed mercurial

Enable the fetch command

When pip installs an editable package from a Mercurial repository, it uses the fetch command with hg. We need to enable it in our ~/.hgrc file because it’s disabled by default.

(TextMate)stewart:~ xdissent$ cat >> ~/.hgrc <<EOF
> [extensions]
> fetch =
> EOF

Get Pygments

Now we can install Pygments from Mercurial. Pip will download the source and unpack it into $VIRTUAL_ENV/src before finally creating an .egg-link file to point our interpreter to the source code.

(TextMate)stewart:~ xdissent$ pip install -e hg+http://dev.pocoo.org/hg/pygments-main@799#egg=Pygments
Checking out Pygments from hg+http://dev.pocoo.org/hg/pygments-main@799#egg=Pygments checkout from hg+http://dev.pocoo.org/hg/pygments-main@799#egg=Pygments
[...]
    Installed /Users/xdissent/.virtualenvs/TextMate/src/pygments
Successfully installed Pygments

Install the reStructuredText sourcecode directive

Activating syntax highlighting in a document requires the sourcecode directive which highlights the following block of code. We can switch out different Pygments lexers using an option after the directive:

.. sourcecode:: python

    from hartzog import VERSION
    print 'Running Hartzog %%s' %% VERSION

The rendered version looks something like this:

from hartzog import VERSION
print 'Running Hartzog %%s' %% VERSION

The sourcecode directive isn’t installed automatically, so we’ll need to add it ourselves. Directives are simply python functions that must be registered with docutils before any parsing is done. That means we can’t use rst2html.py with highlighting yet because the directive won’t be loaded. Pygments does come with the directive already in its source tree, but we’re going to have to let virtualenv know where it is, and then modify rst2html.py to register the directive before it begins parsing.

(TextMate)stewart:~ xdissent$ add2virtualenv $VIRTUAL_ENV/src/pygments/external
(TextMate)stewart:~ xdissent$ sed -i '' '/^publish_cmdline/i\
> __import__("rst-directive")
> ' `which rst2html.py`

Colorize the Preview command

What good is all this work if we can’t see the damn colors? As it stands now, pygments is marking up our code to be colorized, but the CSS to actually make the colors show up isn’t being loaded into our preview window. We have to enhance the Preview command in the reStructuredText bundle to include Pygments stylesheets. You can control which Pygments theme will be used by defining a TM_PYGMENTIZE_STYLE environment variable. I’ve created a patch that we can apply to update our bundle.

Index: Commands/Preview.plist
===================================================================
--- Commands/Preview.plist  (revision 10740)
+++ Commands/Preview.plist  (working copy)
@@ -25,6 +25,13 @@
            border: 1px #888 solid;
            padding: 0 1em;
    }' > $css
+    if [[ -f "$TM_PYGMENTIZE" ]]
+    then
+        if [[ "$TM_PYGMENTIZE_STYLE" = "" ]]
+            then TM_PYGMENTIZE_STYLE=default
+        fi
+        $TM_PYGMENTIZE -S $TM_PYGMENTIZE_STYLE -f html >> $css
+    fi
    stylesheet=$css
    tmpCreated="yes"
 fi

We want to apply the patch into the root of the bundle with patch.

(TextMate)stewart:~ xdissent$ patch -d ~/Library/Application\ Support/TextMate/Bundles/reStructuredText.tmbundle -p0 -i rst_bundle.diff
patching file Commands/Preview.plist

Tell TextMate how to use pygmentize

The patch for the Preview command requires an environment variable called TM_PYGMENTIZE that contains the path to the pygmentize script. We need to add that to our .bash_profile along with any custom style we want to use for highlighting.

(TextMate)stewart:~ xdissent$ echo export TM_PYGMENTIZE=$VIRTUAL_ENV/bin/pygmentize >> ~/.bash_profile
(TextMate)stewart:~ xdissent$ echo export TM_PYGMENTIZE_STYLE=fruity >> ~/.bash_profile
(TextMate)stewart:~ xdissent$ source ~/.bash_profile

Inform TextMate of the updated Preview command

Just like before, we want to nudge TextMate so we can check out the new preview command without restarting.

(TextMate)stewart:~ xdissent$ osascript -e 'tell app "TextMate" to reload bundles'

Patch Pygments for virtualenv

Everything should be working perfectly now in full color, but I went one step further in my setup to tweak the BashSessionLexer in Pygments. Because I use virtualenv all the time, my bash prompt usually has a virtual env name at the beginning, surrounded by parentheses. If I were to use the console option for sourcecode with a virtualenv active in the session, the lexer wouldn’t recognize my prompt and it wouldn’t get highlighted. I put together a quick patch for Pygments that correctly highlights virtualenvwrapper style prompts.

diff -r 9fe544c7818b pygments/lexers/other.py
--- a/pygments/lexers/other.py      Sat Feb 14 13:27:41 2009 +0100
+++ b/pygments/lexers/other.py      Sat Feb 28 22:33:39 2009 -0600
@@ -416,7 +416,7 @@

         for match in line_re.finditer(text):
             line = match.group()
-            m = re.match(r'^((?:|sh\S*?|\w+\S+[@:]\S+(?:\s+\S+)?|\[\S+[@:]'
+            m = re.match(r'^((?:\([A-Za-z_0-9-]+\))?(?:|sh\S*?|\w+\S+[@:]\S+(?:\s+\S+)?|\[\S+[@:]'
                          r'[^\n]+\].+)[$#%%])(.*\n?)', line)
             if m:
                 # To support output lexers (say diff output), the output

Mercurial patches are weird, so be sure to use the -p1 option when applying:

(TextMate)stewart:~ xdissent$ patch -d $VIRTUAL_ENV/src/pygments -p1 -i pygments.diff
patching file pygments/lexers/other.py

Chump and a Hoagie

That’s it. Go buck wild on some reStructuredText in TextMate dawg. You’ve earned it.