Server Hygeine

There’s no reason even a fairly complex system like a Redmine installation can’t play nice with other system services, although it takes some planning. Much misinformation exists on the Web about running Redmine in particular, so we must tread very lightly to sidestep the pitfalls of a shaky system architecture. I’m of the mind that you should fully understand the intricacies of a piece of software before you attempt to run it – or at least understand clearly that you do not need to worry yourself with certain details – and hopefully when this setup is completely we’ll understand not only what we’ve done, but why we’ve done it. That’s the key to building a solid system and is of the highest priority.

Different elements of the system should be reused where appropriate as well. We want to only use the system-wide versions of things like Apache2, since they’ll probably be used for other services too and it makes no sense to double your load when you don’t have to. By the same token, we don’t need to install services which are easily outsourced to existing services or the cloud (like Gmail). In reality, our Redmine server might be moonlighting as a DNS server, firewall, development box – whatever – and we need to respect that. A general policy of software and user isolation along with well controlled shared resource management will ensure we aren’t wasting our time on a server that will need to be rebuilt later to play some additional role.

Stacks

A “stack” is a complete software system packaged so that, once installed, it operates in a self contained environment that will not effect, or be effected by, the rest of the host system. You generally don’t have to know much more than how to run an installer script and you can be the proud admin of your very own LAMP/Rails/etc server!

Stacks as a concept are almost prehistoric yet they remain an effective means of reliable package installation. The more complex the package, the more appealing a stack distribution becomes. The author may finely tune the most fragile system and distribute it as a stack without having to worry about it breaking in most cases.

A Redmine stack is a particularly appealing alternative to fighting Ubuntu’s awkward Rails packages that are notoriously difficult to use and maintain. Bitnami’s Redmine stack was chosen almost randomly, and I’m sure there are other options out there. Bitnami has a great reputation though, and their conventions are very clean and logical in my opinion.

To be honest, I’m a total Ruby-phobe. Not bringing that up would be lying through omission, and it’s a major reason I’ve decided to go with a stack - it’s a turn-key solution to a problem I (care to) know extremely little about otherwise. If you’re not sold on the idea by my humility, feel free to substitute your own Redmine installation process. Most of the general setup steps will be similar anyway.

Fully Qualified Domain Name

Log into the non-root user account on the server via the console or an SSH session. Check the server’s hostname to make sure a FQDN is provided:

xdissent@dev:~$ hostname
dev.local
xdissent@dev:~$ hostname -s
dev
xdissent@dev:~$ hostname -f
dev.local

If for some reason the output of hostname does not give a FQDN, you might have to adjust the contents of your /etc/hosts file and/or your /etc/hostname file. Your host must also be accessible to itself, so a quick ping is in order.

xdissent@dev:~$ head -3 /etc/hosts
127.0.0.1       localhost
127.0.1.1       dev.hsd1.tn.comcast.net.        dev
127.0.1.1       dev.local
xdissent@dev:~$ cat /etc/hostname
dev.local
xdissent@dev:~$ ping dev.local
PING dev.local (127.0.1.1) 56(84) bytes of data.
64 bytes from dev.hsd1.tn.comcast.net. (127.0.1.1): icmp_seq=1 ttl=64 time=0.021 ms
64 bytes from dev.hsd1.tn.comcast.net. (127.0.1.1): icmp_seq=2 ttl=64 time=0.030 ms
^C
--- dev.local ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.021/0.025/0.030/0.006 ms

You may need to tell the server to take note of the changes in /etc/hostname by running sudo hostname -F /etc/hostname. It’s pretty important to get your FQDN set up correctly before you continue the installation, so double check everything now.

Development Machine Access

At this point you should also be able to ssh into the server from your local development machine. If you can’t ssh yet but are logged in via the console, make sure the openssh-server package is installed by running sudo apt-get install openssh-server. If you’re using a FQDN that doesn’t have real-world DNS entries, add the FQDN to your development machine’s /etc/hosts file:

Stewart:~ xdissent$ echo <server-ip> <server-fqdn> | sudo tee -a /etc/hosts > /dev/null

Sudo Configuration

Finally, you must ensure that your non-root user on the server can gain access to other non-root user accounts. Some Ubuntu virtual images come with more restrictive sudoers configurations that only let your user sudo to root. As a simple test, try any random command as a random system user (who has a shell defined). For example, the www-data user is a decent test case:

xdissent@dev:~$ sudo -u www-data ls
Sorry, user xdissent is not allowed to execute '/bin/ls' as www-data on dev.local.

If access is denied, you must change the sudoers file using the command sudo visudo. A common fix is to append %admin ALL=(ALL) ALL to the end of the file, and then add your non-root user to the admin group if not already a member.

Required Packages

The whole idea behind using Ubuntu is to take advantage of its conveniences, like the package system. The Redmine stack comes with its own version of many standard system services which we’re going to disable in favor of the system versions. That way, we don’t have multiple instances of the same service consuming precious resources, and we move further towards a generic system that doesn’t rely on the stack for anything but the most specialized of services.

The standard Ubuntu services are easy to install using the various package management tools. Chances are, you’ve got many of these packages already installed, but there are so many different configurations out there, it’s easiest to just try to install them all in one go. Don’t forget to update your Apt sources:

xdissent@dev:~$ sudo apt-get update
Hit http://security.ubuntu.com karmic-security Release.gpg
Ign http://security.ubuntu.com karmic-security/main Translation-en_US
[...]
Hit http://us.archive.ubuntu.com karmic-updates/mutiverse Packages
Hit http://us.archive.ubuntu.com karmic-updates/mutiverse Sources
Reading package lists... Done
xdissent@dev:~$ sudo apt-get install apache2 mysql-server git-core python-setuptools git-core build-essential acl
Reading package lists...
Reading state information...
The following extra packages will be installed:
[...]

xdissent@dev:~$ cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid -o value -s UUID' to print the universally unique identifier
# for a device; this may be used with UUID= as a more robust way to name
# devices that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
# / was on /dev/sda1 during installation
UUID=0c23e780-3453-4b2b-b42c-a1811722e941 /               ext4    acl,errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
UUID=e3eb3872-b203-46e9-b020-b3caf72ccf99 none            swap    sw              0       0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto,exec,utf8 0       0
/dev/fd0        /media/floppy0  auto    rw,user,noauto,exec,utf8 0       0

xdissent@dev:~$ mount | grep acl
/dev/sda1 on / type ext4 (rw,acl,errors=remount-ro)

xdissent@dev:~$ sudo easy_install pip
Searching for pip
Reading http://pypi.python.org/simple/pip/
[...]
Installed /usr/local/lib/python2.6/dist-packages/pip-0.7.1-py2.6.egg
Processing dependencies for pip
Finished processing dependencies for pip

xdissent@dev:~$ sudo pip install virtualenv
Downloading/unpacking virtualenv
  Downloading virtualenv-1.4.8.tar.gz (1.5Mb): 1.5Mb downloaded
[...]
Successfully installed virtualenv
Cleaning up...

Create a redmine user

A dedicated user is not technically required to run the Bitnami Redmine stack. In fact, I first ran it as www-data and it worked like a charm. Problems arose when trying to integrate Gitosis, and it became obvious that it’s just easier and cleaner to go ahead and set up a new system user for the redmine application. This configuration assumes the Redmine stack, and thus, the dedicated Redmine user’s home directory, will reside at /opt/redmine. Feel free to adjust this path at your whim. The obvious choice for the user name was redmine, but you can change that if you must as well:

xdissent@dev:~$ sudo adduser --system --shell /bin/bash --gecos 'Redmine Administrator' --group --disabled-password --home /opt/redmine redmine
Adding system user `redmine' (UID 106) ...
Adding new group `redmine' (GID 115) ...
Adding new user `redmine' (UID 106) with group `redmine' ...
Creating home directory `/opt/redmine' ...

Install Bitnami Redmine Stack

The Bitnami Stack system is super easy to get running. All you have to do is download and run the installer and away you go. Before we get started, ensure that the system Apache2 and MySQL services have been stopped. That way the default values will be used by the Bitnami installer, which are closer to the values we’ll ultimately want.

xdissent@dev:~$ sudo /etc/init.d/apache2 stop && sudo /etc/init.d/mysql stop
 * Stopping web server apache2
   ...done.
 * Stopping MySQL database server mysqld
   ...done.

Download and run the Bitnami Redmine Stack installer. Choose to install into /opt/redmine, or whichever folder you chose for the redmine user’s home. Do not choose to set up an SMTP server at this time, because we will handle that with Gmail later. Be sure to use sudo -u redmine to run the installer, so the file ownership will be correct.

xdissent@dev:~$ wget http://bitnami.org/files/stacks/redmine/0.9.3-0/bitnami-redmine-0.9.3-0-linux-installer.bin
--2010-05-03 18:34:05--  http://bitnami.org/files/stacks/redmine/0.9.3-0/bitnami-redmine-0.9.3-0-linux-installer.bin
Resolving bitnami.org... 216.235.167.25
Connecting to bitnami.org|216.235.167.25|:80... connected.
[...]
2010-05-03 18:38:48 (550 KB/s) - `bitnami-redmine-0.9.3-0-linux-installer.bin' saved [159459281/159459281]
xdissent@dev:~$ chmod 755 bitnami-redmine-0.9.3-0-linux-installer.bin
xdissent@dev:~$ sudo -u redmine ./bitnami-redmine-0.9.3-0-linux-installer.bin
----------------------------------------------------------------------------
Welcome to the BitNami Redmine Stack Setup Wizard.
Created with an evaluation version of BitRock InstallBuilder

----------------------------------------------------------------------------
Installation folder

Please, choose a folder to install BitNami Redmine Stack

Select a folder [/home/xdissent/redmine-0.9.3-0]: /opt/redmine

----------------------------------------------------------------------------
Create Admin account

BitNami Redmine Stack admin user creation

Login [user]: admin

Password :
Please confirm your password :
Your real name [User Name]: Redmine Administrator
[...]
Please wait while Setup installs BitNami Redmine Stack on your computer.

 Installing
 0% ______________ 50% ______________ 100%
 #########################################

----------------------------------------------------------------------------
Setup has finished installing BitNami Redmine Stack on your computer.

Launch RedMine application. [Y/n]: Y

Info: To access the BitNami Redmine Stack, go to
http://localhost:8080 from your browser.
Press [Enter] to continue :

That’s it! Redmine is running! You could open up http://<server-fqdn>:8080/redmine/ and bask in the glory – or you could grow a pair and just forge ahead with the setup. That’s what I thought, buddy.

A Brief Note on Redmine Management

Bitnami provides a completely self contained Redmine stack with tightly coupled components that require a very specific (but minimal) shell environment to work together properly. For that reason, Bitnami includes a use_redmine script, which sets up the correct environment and runs a shell that is guaranteed to work with the Bitnami system. The use_redmine script should be run once before any Redmine management task. If you invoke a management script from an environment other than the one set up by use_redmine, you run the risk of corrupting your system. All Redmine management should also be done as the redmine user for obvious reasons. The use_redmine script is convenient in this regard, because you only have to run sudo once per administration session:

xdissent@dev:~$ cd ~redmine
xdissent@dev:/opt/redmine$ sudo -H -u redmine ./use_redmine
bash-4.0$ whoami
redmine

The Redmine stack services can be managed using the ~redmine/ctlscript.sh script. It follows the familiar ctlscript.sh <start|stop|restart|status> <service> syntax and, of course, must be run after use_redmine.

Reconfigure the Redmine Stack to Use System Services

Firstly, stop all Bitnami stack services that are currently running. We’re going to completely reconfigure Redmine anyway, and we won’t be needing any of the other services from this point on.

bash-4.0$ pwd
/opt/redmine
bash-4.0$ ./ctlscript.sh stop
/opt/redmine/subversion/scripts/ctl.sh : subversion stopped
Syntax OK
/opt/redmine/apache2/scripts/ctl.sh : httpd stopped
stopping port 3001
stopping port 3002
/opt/redmine/mysql/scripts/ctl.sh : mysql stopped

bash-4.0$ exit
exit
xdissent@dev:/opt/redmine$ sudo /etc/init.d/mysql start
 * Starting MySQL database server mysqld
   ...done.
 * Checking for corrupt, not cleanly closed and upgrade needing tables.
xdissent@dev:/opt/redmine$ mysqladmin --user=root --host=localhost create redmine

production:
  adapter: mysql
  database: redmine
  host: localhost
  username: root
  password:
  socket: /var/run/mysqld/mysqld.sock

xdissent@dev:/opt/redmine$ sudo -H -u redmine ./use_redmine
bash-4.0$ cd apps/redmine
bash-4.0$ rake db:migrate RAILS_ENV=production
(in /opt/redmine/apps/redmine)
==  Setup: migrating ==========================================================
-- create_table("attachments", {:force=>true})
   -> 0.0151s
[...]
==  AddIndexOnChangesetsScmid: migrating ======================================
-- add_index(:changesets, [:repository_id, :scmid], {:name=>:changesets_repos_scmid})
   -> 0.0303s
==  AddIndexOnChangesetsScmid: migrated (0.0305s) =============================

bash-4.0$ exit

ProxyPass /redmine balancer://redminecluster
ProxyPassReverse /redmine balancer://redminecluster

<Proxy balancer://redminecluster>
  BalancerMember http://127.0.0.1:3001/redmine
  BalancerMember http://127.0.0.1:3002/redmine
  Order deny,allow
  Allow from all
</Proxy>

xdissent@dev:/opt/redmine$ sudo a2enmod proxy*
Enabling module proxy.
Considering dependency proxy for proxy_ajp:
Module proxy already enabled
Enabling module proxy_ajp.
Considering dependency proxy for proxy_balancer:
Module proxy already enabled
Enabling module proxy_balancer.
Considering dependency proxy for proxy_connect:
Module proxy already enabled
Enabling module proxy_connect.
Considering dependency proxy for proxy_ftp:
Module proxy already enabled
Enabling module proxy_ftp.
Considering dependency proxy for proxy_http:
Module proxy already enabled
Enabling module proxy_http.
Run '/etc/init.d/apache2 restart' to activate new configuration!
xdissent@dev:/opt/redmine$ echo Include /opt/redmine/apps/redmine/conf/redmine.conf | sudo tee /etc/apache2/conf.d/redmine > /dev/null

Restart Apache2 and Redmine

All that’s left to do is to restart the Apache2 server and the Redmine application. The former must be done from outside of the Redmine management shell, while the latter must be done from within:

xdissent@dev:/opt/redmine$ sudo /etc/init.d/apache2 start
 * Starting web server apache2
   ...done.
xdissent@dev:/opt/redmine$ sudo -H -u redmine ./use_redmine
bash-4.0$ ./ctlscript.sh start redmine
starting port 3001
starting port 3002

Congratulations, Redmine is installed and running on the system Apache2 and MySQL servers!

Load Default Redmine Configuration

Log into Redmine at http://<server-fqdn>/redmine/ using the administrative user which the installer created, in my case, admin. You will be greeted with a screen prompting you to load the Redmine default configuration. This is highly recommended, so do it now. You should probably at least change the URL setting for the site as well.

Set Up Redmine and Gmail

Redmine occasionally needs to send users email to alert them to project changes and other events. We chose not to set up an SMTP server when installing so that we could hopefully interface with Gmail, avoiding yet another redundant service running on our machine. It’s assumed that we have a Gmail or Google Apps for Domains account specifically reserved for Redmine – ideally redmine@<server-fqdn>.

Google’s SMTP servers require a special type of authentication called TLS. The Ruby libraries on which Redmine is built do not support this authentication scheme unfortunately, so we’ll need a Redmine plugin to get it working. Specifically, the ActionMailer class will be extended to support an optional TLS authentication mode. The plugin that provides this extension is (creatively) called action_mailer_optional_tls.

bash-4.0$ cd ~/apps/redmine
bash-4.0$ script/plugin install git://github.com/collectiveidea/action_mailer_optional_tls.git
Initialized empty Git repository in /opt/redmine/apps/redmine/vendor/plugins/action_mailer_optional_tls/.git/
remote: Counting objects: 14, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 14 (delta 2), reused 2 (delta 0)
Unpacking objects: 100% (14/14), done.
From git://github.com/collectiveidea/action_mailer_optional_tls
 * branch            HEAD       -> FETCH_HEAD

production:
  delivery_method: :smtp
  smtp_settings:
    tls: true
    address: "smtp.gmail.com"
    port: 587
    domain: "gmail.com"
    authentication: :plain
    user_name: "redminelover82@gmail.com"
    password: "XXXXXXXX"

Restart Redmine

If the settings are correct we can restart Redmine and the changes will take effect:

bash-4.0$ cd ~
bash-4.0$ ./ctlscript.sh restart redmine
stopping port 3001
stopping port 3002
starting port 3001
starting port 3002

Try the “Send a test email” link from within the “Email Notifications” settings tab in Redmine and you should shortly receive an email, courtesy of Gmail’s servers!

Create the git User

Gitosis absolutely requires a system user – specifically one with a real home directory somewhere. Disabling the password makes sure only those with the Gitosis private key we’ll create will be able to gain access to the box and change Gitosis settings. We’ll start by creating this user, just as we did for the redmine user. Feel free to select a different home directory or name for the user, but note these differences. Also note that Bash is the preferred shell, since it will give us a little more control over the git user environment when running the Gitosis commands, which would otherwise fail to find the Gitosis commands on its path.

xdissent@dev:~$ sudo adduser --system --shell /bin/bash --gecos 'Git Administrator' --group --disabled-password --home /opt/gitosis git
Adding system user `git' (UID 105) ...
Adding new group `git' (GID 114) ...
Adding new user `git' (UID 105) with group `git' ...
Creating home directory `/opt/gitosis' ...

The Master Key Pair

Under the hood, Gitosis access is controlled with the use of private/public key pairs. Each developer will create his own key pair, offering the public key to the Gitosis system through SSH when a Git operation is requested. Gitosis will check the public key against its known key list and grant or deny access appropriately.

Naturally, the key management system must be accessible in some regard in order to actually add developer public keys. Gitosis uses a single master key pair in this case, which will always be allowed to manage keys and the Gitosis configuration. It’s a good idea to generate this key from within a secure shell on the server, and never transfer it over any network. Strict file permissions are applied by default and should not be changed or the SSH system may reject a perfectly good key out of (justified) paranoia.

To generate a master key pair, use the ssh-keygen command. We will be using DSA keys, but any common scheme will work. Be sure to generate the key as the git user as to not totally blow up the file permissions:

xdissent@dev:~$ sudo -H -u git ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/opt/gitosis/.ssh/id_dsa):
Created directory '/opt/gitosis/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /opt/gitosis/.ssh/id_dsa.
Your public key has been saved in /opt/gitosis/.ssh/id_dsa.pub.
The key fingerprint is:
55:d7:4b:4c:c8:a1:f9:c3:4d:99:44:2a:af:03:71:fd git@<server-fqdn>
The key's randomart image is:
+--[ DSA 1024]----+
|            o.B= |
|           .++oo+|
|         ..= o.+.|
|         .o = +. |
|        S.   = E |
|          . . .  |
|           o     |
|            .    |
|                 |
+-----------------+

Create a Virtualenv

Gitosis is “just Python”, so it can easily be installed using Pip. However, it’s a good idea to sequester the Gitosis installation to minimize the risk of impacting our host system’s Python setup. We’ll use Virtualenv to create a standalone Python environment solely used by Gitosis.

xdissent@dev:~$ sudo -u git virtualenv ~git/virtualenv
New python executable in /opt/gitosis/virtualenv/bin/python
Installing setuptools............done.

Now any time a Python script is run from within /opt/gitosis/virtualenv/bin, our Gitosis Python environment will be bootstrapped automatically. That means calling ~git/virtualenv/bin/pip install <package> will install a Python package to the Gitosis virtual environment only. With that in mind, we can install Gitosis straight from its Git repository:

xdissent@dev:~$ sudo -u git ~git/virtualenv/bin/pip install git+git://eagain.net/gitosis.git
Downloading/unpacking git+git://eagain.net/gitosis.git
  Cloning Git repository git://eagain.net/gitosis.git to /tmp/pip-JyRa0m-build
  Running setup.py egg_info for package from git+git://eagain.net/gitosis.git
Requirement already satisfied (use --upgrade to upgrade): setuptools>=0.6c5 in /opt/gitosis/virtualenv/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg (from gitosis)
Installing collected packages: gitosis
  Running setup.py install for gitosis
    Installing gitosis-init script to /opt/gitosis/virtualenv/bin
    Installing gitosis-run-hook script to /opt/gitosis/virtualenv/bin
    Installing gitosis-serve script to /opt/gitosis/virtualenv/bin
Successfully installed gitosis
Cleaning up...

To actually force the virtual environment to take precedence over the system environment, we can load the ~git/virtualenv/bin/activate script. It sets up the user’s PATH environment variable to prepend the special Python scripts, automatically overriding the system versions. Loading the script from a ~/.bashrc file will set up the virtual environment upon login, and in our case, immediately before looking for the Gitosis commands. We’ll need to add a simple ~/.bashrc for the git user to get it all working:

xdissent@dev:~$ echo "source \$HOME/virtualenv/bin/activate" | sudo -u git tee -a ~git/.bashrc > /dev/null

Initialize the Gitosis Repository

Before it can manage any Git repositories, Gitosis needs to be initialized via the gitosis-init script. The master public key should be provided on standard input and the -H flag for sudo must be used:

xdissent@dev:~$ sudo -u git cat ~git/.ssh/id_dsa.pub | sudo -H -u git ~git/virtualenv/bin/gitosis-init
Initialized empty Git repository in /opt/gitosis/repositories/gitosis-admin.git/
Reinitialized existing Git repository in /opt/gitosis/repositories/gitosis-admin.git/

Gitosis is now set up to manage keys and configuration through the special gitosis-admin repository. Assuming they had loaded the master private key, any local or remote user should now be able to clone your Gitosis repository using git clone git@<server-fqdn>:gitosis-admin.git.

Install Redmine Gitosis Plugin

Public key management and automatic project-to-repository linking are provided by the redmine-gitosis plugin. Just like before, plugin installation must be done from within a Redmine management shell. The redmine-gitosis plugin uses the lockfile and inifile Gems, so we’ll need to install those as well:

xdissent@dev:~$ sudo -H -u redmine ~redmine/use_redmine
bash-4.0$ gem install lockfile inifile
Successfully installed lockfile-1.4.3
Successfully installed inifile-0.3.0
2 gems installed
Installing ri documentation for lockfile-1.4.3...
Installing ri documentation for inifile-0.3.0...
Installing RDoc documentation for lockfile-1.4.3...
Installing RDoc documentation for inifile-0.3.0...
bash-4.0$ cd ~/apps/redmine
bash-4.0$ script/plugin install git://github.com/rocket-rentals/redmine-gitosis.git
Initialized empty Git repository in /opt/redmine/apps/redmine/vendor/plugins/redmine-gitosis/.git/
remote: Counting objects: 123, done.
remote: Compressing objects: 100% (120/120), done.
remote: Total 123 (delta 74), reused 0 (delta 0)
Receiving objects: 100% (123/123), 13.19 KiB, done.
Resolving deltas: 100% (74/74), done.
From git://github.com/rocket-rentals/redmine-gitosis
 * branch            HEAD       -> FETCH_HEAD

Don’t try to use the plugin yet though – it requires modification before anything constructive will happen.

Patching the Plugin

Unfortunately, the redmine-gitosis plugin is a definite work in progress (read: hack) and needs to be slightly modified to work in any case. In the future I’d love to have a fork available that will automatically work in this setup, but that presents a sort of chicken-vs-egg problem, doesn’t it?

require 'lockfile'
require 'inifile'
require 'net/ssh'

module Gitosis
  # server config
  GITOSIS_URI = 'git@<server-fqdn>:gitosis-admin.git'
  GITOSIS_BASE_PATH = '/opt/gitosis/repositories/'

  # commands
  ENV['GIT_SSH'] = SSH_WITH_IDENTITY_FILE = File.join(RAILS_ROOT, 'vendor/plugins/redmine-gitosis/extra/ssh_with_identity_file.sh')

#!/bin/bash
LD_LIBRARY_PATH=""
exec ssh -i `dirname $0`/ssh/private_key "$@"

bash-4.0$ ssh <server-fqdn>
The authenticity of host '<server-fqdn> (127.0.1.1)' can't be established.
RSA key fingerprint is d1:fb:2a:0e:53:4f:27:64:63:66:a5:09:28:bd:20:86.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '<server-fqdn>' (RSA) to the list of known hosts.
redmine@<server-fqdn>'s password:
bash-4.0$

bash-4.0$ rake db:migrate_plugins RAILS_ENV=production
(in /opt/redmine/apps/redmine)
Migrating engines...
[...]
Migrating redmine-gitosis...
==  CreateGitosisPublicKeys: migrating ========================================
-- create_table(:gitosis_public_keys)
   -> 0.0136s
==  CreateGitosisPublicKeys: migrated (0.0139s) ===============================

Migrating rfpdf...
Migrating ruby-net-ldap-0.0.4...
bash-4.0$ cd
bash-4.0$ ./ctlscript.sh restart redmine
stopping port 3001
stopping port 3002
starting port 3001
starting port 3002
bash-4.0$ exit

Redmine, the Key Master

It was mentioned that the master Gitosis key pair is used to manage the Gitosis configuration and public keys. Redmine will need access to the private key in order to manage the public keys for us, but we have been very careful to prevent access to the master key from anyone except the git user! This is a perfect use-case for an ACL.

For some reason, the redmine-gitosis plugin comes with a default private key, which should be removed:

xdissent@dev:~$ sudo -u redmine rm ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key

Now we need to create a symlink from the path we just remove to the real Gitosis master key at ~git/.ssh/id_dsa. Note that you won’t currently be able to access the file:

xdissent@dev:~$ sudo -u redmine ln -s ~git/.ssh/id_dsa ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key
xdissent@dev:~$ sudo -u redmine cat ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key
cat: /opt/redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key: Permission denied

We need to install an ACL that will allow the redmine user to read the private key. ACLs are complicated but essentially we will add a rule that only grants the redmine user read-only access to the actual key file and the parent directory, which is also required. The mask parameter must also be modified so the effective ACL permissions will be what we are expecting:

xdissent@dev:~$ sudo setfacl -m user:redmine:r-x,mask:r-x ~git/.ssh
xdissent@dev:~$ sudo setfacl -m user:redmine:r--,mask:r-- ~git/.ssh/id_dsa
xdissent@dev:~$ sudo -u redmine cat ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key
-----BEGIN DSA PRIVATE KEY-----
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXX
-----END DSA PRIVATE KEY-----

The redmine user now has access to the Gitosis private key, but there’s one last ACL we need to implement before we’re done. The redmine-gitosis plugin occasionally manipulates the Gitosis repositories directly, using the path defined as GITOSIS_BASE_PATH above. To ensure the plugin can successfully access the repositories, we’ve got to add a “default” ACL to the repositories directory. In the future, any repositories that are created will inherit the “default” ACL, granting the plugin the required access:

xdissent@dev:~$ sudo setfacl -m default:user::rwx,default:group::r-x,default:other:---,default:user:redmine:rwx,default:mask:rwx ~git/repositories

The redmine-gitosis plugin should now have all the permissions it needs to access the Gitosis repositories, without unnecessarily exposing them to other users. Log into Redmine using the administrator user and verify that the redmine-gitosis plugin is active at http://<server-fqdn>/redmine/admin/plugins.

object instances…

Ok, so that means we’re dealing with an instance of a class, which will obviously require a class definition. That’s a pretty simple concept - define a class and create an instance of it. Somewhere along the line, we’re going to have to define some kind of "descriptor" class, and instantiate it. No big deal. With that in mind, let’s move on: Descriptors are object instances…

posing as dynamic properties…

Dynamic properties are a slightly advanced concept, and usually make use of the __get magic method (and friends), which PHP automatically calls when you try to access a non-existant property (member variable) on an object instance. For example, in the following code, $obj will be an instance of TheClass and the_property will be normal, non-dynamic instance property:

<?php
    // Define the class.
    class TheClass
    {
        // Declare the property.
        public $the_property;
    }

    // Create the object instance.
    $obj = new TheClass();

    // Access the object instance's property.
    echo 'The property: ' . $obj->the_property;

The property is accessed as $obj->the_property. The following call will issue a Notice level error by default, since we attempt to access an undeclared property on $obj:

<?php
    // Access a nonexistent property.
    echo $obj->bogus_property;

However, if the class defines an instance method named __get, it can intercept requests for undefined properties, and return a dynamic value. That means we can calculate a value at runtime for what the world will think is a normal instance property. The end product looks something like this:

<?php
    // Define the class.
    class DynamicClass
    {
        /**
         * Returns the value of a dynamic property.
         *
         * This method will issue a 'Notice' level error if a nonexistent
         * property is requested.
         *
         * @param string $name The name of the property to get.
         *
         * @return mixed
         */
        public function __get($name)
        {
            // Check the name of the requested property.
            if ($name == 'the_property') {

                // Return the dynamic value for 'the_property' property.
                return time();
            }

            // Trigger an error if no property was found with the name.
            trigger_error(sprintf(
                'Undefined property: %s::$%s',
                get_class($this),
                $name
            ));
        }
    }

    // Create the object instance.
    $obj = new DynamicClass();

    // Access the object instance's property.
    echo 'The property: ' . $obj->the_property;

Now every time we access the $obj->the_property property, the __get method will be evaluated, and – specifically – the current time will be returned.

That’s really all there is to the concept of dynamic properties. They’re incredibly convenient though, and indispensable when you want to define a clean and easy to understand API or standard interface to some functionality. There are in fact even more magic methods, allowing you to control property access in contexts other than "getting." We’ll deal with a few of them later.

Let’s get back to unraveling the mystery of descriptors. We know that they’re object instances (duh), and that they themselves somehow pose as dynamic properties. That means that, similarly to the __get method we’ve just seen, descriptors handle access requests for undefined properties. It’s not clear how an object instance actually does this amazing feat yet, but let’s take it one step at a time: Descriptors are object instances posing as dynamic properties…

which are defined on a class…

Holy crap, say what? Defined on a class? How do you define a property on a class? Well, in the biz, we call a property defined on a class, a static property. "Class members", "class properties", "static properties" – it’s all the same. In fact, the PHP documentation can’t even make up its mind what to call them. The premise is simple though. Remember that in PHP, classes are defined globally. It might help to think of the class definition itself as just a block of sourcecode that creates a single global instance of a "class" type object the first time it is run. These global "class" object instances have their own properties and methods, plus they know the how to do the voodoo (that they do) to create a local instance of the class they represent (so well). To declare a static class property, you just use the static keyword in the class definition, and that property will now only be accessible through the class itself and not through instances. The scope resolution operator can be used to get the static property’s value:

<?php
    class StaticPropertyClass
    {
        // Declare the static property.
        public static $the_static_property;

        // Declare the property.
        public $the_property;
    }

    // Create the object instance.
    $obj = new StaticPropertyClass();

    // Access the object instance's property.
    echo 'The property: ' . $obj->the_property;

    // Access the static class property.
    echo 'The class property: ' . StaticPropertyClass::$the_static_property;

    // This will produce an error.
    echo $obj->the_static_property;

    // And so will this.
    echo StaticPropertyClass::$the_property;

As you can see, class properties are completely separate from (and oblivious to) instance properties. They represent a unique global value, associated with the theoretical "global instance" of the "class" type object. Got it? Good. To review, a descriptor is really just an object that happens to be stored in a class property on some random class. It also will intercept and specially handle requests for some sort of dynamic property, which we haven’t really talked about yet. We do know that the value for the hypothetical property will be…

evaluated at runtime by a method on the descriptor object itself…

That means that every time we try to get the value of the dynamic property, the descriptor instance will run an instance method to determine the returned value. So a descriptor has to define some kind of processing method, which is loosely equivalent to the __get method we used earlier. The real difference is that the PHP magic methods only automatically handle access to dynamic properties on the exact instance on which the property is accessed. The descriptor differs in that (somehow) it intercepts property access for a different object, not itself.

Now our picture of a descriptor is complete, even though we don’t really know how it’s going to work or what it’s for yet. According to what we just discussed, we’re simply dealing with an instance of a class, with a method that will calculate and return the dynamic value of the property which the descriptor instance represents. And it goes a little something like this:

<?php
    // Define the descriptor class.
    class DescriptorClass
    {
        /**
         * Calculates and returns the dynamic value for the property
         * associated with the descriptor instance.
         *
         * @return mixed
         */
        public function getDescriptor()
        {
            // Do some special calculations.
            return time();
        }
    }

    // Create a descriptor.
    $the_descriptor = new DescriptorClass();

    // Add the descriptor to the class.
    StaticPropertyClass::$the_static_property = $the_descriptor;

That’s it! Now $the_descriptor object fits all of the criteria of a descriptor so far. It’s stored in a static class property (StaticPropertyClass::$the_static_property), it’s an instance of a class and it has an instance method called getDescriptor that will return a dynamic value for some property somewhere. Ah yes – we have to figure out how the descriptor relates to the property. Recall that descriptors intercept requests for dynamic properties…

that are accessible through either an object instance…

Hold it right there! Forget the "either" for right now and let’s focus on this part. The dynamic properties we’re going to be using with descriptors are accessible through an object instance. So really, when we’ve got our descriptor working correctly, we should be able to access a dynamic instance property just like we did using __get earlier. The important thing to note is that the name of the static class property in which the descriptor is stored should be the name of the instance property we use to get the dynamic value from the descriptor instance. I’ll say that again – store a descriptor instance in a class property, use the name of that class property to access the descriptor in an instance of that class. For example, using the descriptor we stored in StaticPropertyClass::$the_static_property, we would access the descriptor’s calculated property value using $obj->the_static_property:

<?php
    // Create the object instance.
    $obj = new StaticPropertyClass();

    // Access the dynamic property, calculated by the descriptor instance.
    echo 'Current time: ' . $obj->the_static_property;

Every time we access $obj->the_static_property, we need StaticPropertyClass::$the_static_property->getDescriptor() to run. Now that we’re clear, we can actually go about implementing some interfaces and a class that will take care of the actual property request handling. It’s not as hard as it sounds! Let’s start by defining a DescriptorInterface interface, which will need to be implemented by each descriptor class we create.

<?php
    // An interface that all descriptors will implement.
    interface DescriptorInterface
    {
        /**
         * Returns the value of the descriptor when accessed.
         *
         * @param object $instance The object instance on which the descriptor was
         *                         accessed. 'null' will be passed if accessed the
         *                         descriptor was accessed statically.
         * @param string $owner    The name of the class which owns the descriptor.
         *
         * @return mixed
         */
        public function getDescriptor($instance, $owner);
    }

Notice that we added parameters to the getDescriptor method. These are the parameters specified by the Python descriptor interface and they’ll really come in handy later. For now just understand that the first parameter is always the object instance whose descriptor property is being accessed and the second parameter will always be the name of the class on which the descriptor is defined. For example:

<?php
    // Define a descriptor.
    class ChattyDescriptor implements DescriptorInterface
    {
        // Calculate the dynamic value.
        public function getDescriptor($instance, $owner)
        {
            return sprintf(
                "Hey %s, you're a %s.",
                $instance->name,
                $owner
            );
        }
    }

    // Define a simple class.
    class Dude
    {
        public $name;
        public static $descriptor;

        // Assign the name at creation.
        public function __construct($name)
        {
            $this->name = $name;
        }
    }

    // Add the descriptor instance to the class.
    Dude::$descriptor = new ChattyDescriptor();

    // Create an object instance.
    $obj = new Dude('Broseph');

    // Access the descriptor.
    echo $obj->descriptor;

The previous example will output Hey Broseph, You're a Dude. if everything goes according to plan – which it won’t, until we sprinkle some magic code dust on the Dude class. What we need Dude to take care of is the hand-off from the dynamic property request to the descriptor’s getDescriptor method. Since we’re definitely not going to want to add extra code to every single class we use a descriptor with, let’s agree that we’ll need a single base class from which all descriptor-bearing classes will be extended. In this class, we want to intercept requests for undefined instance properties, check for a descriptor with the same name in the static class properties of the instance’s class, and then call the class property’s getDescriptor method if we do in fact find a descriptor object. Here’s one implementation of just such a base class:

<?php
    // Define a base class for objects that will use descriptors.
    abstract class Descriptable
    {
        // Intercepts requests for nonexistent properties.
        public function __get($name)
        {
            // Get the name of this class.
            $class = get_class($this);

            // Get an introspection reflection for the class.
            $class_ref = new ReflectionClass($class);

            // Check for a static class property with the given name.
            $attr = $class_ref->getStaticPropertyValue($name, null);

            // If the class property is an object, check for a descriptor.
            if (is_object($attr)) {

                // Get an introspection reflection for the property.
                $attr_ref = new ReflectionClass(get_class($attr));

                // Check to see if the static property is a descriptor.
                if ($attr_ref->implementsInterface('DescriptorGet')) {

                    // Call the descriptor method and return the value.
                    return $attr->getDescriptor($this, $class);
                }
            }

            // Trigger an error if no descriptor was found.
            trigger_error(sprintf('Undefined property: %s::$%s', $class, $name));
        }
    }

Subclasses of Descriptable will now correctly handle descriptors with no further fiddling. PHP’s new reflection API (speaking of undocumented features) is used to inspect the subclass and the descriptor instance to determine whether descriptor handling should take place. Here’s a final, working example that will correctly implement a descriptor according to the Python rules we’ve been over so far:

<?php
    // Define a simple class.
    class WorkingDude extends Descriptable
    {
        public $name;
        public static $descriptor;

        // Assign the name at creation.
        public function __construct($name)
        {
            $this->name = $name;
        }
    }

    // Add the descriptor instance to the class.
    WorkingDude::$descriptor = new ChattyDescriptor();

    // Create an object instance.
    $obj = new WorkingDude('Brosephus');

    // Access the descriptor (works!)
    echo $obj->descriptor;

Brilliant! We’ve made a descriptor! The only thing we need to remember about using the Descriptable class is that if we override the __get method, we must call the __get method inherited from Descriptable. When we call it in the overridden method will essentially effect the resolution order of dynamic properties, so do tread lightly in these cases.

So far we’ve built a descriptor class that emulates Python’s descriptors in all cases but one. Python descriptor properties may be accessed either through an object instance (as we’ve just shown)…

or statically through a property on the the class itself!

And here’s where we hit the proverbial brick wall. Basically what we need is a way to trigger a descriptor’s dynamic processing when accessing a static class property. The really confusing part is that generally, the static property will actually contain the descriptor instance itself! Mind Blow part II, anyone? So every time we access a static class property that contains a descriptor instance, we want to receive the dynamic value of the descriptor, not the descriptor instance itself. If we could get that to work, it would look like this:

<?php
    // Define a simple class.
    class UnemployedDude extends Descriptable
    {
        public $name;
        public static $descriptor;

        // Assign the name at creation.
        public function __construct($name)
        {
            $this->name = $name;
        }
    }

    // Add the descriptor instance to the class.
    UnemployedDude::$descriptor = new ChattyDescriptor();

    // Access the descriptor on the class (doesn't work!)
    echo UnemployedDude::$descriptor . " You don't even exist, you lazy bum!";

Unfortunately, PHP has a problem. It provides no __get equivalent magic method for static access. Try as we might, we’ll never get a dynamic value for a class property (defined or otherwise). There has been great discussion, a legendary bug ticket and even an RFC dealing with the addition of a __getStatic magic method to the PHP core, but PHP 5.3 shipped without even a hint of progress in this area. Quel bummer, man! So all static class properties must be declared in the class definition, period. In fact, our descriptor implementation subtley relies on this quirk. When __getStatic is finally available our descriptor class will require that the class property be undefined in the class definition, and assigned later. The assignment would take place in the __setStatic magic method, which would be tasked with keeping track of added descriptors, most likely in an array keyed off the property name for each descriptor. Yes, it will be a brave new world for sure! Oh well, it looks like we were just wasting our time on this descriptor pipe dream.

Not so fast! Don’t throw your keyboard in the trashcan yet – there is hope. First, let’s examine precisely how big of a deal this one restriction on our PHP descriptors is. Why would you even want static access to a dynamic descriptor property? Coincidentally, you probably wouldn’t want access to the dynamic value at all! In practice, descriptors rarely define special handling for the class itself, rather they focus on manipulating object instances. So the use cases are few in which you will even be dealing with a descriptor value statically. What you’ll see far more often is a descriptor returning its own instance instead of a value when it is accessed as a static class property. If you think about it, this makes total sense. How else are you supposed to ever get at the descriptor instance otherwise? If UnemployedDude::$descriptor returned a value, there would be no way to get at the descriptor instance at all, since that’s the only way we know how to refer to the damn thing. This is just how PHP works (which we’re stuck with) and it happens to correlate with the most likely use case for a descriptor (luckily) so our descriptor class is still very faithful to the Python equivalent.

One quick thing to note is that the dynamic accessor method required by the descriptor interface accepts an instance parameter which will contain the object instance whose descriptor property was requested. In a class descriptor context, this parameter would always be null since there is no instance in a static context.

Easy (Class-wide) Caching

A classic use case for dynamic properties is value caching. If the value of a property is expensive to calculate, it’s trivial to set up a dynamic property that calculates the value once, store it locally in the object instance and return the calculated value upon subsequent access. The down side to simple caching (probably using the __get method) is that the cache is local to the object. That means in naive implementations the value is calculated and stored each and every time you create an object instance and try to access the dynamic property. This is unneccessary when the calculated value will not differ between object instances. Of course this problem only gets larger the more instances you actually create.

Descriptors open up the possibility of per-class caching, rather than per-object caching since descriptor instances are defined on a class. Even when accessed on different object instances, a single descriptor property instance is handling all of the requests. This allows the descriptor to be used as a class-wide cache with very little effort. Using a descriptor in this way can be a lifesaver for intensive operations like database access:

<?php
    // Define a descriptor to manage retrieving the high score.
    class HighScoreDescriptor
    implements DescriptorGet, DescriptorSet
    {
        protected $_high_score;

        // Returns the current high score.
        public function getDescriptor($instance, $owner)
        {
            // Check to see if we've already retreived the high score.
            if (is_null($this->_high_score)) {

                // Fetch and cache the high score from the db.
                $this->_high_score = get_from_db('high_score');
            }

            // Return the cached high score.
            return $this->_high_score;
        }

        // Sets the current high score.
        public function setDescriptor($instance, $value)
        {
            // Update the cached high score.
            $this->_high_score = $value;

            // Save the new high score to the db.
            update_db('high_score', $this->_high_score);
        }
    }

    // Define a scored game class.
    class ScoredGame extends Descriptable
    {
        // Define a class property to hold the high score descriptor.
        public static $high_score;
    }

    // Add a high score descriptor to the class.
    ScoredGame::$high_score = new HighScoreDescriptor();

    // Create a game instance.
    $game = new ScoredGame();

    // Retrieve the high score for the game (db queried).
    echo 'High Score: ' . $game->high_score;

    // Set the high score, cheater.
    $game->high_score = 31337;

    // Retreive the high score again (db *not* queried).
    echo 'New High Score: ' . $game->high_score;

    // Create another game instance.
    $another_game = new ScoredGame();

    // Retrieve the high score for the new game (db *still* not queried).
    echo 'Same High Score: ' . $another_game->high_score;

Memory Footprint

It’s obvious that caching values with descriptors saves you from executing expensive operations multiple times, but what about the amount of memory used? When you cache a variable locally per-instance, you’re storing that information once for each instance that requests it. Descriptors use far less memory in this case by only storing one copy. This idea can be extended into realms other than simple caching, and will always reap the rewards of leaner memory usage.

Cleaner Than the Alternatives

Implementing a dynamic property with the __get magic method requires checking the name of the requested property to determine whether or not it should be dynamically handled. Once that is determined, the __get method must then figure out what to actually do to calculate the dynamic value. Innumerable approaches to this problem exist in the wild, and that’s a problem in itself. There exists no standard way of determining what should be called, when, and in which context with traditional dynamic variables. Descriptors provide a standardized interface to these concepts, and don’t require hacky switch statements with string checking, or a gazillion setSomething and getSomething stop-gap methods. Descriptions are cleaner and easier.

True Object Oriented Design

Whether or not you’re an OOP lover (or even a hater) doesn’t matter. You have to agree that this confusing middle ground in which PHP has been living sucks. Descriptors promote the concept of PHP classes as objects themselves, which is what they really are. This is a good move for the language if it is even going to try to compete with the Pythons and Rubys of tomorrow. If functionality is class-wide, move it up to the class logic level where it belongs!