Author: xdissent May 4th 2010 4:17 pm
Keywords: Git, Gitosis, Redmine, tools, Ubuntu
I’m in love with GitHub and I don’t mind paying a few bucks a month to host private code repositories there. It’s not without its issues though, and I’ve often had trouble getting others to collaborate with me via GitHub for one reason or another. Desiring more control, I was thrilled when GitHub:FI was announced. Unfortunately, the licensing fees are staggering and put the service far out of my reach. Recently, my buddy Marcus Whitney had been messing around with Redmine and my interest was piqued by his results. I decided to jump in head first and try to build a reliable GitHub:FI alternative using an Ubuntu VPS, Git, Gitosis, Gmail (or Google Apps for Domains), and Bitnami‘s Redmine Stack.
Contents
- Goals and Design Decisions
- Prerequisites
- Install Redmine
- Install Gitosis
- Working With Redmine and Gitosis
- Future Enhancements
Goals and Design Decisions
Redmine itself supports many of the features of GitHub, specifically:
- Project membership
- Issue/ticket tracker
- Wikis
- SCM with Git
The latest stable release even includes the much anticipated Git branch support. With a couple of plugins, Redmine can perform even more neat tricks, like sending outgoing email through Google’s SMTP servers, and managing your Gitosis repositories and public keys automatically. I’ve only just begun to scratch the surface of functionality and I can already sense the power and flexibility of Redmine is going to dramatically increase my productivity and calm sense of well-being.
Server Hygeine
There’s no reason even a fairly complex system like a Redmine installation can’t play nice with other system services, although it takes some planning. Much misinformation exists on the Web about running Redmine in particular, so we must tread very lightly to sidestep the pitfalls of a shaky system architecture. I’m of the mind that you should fully understand the intricacies of a piece of software before you attempt to run it – or at least understand clearly that you do not need to worry yourself with certain details – and hopefully when this setup is completely we’ll understand not only what we’ve done, but why we’ve done it. That’s the key to building a solid system and is of the highest priority.
Different elements of the system should be reused where appropriate as well. We want to only use the system-wide versions of things like Apache2, since they’ll probably be used for other services too and it makes no sense to double your load when you don’t have to. By the same token, we don’t need to install services which are easily outsourced to existing services or the cloud (like Gmail). In reality, our Redmine server might be moonlighting as a DNS server, firewall, development box – whatever – and we need to respect that. A general policy of software and user isolation along with well controlled shared resource management will ensure we aren’t wasting our time on a server that will need to be rebuilt later to play some additional role.
Stacks
A “stack” is a complete software system packaged so that, once installed, it operates in a self contained environment that will not effect, or be effected by, the rest of the host system. You generally don’t have to know much more than how to run an installer script and you can be the proud admin of your very own LAMP/Rails/etc server!
Stacks as a concept are almost prehistoric yet they remain an effective means of reliable package installation. The more complex the package, the more appealing a stack distribution becomes. The author may finely tune the most fragile system and distribute it as a stack without having to worry about it breaking in most cases.
A Redmine stack is a particularly appealing alternative to fighting Ubuntu’s awkward Rails packages that are notoriously difficult to use and maintain. Bitnami’s Redmine stack was chosen almost randomly, and I’m sure there are other options out there. Bitnami has a great reputation though, and their conventions are very clean and logical in my opinion.
To be honest, I’m a total Ruby-phobe. Not bringing that up would be lying through omission, and it’s a major reason I’ve decided to go with a stack – it’s a turn-key solution to a problem I (care to) know extremely little about otherwise. If you’re not sold on the idea by my humility, feel free to substitute your own Redmine installation process. Most of the general setup steps will be similar anyway.
Prerequisites
While this setup procedure should be fairly universal, there are a couple of important things that must be in place before continuing. Firstly it’s assumed that you have an Ubuntu server somewhere and a local development machine, which is capable of reaching the Ubuntu server using a fully qualified domain name (FQDN). You don’t have to have working DNS entries necessarily – you can get away with using your /etc/hosts files. On both of these machines you must have access to a non-root user, and that user must have sudo access on the server.
Fully Qualified Domain Name
Log into the non-root user account on the server via the console or an SSH session. Check the server’s hostname to make sure a FQDN is provided:
xdissent@dev:~$ hostname
dev.local
xdissent@dev:~$ hostname -s
dev
xdissent@dev:~$ hostname -f
dev.local
If for some reason the output of hostname does not give a FQDN, you might have to adjust the contents of your /etc/hosts file and/or your /etc/hostname file. Your host must also be accessible to itself, so a quick ping is in order.
xdissent@dev:~$ head -3 /etc/hosts
127.0.0.1 localhost
127.0.1.1 dev.hsd1.tn.comcast.net. dev
127.0.1.1 dev.local
xdissent@dev:~$ cat /etc/hostname
dev.local
xdissent@dev:~$ ping dev.local
PING dev.local (127.0.1.1) 56(84) bytes of data.
64 bytes from dev.hsd1.tn.comcast.net. (127.0.1.1): icmp_seq=1 ttl=64 time=0.021 ms
64 bytes from dev.hsd1.tn.comcast.net. (127.0.1.1): icmp_seq=2 ttl=64 time=0.030 ms
^C
--- dev.local ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.021/0.025/0.030/0.006 ms
You may need to tell the server to take note of the changes in /etc/hostname by running sudo hostname -F /etc/hostname. It’s pretty important to get your FQDN set up correctly before you continue the installation, so double check everything now.
Development Machine Access
At this point you should also be able to ssh into the server from your local development machine. If you can’t ssh yet but are logged in via the console, make sure the openssh-server package is installed by running sudo apt-get install openssh-server. If you’re using a FQDN that doesn’t have real-world DNS entries, add the FQDN to your development machine’s /etc/hosts file:
Stewart:~ xdissent$ echo <server-ip> <server-fqdn> | sudo tee -a /etc/hosts > /dev/null
Note
The tee command is used because root access is required to change the /etc/hosts file. STDOUT redirection is not granted the same access, so redirecting to a protected file will still result in a permissions error. The tee command simply redirects STDIN to a file, appending the contents of STDIN if the -a option is passed. By default, the contents are also dumped to STDOUT, so we redirect that to /dev/null to get rid of it.
Sudo Configuration
Finally, you must ensure that your non-root user on the server can gain access to other non-root user accounts. Some Ubuntu virtual images come with more restrictive sudoers configurations that only let your user sudo to root. As a simple test, try any random command as a random system user (who has a shell defined). For example, the www-data user is a decent test case:
xdissent@dev:~$ sudo -u www-data ls
Sorry, user xdissent is not allowed to execute '/bin/ls' as www-data on dev.local.
If access is denied, you must change the sudoers file using the command sudo visudo. A common fix is to append %admin ALL=(ALL) ALL to the end of the file, and then add your non-root user to the admin group if not already a member.
Required Packages
The whole idea behind using Ubuntu is to take advantage of its conveniences, like the package system. The Redmine stack comes with its own version of many standard system services which we’re going to disable in favor of the system versions. That way, we don’t have multiple instances of the same service consuming precious resources, and we move further towards a generic system that doesn’t rely on the stack for anything but the most specialized of services.
The standard Ubuntu services are easy to install using the various package management tools. Chances are, you’ve got many of these packages already installed, but there are so many different configurations out there, it’s easiest to just try to install them all in one go. Don’t forget to update your Apt sources:
xdissent@dev:~$ sudo apt-get update
Hit http://security.ubuntu.com karmic-security Release.gpg
Ign http://security.ubuntu.com karmic-security/main Translation-en_US
[...]
Hit http://us.archive.ubuntu.com karmic-updates/mutiverse Packages
Hit http://us.archive.ubuntu.com karmic-updates/mutiverse Sources
Reading package lists... Done
xdissent@dev:~$ sudo apt-get install apache2 mysql-server git-core python-setuptools git-core build-essential acl
Reading package lists...
Reading state information...
The following extra packages will be installed:
[...]
ACL Mount Options
Access Control Lists (ACLs) are an often overlooked security feature that allow additional, more specific access permissions to be applied to a file or directory. They can control automatic file ownership settings inherited from parent paths as well, which is crucial in situations where permissions must be maintained regardless of which user is operating on a shared directory. ACL support in Ubuntu is dependent upon the acl package which we installed earlier.
Simply installing the acl package does not enable ACLs for all disks. Hard disk partitions must be configured in /etc/fstab to load the ACL option and then must be remounted for the changes to take effect. Assuming /opt is on the same partition as the root filesystem, add the acl option to its /etc/fstab entry (using sudo):
xdissent@dev:~$ cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid -o value -s UUID' to print the universally unique identifier
# for a device; this may be used with UUID= as a more robust way to name
# devices that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
# / was on /dev/sda1 during installation
UUID=0c23e780-3453-4b2b-b42c-a1811722e941 / ext4 acl,errors=remount-ro 0 1
# swap was on /dev/sda5 during installation
UUID=e3eb3872-b203-46e9-b020-b3caf72ccf99 none swap sw 0 0
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto,exec,utf8 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto,exec,utf8 0 0
A reboot is required if ACLs were not already enabled and /opt is indeed on the root filesystem. If /opt is on a different (non-root) partition, you may get away with remounting it using mount -o remount,acl /opt or similar. Once you’ve got it working, the following command should give similar output:
xdissent@dev:~$ mount | grep acl
/dev/sda1 on / type ext4 (rw,acl,errors=remount-ro)
ACLs aren’t absolutely required by this setup, but it’s good practice and will probably save us a few headaches down the road, so just bite the bullet and load it up now. If you didn’t already see this coming, we’ll be using ACL features to let Redmine talk to Git repositories without a file ownership nightmare a little later.
Python Packages
We installed the python-setuptools package mere moments ago, which installs the very popular easy_install Python package installer. While it’s a great little dude to have around, I much prefer a new-comer to the Python package tool scene, Pip. It’s one Python package I’m comfortable installing globally, so we’ll use easy_install only once, to fetch its own successor and install it system-wide:
xdissent@dev:~$ sudo easy_install pip
Searching for pip
Reading http://pypi.python.org/simple/pip/
[...]
Installed /usr/local/lib/python2.6/dist-packages/pip-0.7.1-py2.6.egg
Processing dependencies for pip
Finished processing dependencies for pip
Now we can use the pip command to install any Python package we want, which is a much more well-advised notion than attempting to use the Ubuntu package manager. Interestingly, the only thing the Apt system is really bad at is Python and Ruby package management, and this project deals with both.
On the subject of Python, it would be a great time to familiarize yourself with the Virtualenv Python package, and how it’s used to create self contained Python environments. Essentially, the virtualenv <path> command sets up a tiny Python distribution with its own package repository and <path>/bin directory containing special binaries. The special binaries will automatically bootstrap the intended Python environment upon execution, doing away with a lot of path and environment variable nonsense that used to be required when attempting advanced deployments. It really is a life saver if you run more than a single Python app, and the idea of not impacting the system-wide Python installation is in line with the goals we’ve stated.
Install Virtualenv globally, which will be used when installing Gitosis:
xdissent@dev:~$ sudo pip install virtualenv
Downloading/unpacking virtualenv
Downloading virtualenv-1.4.8.tar.gz (1.5Mb): 1.5Mb downloaded
[...]
Successfully installed virtualenv
Cleaning up...
Install Redmine
Bitnami really makes it simple to get up and running with Redmine. If we didn’t care about running “two of everything”, it could be installed and configured in less than an hour with no sweat. Most of the extra steps we’re going to perform will actually disable many parts of the carefully architected Bitnami system, pointing the Redmine install to our standard system services instead.
Create a redmine user
A dedicated user is not technically required to run the Bitnami Redmine stack. In fact, I first ran it as www-data and it worked like a charm. Problems arose when trying to integrate Gitosis, and it became obvious that it’s just easier and cleaner to go ahead and set up a new system user for the redmine application. This configuration assumes the Redmine stack, and thus, the dedicated Redmine user’s home directory, will reside at /opt/redmine. Feel free to adjust this path at your whim. The obvious choice for the user name was redmine, but you can change that if you must as well:
xdissent@dev:~$ sudo adduser --system --shell /bin/bash --gecos 'Redmine Administrator' --group --disabled-password --home /opt/redmine redmine
Adding system user `redmine' (UID 106) ...
Adding new group `redmine' (GID 115) ...
Adding new user `redmine' (UID 106) with group `redmine' ...
Creating home directory `/opt/redmine' ...
Install Bitnami Redmine Stack
The Bitnami Stack system is super easy to get running. All you have to do is download and run the installer and away you go. Before we get started, ensure that the system Apache2 and MySQL services have been stopped. That way the default values will be used by the Bitnami installer, which are closer to the values we’ll ultimately want.
xdissent@dev:~$ sudo /etc/init.d/apache2 stop && sudo /etc/init.d/mysql stop
* Stopping web server apache2
...done.
* Stopping MySQL database server mysqld
...done.
Download and run the Bitnami Redmine Stack installer. Choose to install into /opt/redmine, or whichever folder you chose for the redmine user’s home. Do not choose to set up an SMTP server at this time, because we will handle that with Gmail later. Be sure to use sudo -u redmine to run the installer, so the file ownership will be correct.
xdissent@dev:~$ wget http://bitnami.org/files/stacks/redmine/0.9.3-0/bitnami-redmine-0.9.3-0-linux-installer.bin
--2010-05-03 18:34:05-- http://bitnami.org/files/stacks/redmine/0.9.3-0/bitnami-redmine-0.9.3-0-linux-installer.bin
Resolving bitnami.org... 216.235.167.25
Connecting to bitnami.org|216.235.167.25|:80... connected.
[...]
2010-05-03 18:38:48 (550 KB/s) - `bitnami-redmine-0.9.3-0-linux-installer.bin' saved [159459281/159459281]
xdissent@dev:~$ chmod 755 bitnami-redmine-0.9.3-0-linux-installer.bin
xdissent@dev:~$ sudo -u redmine ./bitnami-redmine-0.9.3-0-linux-installer.bin
----------------------------------------------------------------------------
Welcome to the BitNami Redmine Stack Setup Wizard.
Created with an evaluation version of BitRock InstallBuilder
----------------------------------------------------------------------------
Installation folder
Please, choose a folder to install BitNami Redmine Stack
Select a folder [/home/xdissent/redmine-0.9.3-0]: /opt/redmine
----------------------------------------------------------------------------
Create Admin account
BitNami Redmine Stack admin user creation
Login [user]: admin
Password :
Please confirm your password :
Your real name [User Name]: Redmine Administrator
[...]
Please wait while Setup installs BitNami Redmine Stack on your computer.
Installing
0% ______________ 50% ______________ 100%
#########################################
----------------------------------------------------------------------------
Setup has finished installing BitNami Redmine Stack on your computer.
Launch RedMine application. [Y/n]: Y
Info: To access the BitNami Redmine Stack, go to
http://localhost:8080 from your browser.
Press [Enter] to continue :
That’s it! Redmine is running! You could open up http://<server-fqdn>:8080/redmine/ and bask in the glory – or you could grow a pair and just forge ahead with the setup. That’s what I thought, buddy.
A Brief Note on Redmine Management
Bitnami provides a completely self contained Redmine stack with tightly coupled components that require a very specific (but minimal) shell environment to work together properly. For that reason, Bitnami includes a use_redmine script, which sets up the correct environment and runs a shell that is guaranteed to work with the Bitnami system. The use_redmine script should be run once before any Redmine management task. If you invoke a management script from an environment other than the one set up by use_redmine, you run the risk of corrupting your system. All Redmine management should also be done as the redmine user for obvious reasons. The use_redmine script is convenient in this regard, because you only have to run sudo once per administration session:
xdissent@dev:~$ cd ~redmine
xdissent@dev:/opt/redmine$ sudo -H -u redmine ./use_redmine
bash-4.0$ whoami
redmine
The Redmine stack services can be managed using the ~redmine/ctlscript.sh script. It follows the familiar ctlscript.sh <start|stop|restart|status> <service> syntax and, of course, must be run after use_redmine.
Reconfigure the Redmine Stack to Use System Services
Firstly, stop all Bitnami stack services that are currently running. We’re going to completely reconfigure Redmine anyway, and we won’t be needing any of the other services from this point on.
Warning
These and all management commands must be run from within a Bitnami management shell as the redmine user. Use the following command if you don’t understand and want to be safe: sudo -H -u redmine /opt/redmine/use_redmine.
bash-4.0$ pwd
/opt/redmine
bash-4.0$ ./ctlscript.sh stop
/opt/redmine/subversion/scripts/ctl.sh : subversion stopped
Syntax OK
/opt/redmine/apache2/scripts/ctl.sh : httpd stopped
stopping port 3001
stopping port 3002
/opt/redmine/mysql/scripts/ctl.sh : mysql stopped
Create a Redmine Database
Leave the Redmine management shell and start up the system MySQL service. Then we can create a database to host the Redmine application. The mysqladmin command can be used to create the DB.
bash-4.0$ exit
exit
xdissent@dev:/opt/redmine$ sudo /etc/init.d/mysql start
* Starting MySQL database server mysqld
...done.
* Checking for corrupt, not cleanly closed and upgrade needing tables.
xdissent@dev:/opt/redmine$ mysqladmin --user=root --host=localhost create redmine
Alter Redmine’s Database Configuration
Redmine uses YAML configuration files, which is a fairly common Rails thing from what I understand. We need to point the database configuration towards the system MySQL service with the default settings. Edit the file ~redmine/apps/redmine/config/database.yml and alter the production section to reflect the system settings:
production:
adapter: mysql
database: redmine
host: localhost
username: root
password:
socket: /var/run/mysqld/mysqld.sock
Now we can migrate the database, installing the Redmine application data on the system MySQL server. Rails apps use the rake command for these types of tasks, which I’ll admit is a pretty cool built-in tool. Rails also supports the concept of deployment environments, but we can ignore that for this setup and only pay attention to the production environment.
Note
All rake commands must be run from within the application’s root folder, which is ~redmine/apps/redmine/ in this case.
Migrate the database and exit the Redmine management shell:
xdissent@dev:/opt/redmine$ sudo -H -u redmine ./use_redmine
bash-4.0$ cd apps/redmine
bash-4.0$ rake db:migrate RAILS_ENV=production
(in /opt/redmine/apps/redmine)
== Setup: migrating ==========================================================
-- create_table("attachments", {:force=>true})
-> 0.0151s
[...]
== AddIndexOnChangesetsScmid: migrating ======================================
-- add_index(:changesets, [:repository_id, :scmid], {:name=>:changesets_repos_scmid})
-> 0.0303s
== AddIndexOnChangesetsScmid: migrated (0.0305s) =============================
bash-4.0$ exit
Change Permissions for the Mongrel Cluster Proxy
The Ubuntu Apache2 server has more restrictive proxy security settings than is assumed by the Bitnami Redmine stack. We need to edit ~redmine/apps/redmine/conf/redmine.conf and add the Order and Allow setting keys to permit Apache2 to access the Mongrel cluster:
ProxyPass /redmine balancer://redminecluster
ProxyPassReverse /redmine balancer://redminecluster
<Proxy balancer://redminecluster>
BalancerMember http://127.0.0.1:3001/redmine
BalancerMember http://127.0.0.1:3002/redmine
Order deny,allow
Allow from all
</Proxy>
You don’t need to know much about Mongrel or clusters, which is another perk of an application stack. The default Bitnami setup runs two instances of the application and can load balance between them, which is totally fine for most use-cases. Just change the permissions and forget the word “mongrel” forever.
Configure System Apache2 Server
The Apache2 server only requires a couple of additional modules to be enabled in order to play nice with Redmine. They’re all of the mod_proxy family and come in the default Apache2 package. The Redmine Apache2 configuration file we just edited will need to be “included” in the main Apache2 config, which we’ll accomplish by creating a simple file in /etc/apache2/conf.d/:
xdissent@dev:/opt/redmine$ sudo a2enmod proxy*
Enabling module proxy.
Considering dependency proxy for proxy_ajp:
Module proxy already enabled
Enabling module proxy_ajp.
Considering dependency proxy for proxy_balancer:
Module proxy already enabled
Enabling module proxy_balancer.
Considering dependency proxy for proxy_connect:
Module proxy already enabled
Enabling module proxy_connect.
Considering dependency proxy for proxy_ftp:
Module proxy already enabled
Enabling module proxy_ftp.
Considering dependency proxy for proxy_http:
Module proxy already enabled
Enabling module proxy_http.
Run '/etc/init.d/apache2 restart' to activate new configuration!
xdissent@dev:/opt/redmine$ echo Include /opt/redmine/apps/redmine/conf/redmine.conf | sudo tee /etc/apache2/conf.d/redmine > /dev/null
Restart Apache2 and Redmine
All that’s left to do is to restart the Apache2 server and the Redmine application. The former must be done from outside of the Redmine management shell, while the latter must be done from within:
xdissent@dev:/opt/redmine$ sudo /etc/init.d/apache2 start
* Starting web server apache2
...done.
xdissent@dev:/opt/redmine$ sudo -H -u redmine ./use_redmine
bash-4.0$ ./ctlscript.sh start redmine
starting port 3001
starting port 3002
Congratulations, Redmine is installed and running on the system Apache2 and MySQL servers!
Load Default Redmine Configuration
Log into Redmine at http://<server-fqdn>/redmine/ using the administrative user which the installer created, in my case, admin. You will be greeted with a screen prompting you to load the Redmine default configuration. This is highly recommended, so do it now. You should probably at least change the URL setting for the site as well.
Set Up Redmine and Gmail
Redmine occasionally needs to send users email to alert them to project changes and other events. We chose not to set up an SMTP server when installing so that we could hopefully interface with Gmail, avoiding yet another redundant service running on our machine. It’s assumed that we have a Gmail or Google Apps for Domains account specifically reserved for Redmine – ideally redmine@<server-fqdn>.
Google’s SMTP servers require a special type of authentication called TLS. The Ruby libraries on which Redmine is built do not support this authentication scheme unfortunately, so we’ll need a Redmine plugin to get it working. Specifically, the ActionMailer class will be extended to support an optional TLS authentication mode. The plugin that provides this extension is (creatively) called action_mailer_optional_tls.
Install the Plugin
Redmine (or is it Rails?) comes with a plugin installation script that makes it trivial to try out new extensions. Just like rake commands, the script/plugin script must be run from within the application’s root folder – ~redmine/apps/redmine/ in our case. It accepts an install sub-command and can install plugins directly from Git repositories:
bash-4.0$ cd ~/apps/redmine
bash-4.0$ script/plugin install git://github.com/collectiveidea/action_mailer_optional_tls.git
Initialized empty Git repository in /opt/redmine/apps/redmine/vendor/plugins/action_mailer_optional_tls/.git/
remote: Counting objects: 14, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 14 (delta 2), reused 2 (delta 0)
Unpacking objects: 100% (14/14), done.
From git://github.com/collectiveidea/action_mailer_optional_tls
* branch HEAD -> FETCH_HEAD
Note
It’s standard practice to “migrate” your plugins after adding or removing any packages. The action_mailer_optional_tls plugin does not have its own DB schema, so it’s safe to ignore this step here.
Create Redmine’s Email Configuration
Since we opted out of the SMTP configuration during installation, we probably won’t have a ~redmine/apps/redmine/config/email.yml file, which we’ll need to create. It’s another YAML configuration file that should be fairly self explanatory. Create ~redmine/apps/redmine/config/email.yml (as the redmine user) and add in your own Gmail settings:
production:
delivery_method: :smtp
smtp_settings:
tls: true
address: "smtp.gmail.com"
port: 587
domain: "gmail.com"
authentication: :plain
user_name: "redminelover82@gmail.com"
password: "XXXXXXXX"
The tls: true setting is only allowed after the action_mailer_optional_tls plugin has been installed. If you’re using Google Apps for Domains, you would substitute your domain for gmail.com in the domain and user_name settings.
Restart Redmine
If the settings are correct we can restart Redmine and the changes will take effect:
bash-4.0$ cd ~
bash-4.0$ ./ctlscript.sh restart redmine
stopping port 3001
stopping port 3002
starting port 3001
starting port 3002
Try the “Send a test email” link from within the “Email Notifications” settings tab in Redmine and you should shortly receive an email, courtesy of Gmail’s servers!
Install Gitosis
Gitosis is a great little piece of software that manages Git repository access, magically storing its own configuration in a Git repository as well. Technically, Gitosis relies on a single system user for access and manages more fine grained access policies using public key management. It’s a very solid system, on which GitHub itself is derived, and doesn’t clutter the system up with unnecessary users or daemons.
We’ll be using a Redmine plugin to manage our Gitosis access directly through the Redmine interface. Redmine users may manage their own public keys and they are automatically added to Gitosis’s access control system for project repositories of which they are members. This is similar to the way public keys work in GitHub so it should sound vaguely familiar.
Create the git User
Gitosis absolutely requires a system user – specifically one with a real home directory somewhere. Disabling the password makes sure only those with the Gitosis private key we’ll create will be able to gain access to the box and change Gitosis settings. We’ll start by creating this user, just as we did for the redmine user. Feel free to select a different home directory or name for the user, but note these differences. Also note that Bash is the preferred shell, since it will give us a little more control over the git user environment when running the Gitosis commands, which would otherwise fail to find the Gitosis commands on its path.
xdissent@dev:~$ sudo adduser --system --shell /bin/bash --gecos 'Git Administrator' --group --disabled-password --home /opt/gitosis git
Adding system user `git' (UID 105) ...
Adding new group `git' (GID 114) ...
Adding new user `git' (UID 105) with group `git' ...
Creating home directory `/opt/gitosis' ...
The Master Key Pair
Under the hood, Gitosis access is controlled with the use of private/public key pairs. Each developer will create his own key pair, offering the public key to the Gitosis system through SSH when a Git operation is requested. Gitosis will check the public key against its known key list and grant or deny access appropriately.
Naturally, the key management system must be accessible in some regard in order to actually add developer public keys. Gitosis uses a single master key pair in this case, which will always be allowed to manage keys and the Gitosis configuration. It’s a good idea to generate this key from within a secure shell on the server, and never transfer it over any network. Strict file permissions are applied by default and should not be changed or the SSH system may reject a perfectly good key out of (justified) paranoia.
To generate a master key pair, use the ssh-keygen command. We will be using DSA keys, but any common scheme will work. Be sure to generate the key as the git user as to not totally blow up the file permissions:
xdissent@dev:~$ sudo -H -u git ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/opt/gitosis/.ssh/id_dsa):
Created directory '/opt/gitosis/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /opt/gitosis/.ssh/id_dsa.
Your public key has been saved in /opt/gitosis/.ssh/id_dsa.pub.
The key fingerprint is:
55:d7:4b:4c:c8:a1:f9:c3:4d:99:44:2a:af:03:71:fd git@<server-fqdn>
The key's randomart image is:
+--[ DSA 1024]----+
| o.B= |
| .++oo+|
| ..= o.+.|
| .o = +. |
| S. = E |
| . . . |
| o |
| . |
| |
+-----------------+
Create a Virtualenv
Gitosis is “just Python”, so it can easily be installed using Pip. However, it’s a good idea to sequester the Gitosis installation to minimize the risk of impacting our host system’s Python setup. We’ll use Virtualenv to create a standalone Python environment solely used by Gitosis.
xdissent@dev:~$ sudo -u git virtualenv ~git/virtualenv
New python executable in /opt/gitosis/virtualenv/bin/python
Installing setuptools............done.
Now any time a Python script is run from within /opt/gitosis/virtualenv/bin, our Gitosis Python environment will be bootstrapped automatically. That means calling ~git/virtualenv/bin/pip install <package> will install a Python package to the Gitosis virtual environment only. With that in mind, we can install Gitosis straight from its Git repository:
xdissent@dev:~$ sudo -u git ~git/virtualenv/bin/pip install git+git://eagain.net/gitosis.git
Downloading/unpacking git+git://eagain.net/gitosis.git
Cloning Git repository git://eagain.net/gitosis.git to /tmp/pip-JyRa0m-build
Running setup.py egg_info for package from git+git://eagain.net/gitosis.git
Requirement already satisfied (use --upgrade to upgrade): setuptools>=0.6c5 in /opt/gitosis/virtualenv/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg (from gitosis)
Installing collected packages: gitosis
Running setup.py install for gitosis
Installing gitosis-init script to /opt/gitosis/virtualenv/bin
Installing gitosis-run-hook script to /opt/gitosis/virtualenv/bin
Installing gitosis-serve script to /opt/gitosis/virtualenv/bin
Successfully installed gitosis
Cleaning up...
To actually force the virtual environment to take precedence over the system environment, we can load the ~git/virtualenv/bin/activate script. It sets up the user’s PATH environment variable to prepend the special Python scripts, automatically overriding the system versions. Loading the script from a ~/.bashrc file will set up the virtual environment upon login, and in our case, immediately before looking for the Gitosis commands. We’ll need to add a simple ~/.bashrc for the git user to get it all working:
xdissent@dev:~$ echo "source \$HOME/virtualenv/bin/activate" | sudo -u git tee -a ~git/.bashrc > /dev/null
Initialize the Gitosis Repository
Before it can manage any Git repositories, Gitosis needs to be initialized via the gitosis-init script. The master public key should be provided on standard input and the -H flag for sudo must be used:
xdissent@dev:~$ sudo -u git cat ~git/.ssh/id_dsa.pub | sudo -H -u git ~git/virtualenv/bin/gitosis-init
Initialized empty Git repository in /opt/gitosis/repositories/gitosis-admin.git/
Reinitialized existing Git repository in /opt/gitosis/repositories/gitosis-admin.git/
Note
Many sources would suggest using a command similar to sudo -H -u git ~git/virtualenv/bin/gitosis-init < ~git/.ssh/id_dsa.pub. If your file permissions are correct, this will not work. This is the same IO redirection problem that required the use of tee earlier, and the cat command must be used with sudo to successfully read and output the git user’s public key.
Gitosis is now set up to manage keys and configuration through the special gitosis-admin repository. Assuming they had loaded the master private key, any local or remote user should now be able to clone your Gitosis repository using git clone git@<server-fqdn>:gitosis-admin.git.
Install Redmine Gitosis Plugin
Public key management and automatic project-to-repository linking are provided by the redmine-gitosis plugin. Just like before, plugin installation must be done from within a Redmine management shell. The redmine-gitosis plugin uses the lockfile and inifile Gems, so we’ll need to install those as well:
xdissent@dev:~$ sudo -H -u redmine ~redmine/use_redmine
bash-4.0$ gem install lockfile inifile
Successfully installed lockfile-1.4.3
Successfully installed inifile-0.3.0
2 gems installed
Installing ri documentation for lockfile-1.4.3...
Installing ri documentation for inifile-0.3.0...
Installing RDoc documentation for lockfile-1.4.3...
Installing RDoc documentation for inifile-0.3.0...
bash-4.0$ cd ~/apps/redmine
bash-4.0$ script/plugin install git://github.com/rocket-rentals/redmine-gitosis.git
Initialized empty Git repository in /opt/redmine/apps/redmine/vendor/plugins/redmine-gitosis/.git/
remote: Counting objects: 123, done.
remote: Compressing objects: 100% (120/120), done.
remote: Total 123 (delta 74), reused 0 (delta 0)
Receiving objects: 100% (123/123), 13.19 KiB, done.
Resolving deltas: 100% (74/74), done.
From git://github.com/rocket-rentals/redmine-gitosis
* branch HEAD -> FETCH_HEAD
Don’t try to use the plugin yet though – it requires modification before anything constructive will happen.
Patching the Plugin
Unfortunately, the redmine-gitosis plugin is a definite work in progress (read: hack) and needs to be slightly modified to work in any case. In the future I’d love to have a fork available that will automatically work in this setup, but that presents a sort of chicken-vs-egg problem, doesn’t it?
Gitosis Configuration
Firstly, the plugin is actually configured by editing a Ruby source file rather than a YAML configuration file. We must edit ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/lib/gitosis.rb and change the GITOSIS_URI and GITOSIS_BASE_PATH values to reflect our setup. The GIT_SSH environment variable needs to be slightly tweaked as well:
require 'lockfile'
require 'inifile'
require 'net/ssh'
module Gitosis
# server config
GITOSIS_URI = 'git@<server-fqdn>:gitosis-admin.git'
GITOSIS_BASE_PATH = '/opt/gitosis/repositories/'
# commands
ENV['GIT_SSH'] = SSH_WITH_IDENTITY_FILE = File.join(RAILS_ROOT, 'vendor/plugins/redmine-gitosis/extra/ssh_with_identity_file.sh')
Warning
The default GIT_SSH environment variable is incorrect. The path should have a hypenated redmine-gitosis rather than the underscored default of redmine_gitosis.
Linking Fix
Bitnami stacks are great! The only thing I can think of to make them even better would be if they linked their software differently at compile time. It’s pretty technical (and very boring) but Bitnami could either statically link the more common libraries or at least specify an rpath for the resulting dynamically linked binaries. That way the infamous LD_LIBRARY_PATH environment variable could be avoided entirely, promoting responsible Linux software packaging. In the meantime, we’ll have to edit ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh_with_identity_file.sh to change the LD_LIBRARY_PATH temporarily before calling ssh to add the private key. This prevents a linking error that would otherwise cause all public keys to be rejected. The modified ssh_with_identity_file.sh should look like the following:
#!/bin/bash
LD_LIBRARY_PATH=""
exec ssh -i `dirname $0`/ssh/private_key "$@"
Knowing Your Host
SSH maintains a list of “known hosts” for each user, which that user has determined should be trusted. We should create a quick SSH session to the FQDN of the server as the redmine user, which will prompt SSH to add the hostname to the list of known hosts. Redmine would not be able to connect regardless of private key if the host is not previously known to SSH, specifically for the redmine user. After the password prompt is displayed, simply ctrl-c to cancel the session:
bash-4.0$ ssh <server-fqdn>
The authenticity of host '<server-fqdn> (127.0.1.1)' can't be established.
RSA key fingerprint is d1:fb:2a:0e:53:4f:27:64:63:66:a5:09:28:bd:20:86.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '<server-fqdn>' (RSA) to the list of known hosts.
redmine@<server-fqdn>'s password:
bash-4.0$
Migrate the Plugin
Unlike the action_mailer_optional_tls plugin, redmine-gitosis does use the database internally. For this reason we need to “migrate” the plugin before attempting to use it. The rake command is used again, just like when migrating the initial Redmine application database. After the migration is complete, it’s safe to go ahead and restart Redmine since we’re done altering its configuration:
bash-4.0$ rake db:migrate_plugins RAILS_ENV=production
(in /opt/redmine/apps/redmine)
Migrating engines...
[...]
Migrating redmine-gitosis...
== CreateGitosisPublicKeys: migrating ========================================
-- create_table(:gitosis_public_keys)
-> 0.0136s
== CreateGitosisPublicKeys: migrated (0.0139s) ===============================
Migrating rfpdf...
Migrating ruby-net-ldap-0.0.4...
bash-4.0$ cd
bash-4.0$ ./ctlscript.sh restart redmine
stopping port 3001
stopping port 3002
starting port 3001
starting port 3002
bash-4.0$ exit
Redmine, the Key Master
It was mentioned that the master Gitosis key pair is used to manage the Gitosis configuration and public keys. Redmine will need access to the private key in order to manage the public keys for us, but we have been very careful to prevent access to the master key from anyone except the git user! This is a perfect use-case for an ACL.
For some reason, the redmine-gitosis plugin comes with a default private key, which should be removed:
xdissent@dev:~$ sudo -u redmine rm ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key
Now we need to create a symlink from the path we just remove to the real Gitosis master key at ~git/.ssh/id_dsa. Note that you won’t currently be able to access the file:
xdissent@dev:~$ sudo -u redmine ln -s ~git/.ssh/id_dsa ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key
xdissent@dev:~$ sudo -u redmine cat ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key
cat: /opt/redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key: Permission denied
We need to install an ACL that will allow the redmine user to read the private key. ACLs are complicated but essentially we will add a rule that only grants the redmine user read-only access to the actual key file and the parent directory, which is also required. The mask parameter must also be modified so the effective ACL permissions will be what we are expecting:
xdissent@dev:~$ sudo setfacl -m user:redmine:r-x,mask:r-x ~git/.ssh
xdissent@dev:~$ sudo setfacl -m user:redmine:r--,mask:r-- ~git/.ssh/id_dsa
xdissent@dev:~$ sudo -u redmine cat ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key
-----BEGIN DSA PRIVATE KEY-----
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXX
-----END DSA PRIVATE KEY-----
The redmine user now has access to the Gitosis private key, but there’s one last ACL we need to implement before we’re done. The redmine-gitosis plugin occasionally manipulates the Gitosis repositories directly, using the path defined as GITOSIS_BASE_PATH above. To ensure the plugin can successfully access the repositories, we’ve got to add a “default” ACL to the repositories directory. In the future, any repositories that are created will inherit the “default” ACL, granting the plugin the required access:
xdissent@dev:~$ sudo setfacl -m default:user::rwx,default:group::r-x,default:other:---,default:user:redmine:rwx,default:mask:rwx ~git/repositories
The redmine-gitosis plugin should now have all the permissions it needs to access the Gitosis repositories, without unnecessarily exposing them to other users. Log into Redmine using the administrator user and verify that the redmine-gitosis plugin is active at http://<server-fqdn>/redmine/admin/plugins.
Working With Redmine and Gitosis
Believe it or not, our GitHub clone is finished! Interacting with Redmine should be very familiar if you’ve used GitHub. To get started, we’ll need a Redmine project and at least one project member. Create a project at http://<server-fqdn>/redmine/projects/new/, noting the unique identifier.
The redmine-gitosis plugin will use the project identifier to determine the path each project’s repository. A project whose ID is example will be available at git@<server-fqdn>:example.git. To actually create the project’s repository, a Redmine user (with a valid key pair) must push a branch to the Gitosis server. Create a developer user at http://<server-fqdn>/redmine/users/new who will later create our repo.
Add your new developer user as a member of the project at http://<server-fqdn>/redmine/projects/<project-id>/settings/members/, granting Git access to the project’s repository.
Now we can log out of the administrator account, since it’s really only useful for project and user creation from now on. To enable your developer user to push to the Gitosis repositories, a public key must be assigned. Log in to Redmine as your new developer user and navigate to http://<server-fqdn>/redmine/my/public_keys to add a public key.
To generate a key for use from a development machine, run ssh-keygen for your local user exactly as we did when setting up Gitosis, except without the sudo. We need to paste the public key into the Redmine interface, naming the key appropriately. The general naming convention is <local-user>@<local-fqdn>. When successfully added, the new public key shows up in the key list.
Now our local development user should be able to create the project’s Git repository from a development machine. It’s exactly the same process used by GitHub, which is super convenient since I’m not sure I have the brain space left to remember another command sequence. Once we push a branch to the Gitosis server, the “Repository” tab in Redmine will be populated with our brand new Git repo.
Stewart:Code xdissent$ mkdir example
Stewart:Code xdissent$ cd example/
Stewart:example xdissent$ git init
Initialized empty Git repository in /Users/xdissent/Code/example/.git/
Stewart:example xdissent$ touch README.rst
Stewart:example xdissent$ git add README.rst
Stewart:example xdissent$ git commit -m "Initial commit."
[master (root-commit) 016b344] Initial commit.
0 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 README.rst
Stewart:example xdissent$ git remote add origin git@<server-fqdn>:example.git
Stewart:example xdissent$ git push origin master
Initialized empty Git repository in /opt/gitosis/repositories/example.git/
Counting objects: 3, done.
Writing objects: 100% (3/3), 219 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git@<server-fqdn>:example.git
* [new branch] master -> master
Redmine already includes great SCM tools like diff views and tracker integration, just like GitHub. Git commit users are mapped to Redmine users automatically as well so the interface is often even more intuitive than GitHub’s when you’ve accidentally borked a commit or merge.
Future Enhancements
It’s pretty damn close, but – even with Gitosis integration – Redmine is not GitHub. The feature that’s most likely to be missed is the per-user repository configuration. It might be a sizable undertaking, but this could presumably be implemented in another Redmine plugin. Pull requests, hook management, and pretty graphs are probably next on the list, but should be less complicated to build.
Regardless, Redmine is obviously exciting and it’s my hope that more people get involved in the development. I’ve already brushed up on my Ruby basics to tweak a few things so – who knows – I might even try my hand at putting together a plugin or two of my own. In the meantime, I’ve got more than enough functionality (and the promise of flexibility) to start using Redmine full-time in my development work.