Ivan

build-o-matic build farm

I've released a new version of build-o-matic (actual over a month ago now - just shows how far behind I've got with writing) which includes support for a "build farm".

I've been using it at my current client, Caplin Systems Ltd for their streaming ajax trading platform (very cool) implemented in JavaScript (client), Java and C (server), built using ant and maven, and using Perforce for source control. A free version of the server is now available.

The farm feature of build-o-matic is working fantastically well - exactly what I want.

What should a continuous integration server do?

As far as I'm concerned, the big things are:

1. Tell you whether your build is currently broken or not
2. Tell you when the build gets broken
3. If the build is broken, help you work out why

Where build-o-matic is different from other continuous integration servers is what I've done for (3).

Many continuous integration servers have invested a lot of effort in telling you the symptoms of why the build has failed - for example, with strong integration with the running of your build's tests.

I've taken a quite different approach - I'm not interested in the symptom of the build breakage, but the cause. Therefore, I have concentrated on getting build-o-matic to tell me which commit caused the build to break.

Having your continuous integration server tell you that the build has failed because a particular test has failed is interesting, but I think it's actually more useful to know that the build started failing from commit 5634.

Why this interpretation of "why"?

If the build gets broken, someone has to fix it. The best person/pair to fix it is the person/pair who broke it. Build-o-matic helps find out who broke the build.

The easiest way to find out what caused the failure (rather than the symptom of the failure) is to find out what change caused the build to fail. I expect the build itself to tell you why it is broken, rather than the continuous integration server (which, as far as I'm concerned, should only be running your build).

But all continuous integration servers already tell you which commit broke the build, don't they?

Not quite. Where other continuous integration servers fail is when you have multiple commits between builds. There are a few alternative solutions to this, including:

1. Have an integration token - i.e. never have multiple commits between builds.
2. Use Team City's Pre-tested Commit feature (or equivalent, if there are others)
3. Use build-o-matic's farm feature (or equivalent, if there are others)

I have used approach (1) - it works very well for a co-located and smallish team. You don't need any continuous itegration server at all for that approach.

I haven't tried Team City's "Pre-tested Commit feature" - my concern would be what do you do if the commit fails? You've then got to sort out the problem with the changes you tried to commit, plus any other changes you made since that attempted commit. If the build is successful (as it should be) then you've delayed integrating - it's a bit less continuous than it could be.

The build-o-matic farm does just what I want.

Multiple revisions at the same time

It's easiest to describe what build-o-matic does by example. Say you've got three machines in the farm, and more than three projects being built by the build-o-matic farm.

If there are commits to three different projects all at the same time, then each project will be built on a different machine (just like you'd expect from a build farm). Here's a build-o-matic management page that shows build activity on different machines:

Where build-o-matic is special, is if there are multiple commits to one project at the same time. For example, if there are two commits to project A, and one commit to project B, all at the same time, then build-o-matic will build both commits for project A (in parallel on two different machines) and the commit for project B.

The build overview page in this case looks like this:

If there were three commits to project A and none others, then all three revisions of project A would be built in parallel on different machines.

If there were three commits to project A and one to project B, then project B would be built and two of the three commits to project A. Then, when one of the machines has finished it would build the revision of project A that hasn't been built yet.

The "activity" pages are a bit horrible at the moment, but it works and does just what I want. All of the pages are served rather slowly in the current version of build-o-matic - it'll improve in some future version.

What does your continuous integration server do?

I find it difficult to tell what other continuous integration servers do - there's not yet an agreed "language" to describe their features. The CI Feature Matrix which the Damage Control team produced is a great effort, but doesn't include the features that make build-o-matic special.

Posted by ivan at February 1, 2008 9:25 PM