February 28, 2008

Maven2 - it can be quite good

I've been a bit late to catch on to Maven2 - it's been around for about two and half years now. I've previously been put off by the reputation of Maven1, the language used to describe Maven and some fanatical "fanboys". I know a lot of people of who are generally up-to-date on the latest technologies who haven't given Maven2 much of a chance, I'm guessing for similar reasons.

I'm no "fanboy" - more like a "somewhat in favour of it middle aged man" - so I want to explain to my fellow sceptical oldies who have avoided Maven why they should give it a chance.

Maven fails to sell itself by trying too hard

The very first sentence on the Maven web site is exactly the sort of language that puts me off it.

"Maven is a software project management and comprehension tool."

It doesn't tell what it really is and it doesn't sound like what I'm looking for. I'm going to try to explain what it is in a way that would have got me trying it sooner, so I hope my description is suitable for others too. To be fair to the authors of Better Builds with Maven they do say that calling it a "project management framework" doesn't really tell you what it is. I know the first author, Vincent Massol, from XTC and he's a cool guy so I hope he won't take offense at my somewhat simplistic description of Maven - I'm missing out lots, I'm no expert in Maven, and I'm describing it with a different philosophy than the authors of Maven may have intended - but I am trying to describe what is it in a way that I would have "got it" if it had been described like this to me.

For example, I say that Maven is Java-centric. You can use it for non Java projects, but in many ways it shows it's "Java project" based history. And that's OK.

Maven is like lots of standard "Ant" things refactored

I've seen the same stuff in Ant build files repeated again and again from project to project. Compiling the code, running the unit tests, packaging into a war etc. Either hand-crafted for that particular project, or copied and pasted from a previous project. These Ant build files say much the same stuff but not exactly the same stuff.

Maven is like a set of sensibly written Ant targets which can be used with hardly any effort as long as you use the standard Maven directory structure. Things like compiling, running the unit tests and creating a war. It also provides a suprisingly large number of more advanced build related things, like running code coverage or pmd reports - also very easy to use without having to write a load of stuff.

Standard directory structure

Some people object to Maven "telling them that their directory structure should be". It does seem like the tail is wagging the dog. However, I say "get over it". Personally, as long as the directory structure is sensible then I don't care much exactly what it is. There are lots of directory structures and naming that would be equally OK - the choice, in many cases, seems somewhat arbitrary to me. The Maven standard directory structure is as sensible and good as most, so my advice is don't fight it - just go with it. If you really care that your source is in a folder called "source" rather than "src" - or whatever - then I think you should get out more. If you really care then you can configure Maven to cope with whatever you want - but why bother? I think the benefits outweigh the righteous indignation of "not being dictated to by a tool over the names of folders".

Dependency management - i.e. an alternative to "lib" folders containing jars.

Ant built projects often have a "lib" folder containing the jars that they depend on. Using Maven, instead of having a lib folder per project, you have a "local repository" (just a bunch of files in a particular directory structure) instead where your jars can be used by multiple projects. Each project defines it's dependencies in a sensible declarative way and Maven sorts out the classpath for compiling the code, running the tests, or even packaging everything into a self contained executable jar with all dependencies. This sort of stuff is what Maven means by the phrase "dependency management"; "dependency" is one of those words which means lots of other things to other people.

Once and only once

Having the project's dependencies specified in the Maven way means that you only need this information in one place. For example, you don't need to specify this in your IDE as well (if you use one of the supported IDEs) - Maven can generate the project files for IDEA or Eclipse, so if you add a new dependency, rather than fiddling with both the build and the IDE's classpath, you just specify the depenency in the appropriate Maven file (pom.xml) and it sorts out the classpath for compiling, running the tests etc for both Maven and the IDE.

When what it does is what you want, it's great. Otherwise ...

I've found Maven to be overall pretty good for some smallish projects that I've written recently. However, sometimes I've found it difficult to get it to do what I want. The documentation is patchy (particularly if you want to do something a little bit different from the standard case) - but I would recommend downloading Better Builds with Maven.

New kid on the block

I haven't tried it, but I've heard that ivy does similar "dependency management" stuff to maven - but it's not clear to me whether it does the same other build stuff too.

Posted by ivan at 7:59 AM Copyright (c) 2004-2008 Ivan Moore | Comments (0)

February 1, 2008

build-o-matic build farm

I've released a new version of build-o-matic (actual over a month ago now - just shows how far behind I've got with writing) which includes support for a "build farm".

I've been using it at my current client, Caplin Systems Ltd for their streaming ajax trading platform (very cool) implemented in JavaScript (client), Java and C (server), built using ant and maven, and using Perforce for source control. A free version of the server is now available.

The farm feature of build-o-matic is working fantastically well - exactly what I want.

What should a continuous integration server do?

As far as I'm concerned, the big things are:
  1. Tell you whether your build is currently broken or not
  2. Tell you when the build gets broken
  3. If the build is broken, help you work out why

Where build-o-matic is different from other continuous integration servers is what I've done for (3).

Many continuous integration servers have invested a lot of effort in telling you the symptoms of why the build has failed - for example, with strong integration with the running of your build's tests.

I've taken a quite different approach - I'm not interested in the symptom of the build breakage, but the cause. Therefore, I have concentrated on getting build-o-matic to tell me which commit caused the build to break.

Having your continuous integration server tell you that the build has failed because a particular test has failed is interesting, but I think it's actually more useful to know that the build started failing from commit 5634.

Why this interpretation of "why"?

If the build gets broken, someone has to fix it. The best person/pair to fix it is the person/pair who broke it. Build-o-matic helps find out who broke the build.

The easiest way to find out what caused the failure (rather than the symptom of the failure) is to find out what change caused the build to fail. I expect the build itself to tell you why it is broken, rather than the continuous integration server (which, as far as I'm concerned, should only be running your build).

But all continuous integration servers already tell you which commit broke the build, don't they?

Not quite. Where other continuous integration servers fail is when you have multiple commits between builds. There are a few alternative solutions to this, including:

  1. Have an integration token - i.e. never have multiple commits between builds.
  2. Use Team City's Pre-tested Commit feature (or equivalent, if there are others)
  3. Use build-o-matic's farm feature (or equivalent, if there are others)

I have used approach (1) - it works very well for a co-located and smallish team. You don't need any continuous itegration server at all for that approach.

I haven't tried Team City's "Pre-tested Commit feature" - my concern would be what do you do if the commit fails? You've then got to sort out the problem with the changes you tried to commit, plus any other changes you made since that attempted commit. If the build is successful (as it should be) then you've delayed integrating - it's a bit less continuous than it could be.

The build-o-matic farm does just what I want.

Multiple revisions at the same time

It's easiest to describe what build-o-matic does by example. Say you've got three machines in the farm, and more than three projects being built by the build-o-matic farm.

If there are commits to three different projects all at the same time, then each project will be built on a different machine (just like you'd expect from a build farm). Here's a build-o-matic management page that shows build activity on different machines:

Where build-o-matic is special, is if there are multiple commits to one project at the same time. For example, if there are two commits to project A, and one commit to project B, all at the same time, then build-o-matic will build both commits for project A (in parallel on two different machines) and the commit for project B.

The build overview page in this case looks like this:

If there were three commits to project A and none others, then all three revisions of project A would be built in parallel on different machines.

If there were three commits to project A and one to project B, then project B would be built and two of the three commits to project A. Then, when one of the machines has finished it would build the revision of project A that hasn't been built yet.

The "activity" pages are a bit horrible at the moment, but it works and does just what I want. All of the pages are served rather slowly in the current version of build-o-matic - it'll improve in some future version.

What does your continuous integration server do?

I find it difficult to tell what other continuous integration servers do - there's not yet an agreed "language" to describe their features. The CI Feature Matrix which the Damage Control team produced is a great effort, but doesn't include the features that make build-o-matic special.

Posted by ivan at 9:25 PM Copyright (c) 2004-2008 Ivan Moore