Ivan http://ivan.truemesh.com/ Ivan Moore - Putting the Tea into Team. en-us 2008-04-27T15:43:40+00:00 CITCON 2008 http://ivan.truemesh.com/archives/000734.html I have really enjoyed the CITCON conferences that I've been to. CITCON is an open space conference about continuous integration and testing - excellently run by Jeffrey Fredrick and Paul Julius. CITCON 2006 in London inspired some of the more advanced features of build-o-matic (which I am really happy about). CITCON 2007 prompted me to produce a much simplified version of jester (which I am really happy about and have had some positive feedback about). CITCON 2008 is going to be in Amsterdam on 3 & 4th October this year; I have registered and I'm looking forward to it - even though on past experience it'll end up prompting me to do lots of extra work as a result!

]]>
ivan 2008-04-27T15:43:40+00:00
Programming as if the domain mattered (SPA 2008) http://ivan.truemesh.com/archives/000733.html Keith Braithwaite and I presented a session called Programming as if the domain mattered at SPA 2008. I feel that the part of the session that I prepared code for didn't go great (I hadn't prepared the packaging of the code well enough, so it took longer for people to get set up than it should have). I think the part Keith prepared for went better.

Domainification

The results from the whole session came as a suprise to us. It was an experimental session from which we wanted to encourage participants to include as much domain stuff as possible (I'll call this "domainification") in some code and find how far people think this should be taken.

It seemed from the retrospective at the end of the session that many participants didn't feel it was worth taking the "domainification" as far as myself and Keith had expected.

I now wonder whether this was because of the guidance (or lack of) that we gave, due to the time being too short, or whether I just have a weird programming style (and by implication Keith too, but I shouldn't really say that!).

Therefore, I'm writing this to explain what I thought would happen to the code. I wonder whether people's opinions would have been different if they had had time to go further with the example. I hope this article will show participants of the session (and readers of this blog) some domainification of code to consider whether it is worth "programming as if the domain mattered".

There are more improvements that can be made ... and what has been done could have been done differently ... this article just shows some things that could have been done ...

"Traction control" - a simple continuous integration server

My example code was "Traction control" - written specifically for the session and roughly based on how the very first incarnation of build-o-matic worked (which is now very considerably different!). What "Traction control" does is:

1 an update (svn up)
2 if there are changes, writes a web page with a text area for the build output
3 runs the build putting the build output into the file used on the web page to show the build output
4 once the build has finished, writes either a red or green web page
5 go to 1

The code

Here's the code (with the least interesting bits missed out, and a bogus space in "< script>" due to rendering problem):

public class TractionControl {
	public static void main(String[] args) throws IOException {
		String buildCommand = args[0];
		int previous = -1;
		while (true) {
			exec("svn up");
			String output = exec("svnversion");
			int current = Integer.parseInt(output.trim());
			if (current != previous) {
				StringBuffer stringBuffer = new StringBuffer();
				stringBuffer.append("<html><head>");
				stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
				stringBuffer.append("<title>Traction Control build running</title></head><body>");
				stringBuffer.append("<iframe src=\"buildoutput.txt\" width=\"600\" height=\"180\"/>");
				stringBuffer.append("</body></html>");
				write("results.html", stringBuffer.toString());
				exec(buildCommand + " > buildoutput.txt");
				output = readContents("buildoutput.txt");
				stringBuffer = new StringBuffer();
				boolean success = output.toLowerCase().contains("build successful");
				stringBuffer.append("<html><head>");
				stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
				stringBuffer.append("<title>Traction Control results</title></head><body>");
				stringBuffer.append("< script>document.bgColor=\"");
				stringBuffer.append(success ? "#33ff33" : "#ff3333");
				stringBuffer.append("\";</script>");
				stringBuffer.append("Build " + (success ? "passed" : "failed"));
				stringBuffer.append("</body></html>");
				write("results.html", stringBuffer.toString());
				previous = current;
			}
			try {
				Thread.sleep(10000);
			} catch (InterruptedException e) {
			}
		}
	}
	
	// definitions of "exec", "readContents" and "write" follow - you can guess roughly what they are ...

Procedural refactoring

The most horrible feature of this code is the creation of the web pages - I'll factor those out into methods:

public class TractionControl {
	public static void main(String[] args) throws IOException {
		String buildCommand = args[0];
		int previous = -1;
		while (true) {
			exec("svn up");
			String output = exec("svnversion");
			int current = Integer.parseInt(output.trim());
			if (current != previous) {
				writeBuildingPage();
				exec(buildCommand + " > buildoutput.txt");
				writeResultsPage();
				previous = current;
			}
			try {
				Thread.sleep(10000);
			} catch (InterruptedException e) {
			}
		}
	}

	private static void writeResultsPage() throws IOException {
		String output = readContents("buildoutput.txt");
		StringBuffer stringBuffer = new StringBuffer();
		boolean success = output.toLowerCase().contains("build successful");
		stringBuffer.append("<html><head>");
		stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
		stringBuffer.append("<title>Traction Control results</title></head><body>");
		stringBuffer.append("< script>document.bgColor=\"");
		stringBuffer.append(success ? "#33ff33" : "#ff3333");
		stringBuffer.append("\";</script>");
		stringBuffer.append("Build " + (success ? "passed" : "failed"));
		stringBuffer.append("</body></html>");
		write("results.html", stringBuffer.toString());
	}

	private static void writeBuildingPage() throws IOException {
		StringBuffer stringBuffer = new StringBuffer();
		stringBuffer.append("<html><head>");
		stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
		stringBuffer.append("<title>Traction Control build running</title></head><body>");
		stringBuffer.append("<iframe src=\"buildoutput.txt\" width=\"600\" height=\"180\"/>");
		stringBuffer.append("</body></html>");
		write("results.html", stringBuffer.toString());
	}
		
	// definitions of "exec", "readContents" and "write" follow - you can guess roughly what they are ...

Introducing more domain

Factoring out those methods makes it much clearer, but there is a missing domain concept - the results page, so I'll introduce that:

public class TractionControl {
	public static void main(String[] args) throws IOException {
		String buildCommand = args[0];
		int previous = -1;
		while (true) {
			exec("svn up");
			String output = exec("svnversion");
			int current = Integer.parseInt(output.trim());
			if (current != previous) {
				new ResultsPage().writeBuildingPage();
				exec(buildCommand + " > buildoutput.txt");
				new ResultsPage().writeResultsPage();
				previous = current;
			}
			try {
				Thread.sleep(10000);
			} catch (InterruptedException e) {
			}
		}
	}
	
	// now, definition of "exec" only

and

public class ResultsPage {
	public void writeResultsPage() throws IOException {
		String output = readContents("buildoutput.txt");
		StringBuffer stringBuffer = new StringBuffer();
		boolean success = output.toLowerCase().contains("build successful");
		stringBuffer.append("<html><head>");
		stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
		stringBuffer.append("<title>Traction Control results</title></head><body>");
		stringBuffer.append("< script>document.bgColor=\"");
		stringBuffer.append(success ? "#33ff33" : "#ff3333");
		stringBuffer.append("\";</script>");
		stringBuffer.append("Build " + (success ? "passed" : "failed"));
		stringBuffer.append("</body></html>");
		write("results.html", stringBuffer.toString());
	}

	public void writeBuildingPage() throws IOException {
		StringBuffer stringBuffer = new StringBuffer();
		stringBuffer.append("<html><head>");
		stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
		stringBuffer.append("<title>Traction Control build running</title></head><body>");
		stringBuffer.append("<iframe src=\"buildoutput.txt\" width=\"600\" height=\"180\"/>");
		stringBuffer.append("</body></html>");
		write("results.html", stringBuffer.toString());
	}
		
	// definitions of "readContents" and "write" follow - you can guess roughly what they are ...

Responsibilities mixed up

In the code as shown, the results page is responsible for running the build and deciding whether it passed. This doesn't seem right. Looking at the code, there is no reason why the "success" of the build shouldn't be passed in to the writeResultsPage method, and I feel like it would be better. This refactoring results in:

public class TractionControl {
	public static void main(String[] args) throws IOException {
		String buildCommand = args[0];
		int previous = -1;
		while (true) {
			exec("svn up");
			String output = exec("svnversion");
			int current = Integer.parseInt(output.trim());
			if (current != previous) {
				new ResultsPage().writeBuildingPage();
				exec(buildCommand + " > buildoutput.txt");		
				output = readContents("buildoutput.txt");
				boolean success = output.toLowerCase().contains("build successful");
				new ResultsPage().writeResultsPage(success);
				previous = current;
			}
			try {
				Thread.sleep(10000);
			} catch (InterruptedException e) {
			}
		}
	}

	// definitions of "exec" and "readContents" follow - you can guess roughly what they are ...

and ResultsPage:

public class ResultsPage {
	public void writeResultsPage(boolean success) throws IOException {
		StringBuffer stringBuffer = new StringBuffer();
		stringBuffer.append("<html><head>");
		stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
		stringBuffer.append("<title>Traction Control results</title></head><body>");
		stringBuffer.append("< script>document.bgColor=\"");
		stringBuffer.append(success ? "#33ff33" : "#ff3333");
		stringBuffer.append("\";</script>");
		stringBuffer.append("Build " + (success ? "passed" : "failed"));
		stringBuffer.append("</body></html>");
		write("results.html", stringBuffer.toString());
	}

	public void writeBuildingPage() throws IOException {
		StringBuffer stringBuffer = new StringBuffer();
		stringBuffer.append("<html><head>");
		stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
		stringBuffer.append("<title>Traction Control build running</title></head><body>");
		stringBuffer.append("<iframe src=\"buildoutput.txt\" width=\"600\" height=\"180\"/>");
		stringBuffer.append("</body></html>");
		write("results.html", stringBuffer.toString());
	}

	// definition of "write" follows - you can guess roughly what it is ...

Naming, getting more abstract and other improvements.

During the session, Chris Wallace suggested that instead of writing a web page, the code could usefully be made more abstract. Rather than having a results page, the missing domain concept could be thought of as "build results reporting". Maybe the interface to this would have a "buildStarted" and "buildFinished(boolean)" method - and maybe that is what the names of the methods of ResultsPage should be? Having created the ResultsPage class, it's clear now that it would be easy to introduce new forms of notification without having to significantly change the rest of the code. (Chris also pointed out that the way the code sets the background color is horrible, but for the purposes of the exercise it doesn't matter).

Maybe the magic values "results.html" and "buildoutput.txt" should be passed in on construction? Should TractionControl make one instance of the ResultsPage or is it OK to keep creating new instances? Which is more logical?

One of the benefits of creating ResultsPage is now it's clear that the helper method "write" is only used to write the result page and not for anything else.

Removing duplication

In this article, I won't bother taking ResultsPage any further; there is obvious duplication that could be removed but it'll make the article too long.

During the SPA session, one pair replaced ResultsPage creation with some sort of template thing, which seems sensible. ResultsPage as defined above is a bit horrible because of the duplication of code to do with HTML - so using some templating thing (like Velocity) would be a clean way of refactoring this code.

More procedural refactoring

There are three lines of code that I don't want to continue repeating in this article because they just give you a few more lines to look at without doing anything interesting - it's the Thread.sleep() code. So, I've also factored that out the new method "sleep" - you'll see that soon.

More domainification

The determination of whether the build was successful or not, and the running of the build command, are all a bit ugly. The domain concept missing from the code is the build command. Introducing that, the code becomes (with my little procedural refactoring introducing the sleep method too):

public class TractionControl {
	public static void main(String[] args) throws IOException {
		BuildCommand buildCommand = new BuildCommand(args[0]);
		int previous = -1;
		while (true) {
			exec("svn up");
			String output = exec("svnversion");
			int current = Integer.parseInt(output.trim());
			if (current != previous) {
				new ResultsPage().writeBuildingPage();
				boolean success = buildCommand.runTheBuildAndItPasses();
				new ResultsPage().writeResultsPage(success);
				previous = current;
			}
			sleep();
		}
	}

	private static void sleep() {
		try {
			Thread.sleep(10000);
		} catch (InterruptedException e) {
		}
	}
	
	// definition of "exec" only

and BuildCommand:

public class BuildCommand {
	private String buildCommand;
	
	public BuildCommand(String buildCommand) {
		this.buildCommand = buildCommand;
	}

	public boolean runTheBuildAndItPasses() throws IOException {
		TractionControl.exec(buildCommand + " > buildoutput.txt");		
		String output = readContents("buildoutput.txt");
		boolean success = output.toLowerCase().contains("build successful");
		return success;
	}
		
	// definition of "readContents" only

This is controversial because "runTheBuildAndItPasses" executes the build and returns whether it was successful or not. Many developers like methods that return a boolean to have no side effects, i.e. to separate the executing of the build command from the returning whether that was successful or not. I suspect this is a cargo cult similar to structured programming, but I'm undecided. Some people think I'm a crazy hacker; opinions may vary. Generally, I'll stick with the team style in a day job, and do things like "runTheBuildAndItPasses" when I only have to answer to myself (e.g. my open source projects). I think in this case it's better to do what I've done, but I know some people will absolutely hate it. Overall, I don't care about the details for this - the important thing is the introduction of the BuildCommand - details of it's implementation may vary depending on taste.

Yuk - but I won't fix it here

There is a magic string all around this code - "buildoutput.txt" - I'd want to sort that out before I feel like the code is OK - but I won't bother here - it would just make the article too long.

Last of the really important domain things

So - looking at the code, and thinking about what it's doing - the most important domain concept still missing from this is the stuff related to subversion. There is possibly more to it than one class - but for the time being, I'll introduce the concept of the "working copy" - which is where Traction Control has got the code checked out to. One of the participants, Robert Chatley had a class called "Checkout" (along with lots of other classes - he really got into the spirit of the session - thanks Robert).

public class TractionControl {
	public static void main(String[] args) throws IOException {
		BuildCommand buildCommand = new BuildCommand(args[0]);
		WorkingCopy workingCopy = new WorkingCopy();
		while (true) {
			if (workingCopy.doUpdateAndThereWereIncomingChanges()) {
				new ResultsPage().writeBuildingPage();
				boolean success = buildCommand.runTheBuildAndItPasses();
				new ResultsPage().writeResultsPage(success);
			}
			sleep();
		}
	}

	// definitions of "exec" and "sleep" follow ...

and the working copy is represented by:

public class WorkingCopy {
	private int previous = -1;
	
	public boolean doUpdateAndThereWereIncomingChanges() throws IOException{
		TractionControl.exec("svn up");
		String output = TractionControl.exec("svnversion");
		int current = Integer.parseInt(output.trim());
		boolean thereWereIncomingChanges = current != previous;
		previous = current;
		return thereWereIncomingChanges;
	}
}

Again - controversial use of a method which does something and returns a boolean result. I'd be prepared to do this differently, i.e. with the update and the checking to see if there were updates separately - it's not something I feel as strongly about as "structured programming" - which I think is definitely a cargo cult. Again, the important thing is introducing the concept of the working copy (or something else that encapsulates the use of subversion) rather than the exact details of it's implementation.

Is it any better?

There is more that could be done - the exceptions are nasty, the use of the static "exec" on class TractionControl is nasty. I'd maybe introduce a "CommandLine" class to do the "exec". I would probably make it such that there is one "ResultsPage" that is reused. Some of the names could be better - there are lots of things that could be better (if I was pairing, I'm sure it would be much better!).

I think this code is better than the original; I find it much easier to understand. Based on comments in the session, some people don't think it's worth the effort. Some don't think it's much better. Many think it isn't worth going this far.

I found it interesting that there was such a large variety of what people did in the workshop. Probably the largest group of similar results was where people refactored just within the class "TractionControl" without introducing any more classes - or maybe only one more. I had expected that the domain concepts of the build command, subversion stuff and the web pages would have been represented by classes, as in this article, by a larger number of people (but of course, done slightly differently). Maybe the reason people didn't do this is because each of these doesn't encapsulate much code? I think a lot of people were happy just with the benefits of doing a "procedural" style refactoring.

I'd like to hear from people what they think - but unfortunately, comments are switched off due to high levels of spam. If you are in London, come along to XTC some time when I'm there and talk to me, or email me at ivan at tadmad dot co-dot-uk.

Appendix - the code shown all together as it is the end of this article

public class TractionControl {
	public static void main(String[] args) throws IOException {
		BuildCommand buildCommand = new BuildCommand(args[0]);
		WorkingCopy workingCopy = new WorkingCopy();
		while (true) {
			if (workingCopy.doUpdateAndThereWereIncomingChanges()) {
				new ResultsPage().writeBuildingPage();
				boolean success = buildCommand.runTheBuildAndItPasses();
				new ResultsPage().writeResultsPage(success);
			}
			sleep();
		}
	}

	// definitions of "exec" and "sleep" follow ...
public class ResultsPage {
	public void writeResultsPage(boolean success) throws IOException {
		StringBuffer stringBuffer = new StringBuffer();
		stringBuffer.append("<html><head>");
		stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
		stringBuffer.append("<title>Traction Control results</title></head><body>");
		stringBuffer.append("< script>document.bgColor=\"");
		stringBuffer.append(success ? "#33ff33" : "#ff3333");
		stringBuffer.append("\";</script>");
		stringBuffer.append("Build " + (success ? "passed" : "failed"));
		stringBuffer.append("</body></html>");
		write("results.html", stringBuffer.toString());
	}

	public void writeBuildingPage() throws IOException {
		StringBuffer stringBuffer = new StringBuffer();
		stringBuffer.append("<html><head>");
		stringBuffer.append("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"5\">");
		stringBuffer.append("<title>Traction Control build running</title></head><body>");
		stringBuffer.append("<iframe src=\"buildoutput.txt\" width=\"600\" height=\"180\"/>");
		stringBuffer.append("</body></html>");
		write("results.html", stringBuffer.toString());
	}

	// definition of "write" follows - you can guess roughly what it is ...
public class BuildCommand {
	private String buildCommand;
	
	public BuildCommand(String buildCommand) {
		this.buildCommand = buildCommand;
	}

	public boolean runTheBuildAndItPasses() throws IOException {
		TractionControl.exec(buildCommand + " > buildoutput.txt");		
		String output = readContents("buildoutput.txt");
		boolean success = output.toLowerCase().contains("build successful");
		return success;
	}
		
	// definition of "readContents" only
public class WorkingCopy {
	private int previous = -1;
	
	public boolean doUpdateAndThereWereIncomingChanges() throws IOException{
		TractionControl.exec("svn up");
		String output = TractionControl.exec("svnversion");
		int current = Integer.parseInt(output.trim());
		boolean thereWereIncomingChanges = current != previous;
		previous = current;
		return thereWereIncomingChanges;
	}
}
]]>
ivan 2008-03-21T11:36:43+00:00
Maven2 - it can be quite good http://ivan.truemesh.com/archives/000731.html I've been a bit late to catch on to Maven2 - it's been around for about two and half years now. I've previously been put off by the reputation of Maven1, the language used to describe Maven and some fanatical "fanboys". I know a lot of people of who are generally up-to-date on the latest technologies who haven't given Maven2 much of a chance, I'm guessing for similar reasons.

I'm no "fanboy" - more like a "somewhat in favour of it middle aged man" - so I want to explain to my fellow sceptical oldies who have avoided Maven why they should give it a chance.

Maven fails to sell itself by trying too hard

The very first sentence on the Maven web site is exactly the sort of language that puts me off it.

"Maven is a software project management and comprehension tool."

It doesn't tell what it really is and it doesn't sound like what I'm looking for. I'm going to try to explain what it is in a way that would have got me trying it sooner, so I hope my description is suitable for others too. To be fair to the authors of Better Builds with Maven they do say that calling it a "project management framework" doesn't really tell you what it is. I know the first author, Vincent Massol, from XTC and he's a cool guy so I hope he won't take offense at my somewhat simplistic description of Maven - I'm missing out lots, I'm no expert in Maven, and I'm describing it with a different philosophy than the authors of Maven may have intended - but I am trying to describe what is it in a way that I would have "got it" if it had been described like this to me.

For example, I say that Maven is Java-centric. You can use it for non Java projects, but in many ways it shows it's "Java project" based history. And that's OK.

Maven is like lots of standard "Ant" things refactored

I've seen the same stuff in Ant build files repeated again and again from project to project. Compiling the code, running the unit tests, packaging into a war etc. Either hand-crafted for that particular project, or copied and pasted from a previous project. These Ant build files say much the same stuff but not exactly the same stuff.

Maven is like a set of sensibly written Ant targets which can be used with hardly any effort as long as you use the standard Maven directory structure. Things like compiling, running the unit tests and creating a war. It also provides a suprisingly large number of more advanced build related things, like running code coverage or pmd reports - also very easy to use without having to write a load of stuff.

Standard directory structure

Some people object to Maven "telling them that their directory structure should be". It does seem like the tail is wagging the dog. However, I say "get over it". Personally, as long as the directory structure is sensible then I don't care much exactly what it is. There are lots of directory structures and naming that would be equally OK - the choice, in many cases, seems somewhat arbitrary to me. The Maven standard directory structure is as sensible and good as most, so my advice is don't fight it - just go with it. If you really care that your source is in a folder called "source" rather than "src" - or whatever - then I think you should get out more. If you really care then you can configure Maven to cope with whatever you want - but why bother? I think the benefits outweigh the righteous indignation of "not being dictated to by a tool over the names of folders".

Dependency management - i.e. an alternative to "lib" folders containing jars.

Ant built projects often have a "lib" folder containing the jars that they depend on. Using Maven, instead of having a lib folder per project, you have a "local repository" (just a bunch of files in a particular directory structure) instead where your jars can be used by multiple projects. Each project defines it's dependencies in a sensible declarative way and Maven sorts out the classpath for compiling the code, running the tests, or even packaging everything into a self contained executable jar with all dependencies. This sort of stuff is what Maven means by the phrase "dependency management"; "dependency" is one of those words which means lots of other things to other people.

Once and only once

Having the project's dependencies specified in the Maven way means that you only need this information in one place. For example, you don't need to specify this in your IDE as well (if you use one of the supported IDEs) - Maven can generate the project files for IDEA or Eclipse, so if you add a new dependency, rather than fiddling with both the build and the IDE's classpath, you just specify the depenency in the appropriate Maven file (pom.xml) and it sorts out the classpath for compiling, running the tests etc for both Maven and the IDE.

When what it does is what you want, it's great. Otherwise ...

I've found Maven to be overall pretty good for some smallish projects that I've written recently. However, sometimes I've found it difficult to get it to do what I want. The documentation is patchy (particularly if you want to do something a little bit different from the standard case) - but I would recommend downloading Better Builds with Maven.

New kid on the block

I haven't tried it, but I've heard that ivy does similar "dependency management" stuff to maven - but it's not clear to me whether it does the same other build stuff too.

]]>
ivan 2008-02-28T07:59:44+00:00
build-o-matic build farm http://ivan.truemesh.com/archives/000730.html I've released a new version of build-o-matic (actual over a month ago now - just shows how far behind I've got with writing) which includes support for a "build farm".

I've been using it at my current client, Caplin Systems Ltd for their streaming ajax trading platform (very cool) implemented in JavaScript (client), Java and C (server), built using ant and maven, and using Perforce for source control. A free version of the server is now available.

The farm feature of build-o-matic is working fantastically well - exactly what I want.

What should a continuous integration server do?

As far as I'm concerned, the big things are:
  1. Tell you whether your build is currently broken or not
  2. Tell you when the build gets broken
  3. If the build is broken, help you work out why

Where build-o-matic is different from other continuous integration servers is what I've done for (3).

Many continuous integration servers have invested a lot of effort in telling you the symptoms of why the build has failed - for example, with strong integration with the running of your build's tests.

I've taken a quite different approach - I'm not interested in the symptom of the build breakage, but the cause. Therefore, I have concentrated on getting build-o-matic to tell me which commit caused the build to break.

Having your continuous integration server tell you that the build has failed because a particular test has failed is interesting, but I think it's actually more useful to know that the build started failing from commit 5634.

Why this interpretation of "why"?

If the build gets broken, someone has to fix it. The best person/pair to fix it is the person/pair who broke it. Build-o-matic helps find out who broke the build.

The easiest way to find out what caused the failure (rather than the symptom of the failure) is to find out what change caused the build to fail. I expect the build itself to tell you why it is broken, rather than the continuous integration server (which, as far as I'm concerned, should only be running your build).

But all continuous integration servers already tell you which commit broke the build, don't they?

Not quite. Where other continuous integration servers fail is when you have multiple commits between builds. There are a few alternative solutions to this, including:

  1. Have an integration token - i.e. never have multiple commits between builds.
  2. Use Team City's Pre-tested Commit feature (or equivalent, if there are others)
  3. Use build-o-matic's farm feature (or equivalent, if there are others)

I have used approach (1) - it works very well for a co-located and smallish team. You don't need any continuous itegration server at all for that approach.

I haven't tried Team City's "Pre-tested Commit feature" - my concern would be what do you do if the commit fails? You've then got to sort out the problem with the changes you tried to commit, plus any other changes you made since that attempted commit. If the build is successful (as it should be) then you've delayed integrating - it's a bit less continuous than it could be.

The build-o-matic farm does just what I want.

Multiple revisions at the same time

It's easiest to describe what build-o-matic does by example. Say you've got three machines in the farm, and more than three projects being built by the build-o-matic farm.

If there are commits to three different projects all at the same time, then each project will be built on a different machine (just like you'd expect from a build farm). Here's a build-o-matic management page that shows build activity on different machines:

Where build-o-matic is special, is if there are multiple commits to one project at the same time. For example, if there are two commits to project A, and one commit to project B, all at the same time, then build-o-matic will build both commits for project A (in parallel on two different machines) and the commit for project B.

The build overview page in this case looks like this:

If there were three commits to project A and none others, then all three revisions of project A would be built in parallel on different machines.

If there were three commits to project A and one to project B, then project B would be built and two of the three commits to project A. Then, when one of the machines has finished it would build the revision of project A that hasn't been built yet.

The "activity" pages are a bit horrible at the moment, but it works and does just what I want. All of the pages are served rather slowly in the current version of build-o-matic - it'll improve in some future version.

What does your continuous integration server do?

I find it difficult to tell what other continuous integration servers do - there's not yet an agreed "language" to describe their features. The CI Feature Matrix which the Damage Control team produced is a great effort, but doesn't include the features that make build-o-matic special.

]]>
ivan 2008-02-01T21:25:25+00:00
Simple Jester http://ivan.truemesh.com/archives/000725.html At CITCON I wanted to demo Jester in the session on mutation testing. I tried to run Jester on Nat Pryce and Steve Freeman's JMock2 code but couldn't get it to work in the time I had available, so I gave up and wrote some simple code specifically for the demo.

As a result of this bad experience, I thought I'd make Jester much simpler to use. I've called the result "simple-jester" and the first release of it is available from the Jester sourceforge site.

An example of using simple-jester

This article describes an example of how to run simple-jester, by showing how to run it on JMock2. I have chosen JMock2 because I know that it'll be well written and well tested (because I know Nat Pryce and Steve Freeman), it's not too small and not too big, the build is fast, it wasn't written with Jester in mind, and I didn't write it.

This article describes what I did - on windows, on a machine which I've already got Java, ant, Python and subversion installed on. This is what really happened ... warts and all, so I hope the hints and tips that arise from the problems described here will help you get simple-jester working for yourself.

A note on the style of this article

Sorry if you find the writing style odd (even by my standards) - it's a mix of the instructions that I thought I should write, what is happening as I'm trying to follow them and then instructions to fix the mistakes I made with the earlier instructions. I've written it like this because in practice you will end up going through something similar to get Jester to work - the mistakes weren't deliberate and are instructional.

Download and install Jester

Check that everything is OK by executing, in a command prompt:

  • cd C:\temp\simple-jester-1.0
  • setcp.bat
  • test.bat

Then:

  • wait about a minute for Jester to finish it's run
  • open "jester.html" to see that Jester has done something.

I've just realised that setcp.bat added JUnit 3.8 to the classpath - that might not be good for running the JMock2 build. Jester doesn't need JUnit on it's classpath to run. So:
  • close this command prompt

Get JMock source code

  • open a new command prompt
  • cd c:\temp
  • svn co http://svn.codehaus.org/jmock/trunk/jmock2/
  • cd c:\temp\jmock2
  • execute "ant"

Unfortunately, the build fails:
run.tests:
    [mkdir] Created dir: C:\temp\jmock2\build\reports\tests
    [junit] Testsuite: org.jmock.example.announcer.AnnouncerTests
    [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.031 sec

    [junit] Testsuite: org.jmock.example.qcon.DJTests
    [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0 sec

    [junit] Testcase: warning(junit.framework.TestSuite$1):     FAILED
    [junit] No tests found in org.jmock.example.qcon.DJTests
    [junit] junit.framework.AssertionFailedError: No tests found in org.jmock.ex
ample.qcon.DJTests



BUILD FAILED
C:\temp\jmock2\build.xml:208: Tests failed

Sorry Nat and Steve. Maybe I've got something wrong with the environment. I don't have time to investigate. Almost certainly something I've got wrong and I'm not worried anyway, because the fix is simple and I don't expect the side-effects of the fix to be too bad.

so:

  • delete C:\temp\jmock2\test\org\jmock\example\qcon\DJTests.java
  • execute "ant"
  • BUILD SUCCESSFUL - on my machine it took 14 seconds

Run Jester on JMock2

  • add Jester to the classpath by executing
    "set classpath=C:\temp\simple-jester-1.0\simple-jester.jar;C:\temp\simple-jester-1.0"
  • execute "java jester.TestTester ant src"

Unfortunately, this fails:
Jesting failed!
jester.SourceChangeException: couldn't run tests
        at jester.RealTestRunner.testsRunWithoutFailures(RealTestRunner.java:26)

        at jester.TestTester.run(TestTester.java:144)
        at jester.TestTester.doMain(TestTester.java:127)
        at jester.TestTester.main(TestTester.java:81)
Caused by: java.io.IOException: CreateProcess: ant error=2
        at java.lang.ProcessImpl.create(Native Method)
        at java.lang.ProcessImpl.<init>(ProcessImpl.java:81)
        at java.lang.ProcessImpl.start(ProcessImpl.java:30)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:451)
        at java.lang.Runtime.exec(Runtime.java:591)
        at java.lang.Runtime.exec(Runtime.java:429)
        at java.lang.Runtime.exec(Runtime.java:326)
        at jester.Util.runCommand(Util.java:10)
        at jester.RealTestRunner.testsRunWithoutFailures(RealTestRunner.java:21)

        ... 3 more

Doh! I made the mistake that I now remember documenting in the README.html for Jester. You have to tell Jester to run ant.bat on windows, rather than just ant.

To prove this, execute "java jester.RealTestRunner ant" which runs the code that Jester will run for executing your build command. This produces the same output as above. If you now execute "java jester.RealTestRunner ant.bat" instead, then the result is:

C:\temp\jmock2>java jester.RealTestRunner ant.bat
true

which means that this build passes for Jester.

So - trying again:

  • execute "java jester.TestTester ant.bat src"
  • the "progress" dialog opens, and after 30 seconds or so, you should see something happening in the "progress" dialog
  • lots of stuff gets printed out in the command prompt - it's ugly and alarming. Just ignore it.
  • now go and make a cup of tea; this could take some time.

I feel happy that I've been able to get Jester to work in a very short time. This would have been enough for CITCON - I could have run it on a small subset of the source rather than all the source and it would have all been done, set up and executed, in less than 30 minutes. Here I'm being more ambitious by trying to run Jester on all of the JMock2 source.

It goes horribly wrong

I've gone back to the machine - the progress bar showing not much progress. Not much seems to be happening, but after waiting about 30 seconds I see some more stuff in the "progress" dialog and return to the important task of drinking the tea that I've just made.

I go back to the machine much later - it really does look stuck now. One of the cores is maxed out running a java process, and the "progress" dialog has shown: "src\org\jmock\lib\concurrent\SynchronousScheduledExecutor.java - changed source on line 54 (char index=2029) from if ( to if (false &&" for several minutes.

So:

  • kill jester (ctrl-c)
  • execute "ant" and the result is:
BUILD FAILED
C:\temp\jmock2\build.xml:15: Unable to delete file C:\temp\jmock2\build\jmock-2-
SNAPSHOT\jmock-2-SNAPSHOT.jar

I open the task manager and find a java process that is hanging around that I wouldn't expect so:

  • kill any stray java processes
  • delete the offending jmock-2-SNAPSHOT.jar file.

I suspect a problem with Jester - if a mutation causes the build not to terminate, then Jester times out that run of the build and keeps going. In this case, it looks like that's what has happened, and there is a java process hanging around that is keeping a lock on the jmock-2-SNAPSHOT.jar file which the build needs to delete in order to continue, so now it's hung.

One of the least endearing features of Jester is that if the whole run isn't successful then you can't easily get anything out of Jester - i.e. mostly either the complete run of Jester works or doesn't work, if it crashes or is killed before it's finished then you don't get much useful out of it. I might fix this in the future. In the meantime, it's safest to run Jester on subsets of your code base (e.g. at most a package) at a time. But I won't follow my own advice here because I'm determined to get Jester to work for all the source files of JMock2 in one run.

To try to get Jester to work for JMock2, I'll get Jester to ignore the part of the source file where it made the mutation that caused the hang problem.

  • revert changes to "SynchronousScheduledExecutor.java" by executing: svn revert src\org\jmock\lib\concurrent\SynchronousScheduledExecutor.java
  • add "//stopJesting" as line 58 (if you have a look at the original source you'll see that's immediately before the mutation that was described in the "progress" dialog).
  • add "//resumeJesting" as line 60
  • to check that there are no other problems for Jester with this file, run Jester only on this file rather than on everything, i.e. execute:
    "java jester.TestTester ant.bat src\org\jmock\lib\concurrent\SynchronousScheduledExecutor.java"
  • Jester will complete mutation testing of this (takes about a minute)
  • execute "java jester.TestTester ant.bat src"
  • some time later, a lot later, Jester will finish. Yipee. Here's the last couple of lines of output:
43 mutations survived out of 284 changes. Score = 85
took 42 minutes

I'd expect that having deleted one of the tests (which might not have been the best way to get the build to pass) will have slightly worsened the results. Nevertheless, these results indeed show that the tests for JMock2 are impressively good. Jester often gives lower "scores" than code coverage tools. I'd be interested to know what the results are for a code coverage tool on the same code but my time has run out for this and my tea has gone cold.

However - I still have time for one thing, which is possibly the most important output from Jester:

  • execute "python c:\temp\simple-jester-1.0\makeWebView.py"

This produces web pages that show the modifications that Jester can make where the tests still pass. I'll send these to Nat (here's an example chosen because it's short, which I have hand edited to add his license in for publication). These files are the most interesting output from Jester.

The future of Jester

I'm pleased with both the code and usage simplifications to Jester in "simple-jester". The downside is that it's now even slower than before; but that doesn't worry me unduly because at least now I and others will be able to get it to work for many more projects. There are still some areas of improvement (e.g. if your build calls ant multiple times then Jester can think the build was successful even if it wasn't) - but generally it's much improved thanks to some judicious deletion of code.

If someone wants to pay me to do it, I'd be interested in making Jester faster by having it clustered on multiple machines. There are alternative approaches - e.g. jumble which uses byte code mutation (it's Java specific) which is much faster than Jester's approach.

I'm not planning on spending lots more time on Jester (unless someone pays me to do it) because I'm now happy that Jester is both simplified and improved (as simple-jester) such that it's much more usable, as I hope this article shows with it's "warts and all" commentary, because the warts really weren't very large.

]]>
Tools ivan 2007-12-12T23:39:06+00:00
CITCON 2007 http://ivan.truemesh.com/archives/000720.html I attended CITCON (Continuous Integration and Testing CONference) in Brussels last weekend - and just like last year, it was great! It's an expertly facilitated open space conference by Jeffrey Fredrick and Paul Julius, through their Open Information Foundation. I think I enjoyed it even more this year than last. It really is an exellent conference. I'm already looking forward to the next one.

Mutation testing - Jumble and Jester

Squirrel proposed a session on mutation testing; he wanted to demo Jumble and suggested that I should say something about Jester. It turned out to be a very popular session - it's still an idea that people are interested in.

Mutation testing

In a nutshell:
* Make some change to your code
* Run your tests
* If the tests pass then either you are missing a test or you have unnecessary code.

Jester difficult to use

It's a long time since I've tried to use Jester - I haven't worked on it for a long time - I checked with sourceforge and the last release was February 26, 2005. I spent an hour or so before the session, preparing a demo. It was quite horrible trying to get it to work.

New version of Jester

After finding out the level of interest, and being reminded of the difficulty getting Jester to work, I'm intending to release a new version of Jester - probably a fork because I've handed over project admin to Elliotte Rusty Harold and what I've got in mind is a simplification that has some disadvantages as well as some advantages. It'll be simpler code, simpler to use, but even less efficient. For me, making it simple to use is more important than the performance.

Jumble and others

Since Jester was released, there have been other implementations of the same sort of thing. Squirrel demoed Jumble, which he was keen on. An ex-ThoughtWorks colleague of mine, Stacy Curl, wrote "ajester" which, like Jumble, used byte code mutation rather than source mutation - but I don't know if he ever released it - I can't find a release package. Michael Nyika wrote Grester - a Maven2 Plugin for Jester. There's also a dot net version of Jester, called Nester. Another mutation testing tool is muJava which apparently now has an eclipse plugin. There is an eclipse plugin for Jester, but I'm afraid it never quite got properly released (sorry).

CruiseControl - still the daddy

CITCON is about continuous integration as well as testing. It's very clear from CITCON that cruisecontrol is still by far the most used continuous integration server. There are dozens of other continous integration servers available but none of them have much "market share". I was suprised by how few people had even heard of TeamCity because it seems to be becoming trendy with the London XP crowd.

The second most used continous integration server by CITCON attendees appeared to be Hudson demoed by Eric Lefevre which looks nice enough but still doesn't do what I want.

The only continous integration server that does what I want is build-o-matic - because I wrote most of it! I did a demo which I think went OK - I think the feature people like the most is putting the pictures on the results page. Nat Pryce wrote a plugin called team-piazza that does something similar for TeamCity.

More on the latest developments of build-o-matic in a future post.

]]>
ivan 2007-10-28T13:35:54+00:00
No bug reports != no bugs http://ivan.truemesh.com/archives/000717.html It's difficult getting feedback about software.

Do people log bugs?

Mostly, you get no feedback. Sometimes people log bugs, but not very often. With open source software, or web sites providing some service, if it doesn't work straight away (and very simply), mostly people will ignore it and try the next thing that looks like it might do what they want.

If an open source project has no bugs logged against it, chances are that it's very little used (possibly because it's totally broken) rather than because it is so high quality that it has no bugs. When people do log bugs, it's against software that they want to improve, not software that they can't be bothered with. In fact, I'd say it's generally a good sign if a project has several bugs logged against it and a bad sign if it has none. And yes, I have at least one open source project in the no bugs category (and I think it's good despite what I've just written)!

From "oops" to "ah ha"

I remember a few years ago, I released a new version of Jester (as a zip file). It was downloaded a couple of dozen times before someone (thanks - sorry I've forgotten who it was) told me that they couldn't unzip the file. It turned out that it had got corrupted on upload to sourceforge - I hadn't expected that. Of course, since then, I download and check that the zip wasn't corrupted (a bit of a pain, and I've not had a file corruption problem since, but I guess these things happen once in a while).

Your responsibility to kittens

In his keynote speach at last years ACCU conference, Mark Shuttleworth (Ubuntu founder) said something along the lines of "every time you fail to log a bug, a kitten dies". I like the sentiment. Please do log a bug, even if it seems so obvious that you think that surely the creator of the software (or person running a website) knows about it.

Knives behind your back

In the case of my open source projects, sometimes I'll google to see if someone has said anything about any of them. Sometimes I'll find an article slagging off one of my beautiful creations by someone who hasn't thought to tell me their criticisms or log any bugs.

Ignoring feedback

When using the O2 website I tried to do a search and got the following very obvious error:

o2 shop

Try it yourself. Simply enter anything in the search text box and press the search button. Following my own advice, I sent o2 an email about this (8th September). I tried again tonight (30th September) and got the same error. Never mind. I'll continue to try to save the kittens.

And another thing

There is a whole other article I could write about how to write bug reports ...

]]>
ivan 2007-10-01T07:36:30+00:00
Project management in the small - it isn't all glamourous. http://ivan.truemesh.com/archives/000715.html Following on from my previous article on Project Management in the small, in this article I want to talk about the glamourless, but worthy, side of working as a Project Manager.

The glamourless side of Project Management

One of the most important things for a Project Manager to do is to fix (or cause to be fixed) the sort of problems that the rest of team don't want to, and would be a waste of their talents to spend time on.

For example, if you need to buy a license for some software, then the Project Manager should do that (or have it done), rather than wasting the time of a developer, who probably prefers developing software and probably is quite good at developing software but may not be so motivated at (for example) filling in forms, paying using their own credit card, spending time filling in more forms to claim the money back etc.

The glamourless truth is that part of a Project Manager's job is being a "gopher".

Being a cost-effective gopher.

When working as a Project Manager, if you can get someone else to do this sort of thing (who isn't in your team) then that's even better. Some companies have PAs or other useful employees who can do these things for you, which is probably better value for the company. If your company doesn't have such people (as many very small companies don't) then in my opinion it's overall more cost effective to have the Project Manager do things like this rather than developers (even if the Project Manager is paid more per hour) because of the flow-destroying, and in some cases morale sapping, nature of these tasks. Having one person with lots of interruptions and odd-jobs is more efficient than having these spread throughout the team.

Is that gopher or groundhog?

To be an even better Project Manager you need to be a like a gopher, but able to predict the future, so I guess that means being a groundhog. Rather than just removing obstacles when they appear, a better Project Manager prevents them from appearing. For example, if you are working at a bank and know that someone new is going to be joining the team, then get their security clearance sorted out before they join so they can get their security pass to allow them around the building as soon as possible, rather than getting it sorted out after they have joined and having to wait for a couple of weeks before they can go to the bathroom unattended.

Avoiding groundhog days

If you work as a groundhog you might expect to have many groundhog days. If something is cropping up as a persistent problem then you need to make sure it gets fixed, even if the investment seems high compared to the value. It is very good for morale for things to get better, and morale is very important for productivity, so time spent fixing problems usually not only gets back the time that the problem was wasting, but also some extra by making things better and people happier.

]]>
ivan 2007-09-24T21:21:59+00:00
Project management in the small - it isn't all hard work. http://ivan.truemesh.com/archives/000697.html James Lewis, David Peterson and others have asked if I'm going to follow-up my programming in the small articles with some "project management in the small" articles, because these days I am mostly doing project management; so I'll have a go. I have less experience as a project manager than as a developer, so it might be presumptuous of me to write these articles, but I will anyway. My experience of project management is both on teams with someone else as the project manager, and being a project manager myself, over many years and dozens of projects.

Project management doesn't have to be hard work.

Some project managers seem to think that it's important that they work hard and their team works hard. However, (as Steve Freeman has said recently in conversation too), what is really important are the outputs of the team not it's inputs. That is, what's important is the value the team is delivering (output) rather than how hard they are working (input). Note I say "the value that the team delivers" and not "the amount of software they deliver" - at times the difference can be huge.

Sustainable pace

In some cases, working longer hours increases the output, but not always. Don't confuse the two things. In my experience, a bit of slack* actually increases output, and working too hard decreases output over the long term. You can increase output for a short time by working longer hours, but it doesn't last.

[* - I haven't read this book yet, I will do. From reading the reviews it sounds like the sort of stuff I'm talking about]

If developers get tired, they can also get sloppy, demotivated and stop thinking. Just grinding away hour after hour does nothing for productivity. Furthermore, the most mobile developers will eventually quit and get a job elsewhere if they are working long hours all the time.

Less is more

The biggest gains in productivity often come from doing a lot less work, rather than doing more work. That is, working out a creative way in which you can do something much simpler or using some software that someone has already written to solve the problem.

More is more

The biggest gains in value can come from developers having some slack time to think of new ways of using software for the benefit of the business. At connextra, we introduced gold cards which was a structured way of giving developers some time away from planned stories, and some of the work that came out of these "gold cards" turned out to be enourmously valuable to the company; much more than the cost.

Why long hours are attractive to project managers

It's much easier to work hard and to bully or bribe a team into working harder (or at least longer) than it is to help them work more effectively. Therefore, some project managers take this option - knowing that the hard work of themselves and their team will be noticed by the people they report to, and so they think they might look good. However, what looks better is actually delivering stuff.

Why project managers can't be busy all the time

As a project manager you will often have meetings scattered throughout the day, with half an hour here, and hour there, between meetings. You don't have to feel bad if you aren't busy all the time. You need to have unplanned time in your schedule to be available for helping with things that come up. Being available is more important than being busy. Some project managers seem to book and attend lots of pointless meetings and rush around the office being busy. It's more important to be available to sort out that seemingly minor issue there and then, because what seems like a minor issue can get in the way of the developers' flow. Remember it's the effectiveness of the whole team that matters rather than how busy any particular individual is.

Put your feet up

No. That's not what I'm saying. You shouldn't be a slacker, but a bit of slack is good.

]]>
ivan 2007-07-01T13:11:35+00:00
build-o-matic gets more advanced http://ivan.truemesh.com/archives/000694.html If you have a long enough build and large enough team, then you can quite often have the situation where there are multiple commits between continuous integration server builds. In these circumstances, if the build is broken then you don't know which commit broke the build.

Keith Braithwaite told me that in one team he worked in, someone would manually perform a binary search between the multiple commits to find out which commit broke the build. At a Google event, I was talking to someone about build-o-matic and he said that the feature he'd like for a continuous integration server was to do a binary search to find out which commit broke the build.

blame-o-matic

Build-o-matic is the only continuous integration server that I know of which automatically performs a binary search if the build is broken and there are multiple commits between builds (I call this feature "blame-o-matic"). The binary search is between the last known passing build and the first known broken build. Build-o-matic only performs one step of the binary search at a time - if someone commits while it's running one of these binary search builds, then it'll pick up those changes and run the build. If the build is still broken and there are no other commits, then build-o-matic will do the next binary search, etc.

The fine line between stupid and clever.

The binary search feature is clever but not always ideal. If the build is broken, sometimes it's obvious from the build output (which build-o-matic usefully shows and saves for you). For example, sometimes someone will realise straight away that they forgot to check a file in. If the build can be fixed straight away by (for example) checking a file in then it could be that build-o-matic has set off a blame-o-matic build, so you have to wait for that build to finish before your commit gets picked up and the build gets fixed.

Turning the dials up to eleven.

Having been using the "blame-o-matic" feature of build-o-matic for a while, I've now also added a feature called "filling in the gaps". If there are no incoming changes for build-o-matic to run the build for, and it isn't doing a binary search, then build-o-matic runs the build for the most recent revision that it hasn't run the build for. There are cases where doing a binary search doesn't work - if the build is broken, fixed then broken again within the revisions that are checked in at the same time, then the only way to be sure which revision was the most recent that broke the build, you have to build every revision.

Another benefit of running the build for every revision is that if your build includes any statistics (e.g. the build time, amount of sloppiness, etc) then you get the statistics for every revision.

Smell the glove

You might expect that the "blame-o-matic" and "filling in the gaps" features would result in the build-o-matic machine running the build all the time when you want it to be ready to run the build on a new commit. In practice, it works out well. If the build is fast enough, then it is rare that there are multiple commits between builds and the "blame-o-matic" and "filling in the gaps" features will not run very often. If the build is slow, then there is a very large chance that someone has checked in between builds, in which case, this will take precendence over the "blame-o-matic" or "filling in the gaps" builds. If the build is slow and people are careful not to check in if the build is broken, unless they are fixing the build, then the "blame-o-matic" can help find which commit caused the problem in parallel with people looking to see if they can work out what broke the build. The "filling in the gaps" feature can work over night while no one is checking code in, ready to show you the next day the full history of which builds broke and which passed.

An alternative - killing blame-o-matic builds

A possible improvement to the blame-o-matic feature would be to kill a blame-o-matic build if build-o-matic detects that someone has committed while a blame-o-matic build is running. This would probably be simple enough for build-o-matic to implement, but I didn't implement this because it means you would have to make sure that your build can be killed at any point without leaving the environment in which the build runs in a state in which the build cannot be run again. One common, but not necessarily good, feature of many builds I've seen is for acceptance tests to run against a database, which has to be partially set up and torn down rather than set up from scratch, due to the length of time it would take to do it fully. I didn't want build-o-matic to impose "interuptability" onto the builds it runs yet - maybe a future version will - maybe a future version will support both interruptable and non interruptable builds ... I don't know yet.

Another alternative - only check in if the build passes

I understand that Team City has a feature where you can submit some code for building, and then it is checked in if the build passes (that's my understanding having talked to someone who has used it - but I haven't used it myself). Using that feature as the only way to check code in would mean that the build would never be broken. I didn't implement this for a couple of reasons. It's harder to implement, and I'm not sure I'm keen on it - I want code integrated as soon as possible - a delay in integrating code due to waiting for a build on a continuous integration server doesn't seem ideal to me. I reserve the right to change my mind.

Try build-o-matic yourself

build-o-matic is open source - try it yourself. It's somewhat limited at the moment - it only supports subversion, and ant (or maven) "out of the box". I'd like to add support for perforce some time soon, but have no plans to support any VCS system that I don't want to use myself.

]]>
Tools ivan 2007-06-26T21:32:02+00:00
London to Paris Bike Ride 2007 http://ivan.truemesh.com/archives/000692.html I finished the London to Paris bike ride (NAS write up of last year's) for the National Autistic Society on Sunday.

I'm sure you'll be pleased to know that you can still sponsor me even after the event :-)

Day one - Crystal Palace to Newhaven

We set off from Crystal Palace in South London on Friday morning - 80 participants, mostly from around the UK but also some from further afield. Crystal Palace is close enough to where I live that I rode my bike to the start line. I found it very satisfying that I was cycling not only from London but actually from my home to Paris.

The weather was good, with beautiful scenery through the Kent and Sussex countryside. We rode through some very picturesque villages, with architypical cricket greens and duck ponds. The cycling was challenging with quite a few hills and some poor road surfaces. You notice the quality of the road surface much more on a bike than in a car; particularly on a road race bike, with narrow wheels, high pressure tyres, and a stiff frame.

We earned our lunch with a long steep climb up Turner's Hill; the traffic was pretty busy too which made the ride up Turner's Hill quite unpleasant.

After lunch, the weather deteriorated a bit, particularly with a strong head wind for the final few miles to Newhaven; on a bike, a headwind makes it much harder work. Arriving into Newhaven I was very tired; I'd done 62 miles and had over 100 left to get to Paris - it seemed like it was going to be a very tough trip, particularly given that we had to get up at 4 am the next morning to get the ferry to France.

Day Two - Newhaven to Gournay en Bray

After a horribly early start we got the 6 am ferry to Dieppe, arriving 11 am local time (an hour ahead of UK time). The ferry journey was uneventful - I slept through most of it. We did all look rather odd in our cycling gear on the ferry though - so probably more memorable for the other passengers than for us. The roads in Normandy were fantastic - lovely road surfaces - smooth - like a race track - with only gentle undulations making for high speed and enjoyable cycling. We rode through villages that were so quiet it seemed like they were deserted. The traffic was much less than our first day too - and the weather very pleasant - dry, sunny but not too hot. It was a shorter day of cycling due to the late start after the ferry journey - we did 52 miles - with the good roads and fewer hills it felt like a lot less than the first day. Cycling through Normandy you really appreciate how rural it is - cycling past fields that looked like Monet's poppies painting.

Day Three - Gournay en Bray to Paris

The weather took a turn for the worse - rain and cold. Nevertheless, the roads continued to be great, with one particularly great downhill section where I reached 43.6 mph - which didn't feel quite that fast due to the smooth road surface. Some of the views, e.g. overlooking Paris in the distance, were excellent dispite the weather. When we got into the suburbs of Paris, the roads became more bumpy, and there were lots of traffic lights. Some roads nearer the center were cobbled (cobblestone), including one of the roundabouts (traffic circle), which were treacherous in the rain on a race bike with narrow, slick, high pressure tyres.

We finished at the Eiffel Tower, in the rain, having done 76 miles - the furthest I've ever cycled in a day. My wife and two friends came to Paris for the finish which was great of them. Thanks to my coach, Jason, I was very well prepared. I feel like doing something like this again - again for the National Autistic Society.

]]>
ivan 2007-05-23T20:40:56+00:00
Scrapheap challenge at SPA 2007 http://ivan.truemesh.com/archives/000676.html Nat Pryce and I ran scrapheap challenge again (our Postmodern programming workshop), this time at at SPA 2007.

Nat and I joined in with the three challenges - I hope Nat will write up the other two as he pretty much did them by himself while I drank tea - here's my write up of the one of them where I actually did something.

Slopography part 2

This challenge was to produce a tool that would show you whether you are making code sloppier as you work (compared to how the code was when you checked it out). This was the last challenge of the workshop and we had overrun slightly, so we had only around 30 minutes for it. As a result, we allowed a very simple interpretation of the measure of slop - a simple count of the number of occurences of the string "todo" was enough.

I've called this "part 2" because we did something a little bit similar at a previous scrapheap challenge - to produce a graph showing the sloppiness of code over time - e.g. by measuring the number of "todo" comments in code, by checking out code from a subversion repository one revision at a time. This was enough different so there was very little overlap - having a solution to that challenge wasn't much help for this one.

A solution (in 30 minutes)

Here's the code (in python) for it as typed in on the day - not cleaned up - it could be done more neatly and shorter:

import os, time

def slop(filename):
    f=open(filename)
    return f.read().lower().count('todo')

import sys
rootDir = sys.argv[1]

while(True):
    checkedInSlop = 0
    mySlop = 0

    for root, dirs, files in os.walk(rootDir):
        for fileName in files:
            fullFileName = os.path.join(root, fileName)
            slopcount = slop(fullFileName)
            if fileName.endswith('.svn-base'):
                checkedInSlop = checkedInSlop + slopcount
            else:
                mySlop = mySlop + slopcount

    f=open('slop.html','w')
    s = int(100 * float(mySlop) / float(checkedInSlop))
    if s <= 100:
        color = "#33ff33"
    else:
        color = "#ff3333"
    f.write("""<html>
    <HEAD>
    <META HTTP-EQUIV="Refresh" CONTENT="10">
    </HEAD><body>
    
    You are this sloppy: PERCENT%</body></html>""".replace("PERCENT",str(s)).
            replace("COLOR",color))
    f.close()

    time.sleep(10)

A couple of things about this solution

We just counted the number of occurrences of "todo" - Nat wanted to do a cleverer implementation, but we were rushing to complete the challenge, which included a big friendly display of whether the slop was increasing or not. We put completion of the whole solution, however poor the individual parts, ahead of doing any of the parts of the solution really well. We could return to the worst of the parts of the solution later if we had time, but it turned out that we only just completed this in the time.

For the display, we used a really simple and quick-to-implement solution. Our solution was to write an html file with a meta-refresh, and set the background red or green depending on whether the level of slop has gone up or down. This borrowed from how build-o-matic used to do its build results web page.

Yuk - that's quick and dirty

Possibly. But it does work.

]]>
ivan 2007-04-02T06:30:32+00:00
Using a Coach http://ivan.truemesh.com/archives/000670.html I've been getting help preparing for my London to Paris bike ride (in aid of the National Autistic Society). I have a cycling coach - Jason - he's great. He's the dad of one of my son's friends.

I've worked as an XP coach, on and off, for many years and I feel like there is some level of analogy. Sometimes working as a coach has been better than other times; not everyone seems to know how to use a coach effectively, and I hope to indicate in this article that it isn't all down to the coach.

Your coach has done this before - listen and learn

I'm just cycling a mere 200 miles or so; my coach has cycled across Ecuador. And Mexico. He knows what he's doing; I listen to his advice.

Your coach hasn't done exactly this before

Jason hasn't cycled from London to Paris, but, he's done a lot of cycling. The fact that he hasn't done exactly what I'm doing doesn't matter.

Make time for your coach

I've been going for training rides with Jason every weekend. He coaches at a cycle club on Saturdays, so recommends a ride for me to do by myself on Saturday, then we go for a bike ride together on Sunday mornings. On Sunday mornings I don't tell him that actually I've decided to go shopping instead.

Have whatever resources you need to make best use of your coach.

When Jason arrives at my house on Sunday morning, I don't tell him that actually I don't have a bike so we'll have to go for a run instead.

Don't try to buy your way out of coaching advice

The analogy would have been buying a very expensive Wheelie Pointless Sycle from International Bicycle Makers (a tricycle with octagonal wheels), and then either struggling with it and getting no help, or asking Jason how to make best use of it and insisting on sticking with it because I've just spent loads of money on it.

Instead, I talked to Jason about whether to get a new bike; he suggested keeping my old rubbish bike until I had a better idea of what I need. My old bike was OK for training on, and Jason suggested ways to make it slightly less rubbish without completely replacing it - so I made those changes (things like having the tires actually inflated properly). I've now bought a bike, with his advice, that will be suitable for me and the bike ride.

A good analogy is like cycling downhill on a sunny day.

When employing an XP coach, think about how to get the best use of them. There are things you can do to stop a coach being able to help you; don't do them. There are things you can do to help a coach help you; do them.

]]>
Agile related ivan 2007-02-09T18:03:05+00:00
I've been blogtagged http://ivan.truemesh.com/archives/000665.html I've been blogtagged, by Pip and Rob - here’s my 5 things you probably don’t know about me:

  • I'm bald. No, really. I know you've never noticed, but if you look carefully ...
  • I was a windsurfing instructor on a Greek Island for seven months.
  • I used fix my own car - including once changing the engine (with help from a hydraulic hoist).
  • I came second in a national photography competition (this photo copyright Ivan Moore) (a long time ago).
  • I'm doing a cycle ride from London to Paris in aid of the National Autistic Society (OK - you might alreay know this - but have you sponsored me? I thought I should remind you!).

OK - so to spread the meme even further, my five victims are (and there is some overlap with Pip and Rob, for which I don't apologise):

]]>
ivan 2007-01-14T22:08:55+00:00
Simple and useful Java syntax that you might not know http://ivan.truemesh.com/archives/000663.html Nat Pryce introduced me to a bit of Java syntax that is simple, useful and suprisingly little used or known.

It was a suprise to find this syntax after so long - I've been writing Java on-and-off for 10 years - it isn't new syntax and I've pair-programmed with people who have written more Java than me who didn't use this syntax - I don't know why it isn't used much more frequently.

The syntax

Consider this code:

		ArrayList<String> list = new ArrayList<String>();
		list.add("hello");
		list.add("world");

using the syntax that Nat showed me, can be written as:

		ArrayList<String> list = new ArrayList<String>(){{
			add("hello");
			add("world");
		}};

which is neater, particularly if it's a field declaration with assignment.

Any talk of neat syntax must mention Smalltalk

The language which I think has the neatest syntax of any I've used commercially is Smalltalk - it's lovely. In Smalltalk, there is a syntactic construct called "cascading messages" which is similar but more general. The Java syntax above calls the "add" methods on the newly created ArrayList. You can't send multiple messages to the same receiver in the general case, just on construction. For example:

		ArrayList<String> list = new ArrayList<String>();
		list{{
			add("hello");
			add("world");
		}};

is not legal.

Keep it simple

In lots of cases, I avoid using language features or syntax that is unusual. However, in this case, I think this syntax is easy to understand and deserves to be more commonly used.

And finally

Sponsor me for the National Autistic Society's London to Paris Bike Ride. OK - so it's not relevant to the article, but they've been very good to me and my autistic son, and my fundraising could do with a boost.

]]>
ivan 2006-12-28T17:18:16+00:00