Making a Simple Test Harness

Again and again we get people on the Dream.In.Code board saying something along the lines of “I have this problem with my project, I think it is this function giving me the trouble, but can someone help me figure out what is going on?” What is the first thing the pros do in such a situation? They create what is known as a test harness (aka driver program) to test the function on its own. Lets give one a spin real quick and see how they work on this entry of the Programmer’s Underground!

I am writing this entry in response to a recent post about a function that tests for a maximum value in a 2 dimensional array in Java. The original answer I had given was right off the top of my head and not tested. This resulted in one slight bug in the function and little did I know it. I hadn’t discovered it until I created a quick test program to try it out. Yes even us professionals do make mistakes from time to time… but what do you expect when it comes off the top of your head and expecting for it to be perfect?

Anyways… their problem wasn’t related to the function itself but how the array was passed to the function. So to test things out and, in the process, give them an example showing them how things work I created a small snippet of code which could run on its own but so simple that it only included the main function and the function I was testing. Nothing else. No extra variables or huge libraries and no extraneous code. So lets take a look at it…

public class testharness {
	public static void main(String args[]) {
		double[][] inputCelcius = new double[2][2];
		inputCelcius[0][0] = 10.0;
		inputCelcius[0][1] = 5.78;
		inputCelcius[1][0] = 7.88;
		inputCelcius[1][1] = 6.23;
		
		double topValue = max2(inputCelcius);
		
		System.out.println("Top value in second dimension is: " + topValue);
	
	}
	
	public static double max(double[][] inputCel) {
	     // Set initial value for maximumValue
	     double maximumValue = inputCel[0][1];

	     // Now loop through the array and set maximumValue to a higher value as we come across it
	     for (int i = 0; i < inputCel[i].length; i++) {
	          if (inputCel[i][1] > maximumValue) {
	               maximumValue = inputCel[i][1];
	          }
	     }

	     // Now that maximumValue is defined in function scope, we can return it.
	     return maximumValue;
	}
}

Before we run the test we use predetermined values to load up the array and predict what the outcome will be. Looking from our values, we expect the value 6.23 to be the maximum value in the second dimension. The value 10.0 and 7.88 are in the first dimension so those are not counted. The second dimension has 5.78 and 6.23 with 6.23 obviously being the higher one.

Notice how simple this program is. Just main(), which will call our function and collect the return value, and then our function we want to test. If we get an error in the return value then we know that our function is not working correctly. We can keep working on the function until we run it with values and get correct responses. By using this method we are tightening the testing environment around our function and making something small and quick that will allow us to throw numerous values at a single function and check for weaknesses.

Once we determine that this function is strong and correct, then we can safely cut and paste this from our test app into the big tamale knowing that we have something which is functionally strong. Testing it in the bigger application if we get an error, then we know that something coming into the function is probably the problem and not the function.

You may know where I am going with this. If you do this enough times for each function you can build a rather strong application with fewer bugs. In programming methodology circles you may here this as “unit testing” and then running it in the bigger application with the other functions you have tested would be “integration testing”.

Now running our code above you are going to discover a bug. The bug is with the section inputCel[i].length. Since this is in a loop, there will be a time where the value of “i” is going to be one value higher than the index of the inputCel’s first dimension. This will lead to an “Array index out of bounds” error. So to correct it we put inputCel[0].length Which will get the length of the dimension from the first row of the 2D array and uses that to determine when to stop. This works on 2D arrays and not jagged arrays. So be mindful of that.

With unit testing we were able to isolate this problem at this small functions scope instead of trying to dig through thousands of lines of code in the bigger application and messing with other functions trying to solve the issue. We caught the error in the driver program and were able to correct it quickly and effortlessly. Now that we corrected it and everything seems to work as expected, we can move this function out of our test program and into our bigger program.

We can then save the test program and put in another function for testing. So one test application can be written to test various functions or things like classes etc with just a little work.

So you might be saying “Why do everything twice like this? Why not throw it into the other program, run it, and see where the compiler tells you the error is?” There are two reasons for this. One, you are cutting down the places you need to look to find the error…. just in case the compiler is wrong as to where the error really lies. You are decreasing the code footprint and able to quickly isolate problems closer to their source. The second reason is that when it would take you only 10 minutes to setup the test and find your problems, it is better than looking for 2 hours and trying to find the reason for the failure in the bigger application. This is a great reason for those programming students who need to get something done for tomorrow and want to prevent those late nights of trying to figure out what is going on and having to ask us (after hours of pulling their hair out).

You may have several test cases like this where one is for testing functions, one might be for testing classes and another might be testing whole subsystems. You could run the same code through all three. One in the function testing harness, add that function to a class and test it in the class test harness, and take that class and put it into a test harness for a subsystem.

Design the test programs correctly and they will be reusable from project to project and save you even more time. Then of course you will be spending your time partying with the girls down the hall in your dorm than having to worry about that programming assignment due tomorrow.

Hope it helps! 🙂