At the heart of programming is the test. Without a good test, nothing will work right. How often should you test and what should you test? We will cover a few topics in the world of testing and how you can make tests that tell you the most about what is going on with your programming… on this entry of the Programming Underground!
Every day we get people on the boards asking why something is not working or that they get errors and don’t know how to solve them. Half the time we can just look at the code and see what might be going on, but other times we conduct a few tests on the source code to see what is breaking. Most of the time I just look at the error messages which tell you what is up. Syntax errors are always the easiest to catch and logic ones are bit harder, but using a few methods described below you can quickly nail down most errors.
But before we dive into the methods I want to talk briefly how often one person should test something. The answer is “as much as needed”. What does that mean Martyr? Well it means that if you are doing something a bit fancy or using some new functions or putting something in a completely new way you probably should be testing quite a bit. They say that the average programmer only spends between 30-50% of their time writing the code and the rest dedicated to testing. Now I am not saying test every line you write like testing int myvar = 5; because obviously that is straight forward to most people. However if you are nesting multiple if statements inside one another, declaring complex objects, using multiple function calls… you might want to test and make sure it is doing exactly what you expect it to do. So here are some tips you can use…
1) Print out values so you can see them at different points of the program. – This is the number one tip every programmer should know. If things are not working out quite right, use a cout or a echo or a MessageBox.Show to see what the value of your variables are or what is the result of a function call. In really tough functions I have made a “print” call every other line before. Now in the .NET environment and using the visual studio IDE you can do this in debugger, but in some languages there is no such feature so you can use this method to help.
When you see the values out on the screen, you can watch them change through the program and can quickly isolate when the problem occurs. For instance, if I had a loop that printed one through 5 and something is going wrong, I might put a print statement in there to show the index variable value on each loop.
The value of i is: 1 The value of i is: 2 The value of i is: 3 The value of i is: 5
From the output I can see that something is happening to “i” around the value number 4. So obviously the counter is being incremented one too many times because it is reaching 5 without showing 4.
We can use this trick to show the value of function calls and what they return or the value of state (is the checkbox checked? Return that in an alert statement in javascript and see if it has a value of “on” and see).
2) Comment code out – Once you know an error is in a general area, try commenting what you can out. Do it one line at a time or a group of known “bug free” code and see if you can wipe out the error. Once the error line has been cleared out with a comment, then you know which line is causing the problem. This method would be your number 2 mostly used method. With the print and the commenting out method, you can quickly find problems. Comment out a function call and you neutralize that function from executing and possibly generating an error. Got some package in your code that may cause the problem, comment it out and see if that resolves the problem. In some situations I have almost commented out an entire program and then uncommented things I knew worked until I had only the problem function/line left.
3) Read your error messages and pick out the crucial items… line number, error number, does it mention a name or something you have created like a variable or function name? – With these three pieces of information you can quickly isolate where something is happening. It may not always be on a certain line, but most compilers and debuggers will show you “roughly” where it is happening. Then once there you can look up the error number to see what it means and what in that general area might be causing the message.
4) Put some variables into a known state and see what comes back – Getting some errors with your variables being passed to a function? Is that variable being set with the result of another function call or dynamically? Try setting the variable to an acceptable value. If you have an integer that is suppose to accept a number between 1 and 10 and it is causing problems later, hardcode it to the value 1. Does it work? Hardcode it to the value 10? Does it work? If they all work, then you know the function result which is coming back from another function to set that integer is returning a bad number. Maybe it is returning 20.
5) Test small things until they work flawlessly, then expand on it. (aka Unit testing) – Test small functions by themselves. You can use a driver program (a test program or often called a test harness program) to make sure the function works in all cases. Once you know a function works perfectly, move to another function that may use that function you just tested. Test that function. Go up the chain like that until you run into your main program function like main() or a startup() function or even something like form_load. Whatever the highest program function is (where the program starts its execution). Then test how all the pieces of the code work together and throw everything you can at it to make it break (system testing).
6) Test the range of values that are acceptable, then try values not accepted. – If you have a function that expects an integer that is between the values 1 and 10, throw all ten values at it and see if handles them all. If the range it too big, throw out random ones and always test the end values… in this case the values 1 and 10 (the lowest and highest values). Then if all is working there, throw something like -5 at it and see what it does. That right there will flush out some bugs too.
7) Last but not least, write readable code and then comment as to what it does! – Not just to tell others what you are doing, which is good practice, but also as you write the steps of how things are done you can quickly find logic errors in your code. If you attempt to describe step 3 where you set a variable to 5 and you see your code setting a variable to 7, you can see something is wrong. If you comment as you code you will then be able to go back and read the steps in order and often times it will show you if there is a problem with your logic. It isn’t exactly a test, but it does help you find those bugs that other testing methods may have missed.
By all means these are not all the types of tests, but these are a few of the bigger ones. I always recommend that someone get a book on how to debug in addition to their programming books. Some books I have read on the subject include Code Complete 2 by Steve McConnell and The Science of Debugging by by Yuan Hsieh and Matthew Telles. The debug book you get will generally be very general in the language examples it uses, but it does show you how to identify problems and you can then adapt it to your specific language needs. These types of tests are also very generic and can be applied to just about every language.
So keep on testing and after all tests don’t yield any results, let the DIC staff take a crack at it and perhaps we will be able to setup that one test that will show us where your problem is.
Oh and one last thing…. PLEASE test your code before submitting it to the boards. We are human beings you know, not your compiler. If I ever run across code that I know hasn’t even been run through a compiler, I will tell you to do that before I get into depth on a solution for you. There is no excuse seeing a line like int myvar = “hello there”;
Good luck at the front lines of the programming war! 🙂
