This guide to using unit tests to find and fix bugs in software applications is part 3 of our debugging series, focusing on the practical side of debugging.

Unit tests for debugging

In 1972, Edsger W. Dijkstra famously said, “Program testing can be used to show the presence of bugs, but never to show their absence”. So, tests can tell you that a bug exists. But how does just knowing that a bug exists help you when it can take hours to actually find and fix it?

Well, next, we’re going to present a workflow for debugging using unit tests. In this step-by-step guide, you’ll learn how to use this technique to save a bit of your precious time when debugging your software applications. Unit tests can support your debugging workflow in a number of ways.

Triggering the reproducer

Once you are aware of a bug, you will most likely first try to find a way to reproduce it. This might, for example, consist of running your program with certain inputs. When you dive into your code, you will very likely be running the program with that configuration over and over again to trigger the problem. Writing a test that triggers the bug is an easy way to automate this, even if it might not be known yet what the proper behavior is. At least we now have a way to provoke the error with the press of a button, no matter how complex the reproducer is.

Quickly checking if your fix works

When you finally figured out what the problem is, it’s time to fix it. But before you start trying out different ideas or workarounds, it’s generally a good practice to first finish the “triggering” test you already started. Think about what the correct behavior should be and complete the test case. With a proper test that exercises exactly the broken scenario, it’s easy to check if your fixing attempts work or not.

Avoiding regressions

Finally, just because you found the perfect fix doesn’t necessarily mean you’re done. You might have broken something else that was previously working fine. This “Whac-A-Mole” type of scenario is called a “regression” and should, of course, be avoided. A thorough and well-maintained unit test suite is invaluable here, as it instantly notifies you of such newly introduced problems. And, at last, unit tests might also save you from pushing a bug to production in the first place.

Next up, we’re going to present an example for using generated unit tests to debug software that is based on a workshop hosted by Symflower’s very own Simon Bauer at both the Johannes Kepler University Linz and the University of Freiburg in January 2023. Watch a re-recording of that workshop here:

Example: debugging a hash function

In this example we present a simple hash function. Hash functions are important for many computer science domains from security and cryptography to efficient algorithms and compression. A hash function maps a set of input values to output values according to specific requirements, depending on the context where they are used. For example in cryptography, the output must completely obscure the input values and the hash function must be hard to reverse.

The snippet shows a simple hashing function babyHash that receives an integer and returns a hash value. We might, for example, want to compare two contact lists and find the common entries without compromising the actual numbers. For this, we create a hash of each number and compare the hashes to find identical entries.

public class Hashing {
	public static int babyHash(int in) {
		int div = 3;
		for (int i = 0; i < 3; i++) {
			in = in / div;
			div = div + in;
		}
		return div;
	}
}

We repeatedly divide the input through some divisor and update the divisor by adding the remaining input. After three iterations, we return the divisor as our hash. This, of course, is no perfect hash function. But it looks promising if we throw random inputs at it.

babyHash(5486); // 1831
babyHash(3738); // 1249
babyHash(9541); // 3183

Now, let’s assume we get a bug report for this function, that tells us it crashes for input -17. It’s not immediately clear what’s wrong with this input, which is very common for many real-world bug reports. Let’s take a closer look at the problem, following the steps we outlined above!

Before we start diving in, we need to make sure that we won’t break anything. To avoid regressions, we should first have a reasonable test suite that captures the current behavior of our program. Let’s just make this very cheap and add a single case, using the simplest input: 0.

public class HashingTest {
	@Test
	public void babyHashZero() {
		int in = 0;
		int expected = 3;
		int actual = Hashing.babyHash(in);

		assertEquals(expected, actual);
	}
}

Then, we want to write an incomplete test case that triggers the bug, which we can do with the following code. The important part here is that we really only have a quick way of calling the function with the problematic input, so no further “asserts” are necessary. If we execute this test we get an ArithmeticException.

public class HashingTest {
	@Test
	public void babyHashBug() {
		int in = -17;
		Hashing.babyHash(in);
	}
}

Now, we can start our investigation. For example, we might use the debugger of our IDE to inspect how the hash function behaves with this input. Or, we could just print the values of in and div at the end of each iteration, giving the following results:

It seems like the computations of the hash with -17 lead to a div=0, which is problematic since it’s again used as divisor in the next iteration, leading to a division by zero. To fix this, we could just exchange the divisor and input at the division operation: in = div / in;. This way it doesn’t matter if the divisor ever becomes zero. Even though this would fix the bug for -17, it instantly introduces a new one because now we have a division by zero exception in case the input is zero. Luckily, our unit tests contain exactly this case and fail when we try this fix. One proper solution would be to explicitly catch the case where the divisor turns zero and resetting it.

if (div == 0)
	div = 3;
pin = pin / div;

We still need to complete our incomplete test, since now the case -17 doesn’t crash anymore. This leads to the following complete test suite:

public class HashingTest {
	@Test
	public void babyHashZero() {
		int in = 0;
		int expected = 3;
		int actual = Hashing.babyHash(in);

		assertEquals(expected, actual);
	}
           @Test
	public void babyHashBug() {
		int in = -17;
		int expected = 3;
		int actual = Hashing.babyHash(in);

		assertEquals(expected, actual);
	}
}

Here we have again the simplest case plus a unit test now resembling the original bug where the division by zero happened during the first iteration. This exercises the “reset” mechanism we introduced to fix the problem, leading to higher code coverage as well.

Using auto-generated unit tests for debugging

Even in the simple example detailed above, we could have saved quite a bit of time by automating the creation of our unit test cases rather than writing it by hand. Luckily, there’s a tool for that: Symflower can automatically generate the boilerplate code for your unit tests so all you need to do is fill in the right values to cover all potential cases. In some cases, Symflower will also be able to generate full unit test suites complete with meaningful values to help you explore all potential program paths. A comprehensive test suite with your own test cases complemented by Symflower’s findings will help make sure you have adequate test coverage of your code.

Ready to try Symflower for your own projects? Get the plugin to start automating the creation of unit test templates or complete unit tests with Symflower, and let us know what you think. For more programming wisdom and best practices, join Symflower’s newsletter and follow us on Twitter, LinkedIn or Facebook.