Getting started with Symflower

Quick start guide to Symflower

Symflower offers three main features:

1. Generate test template
2. Generate test suite (BETA)
3. Test-backed diagnostics

We recommend you start with Generating test templates:

• Change the directory to the workspace you want to analyze and run the following command:

symflower unit-test-skeletons

Each source code file receives a generated test file.

• To generate test templates for single directories and files instead of the whole workspace, use:

symflower unit-test-skeletons src/some/directory src/some/file.go src/some/file.java

• For a list of available commands, use:

symflower --help

See our tutorial below for a step-by-step example of using Symflower’s features!

Tutorial example

For this example, we’ll be using a simple copy function. The specification for this task is to create a copy function that copies an array of strings from the parameter from to the parameter to and then returns to so that callers can use the function in an expression.

Let’s start by defining the function’s API as a stub. Create and switch to the directory tutorial and create a file called Copy.java in the directory with the following content:

class Copy {
    static String[] copy(String[] from, String[] to) {
        return to;
    }
}

Per the specification, our first iteration of the function returns to. The copying functionality is not yet implemented. However, since we now have compilable source code, we can already use Symflower to generate our first unit test template that we can use as a starting point for our first unit test as it provides all the necessary boilerplate code.

1. Generate test template

To trigger test template generation, use symflower unit-test-skeletons.

Congratulations, you just generated your first unit test template with Symflower! You’ll see a message with information about the analysis, and you’ll also notice that a unit test file CopyTest.java was generated with the following content:

import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;

public class CopyTest {
        @Test
        public void copy() {
                String[] from = { "abc", "abc", "abc" };
                String[] to = { "abc", "abc", "abc" };
                String[] expected = { "abc", "abc", "abc" };
                String[] actual = Copy.copy(from, to);

                assertArrayEquals(expected, actual);
        }
}

This test case represents the simplest unit test for our function: using abc for the parameters from and to which lets the function return abc. Upon execution, we’ll see that this unit test fully covers our source code. You can use this code template as a boilerplate to create further unit tests, filling it in with new values to create new test scenarios.

2. Generate test suite (BETA)

Let’s implement the actual copy functionality of the specification. There are several ways to do this. For now, let’s go with the simplest one where we copy elements one by one. This code is easy to read and understand, and works the same way in most languages:

class Copy {
    static String[] copy(String[] from, String[] to) {
        for (int i = 0; i < from.length; i++) {
            to[i] = from[i];
        }

        return to;
    }
}

Now that we have actual functionality, we can use Symflower to generate a complete test suite to test all possible paths of our code. Simply use the command: symflower

This will yield 5 unit tests. In your CopyTest.java file, there are now 3 unit tests that fail because the function throws an exception and 2 that pass. Let’s look at the passing tests first to see if it is what we want:

@Test
        public void copy4() {
                String[] from = { null };
                String[] to = { null };
                String[] expected = { null };
                String[] actual = Copy.copy(from, to);

                assertArrayEquals(expected, actual);
        }

We copy from with a single null element into to with a single null element. The expected return argument is an array with a single null as its element. This is definitely according to our specification. Let’s look at the other passing test:

	@Test
	public void copy2() {
		String[] from = {};
		String[] to = null;
		String[] actual = Copy.copy(from, to);

		assertNull(actual);
	}

We copy from with no elements into to which is null and return null. But hold on! Our specification says that we are copying from one array into another. So we need to implement a check to make sure that our parameters are not null. When one of the parameters from or to is null, we found an invalid usage of our API. Java’s IllegalArgumentException is made for cases like this. Let’s add it to our code:

class Copy {
    static String[] copy(String[] from, String[] to) {
        if (from == null || to == null) {
            throw new IllegalArgumentException();
        }

        for (int i = 0; i < from.length; i++) {
            to[i] = from[i];
        }

        return to;
    }
}

Generating new tests with symflower gives us 5 tests once again. The unit test with the empty strings as elements in to and from still exists, but the other unit tests have changed.

There are now 2 unit tests that exercise our new validation logic:

        @Test
        public void copy1() {
                String[] from = null;
                String[] to = null;
                assertThrows(IllegalArgumentException.class, () -> {
                        Copy.copy(from, to);
                });
        }

        @Test
        public void copy2() {
                String[] from = {};
                String[] to = null;
                assertThrows(IllegalArgumentException.class, () -> {
                        Copy.copy(from, to);
                });
        }

We also have a unit test that has empty arrays as parameters and returns an empty array:

        @Test
        public void copy3() {
                String[] from = {};
                String[] to = {};
                String[] expected = {};
                String[] actual = Copy.copy(from, to);

                assertArrayEquals(expected, actual);
        }

Awesome! So far, everything is either according to our specification or throws an IllegalArgumentException. Let’s look at the last generated unit test:

	@Test
	public void copy5() {
		String[] from = { "" };
		String[] to = {};
		// assertThrows(ArrayIndexOutOfBoundsException.class, () -> {
		Copy.copy(from, to);
		// });
	}

Symflower found an out-of-bounds exception since we are copying from an array with 1 element to an array with 0 elements.

What we should do in this case is not defined in the specification: we could either again throw an IllegalArgumentException if the parameters are of different size; we could allocate a new array with the correct size; or we could copy just the elements that fit in the to array. The best solution, of course, is to talk to our team and clarify the specification. We can then adapt the code and use Symflower to make sure that everything works as expected and that all edge cases are covered.

3. Test-backed diagnostics

At this point, you may have already noticed the logs printed by Symflower upon test generation. The log will highlight any uncaught runtime exceptions. To find out what types of values cause the exception, take a look at the generated tests.

As you continue working on your implementation, Symflower will automatically maintain the generated unit test suite to keep your code covered with test cases!

Next steps

For further examples, check out Symflower’s documentation, tutorials repository, and examples repository.

Please send us your feedback on Symflower and your experience with the generated tests! If you are missing a feature or found a problem, open an issue on our community issue tracker or drop us a line at hello@symflower.com. Visit our blog and subscribe to the Symflower newsletter to learn about new features, useful guides, and others topics on software development & testing!