JUnit testing tips and tricks 2/3: Tagging and filtering test cases in JUnit 4 & JUnit 5

In part 2 of our series on JUnit testing tips and tricks, we cover the topic of tagging and filtering test cases in both JUnit 4 and JUnit 5.

In the first part of this series, we covered how to ignore test cases in JUnit 4 and 5.

Tagging and filtering test cases in JUnit 4 and JUnit 5

When running test cases, you may want to run specific tests or groups of test together. That’s the use case for filtering, and tagging/categorizing test cases, which works differently in JUnit 4 and 5. If you want to omit certain test cases from your test runs, check out the following post in this series!

In general, whether you’re using JUnit 4 or 5, you have three options:

• Build system: Using Maven or Gradle either with command line arguments or configuration files for filtering.
• Grouping test classes into test suites: Using @SuiteClasses (JUnit 4) or @SelectClasses (JUnit 5) to create custom suites
• Tagging/Categorizing classes and test methods: Using @Category (JUnit 4) or @Tag (JUnit 5)

If you don’t use Maven or Gradle and still want to execute specific test methods with JUnit 4 using the CLI, you’re out of luck. With JUnit 5, you can still filter via the CLI. Let’s see how each of these ways of tagging and filtering test cases work for both JUnit 4 and JUnit 5!

Using your build system to filter tests with JUnit 4 or JUnit 5

How to filter test cases with Gradle?

As outlined in our article titled How to run JUnit 5 tests with Gradle?, Gradle offers two options for filtered test execution:

• Defining a filter in build.gradle
• Using the --test option in the command line

This latter one is particularly useful in case you want to quickly execute some test cases i.e. for debugging. Here’s the description from Gradle’s documentation.

Both options work with JUnit 4 and JUnit 5. Let’s see a quick example. For instance, if your goal is to only run the test named appHasAGoodbye, you’ll use the following command in the CLI:

gradle test --tests gradle.demo.AppTest.appHasAGoodbye

Where gradle.demo is the package in which the class AppTest resides in, and appHasAGoodbye is the test function to be run.

How to filter test cases with Maven?

Maven also offers options for selective test execution. To execute only the default test phase, use the following command in CLI:

mvn surefire:test -Dtest=appHasAGoodbye

Note that you can also use patterns to select tests to run:

mvn surefire:test -Dtest=*Goodbye

Multiple names and patterns can also be combined:

Dtest=”appHasAHello,*Goodbye”

Here’s the relevant section of the Maven documentation for running a single test with Maven, and including or excluding test cases from test runs.

How to filter test cases in JUnit 4?

Selectively running tests in JUnit 4 using @RunWith and @SuiteClasses

For selecting certain test cases to be run together, you can use @RunWith with @SuiteClasses. @RunWith resides in org.junit.runner.RunWith and lets you have JUnit 4 invoke the referenced class (instead of the built-in runner) to run tests in that class.

For filtered test running, combine it with @SuiteClasses (found in org.junit.runners.Suite) which lets you pass a parameter Suite.class into the annotation @RunWith. This annotation enables you to specify the classes to be run when running tests on a class annotated with @RunWith(Suite.class).

To group test cases together, add the appropriate test class names as a string array (separated by commas) as a parameter to @SuiteClasses. This solution gives you an option to manually group JUnit test classes which can then be run together, providing an option for filtered test running (e.g. building a test suite that only runs selected test classes at execution).

Let’s take a look at a simplified example! First, we have two regular test classes that we want to group together into a test suite. The two regular test classes look as follows:

Test class #1

package org.example;

import org.junit.Test;
import junit.framework.Assert;

public class AdditionTest {
   @Test
   public void onePlusOne(){
       int a = 1, b = 1;
       int result = a + b;
       Assert.assertEquals(2, result);
   }
}

Test class #2

package org.example;

import org.junit.Test;
import junit.framework.Assert;

public class SubtractionTest {
   @Test
   public void oneMinusOne(){
       int a = 1, b = 1;
       int result = a - b;
       Assert.assertEquals(0, result);
   }
}

• Test class #1 verifies whether additions work properly.
• Test class #2 verifies if subtractions work as expected.

(Naturally, you wouldn’t normally write a unit test to verify if Java adds/subtracts correctly, but we do now for the sake of the example).

Next, we want to combine those two test classes into a test suite which always runs all tests of these two classes together. Here’s how to do that:

package org.example;

import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses({AdditionTest.class, SubtractionTest.class })
public class CalculatorTestSuite {}

Note that by running CalculatorTestSuite, for instance via your IDE (in the below case we used IntelliJ), you get a nice overview of the test suite, and which classes and test methods have been included in its execution:

How to categorize tests in JUnit 4 with @Category, @IncludeCategory, and @ExcludeCategory?

To categorize tests into different sets using JUnit 4.12 and up, use the @Category, @IncludeCategory, and @ExcludeCategory annotations.

This is useful as it lets you group together tests for e.g. unit testing, smoke testing, regression testing, etc. To use it, just **annotate the test methods of the classes you want to include in the @SuiteClasses with @Category** (residing in org.junit.experimental.categories.Category) and a name for the category. You’ll also need to define an interface for each category that you want to use. Then, in the test suite file, use @IncludeCategory, and @ExcludeCategory with the category’s name to include or exclude all the tests that belong to the identified category.

For more granularity and control, you can use @Category on the test method level or the class level.

In order to tag tests or test classes with a certain type, we first need to define an interface specifying the type like this:

public interface UnitTest {}

public interface IntegrationTest {}

In this example, we’ll use the above categories to mark some tests as unit tests and others as integration tests:

package org.example.categories;

import org.junit.Test;
import org.junit.experimental.categories.Category;

public class BookTest {
   @Test
   @Category(UnitTest.class)
   public void setAuthor(){
       // logic to set the author of a book
   }

   @Test
   @Category(UnitTest.class)
   public void setPlublicationDate(){
       // logic to set the publication date of a book
   }
}

The test class BookTest includes two unit tests named setAuthor and setPlublicationDate.

Next, there is a test class containing two integration tests:

package org.example.categories;

import org.junit.Test;
import org.junit.experimental.categories.Category;

public class LibraryTest {
   @Test
   @Category(IntegrationTest.class)
   public void lendBook(){
       // logic to test lending a book from the library.
   }

   @Test
   @Category(IntegrationTest.class)
   public void deleteBook(){
       // logic to test deleting a book from the library.
   }
}

Note that a test may belong to more than one category, e.g. @Category(UnitTest.class, SmokeTest.class).

Here’s how you would define a test suite that only executes unit tests:

package org.example.categories;

import org.junit.experimental.categories.Categories;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;

@RunWith(Categories.class)
@Categories.IncludeCategory(UnitTest.class)
@Suite.SuiteClasses({BookTest.class, LibraryTest.class})
public class UnitTestSuite {
}

Note that while the test class LibraryTest has been added to the test suite, none of its methods are executed as they are all integration tests.

The test execution looks as follows:

How to filter test cases in JUnit 5?

How to group test classes? Using @RunWith and @SelectClasses to selectively run test cases in JUnit 5

With the same code for SubtractionTest and AdditionTest, grouping into a test suite works as follows with JUnit 5:

package org.example;

import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.Suite;

@Suite
@SelectClasses({AdditionTest.class, SubtractionTest.class })
public class CalculatorTestSuite {
}

Which again yields the handy overview we saw in JUnit 4 above:

Note that JUnit version 5.10.1 or above is necessary for the above code to work. Also, the following dependencies need to be added to your Maven file:

<dependency>
   <groupId>org.junit.jupiter</groupId>
   <artifactId>junit-jupiter-api</artifactId>
   <version>5.10.1</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.junit.jupiter</groupId>
   <artifactId>junit-jupiter-engine</artifactId>
   <version>5.10.1</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.junit.platform</groupId>
   <artifactId>junit-platform-suite-api</artifactId>
   <version>1.10.1</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.junit.platform</groupId>
   <artifactId>junit-platform-suite-engine</artifactId>
   <version>1.10.1</version>
   <scope>test</scope>
</dependency>

JUnit 5 also offers a @SelectPackages annotation which enables you to group tests from multiple package(s), with the right packages represented as a string array value in the parameters of @SelectPackages.

How to categorize & filter tests in JUnit 5 using @Tag, @IncludeTags and @ExcludeTags?

For organizing and marking tests and test sets for easy filtering, JUnit 5 offers a handy approach with test tagging using @Tag.

Different methods in a class may have different tags, and you can also use @Tag on test classes. There’s also the option of using expressions in @Tags.

Continuing with our library example, here’s how you would tag certain methods as unit or integration tests. Note that it is no longer necessary to define interfaces for each tag:

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

public class BookTest {
    @Test
    @Tag("UnitTest")
    public void setAuthor(){
        // logic to set the author of a book
    }

    @Test
    @Tag("UnitTest")
    public void setPlublicationDate(){
        // logic to set the publication date of a book
    }
}

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

public class LibraryTest {
    @Test
    @Tag("IntegrationTest")
    public void lendBook(){
        // logic to test lending a book from the library.
    }

    @Test
    @Tag("IntegrationTest")
    public void deleteBook(){
        // logic to test deleting a book from the library.
    }
}

You’ll then use @IncludeTags and @ExcludeTags (instead of JUnit 4’s @IncludeCategory and @ExcludeCategory annotations) to selectively run tests.

Alternatively, use @IncludePackages, and @ExcludePackages to include or exclude specified packages when running a test suite.

JUnit 5 also lets you use @IncludeClassNamePatterns and @ExcludeClassNamePatterns to include or exclude classes that match a specific regular expression from the package.

The following example shows you how to include any tests that are tagged with “UnitTest” in the package org.example.tags:

import org.junit.platform.suite.api.IncludeTags;
import org.junit.platform.suite.api.SelectPackages;
import org.junit.platform.suite.api.Suite;

@Suite
@SelectPackages({"org.example.tags"})
@IncludeTags("UnitTest")
public class UnitTestSuite {
}

Like with JUnit 4, we are presented with the following output:

Overall, there are a few differences between JUnit 4 vs JUnit 5 in how you categorize and filter tests:

• JUnit 5 no longer makes it necessary to define interfaces for each tag type.
• You no longer have to list all classes that you want to include in a test suite.
• Selecting individual packages to be included in a test suite is also very easy in JUnit 5.

Summary: tagging and filtering test cases in JUnit 4 and JUnit 5

So that’s all folks, that’s how you tag and filter tests in both JUnit 4 and 5! Make sure your check out our post outlining the best Java unit testing frameworks and tools, and our quick comparison of JUnit and TestNG.

Looking for more JUnit wisdom? Learn about ignoring test cases in JUnit 4 and 5 in part 1 of this series. Stay tuned for part 3 which will cover changing the order of test execution in both JUnit 4 and 5!

Make sure you never miss any of our upcoming content by signing up for our newsletter and by following us on Twitter, LinkedIn or Facebook!