JUnit testing tips and tricks 1/3: How to ignore test cases in JUnit 4 and 5?

Part 1 of our series on JUnit testing tips and tricks is about ignoring test cases in JUnit 4 and JUnit 5.

JUnit is the most popular testing framework used by thousands of developers. If you’re new to the framework, there are a few neat tips and tricks you can learn to make your use of the framework smoother and more efficient. In part 1, we’re covering the topic of ignoring (or skipping) test cases, whether you’re using JUnit 4 or JUnit 5!

Ignoring test cases in JUnit

Ignoring test cases can be useful in cases where you don’t need to run all the test cases in your suite. Ignoring tests helps save testing resources.

For instance, if smaller code changes don’t affect certain test cases, there’s no point in running them again (provided those tests have already run on the affected sections of code). In other cases, the code for the corresponding test case may still be under development, so you want to avoid running them for now. In these situations, you’ll want to skip a few test cases to only run selected tests.

Depending on whether you’re using JUnit 4 or 5, there are different ways to do this.

How to ignore test cases in JUnit 4: @Ignore

JUnit 4 offers the @Ignore annotation that you can use to tag the test cases you want to skip. You’ll use this annotation like any other annotation:

import package org.junit.Ignore;

...

@Ignore
@Test
public void testToSkip() { ... }

Note that @Ignore takes an optional default parameter which enables you to add a note about why the test case is skipped:

@Ignore("code under development")
@Test
public void testToSkip() { ... }

The neat thing about @Ignore is that you can also apply it to an entire test class, providing more flexibility around managing tests:

@Ignore
public class IgnoreTest {
    @Test
    public void test1() { ... }

    @Test
    public void test2() { ... }
}

How to ignore test cases in JUnit 5: @Disabled

JUnit 5 changes how test cases are skipped. Instead of @Ignore, JUnit 5 provides the @Disabled annotation.

While the annotation itself has changed in JUnit 5 compared to JUnit 4, you’ll use it in a similar way to skip tests:

import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;

...

@Disabled
@Test
public void testToSkip() { ... }

@Disabled also takes an optional parameter to provide information on why the test is skipped:

@Disabled("code under development")
@Test
public void testToSkip() { ... }

And like @Ignore in JUnit 4, you can use @Disabled with entire test classes:

@Disabled("code under development")
class IgnoreTest {
    @Test
    void testToSkip() {
    }
}

If you’re looking to skip test methods within a given test class, use it the following way:

class IgnoreTest {
    @Disabled("Disabled until bug #31 is resolved")
    @Test
    void testToSkip() {
    }

    @Test
    void testToExecute() {
    }
}

+1: Conditionally running test cases in JUnit

There are situations where you may want to skip running certain tests only if certain conditions are true. You could of course use a bunch of if statements to get there. But both JUnit 4 and 5 support a number of neat, elegant ways to do this without using a number of if statements!

To conditionally run or ignore test cases in JUnit 4, you’ll need to use the Assume class. In JUnit 5, you have a few options. You can either extend the functionality of @Disabled with the @DisabledIf annotation (@EnabledIf also works, but note that these are both deprecated in JUnit 5.5), use one of the built-in conditions (see a list below), or create a custom implementation of ExecutionCondition which resides in JUnit Jupiter.

Let’s see how each of these should be used!

How to conditionally run tests in JUnit 4: the Assume class

The Assume class lets you state assumptions about the conditions for test execution: If the conditions you specify are passed, the test will run. If the conditions are not passed, JUnit will skip execution (Note: that only applies if you’re using JUnit’s default runner. There’s no guarantee that other or custom runners will behave the same way, so make sure you read the documentation or test things beforehand). Skipped tests will be marked as passed in the test report.

Using the Assume class in JUnit 4 gives you the following options to state assumptions about a test being meaningful: assumeFalse() assumeNoException() assumeNotNull() assumeThat() assumeTrue() While you can use these methods directly, e.g.: Assume.assumeTrue(...), it’s just nicer if you choose to reference them through static import:

import static org.junit.Assume.assumeTrue;

class AssumeTestJUnit4 {
    @Test
    void TestAssumeExampleSkippedTest() {
        assumeTrue(1 > 5) // A non meaningful test.
    }

    @Test
    void TestAssumeExampleRunTest() {
        assumeTrue(1 < 5) // A meaningful test.
    }
}

Note that the assumeFalse and assumeTrue functions will still work in JUnit 5, but there are better ways to get there in JUnit 5.

How to conditionally run tests in JUnit 5: assume*, @DisabledIf, built-in annotations, and ExecutionCondition

JUnit 5 offers a variety of additional ways to tackle conditional test execution. Like with JUnit 4, you can still use assume(...) with JUnit 5. This is an elegant solution as it allows you to reuse these conditions in other functions once an assumption is defined:

import static org.junit.jupiter.api.Assumptions.assumeTrue;

class AssumeTestJUnit5 {
    @Test
    void TestAssumeTrueSkippedTest() {
        // For this case we define that our Order quantity is 4.
        assumeTrue(Order.minimumQuantity(6)) // We do have a lower quantity (4 < 6) so it does not make sense to run this test.
    }

    @Test
    void TestAssumeTrueRunTest() {
        // For this case we define that our Order quantity is 4.
        assumeTrue(Order.minimumQuantity(2)) // Our quantity is 4 so running this test would make sense.
    }
}

Alternatively, you could just go with @DisabledIf (and its counterpart @EnabledIf), which still both work in JUnit 5, but note that these are deprecated in JUnit version 5.5 and removed from 5.6!

Instead, it’s recommended that you use one of the conditions that come built into JUnit 5, or implement a custom solution using ExecutionCondition if your project requires a more sophisticated way for conditional execution.

In JUnit 5, built-in conditions with annotations include:

In case you need a more sophisticated solution, you can implement custom conditions using ExecutionCondition.

If you’re registering several ExecutionCondition extensions, the container or test will be disabled in case either one of the conditions returns disabled. Note: that means not each and every condition may be evaluated, since there is a chance the test will already be disabled due to another condition returning disabled earlier.

Let’s take a look at a custom execution condition, and how you can utilize them in your tests.

For the sake of brevity, let’s assume there is a production class Order that defines the minimum order quantity like this:

public class Order {
    static final int minimumQuantity = 4;
}

For the custom execution condition we first need to define a new annotation type that can be used on methods and types:

import org.junit.jupiter.api.extension.ExtendWith;

import java.lang.annotation.Retention;
import java.lang.annotation.Target;

import static java.lang.annotation.ElementType.*;
import static java.lang.annotation.RetentionPolicy.RUNTIME;

@Target({ METHOD, TYPE, ANNOTATION_TYPE })
@Retention(RUNTIME)
@ExtendWith(EnabledIfMinimumQuantityCondition.class)
public @interface EnabledIfMinimumQuantity {
    int quantity();
}

Next, we define the actual logic of the custom ExecutionCondition by overriding the method evaluateExecutionCondition, specifying that a test method should only be run in case the order quantity specified in the used annotation exceeds the defined Order.minimumQuantity:

package org.example;

import org.junit.jupiter.api.extension.ConditionEvaluationResult;
import org.junit.jupiter.api.extension.ExecutionCondition;
import org.junit.jupiter.api.extension.ExtensionContext;

import java.lang.reflect.AnnotatedElement;

import static org.junit.jupiter.api.extension.ConditionEvaluationResult.disabled;
import static org.junit.jupiter.api.extension.ConditionEvaluationResult.enabled;
import static org.junit.jupiter.params.shadow.com.univocity.parsers.annotations.helpers.AnnotationHelper.findAnnotation;

public class EnabledIfMinimumQuantityCondition implements ExecutionCondition {
    @Override
    public ConditionEvaluationResult evaluateExecutionCondition(ExtensionContext context) {
        AnnotatedElement element = context
                .getElement()
                .orElseThrow(IllegalStateException::new);

        return checkQuantity(findAnnotation(element, EnabledIfMinimumQuantity.class), element);
    }

    private ConditionEvaluationResult checkQuantity(EnabledIfMinimumQuantity annotation, AnnotatedElement element) {
         if (Order.minimumQuantity<=annotation.quantity()) {
            return enabled("Minimum quantity available");
        } else {
            return disabled("Minimum quantity not available");
        }
    }
}

Finally, the new custom execution condition can be used like this:

import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertTrue;

public class ConditionEvaluationJUnit5 {
   @Test
   @EnabledIfMinimumQuantity(quantity = 6)
   void TestRunTest() {
       // For this case we assume that our Order quantity is 4.
       // Because of that this test would be run.
       assertTrue(true);
   }

   @Test
   @EnabledIfMinimumQuantity(quantity = 2)
   void TestSkippedTest() {
       // For this case we assume that our Order quantity is 4.
       // Because of that this test would not be run.
       assertTrue(true);
   }
}

Note that TestRunTest will be run, while test TestSkippedTest will be skipped upon test execution.

So there you have it! That’s about everything you need to know about the basics of skipping tests in JUnit 4 and 5, which we hope will help save you a bunch of time and testing resources. Curious about another way to boost your testing efficiency?

Symflower integrates into your workflow to generate test templates that you can reuse as boilerplate code to make the creation of tests faster. Better still, Symflower has a beta feature that also calculates the right values for you! Try Symflower in your IDE free of charge.

Make sure you never miss any of our upcoming content by signing up for our newsletter and by following us on Twitter, LinkedIn or Facebook!