com.carrotsearch.randomizedtesting

Class RandomizedRunner

java.lang.Object

org.junit.runner.Runner

com.carrotsearch.randomizedtesting.RandomizedRunner

All Implemented Interfaces:

Describable, Filterable

public final class RandomizedRunner
extends Runner
implements Filterable

A Runner implementation for running randomized test cases with predictable and repeatable randomness.

Supports the following JUnit4 features:

• BeforeClass-annotated methods (before all tests of a class/superclass),
• Before-annotated methods (before each test),
• Test-annotated methods,
• After-annotated methods (after each test),
• AfterClass-annotated methods (after all tests of a class/superclass),
• Rule-annotated fields implementing MethodRule and TestRule.

Contracts:

• BeforeClass, Before methods declared in superclasses are called before methods declared in subclasses,
• AfterClass, After methods declared in superclasses are called after methods declared in subclasses,
• BeforeClass, Before, AfterClass, After methods declared within the same class are called in randomized order derived from the master seed (repeatable with the same seed),

Deviations from "standard" JUnit:

• test methods are allowed to return values (the return value is ignored),
• hook methods need not be public; in fact, it is encouraged to make them private to avoid accidental shadowing which silently drops parent hooks from executing (applies to class hooks mostly, but also to instance hooks).
• all exceptions raised during hooks or test case execution are reported to the notifier, there is no suppression or chaining of exceptions,
• a test method must not leave behind any active threads; this is detected using ThreadGroup active counts and is sometimes problematic (many classes in the standard library leave active threads behind without waiting for them to terminate). One can use the ThreadLeaks annotation to control how aggressive the detection strategy is and if it fails the test or not.
• uncaught exceptions from any of children threads will cause the test to fail.

See Also:

RandomizedTest, ThreadLeaks, Validators, Listeners, RandomizedContext, TestMethodProviders

Field Summary

Fields

Modifier and Type	Field and Description
static String	AUGMENTED_SEED_PACKAGE Fake package of a stack trace entry inserted into exceptions thrown by test methods.
static int	DEFAULT_ITERATIONS The default number of test repeat iterations.
static int	DEFAULT_KILLATTEMPTS The default number of first interrupts, then Thread.stop attempts.
static int	DEFAULT_KILLWAIT Time in between interrupt retries or stop retries.
static int	DEFAULT_TIMEOUT Default timeout for a single test case.
static int	DEFAULT_TIMEOUT_SUITE Default timeout for an entire suite.

Constructor Summary

Constructors

Constructor and Description
RandomizedRunner(Class<?> testClass) Creates a new runner for the given class.

Method Summary

Methods

Modifier and Type	Method and Description
ArrayList<Object[]>	collectFactoryParameters() Collect parameters from factory methods.
void	filter(Filter filter) Implement Filterable because GUIs depend on it to run tests selectively.
Description	getDescription() Return the current tree of test descriptions (filtered).
TraceFormatting	getTraceFormatting() Stack trace formatting utilities.
static String	methodName(Description description) Attempts to extract just the method name from parameterized notation.
void	run(RunNotifier notifier) Runs all tests and hooks.
static String	seedFromThrowable(Throwable t) RandomizedRunner augments stack traces of test methods that ended in an exception and inserts a fake entry starting with AUGMENTED_SEED_PACKAGE.

Methods inherited from class org.junit.runner.Runner

testCount

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

AUGMENTED_SEED_PACKAGE

public static final String AUGMENTED_SEED_PACKAGE

Fake package of a stack trace entry inserted into exceptions thrown by test methods. These stack entries contain additional information about seeds used during execution.

See Also:

Constant Field Values

DEFAULT_TIMEOUT

public static final int DEFAULT_TIMEOUT

Default timeout for a single test case. By default the timeout is disabled. Use global system property SysGlobals.SYSPROP_TIMEOUT or an annotation Timeout if you need to set timeouts or expect some test cases may hang. This will slightly slow down the tests because each test case is executed in a forked thread.

See Also:

SysGlobals.SYSPROP_TIMEOUT(), Constant Field Values

DEFAULT_TIMEOUT_SUITE

public static final int DEFAULT_TIMEOUT_SUITE

Default timeout for an entire suite. By default the timeout is disabled. Use the global system property SysGlobals.SYSPROP_TIMEOUT_SUITE or an annotation TimeoutSuite if you need to set timeouts or expect some tests (hooks) may hang.

See Also:

SysGlobals.SYSPROP_TIMEOUT_SUITE(), Constant Field Values

DEFAULT_KILLATTEMPTS

public static final int DEFAULT_KILLATTEMPTS

The default number of first interrupts, then Thread.stop attempts.

See Also:

Constant Field Values

DEFAULT_KILLWAIT

public static final int DEFAULT_KILLWAIT

Time in between interrupt retries or stop retries.

See Also:

Constant Field Values

DEFAULT_ITERATIONS

public static final int DEFAULT_ITERATIONS

The default number of test repeat iterations.

See Also:

Constant Field Values

Constructor Detail

RandomizedRunner

public RandomizedRunner(Class<?> testClass)
                 throws InitializationError

Creates a new runner for the given class.

Throws:

InitializationError

Method Detail

getDescription

public Description getDescription()

Return the current tree of test descriptions (filtered).

Specified by:

getDescription in interface Describable

Specified by:

getDescription in class Runner

filter

public void filter(Filter filter)
            throws NoTestsRemainException

Implement Filterable because GUIs depend on it to run tests selectively.

Specified by:

filter in interface Filterable

Throws:

NoTestsRemainException

run

public void run(RunNotifier notifier)

Runs all tests and hooks.

Specified by:

run in class Runner

collectFactoryParameters

public ArrayList<Object[]> collectFactoryParameters()

Collect parameters from factory methods.

getTraceFormatting

public TraceFormatting getTraceFormatting()

Stack trace formatting utilities. These may be initialized to filter out certain packages.

seedFromThrowable

public static String seedFromThrowable(Throwable t)

RandomizedRunner augments stack traces of test methods that ended in an exception and inserts a fake entry starting with AUGMENTED_SEED_PACKAGE.

Returns:

A string is returned with seeds combined, if any. Null is returned if no augmentation can be found.

methodName

public static String methodName(Description description)

Attempts to extract just the method name from parameterized notation.