= Test Engines

A `TestEngine` facilitates _discovery_ and _execution_ of tests for a particular
programming model.

For example, JUnit provides a `TestEngine` that discovers and executes tests written using
the JUnit Jupiter programming model (see xref:writing-tests/intro.adoc[] and xref:extensions/overview.adoc[]).

[[junit]]
== JUnit Test Engines

JUnit provides three `TestEngine` implementations.

* `{junit-jupiter-engine}`: The core of JUnit Jupiter.
* `{junit-vintage-engine}`: A thin layer on top of JUnit 4 to allow running _vintage_
  tests (based on JUnit 3.8 and JUnit 4) with the JUnit Platform launcher infrastructure.
* `{junit-platform-suite-engine}`: Executes declarative suites of tests with the JUnit
  Platform launcher infrastructure.

[[custom]]
== Custom Test Engines

You can contribute your own custom `{TestEngine}` by implementing the interfaces in the
{junit-platform-engine} module and _registering_ your engine.

Every `TestEngine` must provide its own _unique ID_, _discover_ tests from an
`EngineDiscoveryRequest`, and _execute_ those tests according to an `ExecutionRequest`.

[WARNING]
.The `junit-` unique ID prefix is reserved for TestEngines from the JUnit Team
====
The JUnit Platform `Launcher` enforces that only `TestEngine` implementations published
by the JUnit Team may use the `junit-` prefix for their `TestEngine` IDs.

* If any third-party `TestEngine` claims to be `junit-jupiter` or `junit-vintage`, an
  exception will be thrown, immediately halting execution of the JUnit Platform.
* If any third-party `TestEngine` uses the `junit-` prefix for its ID, a warning message
  will be logged. Later releases of the JUnit Platform will throw an exception for such
  violations.
====

In order to facilitate test discovery within IDEs and tools prior to launching the JUnit
Platform, `TestEngine` implementations are encouraged to make use of the `@Testable`
annotation. For example, the `@Test` and `@TestFactory` annotations in JUnit Jupiter are
meta-annotated with `@Testable`. Consult the Javadoc for `{Testable}` for further details.

If your custom `TestEngine` needs to be configured, consider allowing users to supply
configuration via xref:running-tests/configuration-parameters.adoc[configuration parameters]. Please note,
however, that you are strongly encouraged to use a unique prefix for all configuration
parameters supported by your test engine. Doing so will ensure that there are no conflicts
between the names of your configuration parameters and those from other test engines. In
addition, since configuration parameters may be supplied as JVM system properties, it is
wise to avoid conflicts with the names of other system properties. For example, JUnit
Jupiter uses `junit.jupiter.` as a prefix of all of its supported configuration
parameters. Furthermore, as with the warning above regarding the `junit-` prefix for
`TestEngine` IDs, you should not use `junit.` as a prefix for the names of your own
configuration parameters.

Although there is currently no official guide on how to implement a custom `TestEngine`,
you can consult the implementation of <<junit>> or the implementation of
third-party test engines listed in the
https://github.com/junit-team/junit-framework/wiki/Third-party-Extensions#junit-platform-test-engines[JUnit wiki].
You will also find various tutorials and blogs on the Internet that demonstrate how to
write a custom `TestEngine`.

NOTE: `{HierarchicalTestEngine}` is a convenient abstract base implementation of the
`TestEngine` SPI (used by the `{junit-jupiter-engine}`) that only requires implementors to
provide the logic for test discovery. It implements execution of `TestDescriptors` that
implement the `Node` interface, including support for parallel execution.

[[registration]]
== Registering a TestEngine

`TestEngine` registration is supported via Java's `{ServiceLoader}` mechanism.

For example, the `junit-jupiter-engine` module registers its
`org.junit.jupiter.engine.JupiterTestEngine` in a file named
`org.junit.platform.engine.TestEngine` within the `/META-INF/services` folder in the
`junit-jupiter-engine` JAR.

[[requirements]]
== Requirements

NOTE: The words "must", "must not", "required", "shall", "shall not", "should", "should
not", "recommended",  "may", and "optional" in this section are to be interpreted as
described in https://www.ietf.org/rfc/rfc2119.txt[RFC 2119.]

[[requirements-mandatory]]
=== Mandatory requirements

For interoperability with build tools and IDEs, `TestEngine` implementations must adhere
to the following requirements:

* The `TestDescriptor` returned from `TestEngine.discover()` _must_ be the root of a tree
  of `TestDescriptor` instances. This implies that there _must not_ be any cycles between
  a node and its descendants.
* The hierarchy of test descriptors returned from `TestEngine.discover()` _must_ be
  mutable, but the test descriptors _must_ otherwise be immutable after discovery.
* A `TestEngine` _must_ be able to discover `UniqueIdSelectors` for any unique ID that it
  previously generated and returned from `TestEngine.discover()`. This enables selecting a
  subset of tests to execute or rerun.
* The `executionSkipped`, `executionStarted`, and `executionFinished` methods of the
  `EngineExecutionListener` passed to `TestEngine.execute()` _must_ be called for every
  `TestDescriptor` node in the tree returned from `TestEngine.discover()` at most
  once. Parent nodes _must_ be reported as started before their children and as finished
  after their children. If a node is reported as skipped, there _must not_ be any events
  reported for its descendants.

[[requirements-enhanced-compatibility]]
=== Enhanced compatibility

Adhering to the following requirements is optional but recommended for enhanced
compatibility with build tools and IDEs:

* Unless to indicate an empty discovery result, the `TestDescriptor` returned from
  `TestEngine.discover()` _should_ have children rather than being completely dynamic.
  This allows tools to display the structure of the tests and to select a subset of tests
  to execute.
* When resolving `UniqueIdSelectors`, a `TestEngine` _should_ only return `TestDescriptor`
  instances with matching unique IDs including their ancestors but _may_ return additional
  siblings or other nodes that are required for the execution of the selected tests.
* `TestEngines` _should_ support xref:running-tests/tags.adoc[tagging] tests and containers so
  that tag filters can be applied when discovering tests.
* [[requirements-cancellation]] A `TestEngine` _should_ cancel its execution
  when the `{CancellationToken}` it is passed as part of the `ExecutionRequest` indicates
  that cancellation has been requested. In this case, it _should_ report any remaining
  `TestDescriptors` as skipped but not report any events for their descendants. It _may_
  report already started `TestDescriptors` as aborted in case they have not been executed
  completely. If a `TestEngine` supports cancellation, it should clean up any resources
  that it has created just like if execution had finished regularly.

[[discovery-issues]]
== Reporting Discovery Issues

Test engines should report xref:running-tests/discovery-issues.adoc[discovery issues] if they
encounter any problems or potential misconfigurations during test discovery. This is
especially important if the issue could lead to tests not being executed at all or only
partially.

In order to report a `{DiscoveryIssue}`, a test engine should call the
`issueEncountered()` method on the `{EngineDiscoveryListener}` available via the
`{EngineDiscoveryRequest}` passed to its `discover()` method. Rather than passing the
listener around, the `{DiscoveryIssueReporter}` interface should be used. It also provides
a way to create a `Condition` that reports a discovery issue if its check fails and may
be used as a `Predicate` or `Consumer`. Please refer to the implementations of the
<<junit, test engines provided by JUnit>> for examples.

Moreover, xref:advanced-topics/testkit.adoc#engine-discovery[Engine Test Kit] provides a way to write tests for
reported discovery issues.
