Skip navigation links

Package org.apache.axis2.transport.testkit

Transport test kit base package.

See: Description

Package org.apache.axis2.transport.testkit Description

Transport test kit base package.

Introduction and high level overview

In general a good test suite for an Axis2 transport should contain test cases that In addition, the test suite should cover Also for some transports it is necessary to execute the tests with different transport configurations or with different protocol providers. For example, HTTP transport implementations are tested in HTTP 1.0 and HTTP 1.1 mode, and the JMS transport is tested with different JMS providers (currently Qpid and ActiveMQ).

The test kit grew out of the idea that is should be possible to apply a common set of tests (with different MEPs and content types) to several transports with a minimum of code duplication. By providing non Axis2 test clients and endpoints as well as the code that sets up the necessary environment as input, the framework should then be able to build a complete test suite for the transport.

It is clear that since each transport protocol has its own specificities, a high level of abstraction is required to achieve this goal. The following sections give a high level overview of the various abstractions that have been introduced in the test kit.

Integration with JUnit

One of the fundamental requirements for the test kit is to integrate well with JUnit. This requirement ensures that the tests can be executed easily as part of the Maven build and that other available tools such as test report generators and test coverage analysis tools can be used.

The usual approach to write JUnit tests is to extend junit.framework.TestCase.TestCase and to define a set of methods that implement the different test cases. Since the goal of the framework is to build test suites in an automated way and the number of test cases can be fairly high, this approach would not be feasible. Fortunately JUnit supports another way to create a test suite dynamically. Indeed JUnit scans the test code for methods with the following signature:

public static TestSuite suite()
A typical transport test will implement this method and use TransportTestSuiteBuilder to let the framework create the test suite.

Test case naming

One problem that immediately arises when building a test suite dynamically is that each test case must have a name (which should be unique) and that this name should be sufficiently meaningful so that when it appears in a report a human should be able to get a basic idea of what the test case does. The names generated by the test kit have two parts: Example:
0076:test=REST,client=java.net,endpoint=axis
The algorithm used by the test kit to collect the key-value pairs is described in the documentation of the org.apache.axis2.transport.testkit.name package.

Resource management

In general setting up the environment in which a given test case is executed may be quite expensive. For example, running a test case for the JMS transport requires starting a message broker. Also every test case requires at least an Axis2 client and/or server environment to deploy the transport. Setting up and tearing down the entire environment for every single test case would be far too expensive. On the other hand the environments required by different test cases in a single test suite are in general very different from each other so that it would not possible to set up a common environment used by all the test cases.

To overcome this difficulty, the test kit has a mechanism that allows a test case to reuse resources from the previous test case. This is managed in an entirely transparent way by a lightweight dependency injection container (see [TODO: need to regroup this code in a single package]), so that the test case doesn't need to care about it.

The mechanism is based on a set of simple concepts: [TODO: this is too detailed for a high level overview and should be moved to the Javadoc of the relevant package]

Resources required by a transport test case

Every transport test case (extending MessageTestCase) at least requires three resources:

The test kit provides the following Axis2 based test client and endpoint implementations:

One-way Request-response
Client AxisAsyncTestClient AxisRequestResponseTestClient
Endpoint AxisAsyncEndpoint AxisEchoEndpoint

Message encoders and decoders

Different clients, endpoints and test cases may have fairly different ways to "naturally" represent a message: Since defining a message representation that would be suitable for all clients, endpoints and test cases (and make their implementation simple) is impossible, a different approach has been chosen in the framework. Every client, endpoint or test case implementation chooses the Java type that is considers as best suited to represent the message. When invoking the test client, a test case uses a MessageEncoder to transform the message from its own representation to the representation used by the test client. In the same way, a MessageDecoder is used to transform the message intercepted by the endpoint (in the one-way case) or the response message received by the test client (in the request-response case).

[TODO: currently message encoders and decoders are chosen at compile time and the transformation is is invoked indirectly by adapters; this will change in the future so that encoders and decoders are selected dynamically at runtime]

Exclusion rules

Sometimes it is necessary to exclude particular test cases (or entire groups of test cases) from the test suite generated by the test kit. There are various reasons why one would do that: The test kit allows to specify exclusion rules using LDAP filter expressions. It takes advantage of the fact that each test case has a set of key-value pairs used to build the test case name. The LDAP filters are evaluated against this set. For example, TransportTestSuiteBuilder defines the following default exclusion rule:
(&(client=*)(endpoint=*)(!(|(client=axis)(endpoint=axis))))
This rule excludes all test cases that would use a non Axis2 client and a non Axis2 endpoint.

Logging

Transport test cases generally involve several interacting components and some of these components may use multithreading. Also experience has shown that some test cases may randomly fail (often with a failure probablity highly dependent on the execution platform) because of subtle problems in the transport under test or in the tests themselves. All this can make debugging extremely difficult. To simplify this task, the test kit collects (or provides the necessary infrastructure to collect) a maximum of information during the execution of each test case.

The collected information is written to a set of log files managed by LogManager. An instance is added automatically to the resource set of every test case and other resources can acquire a reference through the dependency injection mechanism described above. This is the recommended approach. Alternatively, the log manager can be used as a singleton through LogManager.INSTANCE.

Logs files are written to subdirectories of target/testkit-logs. The directory structure has a two level hierarchy identifying the test class (by its fully qualified name) and the test case (by its ID). It should be noted that the test results themselves (in particular the exception in case of failure) are still written to the standard JUnit/Surefire logs and that these logs should be consulted first. The test kit specific log files are only meant to provide additional information.

Each test case at least produces a 01-debug.log file with the messages that were logged (using JCL) at level DEBUG during the execution of the test case. In addition, depending on the components involved in the test, the test kit will produce the following logs (XX denotes a sequence number which is generated automatically):

XX-formatter.log
XX-builder.log

These files are produced when Axis2 test clients and endpoints are used. XX-formatter.log will contain the payload of an incoming message as seen by the MessageFormatter. XX-builder.log on the other hand will contain the payload of an outgoing message as produced by the Builder. Note that the number of log files depends on serveral factors, such as the MEP, whether the client or endpoint is Axis2 based or not and whether the transport chooses to use message builders and formatters or not.

These files provides extremely valuable information since it is very difficult to get this data using other debugging techniques. Note that the files are created by LogAspect which relies on Aspect/J to intercept calls to message formatters and builders. This will only work if the tests are run with the Aspect/J weaver.

XX-service-parameters.log

If the test case uses an Axis2 based endpoint, this file will contain the parameters of the AxisService implementing this endpoint. This information is useful since the service configuration is in general determined by different components involved in the test.

Skip navigation links

Copyright © The Apache Software Foundation. All Rights Reserved.