Testrecorder

Testrecorder ist a unit test generator for Java. Testrecorder is based on capturing and replaying the states before and after method calls. The generated result is a pure Java JUnit test.

Get Testrecorder Github

You have code, you have a running system - and you have no tests. This is where Testrecorder jumps in. The most relevant tests of a system are those, that resemble the workflow of the users of the system. Testrecorder records every user interaction a selected method is involved in and generates executable JUnit-Tests for this interaction.

testrecorder is as capture-replay test generator. In contrast to most other capture-replay test generators testrecorder does not generate gui/web integration tests but pure Java unit tests.

Basic Usage

1. Annotate the method to record

Annotate the method to record with @Recorded. For example you want to record this simple example

package fizzbuzz;

public class FizzBuzz {
    @Recorded
    public String fizzBuzz(int i) {
        if (i % 15 == 0) {
            return "FizzBuzz";
        } else if (i % 3 == 0) {
            return "Fizz";
        } else if (i % 5 == 0) {
            return "Buzz";
        } else {
            return String.valueOf(i);
        }
    }
}

2. Configure the test serialization

Create a directory agentconfig (either on your classpath or in the directory you run from). This directory should contain at least two files:

  • net.amygdalum.testrecorder.profile.SerializationProfile containing a single line:
    fizzbuzz.SerializationProfile
  • net.amygdalum.testrecorder.SnapshotConsumer containing a single line:
    fizzbuzz.TestGenerator

These files refer two the configuration classes. A SerializationProfile configures which classes/methods/fields should be analyzed, included or excluded from recording. A SnapshotConsumer is notified if some recorded ContextSnapshot is available. Typically we want to generate Tests from the snapshot, so we plug in a configured instance of ScheduledTestGenerator.

Now implement the two configuration classes:

package fizzbuzz;
 
public class SerializationProfile extends DefaultSerializationProfile {

    @Override
    public List<Classes> getClasses() {
        return asList(Classes.byPackage("fizzbuzz"));
    }

}

getClasses defines all classes that are analyzed. Any method of an analyzed class may be recorded.

package fizzbuzz;

public class TestGenerator extends ScheduledTestGenerator {

    public TestGenerator(AgentConfiguration config) {
        super(config);
        this.generateTo = Paths.get("target/generated");
        this.dumpOnShutdown(true);
    }

}

We use a typical ScheduledTestGenerator modifying it two have following properties:

  • all tests will be generated to the directory target/generated
  • shutdown of the host program should trigger test generation

3. Run your program with TestRecorderAgent

To run your program with test recording activated you have to call it with an agent

-javaagent:testrecorder-agent-[version]-agent.jar

testrecorder-agent-[version]-agent.jar is an artifact provided by the maven build (available in maven repository).

4. Interact with the program and check results

You may now interact with your program and every call to a @Recorded method will be captured. After shutdown of your program all captured recordings will be transformed to executable JUnit tests, e.g.

@Test
public void testFizzBuzz0() throws Exception {

    //Arrange
    FizzBuzz fizzBuzz1 = new FizzBuzz();
    
    //Act
    String string1 = fizzBuzz1.fizzBuzz(1);
    
    //Assert
    assertThat(string1, equalTo("1"));
    assertThat(fizzBuzz1, new GenericMatcher() {
    }.matching(FizzBuzz.class));
}

...

Benefits

Use the generated tests as characterization tests for test regression. Many refactoring will directly adjust the test, few will need some manual corrections and none will force you to dig in xml (or java serialization code) to correct the data of the test.

The generated test can also give you insights about what is changed in the tested scenario - involving not only primitive arguments and results, but also side effects on arguments, global variables and exceptions.

And if you think testrecording is cool, but byte code instrumetation is not: the api of testrecorder is open. You may generate setup code (the code that generates a given object) and matcher code (the code that validates a given object) as part of your debugging (or runtime system).

Limitations

TestRecorder serialization (for values and tests) does not cover all of an objects properties. Problems might occur with:

  • static fields
  • synthetic fields (e.g. added by some bytecode rewriting framework)
  • native state
  • proxies
  • state that influences object access (e.g. modification counter in collections)

Such scenarios should be solved with custom serializers and/or custom desersializers.

Examples

Examples can be found at testrecorder-examples