Testrecorder

You have code, you have a running system - and you have no tests. This is where Testrecorder jumps in. The most relevant tests of a system are those, that resemble the workflow of the users of the system. Testrecorder in an automatic test case generator from user inputs. It records every user interaction in which the previously selected method is involved and generates executable JUnit-Tests for this recorded interaction. Thus

TL;DR?

Testrecorder generates tests with a capture-replay technique. It differs in some ways from other capture-replay or test generation tools:

• Testrecorder records method calls not GUI/Web interactions
• Testrecorder generates ready-to-run-unit tests (java files) not only specifications
• Testrecorder records not only arguments and results, but also fields, global variables, exceptions and input/output
• Testrecorder records not only input data but also changes
• Testrecorder is configurable and extensible with custom components
• Testrecorder may be used as agent at load time, as agent on an already running system, or as library dependency
• Testrecorder is strictly about recording real interactions (no fuzzing, no randomness, no static analysis)
• Testrecorder supports different testing frameworks (e.g. JUnit 4, JUnit 5) and is extensible for others

Use Testrecorder as an automatic test generator by just recording all interactions of a certain method. Or you can explicitly reproduce buggy scenarios and record them (semi-automatic since such tests afford the attention of an analyzing tester).

Basic Usage

1. Annotate the method to record

Annotate the method to record with @Recorded. For example you want to record this simple example

package fizzbuzz;

public class FizzBuzz {
    @Recorded
    public String fizzBuzz(int i) {
        if (i % 15 == 0) {
            return "FizzBuzz";
        } else if (i % 3 == 0) {
            return "Fizz";
        } else if (i % 5 == 0) {
            return "Buzz";
        } else {
            return String.valueOf(i);
        }
    }
}

2. Configure the test serialization

Create a directory agentconfig (either on your classpath or in the directory you run from). This directory should contain at least two files:

• net.amygdalum.testrecorder.profile.SerializationProfile containing a single line:

    fizzbuzz.SerializationProfile

• net.amygdalum.testrecorder.profile.SnapshotConsumer containing a single line:

    fizzbuzz.TestGenerator

These files refer two the configuration classes. A SerializationProfile configures which classes/methods/fields should be analyzed, included or excluded from recording. A SnapshotConsumer is notified if some recorded ContextSnapshot is available. Typically we want to generate Tests from the snapshot, so we plug in a configured instance of ScheduledTestGenerator.

Now implement the two configuration classes:

package fizzbuzz;
 
public class SerializationProfile extends DefaultSerializationProfile {

    @Override
    public List<Classes> getClasses() {
        return asList(Classes.byPackage("fizzbuzz"));
    }

}

getClasses defines all classes that are analyzed. Any method of an analyzed class may be recorded.

package fizzbuzz;

public class TestGenerator extends ScheduledTestGenerator {

    public TestGenerator(AgentConfiguration config) {
        super(config);
        this.generateTo = Paths.get("target/generated");
        this.dumpOnShutdown(true);
    }

}

We use a typical ScheduledTestGenerator modifying it two have following properties:

• all tests will be generated to the directory target/generated
• shutdown of the host program should trigger test generation

3. Run your program with TestRecorderAgent

To run your program with test recording activated you have to call it with an agent

-javaagent:testrecorder-agent-[version]-agent.jar

testrecorder-agent-[version]-agent.jar is an artifact provided by the maven build (available in maven repository).

4. Interact with the program and check results

You may now interact with your program and every call to a @Recorded method will be captured. After shutdown of your program all captured recordings will be transformed to executable JUnit tests, e.g.

@Test
public void testFizzBuzz0() throws Exception {

    //Arrange
    FizzBuzz fizzBuzz1 = new FizzBuzz();
    
    //Act
    String string1 = fizzBuzz1.fizzBuzz(1);
    
    //Assert
    assertThat(string1, equalTo("1"));
    assertThat(fizzBuzz1, new GenericMatcher() {
    }.matching(FizzBuzz.class));
}

...