Page Comparison

Transform SDK allow users to build custom transforms. As a step, well developed custom transforms include tests that validates the functionality. This is a how-to guide that provides steps on how to define a test for a transform using the testing framework.

1. Configure POM File

CDAP provides a hydrator-test module that contains several mock plugins for you to use in tests with your custom plugins. To use this module, add a dependency to your pom.xml.

...

Specify the SNAPSHOT or release version of cdap as value of property cdap.version.

2. Create input schema

In order to test a transform, you would need to specify the input schema. The input schema specifies the structure of record records that is are passed to the transform. The example below shows how an a schema can be created within a test.

...

All schemas are defined as a record (Schema.recordOf).

3. Create Transform Config object

Creates Create an instance of a plugin configuration object that is used as the configuration for Transform. Make sure the plugin configuration has a constructor (no private constructors).

ExampleTransformPlugin.Config config 
      = new ExampleTransformPlugin.Config("SomeValue", null, INPUT.toString());

4. Create Custom Transform object

Create an instance of a custom transform plugin by passing the configuration created above to the plugin. The plugin should also have a public constructor.

Transform<StructuredRecord, StructuredRecord> transform 
      = new ExampleTransformPlugin(config);

5. Initialize Transform

Initialize the transform, passing null as TransformContext.

transform.initialize(null);

6. Create A Mock Emitter

Transform needs an emitter, so create a MockEmitter.

MockEmitter<StructuredRecord> emitter = new MockEmitter<>();

7. Create A Structured Record based on Schema

Use the schema that was defined in Step - 2 as the schema for creating a StructuredRecord.

...

StructuredRecord record = StructuredRecord.builder(INPUT)
                             .set("a", "1")
                             .set("b", "2")
                             .set("c", "3")
                             .set("d", "4")
                             .set("e", "5")
                             .build();
transform.transform(record, emitter);                             

8. Test Results from Transform

The MockEmitter contains all the records emitted by the custom transform plugin.

Assert.assertEquals("1", emitter.getEmitted().get(0).get("a"));
Assert.assertEquals("2", emitter.getEmitted().get(0).get("b"));
Assert.assertEquals("3", emitter.getEmitted().get(0).get("c"));
Assert.assertEquals("4", emitter.getEmitted().get(0).get("d"));
Assert.assertEquals("5", emitter.getEmitted().get(0).get("e"));

Full End to End Transform JUnit 4 Test

Full code is available here.

...