Info |
---|
This article is posted on the CDAP Doc wiki and will be maintained here: https://cdap.atlassian.net/wiki/spaces/DOCS/pages/1162641656/Testing+a+Custom+Transform+Plugin |
Transform SDK allow users to build custom transforms. As a step, well Well developed custom transforms include tests that validates the functionality. This is a how-to guide that provides steps on how to define a test for a transform using the testing framework.
1. Configure POM File
CDAP provides a hydrator-test
module that contains several mock plugins for you to use in tests with your custom plugins. To use this module, add a dependency to your pom.xml
.
...
Specify the SNAPSHOT or release version of cdap as value of property cdap.version
.
2. Create input schema
In order to test a transform, you would need to specify the input schema. The input schema specifies the structure of record records that is are passed to the transform. The example below shows how an a schema can be created within a test.
...
All schemas are defined as a record (Schema.recordOf
).
3. Create Transform Config object
Creates Create an instance of a plugin configuration object that is used as the configuration for Transform. Make sure the plugin configuration has a constructor (no private constructors).
Code Block |
---|
ExampleTransformPlugin.Config config = new ExampleTransformPlugin.Config("SomeValue", null, INPUT.toString()); |
4. Create Custom Transform object
Create an instance of a custom transform plugin by passing the configuration created above to the plugin. The plugin should also have a public constructor.
Code Block |
---|
Transform<StructuredRecord, StructuredRecord> transform = new ExampleTransformPlugin(config); |
5. Initialize Transform
Initialize the transform, passing null
as TransformContext
.
Code Block |
---|
transform.initialize(null); |
6. Create A Mock Emitter
Transform needs an emitter, so create a MockEmitter
.
Code Block |
---|
MockEmitter<StructuredRecord> emitter = new MockEmitter<>(); |
7. Create A Structured Record based on Schema
Use the schema that was defined in Step - 2 as the schema for creating a StructuredRecord
.
...
Code Block |
---|
StructuredRecord record = StructuredRecord.builder(INPUT) .set("a", "1") .set("b", "2") .set("c", "3") .set("d", "4") .set("e", "5") .build(); transform.transform(record, emitter); |
8. Test Results from Transform
The MockEmitter
contains all the records emitted by the custom transform plugin.
Code Block |
---|
Assert.assertEquals("1", emitter.getEmitted().get(0).get("a")); Assert.assertEquals("2", emitter.getEmitted().get(0).get("b")); Assert.assertEquals("3", emitter.getEmitted().get(0).get("c")); Assert.assertEquals("4", emitter.getEmitted().get(0).get("d")); Assert.assertEquals("5", emitter.getEmitted().get(0).get("e")); |
Full End to End Transform JUnit 4 Test
Full code is available here.
...