Use Cases:
- Validator Filter: All records of a transform that are invalid go into one dataset; the remainder go into another.
API:
Existing APIs (in MapReduceContext):
context.setOutput(String datasetName);
context.setOutput(String datasetName, Dataset dataset);
Additional APIs (in MapReduceContext):
context.addOutput(String datasetName);
context.addOutput(String datasetName, Dataset dataset);
Approach:
Take an approach similar to org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.