Page Comparison

...

In initialize() of ETLMapReduce, PipelinePluginInstantiator instantiates a stage at runtime. To set substituted properties for a stage, we can call context.setPluginProperties() We can substitute the properties with actual runtime arguments in ETLMapReduce.initialize() for MapReduceContext or ETLSpark.beforeSubmit() for SparkClientContext. DefaultPluginContext can set substituted properties for a stage and instantiate a stage.

 
PipelinePluginInstantiator pluginInstantiator = new PipelinePluginInstantiator(context, phaseSpec);
// substitute properties of a plugin and set it in the context
context.setPluginProperties(sourceName,
properties);
BatchConfigurable<BatchSourceContext> batchSource = pluginInstantiator.newPluginInstance(sourceName, substitutedProperties);

Limitations:

- The runtime argument name should not contain '$'
- Rumtime argument name should be unique through out pipeline

...