Goals

Ability to specify condition for execution-path in the hydrator pipelines.

Checklist

• User stories documented (Shankar/Sagar)
• User stories reviewed (Nitin)
• Design documented (Shankar/Sagar)
• Design reviewed (Terence/Andreas)
• Feature merged ()
• Examples and guides ()
• Integration tests () 
• Documentation for feature ()
• Blog post

Use-case

User has a multi job pipeline, set of transforms run as a part of MR job doing data cleansing and a spark ML job on training data to output model. wants to add a conditional node after data cleansing, if there were less than 100 records that failed the preprocessing, he/she likes to continue running spark ML job, else he/she wants to send an email to the data engineer.

User-Stories

• Pipeline developer wants to add a conditional node after one job before running the next job. 
• Pipeline developer wants to add a conditional node at the end of the pipeline and optionally send an email if there were any errors in the final stage.
• Pipeline developer wants to configure the conditional node with his custom code logic.   
• Pipeline developer wants to configure the next job to run based on a condition, based on condition pipeline can run either Job-A or Job-B  
• Plugin developer wants to add a new conditional-plugin supporting a new language or expression for specifying conditional logic.
• Pipeline developer wants to use workflow tokens and use that information to take action in the conditional node.
• Pipeline developer wants to test that their custom condition logic is correct without having to deploy and run their pipeline.