A rules engine transform will apply predefined rules to incoming data (realtime as well as batch). Rules must be generic enough to allow updates to a dataset, posting to an HTTP endpoint, sending an email, etc.
Use case(s)
CompanyA wants to develop a streaming pipeline to read and process signals from wearable and non-wearable devices, and apply rules on incoming signals. Based on the rules, it wants to send notifications to configured mobile devices to provide concierge and/or healthcare services.
CompanyA has a rules management system that can allow users to feed in rules for devices. Rules can state actions to be taken if certain conditions are met in the signals from the provided devices. These rules are stored in a CDAP dataset. A streaming pipeline will then read the rules dataset and apply rules applicable for incoming signals to trigger appropriate notifications.
CompanyA would like a selection criteria for applying rules to incoming signals. Selection criteria could be based on device hierarchy, but could also be arbitrary. Hydrator should allow the user to specify a rule selection criteria to select the set of rules to apply on an incoming signal.
User Storie(s)
As a Hydrator user, I would like to apply rules on the incoming stream to trigger notifications or take any other appropriate actions if necessary.
As a Hydrator user, I would like to be able to look up rules from a rules repository
As a Hydrator user, I would like to apply only those rules that are applicable to the incoming record. For achieving this, I would like to specify a selection criteria for rules. The selection criteria may be based on input schema.
Plugin Type
Batch Source
Batch Sink
Real-time Source
Real-time Sink
Action
Post-Run Action
Aggregate
Join
Spark Model
Spark Compute
Transform
Configurables
This section defines properties that are configurable for this plugin.
User Facing Name
Type
Description
Constraints
Input Schema
String
Schema of the input data
Specified as a CDAP schema
Ruleset Repository
String
Name of the dataset to be used as a repository of rules
Ruleset Selection Criteria
String
Using some grammar, this string specifies the logic of selecting a rule from the rules dataset
Design / Implementation Tips
Tip #1
Tip #2
Design
Approach(s)
Properties
Security
Limitation(s)
Future Work
Some future work – HYDRATOR-99999
Another future work – HYDRATOR-99999
Test Case(s)
Test case #1
Test case #2
Sample Pipeline
Please attach one or more sample pipeline(s) and associated data.