...
As a pipeline developer, I want to create realtime ETL pipelines that run using Spark Streaming.
As a pipeline developer, I want to enrich streaming events by joining to other datasets.
As a pipeline developer, I want to be able to window group events into time windows in my streaming pipeline.
As a plugin developer, I want to write a my transform, aggregate, and join plugins that will to work in both Spark Streaming and the Data Pipelines.
Design
...