Introduction

Streams are the primary means for ingesting data into CDAP. A realtime CDAP Stream source will provide a way to read from CDAP streams in a Streaming pipeline. 

Use case(s)

• CompanyA is developing an IoT application to analyze signals from devices in realtime using Cask Hydrator. The device operator can send signals from various devices to a data processing system over HTTP. This data needs to be processed and analyzed in realtime and notifications need to be sent when unusual activity is suspected. Since CDAP Streams are provide high throughput ingestion over HTTP already, they are a good fit for this use case. 
• CompanyA does not want to use another external system (such as Apache Kafka) for their solution, since they have deployed CDAP for processing already, and CDAP provides streams for high throughput ingestion.
• CompanyA wants to develop a pipeline to consume signals from operators of various IoT wearable (smartwatch, etc) and non-wearable (antennas, etc) devices. These signals need to be processed in realtime, and notifications need to be sent out to devices for concierge services, health assistance, etc.

User Storie(s)

• User story #1
• User story #2
• User story #3
• User story #m

Plugin Type

• Batch Source
• Batch Sink 
• Real-time Source
• Real-time Sink
• Action
• Post-Run Action
• Aggregate
• Join
• Spark Model
• Spark Compute

Configurables

This section defines properties that are configurable for this plugin. 

Design / Implementation Tips

• Tip #1
• Tip #2

Design

Approach(s)

Properties

Security

Limitation(s)

Future Work

• Some future work – HYDRATOR-99999
• Another future work – HYDRATOR-99999

Test Case(s)

• Test case #1
• Test case #2

Sample Pipeline

Please attach one or more sample pipeline(s) and associated data. 

Pipeline #1

Pipeline #2