...

Note: Moved to Views.

Requirements

• CDAP exposes the API for developers to build their own plugin for parsing data in a Stream.
• Developer should have the ability to build his own parser using the CDAP provided API for parsing events in the stream.
• Developer/Operations should then have the ability to deploy the parser implemented into a directory with a configuration
• User should specify at minimum a name and description for the plugin in a configuration
• User should have the ability to list the available plugins using REST API / CLI
• User should have the ability to view using REST API / CLI the pre-defined schema of the plugin in case the plugin defines one.
• User should have the ability to list the views associated with a Stream using REST API / CLI / UI
• User should have the ability to apply the plugin to a Stream and create a view
• User specified view name should be registered in a catalog allowing one to query (SQL) using the view name.
• User should have the ability to apply different plugins on the same Stream creating different view
• User should have the ability to change the plugin associated with a view
• CDAP should provide a text wrangler plugin that allows one to create rules for parsing mostly text files.

...

1. User wants to create a stream "stream1" that contains CSV data and read via Explore through two views "view1" and "view2".
   1. create stream stream1
   2. send stream stream1 "a,b,c"
      send stream stream1 "d,e,f" 

   3. execute "select * from stream_stream1" // this is the default table, will be deprecated and later removed

      body
      a,b,c
      d,e,f

   4. create stream-view stream1 view1 format csv "ticker string, num_traded int, price double"

   5. execute "select * from stream_stream1_ view_view1"

      ticker	num_traded	price
      a	b	c
      d	e	f

   6. create stream-view stream1 view2 format csv "drop=num_traded"

   7. execute "select * from stream_stream1_ view_view2"

      ticker	price
      a	c
      d	f