Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Now CDAP provides the interface for users to handle their datasets in BigQuery. 

 


Use-case

Users want to integrate CDAP with their already stored dataset in Google BigQuery. 

...

  1. User should provide the correct project id which he has access to. 
  2.   

...

  1. User should provide the SQL query against a dataset inside his project.
  2. User should specify the limit time for the querying. 

...

  1. User should specify the limit

...

  3. The schema is automatically pulled from the table. 

  4. User can pull the field names from the query.

 

  1. time for the querying. 

Example

Following is a simple example showing how BigQuery Source would work.

...

namecount
Jay1123
Nicolas764
Oscar334


Implementation Tips

 

...

CDAP provides two type of operations on the dataset stored in BigQuery: Query and Poll Results. 

Users can use Query operation to do SQL query on specified dataset in BigQuery.

For Poll Results, user can fetch the result using specified job ID or fetch the a specified number of latest query results. 

...

 

Design

 

 

Inputstyperequireddefault
ProjectIdStringYes

 

CredencialStringYes 
QueryStringYes 
Limit TimeInteger (min)No10
Limit SizeInteger (GB)No50

...