...
Users want to integrate CDAP with their already stored dataset in Google BigQuery.
User Stories 1. As
- As a User, I would like to run arbitrary queries synchronously against my datasets in BigQuery and pull those records into a hydrator pipeline.
...
- As a User, I would like to store data from a Hydrator pipeline into a table (dataset) in BigQuery. If the table doesn't exist, it should be created.
Requirements
- User should specify the limit time for the querying.
...
- User is able to specify the limit size of the dataset to query.
...
- User is able to poll for the result.
...
- User can list the query result history for a duration of time.
...
- The schema is automatically pulled from the table.
...
- User can pull the field names from the query.
Example
Following is a simple example showing how BigQuery Source would work.
...