...
Now CDAP provides the interface for users to handle their datasets in BigQuery.
Use-case
Users want to integrate CDAP with their already stored dataset in Google BigQuery.
...
- User should provide the correct project id which he has access to.
...
- User should provide the SQL query against a dataset inside his project.
- User should specify the limit time for the querying.
...
- User should specify the limit
...
3. The schema is automatically pulled from the table.
4. User can pull the field names from the query.
- time for the querying.
Example
Following is a simple example showing how BigQuery Source would work.
...
name | count |
---|---|
Jay | 1123 |
Nicolas | 764 |
Oscar | 334 |
Implementation Tips
...
CDAP provides two type of operations on the dataset stored in BigQuery: Query and Poll Results.
Users can use Query operation to do SQL query on specified dataset in BigQuery.
For Poll Results, user can fetch the result using specified job ID or fetch the a specified number of latest query results.
...
Design
Inputs | type | required | default |
---|---|---|---|
ProjectId | String | Yes |
|
Credencial | String | Yes | |
Query | String | Yes | |
Limit Time | Integer (min) | No | 10 |
Limit Size | Integer (GB) | No | 50 |
...