...

• Periodically fetch Omniture Click Data from SFTP
• Load the raw data into Cloud BigQuery staging table
• Apply a series of SQL queries to transform the data
• Generate or update 5-6 auxiliary tables and update main fact table. 

Requirements

1. User can specify Legacy SQL or Non-Legacy SQL to execute
2. User can set the priority for the query
3. User can specify the mode - interactive or batch mode for the query execution
4. User can specify a non-temporary (permanent) table in which the results can be written
5. User can specify the query cache to be used
6. User can specify different retry options in case of failures
7. User can specify whether to continue further in case of failure
8. User can use the plugin in a non-GCP environment
9. Plugin passes the query and job id to the next node in the DAG 

Limitations

1. No records are retrieved and/or stored.