A source Plugin in Hydrator is needed to efficiently export data from Netezza. Netezza includes strategies for exporting data including the use of transient external tables, and a command line tool NZSQL that can export data to a flat file.
Use-case
A Hydrator User would like to incorporate Netezza data into a pipeline using a source plugin that does not require a JDBC connection to perform the export from Netezza.
User Stories
As a Hydrator User I want a plugin that I can use to export data from Netezza so that I can use the data in my pipeline.
As a Hydrator User I want a Netezza plugin that works exports data efficiently
As a Hydrator User I want the export data capability of the Netezza plugin to be based on a sql query that I issue.
Example
Example for how the plugin should work
Implementation Tips
NZSQL is a command line utility that can run remotely to allow exporting. The output format can be specified to a flat file with specified delimiter or an external table.
Command example to output to a flat file: nzsql -d {dbname} -u {username} -h {host} -c "select * from table_with_90_columns " -t -F -o output.csv
Command example to use External Table: CREATE EXTERNAL TABLE '/tmp/export.csv' USING (DELIM ',') AS
SELECT foo.x, bar.y, bar.dt FROM foo, bar WHERE foo.x = bar.x;