...
PostgreSQL connector reference: https://jdbc.postgresql.org/download/postgresql-9.4.1211.jar
Existing database plugins: https://github.com/cdapio/hydrator-plugins/tree/develop/database-plugins
PostgreSQL datatypes mappings and conversions:
Design
Sink Properties
| User Facing Name | Type | Description | Constraints |
|---|---|---|---|
| Label | String | Label for UI | |
| Reference Name | String | Uniquely identified name for lineage | |
| Host | String | PostgreSQL host | Required (defaults to localhost on UI) |
| Port | Number | Specific port where PostgreSQL running on | Optional (default 5432) |
| Database | String | Database name to connect | Required |
| Import Query | String | Query for import data | Valid SQL query |
| Username | String | DB username | Required |
| Password | Password | User password | Required |
| Bounding Query | String | Returns max and minof split-By Filed | Valid SQL query |
| Split-By Field Name | String | Field name which will be used to generate splits | |
| Number of Splits to Generate | Number | Number of splits to generate | |
| Transaction Isolation Level | Select | Transaction isolation level for queries run by this sink | |
| Connection Arguments | Keyvalue | A list of arbitrary string tag/value pairs as connection arguments, list of properties https://jdbc.postgresql.org/documentation/head/connect.html#connection-parameters | |
| Table Name | String | Name of a database table to write to |
Source Properties
| User Facing Name | Type | Description | Constraints |
|---|---|---|---|
| Label | String | Label for UI | |
| Reference Name | String | Uniquely identified name for lineage | |
| Host | String | PostgreSQL host | Required (defaults to localhost on UI) |
| Port | Number | Specific port where PostgreSQL running on | Optional (default 5432) |
| Database | String | Database name to connect | Required |
| Import Query | String | Query for import data | Valid SQL query |
| Username | String | DB username | Required |
| Password | String | User password | Required |
| Bounding Query | String | Returns max and minof split-By Filed | Valid SQL query |
| Split-By Field Name | String | Field name which will be used to generate splits | |
| Number of Splits to Generate | Number | Number of splits to generate | |
| Transaction Isolation Level | Select | Transaction isolation level for queries run by this sink | |
| Connection Arguments | Keyvalue | A list of arbitrary string tag/value pairs as connection arguments, list of properties https://jdbc.postgresql.org/documentation/head/connect.html#connection-parameters |
Action Properties
| User Facing Name | Type | Description | Constraints |
|---|---|---|---|
| Label | String | Label for UI | |
| Host | String | PostgreSQL host | Required (defaults to localhost on UI) |
| Port | Number | Specific port where PostgreSQL running on | Optional (default 5432) |
| Database | String | Database name to connect | Required |
| Username | String | DB username | Required |
| Password | String | User password | Required |
| Connection Arguments | Keyvalue | A list of arbitrary string tag/value pairs as connection arguments, list of properties https://jdbc.postgresql.org/documentation/head/connect.html#connection-parameters | |
| Database Command | String | Database command to run | Valid SQL query |
Approach
Create a module postgresql-plugin in database-plugins project, reuse existing database-plugins code if possible. Add PostgreSQL-specific properties to configuration, add support for PostgreSQL-specific datatypes. Update UI widgets JSON definitions.
...