...
Design (WIP - dont review yet)
...
Macros
...
We can introduce macro syntax that can be used in plugin configs that the Hydrator app will substitute before any plugin code is run. For example:
Syntax
| Code Block |
|---|
Expanded Syntax :
${macro-type(macro)}
Shorthand notation:
${macro}
Example Usage:
${runtime(hostname)) - get hostname from runtime arguments
${token(hostname)) - get hostname from workflow token
${secure(access_key)) - get access key from secure store
${function_time(time_format)) - apply time function on the time_format provided and use the value.
The Default (short-hand) usage will read from runtime arguments, having an expanded notation gives user option for using more macro types.
Examples :
ipConfig : ${hostname}:${port}
JDBC connection string : jdbc:{jdbc-plugin}://{hostname}:{sql-port}/{db-name}
|
| Code Block |
|---|
{
"stages": [
{
"name": "customers",
"plugin": {
"name": "File",
"type": "batchsource",
"properties": {
"path": "hdfs://host:port/${inputpath}" // ${inputpath} will get replaced with the value of the 'customers.inputpath' runtime argument
}
}
},
{
"name": "items",
"plugin": {
"name": "File",
"type": "batchsource",
"properties": {
"path": "hdfs://host:port/${inputpath}" // ${inputpath} will get replaced with the value of the 'items.inputpath' runtime argument
}
}
}
]
} |
...