...
| Code Block |
|---|
Expanded Syntax :
${macro-type(macro)}
Shorthand notation:
${macro}
Example Usage:
${runtime(hostname)) - get hostname from runtime arguments
${token(hostname)) - get hostname from workflow token
${secure(access_key)) - get access key from secure store
${function_time(time_format)) - apply time function on the time_format provided and use the value.
The Default (short-hand) usage will read from runtime arguments, having an expanded notation gives user option for using more macro types.
Examples :
ipConfig : ${hostname}:${port}
JDBC connection string : jdbc:${jdbc-plugin}://${hostname}:${sql-port}/${db-name}
|
| Code Block | ||
|---|---|---|
| ||
"stages": [
{
"name": "Database",
"plugin": {
"name": "Database",
"type": "batchsource",
"properties": {
...
"user": "${username}",
"password": "${secure(sql-password)}",
"jdbcPluginName": "jdbc",
"jdbcPluginType": "${jdbc-type}",
"connectionString": "jdbc:${jdbc-type}//${hostname}:${port}/${db-name}",
"importQuery": "select * from ${table-name};"
}
}
},
{
"name": "Table",
"plugin": {
"name": "Table",
"type": "batchsink",
"properties": {
"schema": "{\"type\":\"record\",\"name\":\"etlSchemaBody\",
\"fields\":[{\"name\":\"name\",\"type\":\"string\"},
{\"name\":\"age\",\"type\":\"int\"},{\"name\":\"emp_id\",\"type\":\"long\"}]}",
"name": "${table-name}",
"schema.row.field": "name"
}
}
}
] |
...