Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
titleMacro-Enabled Plugin Property
public class TableSinkConfig extends PluginConfig {
  @Name(Properties.Table.NAME)
  @Description("Name of the table. If the table does not already exist, one will be created.")
  // The name of the table can be specified by a runtime macro
  @Macro 
  private String name;

  @Name(Properties.Table.PROPERTY_SCHEMA)
  @Description("schema of the table as a JSON Object. If the table does not already exist, one will be " +
    "created with this schema, which will allow the table to be explored through Hive. If no schema is given, the " +
    "table created will not be explorable.")
  @Nullable
  private String schemaStr;

  @Name(Properties.Table.PROPERTY_SCHEMA_ROW_FIELD)
  @Description("The name of the record field that should be used as the row key when writing to the table.")
  private String rowField;
}

 

Syntax

In addition to a plugin property being annotated with @Macro, proper macro syntax must be provided to the property field at configure time. There are two valid macro syntaxes, property lookups and macro functions.

...

Code Block
titleExpanded Server Property
my-demo-host.example.com:9991

Lookup Precedence

For property lookup macros, substitutions set through workflow tokens take precedence over those set through runtime arguments and preferences.

Macro Function

Macro functions allow more complex logic to be run before a substitution occurs and use the following syntax:

...

which will pull "mysql-password" from the secure store at runtime.

 

Expansion Capabilities

Macro parsing supports the following capabilities:

Multiple Macros

A single property can have multiple macros either consecutive or spread out.

Code Block
titleMultiple Macros
${host}/${path}:${port} --> example.com/index.html:80

 

Nested Macros

A macro can have another macro nested inside. Nested macros are expanded before the outer macro. If there are multiple nested macros within a single enclosing macro, they are evaluated from right-to-left. Based on the specified substitutions, the following substitutions could be possible based on the value of "host-suffix":

Code Block
titleNested Macros
Option 1. ${hostname${host-suffix}} --> ${hostname-use-suffix} --> example.com/index.html:80
Option 2. ${hostname${host-suffix}} --> ${hostname-dont-use-suffix} --> example.com

 

Recursive Macros

A macro can be substituted with another macro if specified as such. For example, the following key-value pair:

Code Block
titleRecursive Macros Arguments
full-server-address: ${hostname}/${path}:${port}

would lead to the following expansion:

Code Block
titleRecursive Macros
${full-server-address} --> ${hostname}/${path}:${port} --> example.com/index.html:80

In the case that a single macro expands to multiple macros, the new macros will be substituted from right-to-left. Recursive expansion is currently supported up until a maximum depth of 10.

 

Macro Syntax Escaping

Macro syntax can be escaped. To escape existing syntax, use the backslash '\' character. Backslashes themselves must also be escaped, so to use a backslash, use two consecutive backslashes '\\'. For example, the following syntax:

Code Block
titleRecursive Macros
${\${escaped-macro-literal\}}

would be look up the specified property with a key of: ${escaped-macro-literal}.