Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

User has configured a pipeline with multiple plugins , User wants to use REST endpoint to see the plugins used in the app along with programs, streams and datasets used by the pipeline when getting application detail.

User is populating runtime arguments, for the DBSource plugin in his pipeline, he has configured DBSource plugin with macros for the fields "connectionString",  "username" and uses macro function secure for the "password" field. UI wants to get all the configured macros macro properties for that plugin stage, in this case "connectionString" and "username", so its easy for user to provide the values for those values macros as runtime arguments before starting the run.

User Story:


1) User wants to see list of plugins in the app when calling application detail REST endpoint.
2) For a plugin, user wants to get the details on plugin, including plugin properties and list of macros used in properties.

3) UI wants to get properties macro used by a plugin.

Design:


Code Block
 
class PluginDetail {
	String pluginName;
	String pluginType;
}
 
Example : PluginDetail {pluginName : "source1", pluginType: "DBSource"}

...

Code Block
GET: /v3/namespaces/{namespace-id}/apps/{app-id}/plugins/{plugin-id}
 
{plugin-id} is unique in the
app.
this will return JSONlist of Plugin object's correspondingused toby the plugin-id. app.

 

 

Code Block
// No changes to existing Plugin class in CDAP-API.
public final class Plugin {
  private // Existingfinal ArtifactId artifactId;
  private final ArtifactIdPluginClass artifactIdpluginClass;
  private final PluginProperties properties;
  	Plugin(ArtifactId artifactId, PluginClass pluginClass, PluginProperties) {
	...
  }
  ...
}
Code Block
/**
 * Plugin instance properties.
 */
@Beta
public class PluginProperties implements Serializable {

  private static final long serialVersionUID = -7396484717511717753L;

  // Currently only support String->String map.
  private final PluginPropertiesMap<String, String> properties;
  // new field addition in PluginProperties to add macro
  private final Set<String> macros;

  public static Builder builder() {
    return new Builder();
  }

  private PluginProperties(Map<String, String> properties) {
    this(properties, null);
  }
 
   private PluginProperties(Map<String, String> properties, @Nullable Set<String> macros) {
    this.properties = properties;
    this.macros = macros;
  }
  
  /**
   * return the set of macros used atby this plugin 
   * Examples : the plugin
   */ 
  @Nullable 
  public Set<String> getMacros() {
   return macros;
  }
  
  /**
   * internal method used by platform to set macros used in properties. its not advisable for users to use this. 
   * returns new PluginProperties instance with macros set
   */ for the field value ${hostname}:${port}/${path}, the set would be[hostname, port, path]
   * for the field value ${secure(accessKey)}, the set would be [accessKey]
   * for the field value ${key1:${key2}}, the set would be [key2, key1:${key2}]
   * for the field value \${u-name}, the set would be empty as macro is escaped    
   * for the field value ${key1\${key2}}, the set would be [key1\${key2}] so the user understands key2 is not a macro and will provide value for the key, "key1${key2}" 
  public PluginProperties setMacros(Set<String> macros) {
    return new PluginProperties(getProperties(), macros);
  }
 
  // no changes to existing methods or Builder
  public Map<String, String> getProperties() {
    return properties;
  }

  /**
   * A builder to create {@link PluginProperties} instance.
   */
  public static final class Builder {

    private final Map<String, String> properties;

    private Builder() {
      this.properties = new HashMap<>();
    }

    /**
     * Adds multiple properties.
     *
     * @param properties map of properties to add.
     * @return this builder
     */
    public Builder addAll(Map<String, String> properties) {
      this.properties.putAll(properties);
      return this;
    }

    /**
     * Adds a property
     * @param key the name of the property
     * @param value the value of the property
     * @return this builder
     */
    public Builder add(String key, String value) {
      this.properties.put(key, value);
      return this;
    }

    /**
     * Creates a new instance of {@link PluginProperties} with the properties added to this builder prior to this call.
     */
   private finalpublic Set<String> macros;
PluginProperties build() {
  Plugin(ArtifactId artifactId, PluginClass pluginClass, PluginProperties,return @Nullable Set<String> macros) {
	...new PluginProperties(Collections.unmodifiableMap(new HashMap<>(properties)));
    }
  ...}
}

Notes:

macros can either be nullable to support backward compatibility
or we need upgrade step to provide empty set for macros when upgrading pipeline.