Uploading a plugin using REST API

This article describes how to manually upload a plugin using the CDAP REST APIs.

Requirements

• gcloud CLI

• Python

• Cloud Data Fusion instance URL

• Access to the CDF Instance

• Plugin jar and json files (ex. dlp-1.2.0.jar and dlp-1.2.0.json)

Steps

1. Fill in and export the following environment variables. The script assumes plugin file names are in format <plugin_name>-plugin_version.jar

   URL= PLUGIN_NAME= PLUGIN_VERSION=

2. To upload a custom plugin to a CDAP instance, run the following command:

   PLUGIN_PROPERTIES=$(cat $PLUGIN_NAME-$PLUGIN_VERSION.json | python -c "import sys, json; print(json.dumps(json.load(sys.stdin)['properties']))") PLUGIN_PARENTS=$(cat $PLUGIN_NAME-$PLUGIN_VERSION.json | python -c "import sys, json; print('/'.join(json.load(sys.stdin)['parents']))") curl -v -i -H "Authorization: Bearer $(gcloud auth print-access-token)" -X POST "${URL}api/v3/namespaces/default/artifacts/$PLUGIN_NAME" -H "Artifact-Extends: $PLUGIN_PARENTS" -H "Artifact-Version: $PLUGIN_VERSION" -H "Content-Type: application/octet-stream" --data-binary "@$PLUGIN_NAME-$PLUGIN_VERSION.jar" curl -v -i -H "Authorization: Bearer $(gcloud auth print-access-token)" -X PUT "${URL}api/v3/namespaces/default/artifacts/$PLUGIN_NAME/versions/$PLUGIN_VERSION/properties" --data "$PLUGIN_PROPERTIES"

3. Open the Cloud Data Fusion instance and confirm that the plugin appears in the Pipeline Studio. Click on the plugin to ensure the properties and documentation loads correctly.