This page describes the steps required to create a new custom plugin, generate the jars, and subsequent deployment of the custom plugin into CDF.
Before you begin
N/A
Steps
The creation and deployment of custom plugins involves 3 broad steps.
One time setup on your local machine to write the java code
Writing the plugin code and building the jar
Deploying the plugin in Data Fusion
One Time Setup
Clone Cloud Source repository to local machine
A&G Cloud source repository can be accessed through this link
https://source.cloud.google.com/cloud-professional-services/autogeneral-data-lake
Install Google Cloud SDK on your local machine following instructions available at
https://cloud.google.com/sdk/docs/quickstart-macos
Run
gcloud initto initialise your session with the right user name and project
Set user name as the appropriate
Set project id to the project id hosting the cloud source repository
Authenticate using
gcloud auth logincommand
https://cloud.google.com/source-repositories/docs/authentication
Follow instructions on the new web page that opens up, and choose the account using which you want to authenticate
When successfully authenticated, you will see a web page with below message
You are now authenticated with the Google Cloud SDK!
Clone Cloud Source Repository to local machine using the following command
gcloud source repos clone <repository_name> --project=<project-id>
Install IDE, Maven
Install VS Code IDE or any other IDE of your choice
https://code.visualstudio.com/
Install JDK 8
Download and install Java
Set JAVA_HOME
Run below command to get java home directory
/usr/libexec/java_home -V
Set JAVA HOME
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_251.jdk/Contents/Home
Install Maven
Download installable binary apache-maven-3.6.3-bin.tar.gz from https://maven.apache.org/download.cgi#
Follow installation instructions here https://www.baeldung.com/install-maven-on-windows-linux-mac
Build Plugin Jar
In VS Code, on the Terminal pane,
Write plugin code in the suitable path in the cloned directory structure
e.g., /Users/<username>/<repo_name>/<plugin-name>
Navigate to the directory where your plugin code is located in the repository path e.g., /Users/<username>/<repo_name>/<plugin-name>
Run the following command to build plugin jar
/opt/apache-maven/bin/mvn install
This will create a new sub-directory ‘Target’ with the jar and json files for the plugin as below -
<plugin_name>-1.25-SNAPSHOT.jar
<plugin_name>-1.25-SNAPSHOT.json