Page Comparison

...

$ git --version
git version 2.8.2

Please refer to Build System & CI on how to setup building of CDAP.

Build

Git clone the CDAP repository and the Hydrator plugins:

...

cd cdap
mvn clean packageinstall -DskipTests

If you get compile errors similar to this one:

...

this is likely due to the version of maven that does not work properly with latest Java versions. Upgrading to maven 3.6.1 will most likely fix the problem. 

If you encounter an error like this one

[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project io.cdap.cdap:cdap:6.5.0-SNAPSHOT (/workspace/cdap-build/cdap/pom.xml) has 1 error
[ERROR]     Child module /workspace/cdap-build/cdap/cdap-ui of /workspace/cdap-build/cdap/pom.xml does not exist

Run this to update the submodule

git submodule update --init --recursive --remote

Please refer to Build System & CI for more details on building CDAP.

Build and run Local Standalone CDAP

Git clone the CDAP repository (if you haven't yet), and the Hydrator Follow the steps above to build CDAP and install it to the local maven repository. Then cd out of the cdap directory and clone hydrator-plugins:

gitcd clone git@github.com:cdapio/cdap.git
git clone git@github.com:cdapio/hydrator-plugins.git

First build Build the plugins:

note: --remote pulls the latest versions of every submodule, rather than the versions locked in the hydrator-plugins git. This is intentional.

cd hydrator-plugins
git submodule update --init --recursive --remote
mvn clean install -DskipTests 

If you run into errors like this

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project google-cloud: Compilation failure: Compilation failure:
[ERROR] /Users/USR/goog/hydrator-plugins/google-cloud/src/main/java/io/cdap/plugin/gcp/bigquery/connector/BigQueryConnector.java:[45,38] cannot find symbol
[ERROR]   symbol:   class ConnectorContext
[ERROR]   location: package io.cdap.cdap.etl.api.connector

You may need to run the following command in your CDAP directory and then try to build hydrator plugins again

mvn clean install -DskipTests -P templates,spark-dev

Then cd out of hydrator-plugins back to the cdap directory, and build the CDAP sandbox, including the plugins:

cd ../cdap
HYDRATOR_PLUGINS=../hydrator-plugins

If you're on a mac you would do ```export HYDRATOR_PLUGINS=[path-to-hydrator-plugins]``` with the full path instead of the above variable assignment otherwise it won't be able to find the hydrator plugins.

cd cdap
MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=128m" mvn clean package \
    -pl cdap-standalone,cdap-app-templates/cdap-etl \
    -am -amd -DskipTests \
    -P templates,dist,release,unit-tests \
    -Dadditional.artifacts.dir=<path to hydrator-plugins>$HYDRATOR_PLUGINS
cd cdap-standalone/target
unzip cdap-sandbox-<version>-SNAPSHOT.zip 
cd cdap-sandbox-<version>-SNAPSHOT
cd bin
./cdap sandbox start

...