...
This document describes the CDAP build pipelines on Cask's build system. There There are two main pipelines, the CDAP development pipeline, and the CDAP release pipeline. Both pipelines start with a single build, CDAP-DRC, which performs RAT and the CDAP development pipeline.
Release Pipeline
Builds: CDAP-BUT, CDAP-DA
The release pipeline consists of two builds, to be run in sequence. The first build is CDAP-BUT, or Build and Unit Test, which produces all of the checkstyle checks on all branches in the CDAP repository. On success, the build triggers the child builds, which are based on the branch name.
Development Pipeline
Branches: (bugfix|feature)/CDAP.* and (bugfix|feature)_release/CDAP.*, or for UI: bugfix|feature)-ui/CDAP.* and (bugfix|feature)_release-ui/CDAP.*
The development pipeline is for pre-merge testing and will perform unit tests on the given branch. There are two builds which cover this: CDAP-DUT, which will match (bugfix|feature)/.* branches and merge with the develop branch before building, and CDAP-RUT, which will match any (bugfix|feature)_release/CDAP.* branches and merge with the latest release/.* branch of CDAP. If a release bugfix or feature branch needs to merge against a different parent release, the plan branch will need to be configured manually to do so.
Similarly for UI code branches, there are two UI-specific builds: CDAP-UDUT will match bugfix|feature)-ui/CDAP.* branches and merge with develop before building, and CDAP-URUT will match any (bugfix|feature)_release-ui/CDAP.* branches and merge with the latest release/.* branch of CDAP. If a release bugfix or feature branch needs to merge against a different parent release, the plan branch will need to be configured manually to do so.
Release Pipeline
Branches: develop and release/.*
The release pipeline is for post-merge building and testing. It uses the cdap-build repository to include projects outside the main CDAP repository, such as Cask Tracker, which are shipped with CDAP releases. This is done in the CDAP-BUT build, which performs unit tests on the given branch, then builds all of the Maven artifacts for a CDAP release and deploys JARs to Sonatype OSS for public consumption. Snapshot JARs are available immediately. Release JARs are stored in a staging repository and must be released manually by Cask before they are accessible. The release pipeline only processes the develop branch and release branches, which match a release/.* pattern. The release branches must be created manually when the release branch is created. The second build, CDAP-DA, or Deploy Artifacts, deploys the release artifacts created by CDAP-BUT to the appropriate remote storage locations for public consumptionAfter completion of the main artifacts build, CDAP-BUT will trigger CDAP-BA, an assembly build which creates secondary artifacts such as the SDK Docker Image and Virtual Machine Image, and IT-ITS, an integration test build using in-memory CDAP.
To release artifacts to the public, the CDAP-DA build will download them from CDAP-BUT and CDAP-BA and deploy them to the appropriate public repository. CDAP-DA requires artifacts from CDAP-BUT and CDAP-BA, but it is not triggered by CDAP-BUT and must be run manually to deploy.
Development Pipeline
Builds: CDAP-DRC, CDAP-DUT, CDAP-RUT
The development pipeline consists of three builds, in two stages. The first build is CDAP-DRC, or RAT and Checkstyle, which performs RAT and checkstyle checks on the given branch.
The development pipeline only processed the develop branch and bugfix or feature branches, which match a (bugfix|feature).* pattern. Branches which match (bugfix|feature)/.* will trigger CDAP-DUT whereas branches which match (bugfix|feature)_release.* will trigger CDAP-RUT. The CDAP-DUT build merges against develop, whereas the CDAP-RUT build merges against the latest CDAP release branch, which is currently "release/3.5". If a branch needs to merge against a different release, the plan branch will need to be configured manually to do so.