...
The provisioner is the component that intereacts with cloud providers for resources provisioning and de-provisioning. Generally it has the following properties:
- Resilient It must be able to handle service interruptionsto failures
- CDAP master interruptions, including restart, failover, upgrade, etc.
- Cloud provider interruptions, which can be caused by network outage, disconnection or cloud provider availability issues.
- By resilientGenerally speak, it means the provisioner progress might be halt during service interruptions, but should be able to recover and continue correctly after interruptions.
- Generally it is handled with intent log and retry
- First log (persist) the intented action before performing the action
- Based on intent log, the action will be retried upon failure, even after process termination and restart
- The retry should contain logic to validate if the action has been performed or not by consulting the source of truth
- Once the action completed, log the completion state as well as the next intended action (if any).
- Pluggable and Extensible
- It is defined as part of the Runtime Extension SPI
- Horizontally Scalable
- A CDAP instance could be launching thousands or even more programs at the same time
Detail design is in CDAP Provisioning
Program Launcher
The Program Launcher is responsible to launch the CDAP runtime in the cluster being provisioned by the Provisioner.
Runtime Monitor
Program Runtime
...