...
- A stable and scalable core platform that provides essential functionalities
- An application API for Data Application development
- Manages and operates Data Application
- A central catalog for entities' metadata
- Provide a well defined API to support addition capabilities
- Enable fast iteration of new ideas
- Allow running data / compute intensive jobs for the system (e.g. Spark, Elastic Search, etc.)
- Individual extended system can be turn on / off independently
CDAP Core System
The major components of the CDAP core are as follows:
- Metadata Catalog
- Responsible for collecting, storing, serving and exporting metadata as defined by users and applications
- Artifacts and Application Repository
- Responsible for artifacts and applications deployment, update (versioning), removal and surfacing of those information
- Runtime Manager
- Responsible for all aspects about program lifecycle, including resource provisioning, execution, run records, etc.
- Scheduler
- Responsible for launching programs based on triggers and constraints
- Transaction
- Apache Tephra to provide transactional operations on HBase
- #Transaction Service may not be available to application runtime, depending on the target execution cluster
...
CDAP Core System
...
- and we may eventually removing it
- Metrics and Monitoring
- Responsible for collecting and querying metrics
- *May integrate with external metrics system in future
- TMS
- Provides messaging service for event based design to decouple components
CDAP Extended System
Metadata Management
...