...
- A stable and scalable core platform that provides essential functionalities
- An application API for Data Application development
- Manages and operates Data Application
- A central catalog for entities' metadata
- Provide a well defined API to support addition capabilities
- Enable fast iteration of new ideas
- Allow running data / compute intensive jobs for the system (e.g. Spark, Elastic Search, etc.)
- Individual extended system can be turn on / off independently
# Transaction Service may not be available to application runtime, depending on the target execution cluster and we may eventually not needing it at all.
CDAP Core System
The major components of the CDAP core are as follows:
- Metadata Catalog
- Responsible for collecting, storing, serving and exporting metadata as defined by users and applications
- Artifacts and Application Repository
- Responsible for artifacts and applications deployment, update (versioning), removal and surfacing of those information
- Runtime Manager
- Responsible for all aspects about program lifecycle, including resource provisioning, execution, run records, etc.
- Scheduler
- Responsible for launching programs based on triggers and constraints
- Transaction
- Apache Tephra to provide transactional operations on HBase
- #Transaction Service may not be available to application runtime, depending on the target execution cluster and we may eventually removing it
- Metrics and Monitoring
- Responsible for collecting and querying metrics
- *May integrate with external metrics system in future
- TMS
- Provides messaging service for event based design to decouple components
CDAP Extended System
Metadata Management
...