Goals:
- Provide a great out of the box installation experience
- CDAP services on startup should check necessary pre-reqs and provide meaningful error messages if pre-reqs are not met
- Provide meaningful messages in the log files for services on failure
- Improve the documentation for installation
Checklist
- User stories documented (Albert/Alvin)
- User stories reviewed (Nitin)
- Design documented (Albert/Alvin)
- Design reviewed (Andreas)
- Feature merged (Albert/Alvin)
-
Examples and guides (Albert/Alvin) (not needed for the feature) - Integration tests (Albert/Alvin)
- Documentation for feature (Albert/Alvin)
- Blog post
User Stories:
- User should be able to easily navigate to the installation docs for Ambari, Cloudera manager, Installing via package managers, MapR
- User should be able to perform necessary preparation step for the installation using command line or UI
- User should be able to perform step by step installation using command line or UI
Users should have section for installing on HA clusters
HA secure clusters
HA YARN
HA HDFS
HA CDAP
- User should be able to install non standard components (nodejs) using instructions in docs
- Users should be able to easily determine the configurations needed for CDAP installation by reading the docs
- Users should have section for installing on secure clusters
- CDAP master services should not start up if required pre-requisites are not met
- Pre-requisites: YARN containers required
- HDFS permissions
- CDAP system services running on Twill
- Kafka services [Low Priority]
- CDAP master service should log appropriate error messages if it fails to start and prescribe corrective actions where possible
- CDAP master should start successfully only if all the corresponding system service can start fine
- CDAP master service should serve requests only if app fabric and corresponding system services start up fine
- Error logs in master logs should be meaningful to the end user and the end user should understand the behavior of the service easily
- Master log should contain only app fabric logs and system services logs and not application logs
- User should be able to determine the complete class path of the services (master, router, auth) by looking at the logs
- Users should be able to determine what components are enabled and disabled by looking at the logs
- Users should be able to determine the versions of underlying infra components by looking at the master logs
- Users should be able to query for versions of the underlying infra components using HTTP REST API calls (Low)
- CDAP router should not startup if it cannot bind to configured port
- CDAP router should log appropriate error in the logs if it fails to start
- CDAP Auth should not startup if it cannot bind to configured port
- CDAP auth service should log appropriate error in the logs if it fails to start
- Services that are bound to 0.0.0.0 should log all the interfaces that are bound (Low)
- CDAP UI should not serve services are starting up content up if the downstream services are not up and not make any other calls to the backend
- CDAP UI should log meaningful messages on failure to start
- CDAP UI should should not start up if the version of nodejs installed is not compatible
- Users should be able to optionally disable the checks performed during startup
- Service (/etc/init.d/<service> status should work even if service is stopped
Design:
Additional Notes:
Brainstorming notes (Andreas/Nitin/Albert/Sree):
...