Checklist
- User Stories Documented
- User Stories Reviewed
- Design Reviewed
- APIs reviewed
- Release priorities assigned
- Test cases reviewed
- Blog post
Introduction
Goals
- Make CDAP authorization policy consistent across all entities and permissions
- Allow setting granular permissions at dataset level, application level etc.
- Ranger integration for CDAP authorization
- Improve Sentry data model to fix existing issues seen on customer environment
- Allow admins to use existing role/groups for authorization
User Stories
- TBD
Design
CDAP Authorization Model
The existing CDAP Authorization Model has the following drawbacks:
Granular permissions
- Cannot grant a privilege to a user to read only one dataset or one stream in a namespace.
- Cannot grant a privilege to a user to deploy an application/artifact/dataset/stream without granting write on the namespace.
- Cannot grant a privilege to a user to start/stop a program without granting READ on the namespace.
- Visibility
- User who has a privilege on a program cannot see the program in the UI or CLI without having any privilege on the namespace.
- Inconsistencies
- To write to a dataset user needs to have WRITE privilege on the dataset but to write to a stream user needs to have WRITE on the the stream and READ on the namespace.
- To retrieve dataset properties READ on dataset is required whereas to read stream properties any privilege (READ/WRITE/EXECUTE/ADMIN) is sufficient.
- ADMIN on an entity allows to delete the entity where ADMIN on entity doesn't allow to CREATE.
- Dataset read needs namespace READ but dataset write does not need namespace WRITE.
- TBD Dataset Module Delete All.
- Redundancy
- List and View operations are equivalent but are listed separately in documentation.
- Dataset READ and Stream READ are redundant because they need Namespace READ permission to be meaningful.
- Allow granting privileges to entities beforehand
Namespaces
| Operation | Privileges Required (Existing) | Privileges Required (Proposed) |
|---|---|---|
| Create | WRITE (on the CDAP instance) |
|
| Update | ADMIN | |
| Delete | ADMIN | |
| List | Only returns those namespaces on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN | At least one of READ, WRITE, EXECUTE, or ADMIN on the namespace or any of its descendants. |
Artifacts
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Add | W (on the namespace) | WRITE (on the namespace) | ADMIN (on the namespace) | ADMIN (on the artifact being deployed) |
| Add a property | A (on namespace) | A (on artifact) | |
| Remove a property | A (on namespace) | A (on artifact) | |
| Delete | A (on namespace) | A (on artifact) | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | Any of READ, WRITE, EXECUTE, or ADMIN (on namespace) | Any of READ, WRITE, EXECUTE, or ADMIN (on artifact) |
Applications
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Add | WRITE (on the namespace) and READ (on the artifact if deployed from an artifact) | WRITE (on the namespace) | A (namespace) | A (app)
|
| Delete | ADMIN | |
| List | Only returns those applications on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | Any of READ, WRITE, EXECUTE, or ADMIN (on namespace) | Any of READ, WRITE, EXECUTE, or ADMIN (on application) |
Programs
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Start, Stop, or Debug | (EXECUTE (on the program) | EXECUTE (on the application) | EXECUTE (on the namespace)) & READ (on the namespace) | EXECUTE (on the program) | | EXECUTE (on the application) | EXECUTE (on the namespace) |
| Set instances | ADMIN | |
| Set runtime arguments | ADMIN | |
| Retrieve runtime arguments | READ | |
| Retrieve status | At least one of READ, WRITE, EXECUTE, or ADMIN | |
| List | Only returns those programs on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
Datasets
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Create | WRITE (on the namespace) | WRITE (on the namespace) | ADMIN (on the dataset being created) | ADMIN (on the namespace) |
| Read | (READ (on the dataset) and READ (namespace)) | READ (on the namespace) | READ (on the namespace) | READ (on the the dataset) | |
| Write | WRITE (on the dataset) | WRITE (on the namespace) | WRITE (on the the namespace) | WRITE (on the the dataset) | |
| Update | (ADMIN (on the dataset) and READ (on the namespace)) | (ADMIN (on the namespace) and READ (on the namespace)) | ADMIN (on the dataset) | ADMIN (on the namespace) |
| Upgrade | ADMIN | |
| Truncate | ADMIN | |
| Drop | ADMIN | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
Dataset Modules
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Deploy | WRITE (on the namespace) | WRITE (on the namespace) | ADMIN (on the module being deployed) | ADMIN (on the namespace) |
| Delete | ADMIN (on the dataset module) | ADMIN (on the namespace) | |
| Delete-all in the namespace | ADMIN (on the namespace) | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
Dataset Types
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
Secure Keys
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Create | WRITE (on the namespace) | WRITE (on the namespace) | ADMIN (on the key being created) | ADMIN (on the namespace) |
| Delete | ADMIN | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN | |
| Read | READ (on the namespace) | READ (on the key) |
Streams
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Create | WRITE (on the namespace) | WRITE (on the namespace) | ADMIN (on the stream being created) | ADMIN (on the namespace) |
| Retrieving events | READ (on the stream) & READ (on the namespace) | READ (on the stream) | READ (on namespace) |
| Retrieving properties | At least one of READ, WRITE, ADMIN, or EXECUTE | TBD |
| Sending events to a stream (sync, async, or batch) | (WRITE (on the stream) and READ (on the namespace)) | WRITE (on namespace & READ (on the namespace)) | WRITE (on the stream) | WRITE (on namespace) |
| Drop | ADMIN (on dataset) | ADMIN (on namespace) | |
| Drop-all in the namespace | ADMIN (on the namespace) | |
| Update | ADMIN | | |
| Truncate | ADMIN | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
CDAP Ranger Integration
CDAP Sentry Extension Improvements
- Has no grant
Existing Roles/Groups for Authorization
API changes
New Programmatic APIs
New Java APIs introduced (both user facing and internal)
Deprecated Programmatic APIs
New REST APIs
| Path | Method | Description | Response Code | Response |
|---|---|---|---|---|
| /v3/apps/<app-id> | GET | Returns the application spec for a given application | 200 - On success 404 - When application is not available 500 - Any internal errors |
|
Deprecated REST API
| Path | Method | Description |
|---|---|---|
| /v3/apps/<app-id> | GET | Returns the application spec for a given application |
CLI Impact or Changes
- Impact #1
- Impact #2
- Impact #3
UI Impact or Changes
- Impact #1
- Impact #2
- Impact #3
Security Impact
What's the impact on Authorization and how does the design take care of this aspect
Impact on Infrastructure Outages
System behavior (if applicable - document impact on downstream [ YARN, HBase etc ] component failures) and how does the design take care of these aspect
Test Scenarios
| Test ID | Test Description | Expected Results |
|---|---|---|
Releases
Release X.Y.Z
Release X.Y.Z
Related Work
- Work #1
- Work #2
- Work #3