| Table of Contents |
|---|
Checklist
- User Stories Documented
- User Stories Reviewed
- Design Reviewed
- APIs reviewed
- Release priorities assigned
- Test cases reviewed
- Blog post
Introduction
Goals
- Make CDAP authorization policy consistent across all entities and permissions
- Allow setting granular permissions at dataset level, application level etc.
- Ranger integration for CDAP authorization
- Improve Sentry data model to fix existing issues seen on customer environment
- Allow admins to use existing role/groups for authorization
User Stories
- TBD
Design
CDAP Authorization Model
Existing CDAP Authorization Model
The existing CDAP Authorization Model has the following drawbacks:
Granular permissions
- Cannot grant a privilege to a user to read only one dataset or one stream in a namespace.
- Cannot grant a privilege to a user to deploy an application/artifact/dataset/stream without granting write on the namespace.
- Cannot grant a privilege to a user to start/stop a program without granting READ on the namespace.
- Visibility
- User who has a privilege on a program cannot see the program in the UI or CLI without having any privilege on the namespace.
- Inconsistencies
- To write to a dataset user needs to have WRITE privilege on the dataset but to write to a stream user needs to have WRITE on the the stream and READ on the namespace.
- To retrieve dataset properties READ on dataset is required whereas to read stream properties any privilege (READ/WRITE/EXECUTE/ADMIN) is sufficient.
- ADMIN on an entity allows to delete the entity where ADMIN on entity doesn't allow to CREATE.
- Dataset read needs namespace READ but dataset write does not need namespace WRITE.
- TBD Dataset Module Delete All.
- Redundancy
- List and View operations are equivalent but are listed separately in documentation.
- Dataset READ and Stream READ are redundant because they need Namespace READ permission to be meaningful.
Overview of Proposed Model
We propose the following CDAP Authorization policy which can be defined by the following three principles:
Access:
Access defines who can perform an action (READ, WRITE, EXECUTE, ADMIN) on an entity.
Access flows top-down i.e. if an user has READ on namespace it implies that the user has READ on all entities inside the namespace.
Visibility
- Visibility defines whether an entity is visible to a user or not.
- If a user has any privilege on an entity, it is visible to the user.
- Visibility flows bottom-up i.e. if a user has any privilege on a program then the user will be able to see the application that contains the program and namespace that contains the application.
Grant
- Grant is defined as action of giving a privilege on an entity to a user.
- To grant privileges on an entity ADMIN on the entity is required.
- Grant flows top-down i.e. if a user has ADMIN on namespace then the user can grant privileges on all entities inside the namespace.
- Note: CDAP Instance is not part of the privilege hierarchy.
Decouple entity existence from privilege:
In addition CDAP will now support creating privileges for entities that are yet to be created. This will allow admins to grant fine grained privileges on entities. For example, an admin can grant a user ADMIN on an application before the application is deployed. This will allow the user to deploy only this specific application without having any other access to the namespace.
Decouple entity creation and granting privileges:
TBD
Based on the above, we propose the following changes to the authorization matrix:
Instance
ADMIN on an Instance allows user to create Namespaces in the instance. No other operations are defined as of now.
Also Instance is not a part of privilege hierarchy.
Namespaces
| Operation | Privileges Required (Existing) | Privileges Required (Proposed) |
|---|---|---|
| Create | WRITE (on the CDAP instance) | WRITE (on the CDAP instance) | ADMIN (on the namespace) |
| Update | ADMIN | |
| Delete | ADMIN | |
| List | Only returns those namespaces on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN | At least one of READ, WRITE, EXECUTE, or ADMIN on the namespace or any of its descendants. |
| Grant | ADMIN (on the namespace) |
Artifacts
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Add | W (on the namespace) | WRITE (on the namespace) | ADMIN (on the namespace) | ADMIN (on the artifact being deployed) |
| Add a property | A (on namespace) | A (on artifact) | |
| Remove a property | A (on namespace) | A (on artifact) | |
| Delete | A (on namespace) | A (on artifact) | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | Any of READ, WRITE, EXECUTE, or ADMIN (on namespace) | Any of READ, WRITE, EXECUTE, or ADMIN (on artifact) |
Applications
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Add | WRITE (on the namespace) and READ (on the artifact if deployed from an artifact) | WRITE (on the namespace) | A (namespace) | A (app)
|
| Delete | ADMIN | |
| List | Only returns those applications on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | Any of READ, WRITE, EXECUTE, or ADMIN (on namespace) | Any of READ, WRITE, EXECUTE, or ADMIN (on application) |
Programs
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Start, Stop, or Debug | (EXECUTE (on the program) | EXECUTE (on the application) | EXECUTE (on the namespace)) & READ (on the namespace) | EXECUTE (on the program) | | EXECUTE (on the application) | EXECUTE (on the namespace) |
| Set instances | ADMIN | |
| Set runtime arguments | ADMIN | |
| Retrieve runtime arguments | READ | |
| Retrieve status | At least one of READ, WRITE, EXECUTE, or ADMIN | |
| List | Only returns those programs on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
Datasets
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Create | WRITE (on the namespace) | WRITE (on the namespace) | ADMIN (on the dataset being created) | ADMIN (on the namespace) |
| Read | (READ (on the dataset) and READ (namespace)) | READ (on the namespace) | READ (on the namespace) | READ (on the the dataset) | |
| Retrieving properties | Not Documented | At least one of READ, WRITE, ADMIN, or EXECUTE |
| Write | WRITE (on the dataset) | WRITE (on the namespace) | WRITE (on the the namespace) | WRITE (on the the dataset) | |
| Update | (ADMIN (on the dataset) and READ (on the namespace)) | (ADMIN (on the namespace) and READ (on the namespace)) | ADMIN (on the dataset) | ADMIN (on the namespace) |
| Upgrade | ADMIN | |
| Truncate | ADMIN | |
| Drop | ADMIN | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
Dataset Modules
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Deploy | WRITE (on the namespace) | WRITE (on the namespace) | ADMIN (on the module being deployed) | ADMIN (on the namespace) |
| Delete | ADMIN (on the dataset module) | ADMIN (on the namespace) | |
| Delete-all in the namespace | ADMIN (on the namespace) | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
Dataset Types
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | Will be removed |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
Secure Keys
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Create | WRITE (on the namespace) | WRITE (on the namespace) | ADMIN (on the key being created) | ADMIN (on the namespace) |
| Delete | ADMIN | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN | |
| Read | READ (on the namespace) | READ (on the key) |
Streams
| Operation | Privileges Required | Privileges Required (Proposed) |
|---|---|---|
| Create | WRITE (on the namespace) | WRITE (on the namespace) | ADMIN (on the stream being created) | ADMIN (on the namespace) |
| Retrieving events | READ (on the stream) & READ (on the namespace) | READ (on the stream) | READ (on namespace) |
| Retrieving properties | At least one of READ, WRITE, ADMIN, or EXECUTE | |
| Sending events to a stream (sync, async, or batch) | (WRITE (on the stream) and READ (on the namespace)) | WRITE (on namespace & READ (on the namespace)) | WRITE (on the stream) | WRITE (on namespace) |
| Drop | ADMIN (on dataset) | ADMIN (on namespace) | |
| Drop-all in the namespace | ADMIN (on the namespace) | |
| Update | ADMIN | | |
| Truncate | ADMIN | |
| List | Only returns those artifacts on which user has at least one of READ, WRITE, EXECUTE, or ADMIN | |
| View | At least one of READ, WRITE, EXECUTE, or ADMIN |
CDAP Ranger Integration
Introduction
Apache Ranger is authorization store.
Design
CDAP Sentry Extension Improvements
- Has no grant
Existing Roles/Groups for Authorization
API changes
New Programmatic APIs
New Java APIs introduced (both user facing and internal)
Deprecated Programmatic APIs
New REST APIs
| Path | Method | Description | Response Code | Response |
|---|---|---|---|---|
| /v3/apps/<app-id> | GET | Returns the application spec for a given application | 200 - On success 404 - When application is not available 500 - Any internal errors |
|
Deprecated REST API
| Path | Method | Description |
|---|---|---|
| /v3/apps/<app-id> | GET | Returns the application spec for a given application |
CLI Impact or Changes
- Impact #1
- Impact #2
- Impact #3
UI Impact or Changes
- Impact #1
- Impact #2
- Impact #3
Security Impact
What's the impact on Authorization and how does the design take care of this aspect
Impact on Infrastructure Outages
System behavior (if applicable - document impact on downstream [ YARN, HBase etc ] component failures) and how does the design take care of these aspect
Test Scenarios
| Test ID | Test Description | Expected Results |
|---|---|---|
Releases
Release X.Y.Z
Release X.Y.Z
Related Work
- Work #1
- Work #2
- Work #3