Goals

1. Performance improvements (caching authorization policies)

2. Authorization of dataset and stream access

3. Authorization for listing and viewing entities

Checklist

• User stories documented (Bhooshan)
• User stories reviewed (Nitin)
• Design documented (Bhooshan)
• Design reviewed (Andreas/Terence)
• Feature merged (Bhooshan)
• Documentation (Bhooshan)
• Blog post 

User Stories

1. As a CDAP security admin, I want all operations on datasets/streams to be governed by my configured authorization system.
2. As a CDAP security admin, I want list operations for all CDAP entities to only return entities that the logged-in user is authorized to view.
3. As a CDAP security admin, I want view operations for a CDAP entity to only succeed if the logged-in user is authorized to view that entity

Scenarios

Scenario #1

Derek is an IT Operations Extraordinaire at a corporation that uses CDAP to manage datasets with varying degrees of sensitivity. He would like to implement authorization policies for all data stored in CDAP across datasets and streams, so only authorized users have access to such data. He would like to control both read as well as write access. 

Scenario #2

Derek would like to be able to use external authorization systems like Apache Sentry to manage authorization policies. Given that Apache Sentry could be installed in a different environment from CDAP, he would like to minimize the impact of verifying authorization while accessing data. Derek expects that performance improvement does not result in security breaches. For example, if authorization policies are cached in CDAP, Derek expects that they be refreshed regularly at configurable time intervals.

Scenario #3

In the said organization, CDAP is used to store data belonging to various business units. These business units are potentially completely disparate, and do not share information. Some of their data or applications may be extremely sensitive. As a security measure, Derek would also like to enforce authorization for operations that list CDAP entities, so that a user can only see the entities that he is authorized to read or write.

Design

Authorizing Dataset and Stream operations

The most critical requirement to address in 3.5 is to authorize dataset and stream operations. These operations can be categorized into data access (read/write) and admin (create, update, delete). Admin operations can be presumed to occur less often than data access operations, and are not in the data path. As a result, even though performance is important, it is less critical for admin operations compared to data access operations. For data access operations, it is not practical to communicate with an external authorization system like Apache Sentry for every single operation, since that would lead to major performance degradation. As a result, authorization policies need to be cached in CDAP potentially for all operations, but especially for data access operations.

One of the major concerns about caching is freshness or invalidation. It is especially important in a security/authorization context, because it could result in security breaches. For example, suppose we've cached all authorization policies. An update, especially a rollback of privileges in the external authorization system should result in an immediate refresh of the cache, otherwise there could be security breaches by the time refresh takes place.

For such an authorization policy cache, the major design goals are:

1. Minimal refresh time 
   1. The refresh operation should be fast. 
   2. It should make minimal RPC calls
   3. It should transfer only necessary data
2. Configurable refresh interval
   1. The refresh operation should happen at configurable time intervals so users can tune it per their requirement.

To satisfy these goals, the data structure that should be cached can be defined as follows:

// TODO: Explore using Guava Cache
class PrivilegeCache {
  private final Table<Principal, EntityId, Set<Action>> privileges = HashBasedTable.create();

  public void addPrivileges(Principal principal, EntityId entityId, Set<Action> actionsToAdd) {
    Set<Action> actions = privileges.get(principal, entityId);
    if (actions == null) {
      actions = new HashSet<>();
    }
    actions.addAll(actionsToAdd);
    privileges.put(principal, entityId, actions);
  }

  public void revokePrivileges(Principal principal, EntityId entityId, Set<Action> actionsToRemove) {
    Set<Action> actions = privileges.get(principal, entityId);
    if (actions == null) {
      throw new NoSuchElementException();
    }
    actions.removeAll(actionsToRemove);
    privileges.put(principal, entityId, actions);
  }
}

The above cache would be re-populated asynchronously from the configured Authorization Provider (Apache Sentry/Apache Ranger, etc) at a configurable time interval, using an AbstractScheduledService. Instead of querying these authorization providers every time an authorization check is required, various CDAP sub-components will instead query this cache.

: authorization providers may have their own caching mechanisms. e.g. Sentry has PrivilegeCache. Is any integration with these possible

Cache Freshness

Like mentioned above, the policy cache in CDAP can be made consistent with authorization providers at regular scheduled intervals. However, this has the following race: Suppose Alice and Bob have been given READ access to Dataset1, and this state is consistent in both the external system (e.g. Apache Sentry) and the cache. Now, ACLs are updated to remove Alice's permissions. Until the time when the refresh thread mentioned above runs, the cache will be inconsistent with the external system, and CDAP will still think that both Alice and Bob have READ access to Dataset1. The severity of this may vary depending on the situation, but it is a security loophole nonetheless. There are two possible ways in which this situation may arise:

1. User uses CDAP (CLI/REST APIs) to update ACLs: In this scenario, we can have a callback to the revoke APIs in CDAP to also update the cache. As long as both updating the store and the cache is done transactionally , there would not be an inconsistency between the external system and the CDAP cache.
2. User uses an external interface (e.g. Hue, Apache Ranger UI) to update ACLs: In this scenario, we may have to depend upon the external system providing a callback mechanism. Even if such a mechanism is provided, the interface for the cache to be updated (e.g. from a message queue), will have to be built in CDAP. The external system can then add events to such an interface, and the cache could keep itself up-to-date. In the first release, however, it is likely that there may be an inconsistency if this method is chosen to update ACLs.

Caching in Apache Sentry

Apache Sentry has some active work going on to enable client-side caching as part of SENTRY-1229. It will likely suffer from the same drawbacks mentioned above regarding cache freshness, but CDAP should re-use such caching mechanisms if they're available in authorization providers. Another drawback of this approach though is that it is active work, and there are no timelines yet as to when this change will make it to a CDH distro.

Authorizing list operations

Operations that list entities (namespaces, artifacts, apps, programs, datasets, streams) should be updated so that they only return the entities that the currently logged-in user has privileges on. 

• Listing namespaces, apps, artifacts, datasets and streams should return only those respective entities that the user has READ, WRITE or ALL permissions on
• Listing programs should return programs that the user has READ, WRITE, EXECUTE or ALL permissions on

To achieve this, the corresponding list APIs in CDAP (e.g. NamespaceAdmin, AppLifecycleService, ProgramLifecycleService, DatasetFramework, StreamAdmin) should be updated with a filter to only return elements that users have access to.

Dependencies

Ability to distinguish between read and write operations in datasets

Entities, Operations and Required Privileges

Entity	Operation	Required Privileges	Resultant Privileges
Namespace	create	ADMIN (Instance)	ADMIN (Namespace)
	update	ADMIN (Namespace)
	list	READ (Instance)
	get	READ (Namespace)
	delete	ADMIN (Namespace)
	set preference	WRITE (Namespace)
	get preference	READ (Namespace)
	search	READ (Namespace)
Artifact	add	WRITE (Namespace)	ADMIN (Artifact)
	delete	ADMIN (Artifact)
	get	READ (Artifact)
	list	READ (Namespace)
	write property	ADMIN (Artifact)
	delete property	ADMIN (Artifact)
	get property	READ (Artifact)
	refresh	WRITE (Instance)
	write metadata	ADMIN (Artifact)
	read metadata	READ (Artifact)
Application	deploy	WRITE (Namespace)	ADMIN (Application)
	get	READ (Application)
	list	READ (Namespace)
	update	ADMIN (Application)
	delete	ADMIN (Application)
	set preference	WRITE (Application)
	get preference	READ (Application)
	add metadata	ADMIN (Application)
	get metadata	READ (Application)
Programs	start/stop/debug	EXECUTE (Program)
	set instances	ADMIN (Program)
	list	READ (Namespace)
	set runtime args	EXECUTE (Program)
	get runtime args	READ (Program)
	get instances	READ (Program)
	set preference	ADMIN (Program)
	get preference	READ (Program)
	get status	READ (Program)
	get history	READ (Program)
	add metadata	ADMIN (Program)
	get metadata	READ (Program)
	emit logs	WRITE (Program)
	view logs	READ (Program)
	emit metrics	WRITE (Program)
	view metrics	READ (Program)
Streams	create	WRITE (Namespace)	ADMIN (Stream)
	update properties	ADMIN (Stream)
	delete	ADMIN (Stream)
	truncate	ADMIN (Stream)
	enqueue asyncEnqueue batch	WRITE (Stream)
	get	READ (Stream)
	list	READ (Namespace)
	read events	READ (Stream)
	set preferences	ADMIN (Stream)
	get preferences	READ (Stream)
	add metadata	ADMIN (Stream)
	get metadata	READ (Stream)
	view lineage	READ (Stream)
	emit metrics	WRITE (Stream)
	view metrics	READ (Stream)
Datasets	list	READ (Namespace)
	get	READ (Dataset)
	create	WRITE (Namespace)	ADMIN (Dataset)
	update	ADMIN (Dataset)
	drop	ADMIN (Dataset)
	executeAdmin (exists/truncate/upgrade)	ADMIN (Dataset)
	add metadata	ADMIN (Dataset)
	get metadata	READ (Dataset)
	view lineage	READ (Dataset)
	emit metrics	WRITE (Dataset)
	view metrics	READ (Dataset)

NOTE: Cells marked green were done in 3.4. Cells marked in yellow are in scope for 3.5.

References