Introduction 

Briefly write the need for this feature

Goals

1. Simplify and standardize the process of authorization enforcement in CDAP.
2. Support Authorization all across CDAP (Logs, Metrics, Stream Views, Metadata, Preferences, Kafka, Explore)

3. Enable/Disable operations on the fly for the logged in user (Out of scope for 4.0)

User Stories 

1. As a CDAP developer, I would like to easily enforce authorization for a code block with minimal code in a standard way.
2. As a CDAP security admin, I will like to enforce authorization on all entities in CDAP.
3. As a CDAP security admin, I will like users to access/view only the operation for which they have appropriate privileges. (Out of scope of 4.0)

Scenarios

• Scenario #1

  Bhooshan is a ninja CDAP developer who is working on a cool new feature, adding lots of new publicly exposed APIs and he want  support authorization enforcement for his new feature. He looks around in CDAP code base and finds out that he will need to get some classes through injector and put conditional checks at various places in his new code. He feels overwhelmed by all this new changes and will like to have a simplistic and standard to add authorization.
  
  

• Scenario #2

  Derek is an IT Operations Extraordinaire at a corporation that uses CDAP. He just upgraded CDAP to use the new authorization features. He granted all the appropriate privileges to different users of CDAP in his corporation and left for the day. When he comes back to the office next day he sees a new ticket assigned to him created by Mr. Little Fingers who is angry because Mr. Tim can see all the logs and metrics which his CDAP programs are generating. Derek is furious and after some digging he identifies that not all entities in CDAP supports authorization and he will like to enforce authorization on all entities in CDAP. 
  
  

• Scenario #3 (Out of scope for 4.0)

  Derek (IT Operations) has been appreciating the authorization feature of CDAP as it has restricted unintentional usage of CDAP entities among the users in his coporation. But he does not like that users can see the UI button to run a CDAP program even though the user does not have privileges for it. This creates an unnecessary confusion among users. He thinks that it will be helpful if an user can only see operations for which he has privileges. 

Motivation

After adding support for Authorization in CDAP we are are enforcing authorization for operation on different cdap entities through AuthorizationEnforcer in the below way:

public synchronized void create(final NamespaceMeta metadata) throws Exception {
  // Namespace can be created. Check if the user is authorized now.
  Principal principal = authenticationContext.getPrincipal();
  authorizationEnforcer.enforce(instanceId, principal, Action.ADMIN);
  ....
  ...
}

The AuthorizationEnforcer is also used to create filters for entities which is accessible to an user. This is typically used in list operations:

public List<NamespaceMeta> list() throws Exception {
  List<NamespaceMeta> namespaces = nsStore.list();
  Principal principal = authenticationContext.getPrincipal();
  final Predicate<EntityId> filter = authorizationEnforcer.createFilter(principal);
  return Lists.newArrayList(
    Iterables.filter(namespaces, new com.google.common.base.Predicate<NamespaceMeta>() {
      @Override
      public boolean apply(NamespaceMeta namespaceMeta) {
        return filter.apply(namespaceMeta.getNamespaceId());
      }
    })
  );
}

The above process is very cumbersome because of the following reason:

1. A developer will first need to make sure he has all the needed module injected to use AuthorizationEnforcer and AuthenticationContext. 
2. Create and initialize an instance of AuthorizationEnforcer and AuthenticationContext
3. Use them at various place in their code to enforce Authorization.

Moreover, we require authorization enforcement all over cdap code base which requires the above to be done at a lot of places.

Also, in coming releases we also want to support enabling/disabling operation depending on user privileges. For example if an user does not have "ADMIN" privilege on a program the "START" button in the UI should be disabled for that user. Similar feature is needed in CLI too where the auto complete should only show options for which the user have privileges. Although, this requirement is out of score for 4.0 we will like to design the Authorization Enforcement mechanism to support this in near future.

The above calls for some redesign in the authorization enforcement to simplify and standardize authorization enforcement across CDAP code base and also provide a good base for further UI enhancement. The design in following section present an approach to solve the above issues.

Design

In our current code base we do authorization enforcement where the entity id on which enforcement is being done is one of the following: 

StreamQueueReader.dequeue

ArtifactRepository.deleteArtifactProperties

ArtifactRepository.addArtifact

Approach

We propose the following authorization annotation to standarize the process of authorization enforcement. This will also allow us to enable/disable actions in UI and CLI on the fly since we will know required privilege for an action in advance.

@AuthEnforce(

     entity("pid": EntityId)

     enforceOn(ApplicationId.class)

     privileges({ADMIN, EXECUTE})

)

@AuthEnforce(

     entity(String[] = {"ns":, "appid"})

     enforceOn(ApplicationId.class)

     privileges({ADMIN, EXECUTE})

) 

entity: Specifies the entity on which authorization will be enforced. It can either be an EntityId or an array of Strings from which the entity id can be constructed. 

enforeOn: Specifies the entityId on which authorization enforcement will be done. It is possible to pass ProgramId in "entity" and enforce authorization on NamespaceId by passing NamespaceId.class here. This allows developers to enforce on parents of an entity. In the case of "entity" being being an array of String enforceOn specifies what entityId will be created.

privileges: The privileges to check for during authorization enforcement on this entity. This is an array of Actions to allow enforcing multiple checks.

API changes

New Programmatic APIs

New Java APIs introduced (both user facing and internal)

Deprecated Programmatic APIs

New REST APIs

Deprecated REST API

CLI Impact or Changes

• Impact #1
• Impact #2
• Impact #3

UI Impact or Changes

• Impact #1
• Impact #2
• Impact #3

Security Impact 

What's the impact on Authorization and how does the design take care of this aspect

Impact on Infrastructure Outages 

System behavior (if applicable - document impact on downstream [ YARN, HBase etc ] component failures) and how does the design take care of these aspect

Test Scenarios

Releases

Release 4.0.0

• Goal 1: Simplify and standardize the process of authorization enforcement in CDAP.
• Goal 2: Support Authorization all across CDAP (Logs, Metrics, Stream Views, Metadata, Preferences, Kafka, Explore)

Release 4.1.0

• Goal 3: As a CDAP security admin, I will like users to access/view only the operation for which they have appropriate privileges. (Out of scope of 4.0)

Related Work

• Error rendering macro 'jira' : Unable to locate Jira server for this macro. It may be due to Application Link configuration.
  
  
• Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Future work

Enabling/Disabling action in UI and CLI

The future work involves taking advantage of authorization annotation to enable/disable actions in UI and CLI for an user on the fly. Below we discuss an initial design to showcase that this is possible with authorization annotation.

We will add an unique name called actionId to every annotation. This will serve as a map between UI and the backend so that for every action in UI we will know what privileges needs to be enforced and on which entity.

@AuthEnforce(

     entity("pid": EntityId)

     enforceOn(ApplicationId.class)

     privileges({ADMIN, EXECUTE})
 
     actionId("uniqueActionName")
)

PrivilegeInspector: When CDAP  Master starts privilege inspector can inspect all the authorization annotation to create a map of  privileges needed for a given action.

Querying Privileges: Once the action to required privilege map is build by the PrivilegeInspector this can be served to the  UI. We will introduce new endpoints which which will map every page in UI (like ns page, app page etc) which can be used to construct entityId on which authorization needs to be enforced and the UI will also pass set of unique action names for which privileges needs to be checked.

For example let's say a user is on a program page  (/ns/appid/program/checkactions)

/programs end point can be hit with  /program {ns, appid} {set of actions: start, stop}

Now the handler can use the map created by the privilegeInspector to know what privileges are needed for the requested action.