Introduction 

Briefly write the need for this feature

Goals

1. Simplify and standardize the process of authorization enforcement in CDAP.
2. Support Authorization all across CDAP (Logs, Metrics, Stream Views, Metadata, Preferences, Kafka, Explore)

3. Enable/Disable operations on the fly for the logged in user (Out of scope for 4.0)

User Stories 

1. As a CDAP developer, I would like to easily enforce authorization for a code block with minimal code in a standard way.
2. As a CDAP security admin, I will like to enforce authorization on all entities in CDAP.
3. As a CDAP security admin, I will like users to access/view only the operation for which they have appropriate privileges. (Out of scope of 4.0)

Scenarios

Scenario #1

Scenario #2

Motivation

After adding support for Authorization in CDAP, we are are enforcing authorization for operation authorizing operations on different cdap entities through AuthorizationEnforcer in the below following way:

public synchronized void create(final NamespaceMeta metadata) throws Exception {
  // Namespace can be created. Check if the user is authorized now.
  Principal principal = authenticationContext.getPrincipal();
  authorizationEnforcer.enforce(instanceId, principal, Action.ADMIN);
  ....
  ...
}

The AuthorizationEnforcer is also used to create filters for entities which is are accessible to an a user. This is typically used in list operations:

public List<NamespaceMeta> list() throws Exception {
  List<NamespaceMeta> namespaces = nsStore.list();
  Principal principal = authenticationContext.getPrincipal();
  final Predicate<EntityId> filter = authorizationEnforcer.createFilter(principal);
  return Lists.newArrayList(
    Iterables.filter(namespaces, new com.google.common.base.Predicate<NamespaceMeta>() {
      @Override
      public boolean apply(NamespaceMeta namespaceMeta) {
        return filter.apply(namespaceMeta.getNamespaceId());
      }
    })
  );
}

The above This process is very cumbersome because of the following reasonreasons:

1. A developer will first need to make sure he has all the needed module modules injected to use AuthorizationEnforcer and AuthenticationContext. 
2. Create and initialize an instance of AuthorizationEnforcer and AuthenticationContext
3. Use them at various place places in their code to enforce Authorization.

Moreover, we require authorization enforcement all over cdap code base which requires the above to be done at a lot of places.

Also, in coming releases, we also want to support enabling/disabling operation operations depending on user privileges. For example, if an a user does not have "ADMIN" privilege on a program, the "START" button in the UI should be disabled for that user. Similar A similar feature is needed in the CLI too, where the auto complete should only show options for which the user have has privileges. Although , this requirement is out of score scope for 4.0 we will would like to design the Authorization Enforcement mechanism to support this in near the future.

The above This calls for some redesign in of the authorization enforcement to simplify and standardize authorization enforcement across CDAP code base and also provide a good base for further UI enhancementenhancements. The design in the following section present presents an approach to solve the above issues.

Goals

1. Simplify and standardize the process of authorization enforcement in CDAP.
2. Support Authorization all across CDAP (Logs, Metrics, Stream Views, Metadata, Preferences, Kafka, Explore)

3. Enable/Disable operations on the fly for the logged in user (Out of scope for 4.0)

User Stories 

1. As a CDAP developer, I would like to easily enforce authorization for a code block with minimal code in a standard way.
2. As a CDAP security admin, I would like to enforce authorization on all entities in CDAP.
3. As a CDAP security admin, I would like users to access/view only the operation for which they have appropriate privileges. (Out of scope of 4.0)

Scenarios

• Scenario #1

  Bhooshan is a ninja CDAP developer who is working on a cool new feature, adding lots of new publicly exposed APIs and he wants to support authorization enforcement for his new feature. He looks around in CDAP code base and finds out that he will need to get some classes through the injector and put conditional checks at various places in his new code. He feels overwhelmed by all these new changes and will like to have a simplistic and standard way to add authorization.
  
  

• Scenario #2

  Derek is an IT Operations Extraordinaire at a corporation that uses CDAP. He just upgraded CDAP to use the new authorization features. He granted all the appropriate privileges to different users of CDAP in his corporation and left for the day. When he comes back to the office next day he sees a new ticket assigned to him created by Mr. Little Fingers who is angry because Mr. Tim can see all the logs and metrics which his CDAP programs are generating. Derek is furious and after some digging, he identifies that not all entities in CDAP support authorization and he would like to enforce authorization on all entities in CDAP. 
  
  

• Scenario #3 (Out of scope for 4.0)

  Derek (IT Operations) has been appreciating the authorization feature of CDAP as it has restricted unintentional usage of CDAP entities among the users in his corporation. Although he does not like that users can see the UI button to run a CDAP program even though the user does not have privileges for it. This creates unnecessary confusion among users. He thinks that it will be helpful if a user can only see operations for which he has privileges. 

Design

In our current code base, we do authorization enforcement where the entity id on which enforcement is being done is one of the following: 

StreamQueueReader.dequeue

ArtifactRepository.deleteArtifactProperties

ArtifactRepository.addArtifact

Approach

We To represent all the above possibilities we propose the following authorization annotation to standarize the process of authorization enforcement. This will also allow us to enable/disable actions in UI and CLI on the fly since we will know required privilege for an action in advance.

/**
 * Annotation

@AuthEnforce(

     entity("pid": EntityId)

     enforceOn(ApplicationId.class)

     privileges({ADMIN, EXECUTE})

)

@AuthEnforce(

     entity(String[] = {"ns":, "appid"})

     enforceOn(ApplicationId.class)

     privileges({ADMIN, EXECUTE})

)  for a method that needs Authorization
 * <p>
 * {@link AuthEnforce#entity()}: Specifies the entity on which authorization will be enforced.
 * It can either be a variable name of the EntityId or an array of Strings from which the entity id can be constructed.
 * This variable will be first looked up in the method parameter and if not found it will be looked up in the class
 * member variable.
 * <p>
 * {@link AuthEnforce#enforceOn()}: Class name of one of the CDAP entities on which enforcement will be done. If you
 * want to enforce on the parent of the entity specify that EntityId class here
 * <p>
 * {@link AuthEnforce#privileges()}: Array of Action to be checked during enforcement
 */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface AuthEnforce {

  // Specifies the entity on which authorization will be enforced. It can either be a variable name of the EntityId or
  // an array of Strings from which the entity id can be constructed. These variable names will be first looked up in the 
  // method parameter and if not found it will be looked up in the class member variable.
  String[] entity();

  // Class name of one of the CDAP entities on which enforcement will be done. If you want to enforce on the parent
  // of the entity specify that EntityId class here
  Class enforceOn();

  // Array of Action to be checked during enforcement
  Action[] privileges();
}

entity: Specifies the entity on which authorization will be enforced. It can either be an a variable name of the EntityId or an array of Strings from which the entity id can be constructed.  This variable will be first looked up in the method parameter and if not found it will be looked up in the class member variable.

enforeOn: Specifies the entityId on which authorization enforcement will be done. It is possible to pass ProgramId in "entity" and enforce authorization on NamespaceId by passing NamespaceId.class here. This allows developers to enforce on parents parent of an entity. In the case of "entity" being being an array of String Strings, enforceOn specifies what the entityId which will be created.

privileges: The privileges to check for during authorization enforcement on this entity. This is an array of Actions to allow enforcing multiple checks.

API changes

New Programmatic APIs

New Java APIs introduced (both user facing and internal)

Deprecated Programmatic APIs

New REST APIs

Deprecated REST API

CLI Impact or Changes

• Impact #1
• Impact #2
• Impact #3

UI Impact or Changes

• Impact #1
• Impact #2
• Impact #3

Security Impact 

What's the impact on Authorization and how does the design take care of this aspect

Impact on Infrastructure Outages 

System behavior (if applicable - document impact on downstream [ YARN, HBase etc ] component failures) and how does the design take care of these aspect.

Advantage of Authorization Annotations:

• Authorization annotation will remove the need of adding AuthorizationEnforcer and AuthenticationContext all over the CDAP code base to do authorization.
• Although, this is not an immediate goal and hence its not documented in the design but later we can also remove the need of injecting Authorization and Authentication module for authorization. This will be possible by making the implementation of these two classes singleton and using them in during the class rewrite.
• Standardize the way we do authorization enforcement across CDAP
• Drives us towards Aspect Oriented Programming. Since, authorization annotation will allow us to add additional behavior to existing code without modifying the code. This will also make our code base more modular.
• Allow us to enable/disable actions in UI/CLI for a logged in user on the fly.

Any method that needs authorization enforcement can be annotated with the above annotation. Below are some examples:

@AuthEnforce(entity = "artifactId", enforceOn = ArtifactId.class, privileges = {Action.ADMIN})
public void writeArtifactProperties(Id.Artifact artifactId, final Map<String, String> properties) throws Exception {
  artifactStore.updateArtifactProperties(artifactId, new Function<Map<String, String>, Map<String, String>>() {
	...
  });
}

@GET
@Path("/{stream}/events")
@AuthEnforce(entity = {"namespaceId", "stream"}, enforceOn = StreamId.class, privileges = {Action.READ})
public void fetch(HttpRequest request, final HttpResponder responder,
                  @PathParam("namespace-id") String namespaceId,
                  @PathParam("stream") String stream,
                  @QueryParam("start") @DefaultValue("0") String start,
                  @QueryParam("end") @DefaultValue("9223372036854775807") String end,
                  @QueryParam("limit") @DefaultValue("2147483647") final int limitEvents) throws Exception {

Image Added

The MainClassLoader which extends an InterceptableClassLoader and is responsible for loading CDAP Master can intercept all the CDAP classes (we will look for package name to start with co.cask.cdap) which need class rewrite by looking for the presence of AuthEnforce annotation. During the class rewrite phase all methods which have AuthEnforce annotation will be rewritten where the first line of the method will be authorization enforcement something like below:

public void writeArtifactProperties(Id.Artifact artifactId, final Map<String, String> properties) throws Exception {
  authorizationEnforcer.enforce(artifactId.toEntityId(), authenticationContext.getPrincipal(), Action.ADMIN);
  artifactStore.updateArtifactProperties(artifactId, new Function<Map<String, String>, Map<String, String>>() {
	...
  });
}

The authorization enforcement line which will be generated during class rewrite will always be the the first line of the method with AuthEnforce annotation. There are some places in our code base where we do enforcement in middle of the function and we will need to refactor it for standardization. 

Note: AuthorizationEnforcer.enforce() will not be deprecated even after introducing AuthEnforce annotation as after class rewrite we will generate AuthorizationEnforcer.enforce() and also program container will use them rather than the annotation.

API changes

New Programmatic APIs

AuthEnforce annotation as mentioned above.

Deprecated Programmatic APIs

None

New REST APIs

None

Deprecated REST API

None

CLI Impact or Changes

None

UI Impact or Changes

None

Security Impact 

We will replace authorization enforcement all across CDAP with the new AuthEnforce annotation. We expect to write a good Authorization Integration test suite which we can depend on to make sure nothing getting affected by these changes. 

Impact on Infrastructure Outages 

None

Test Scenarios

Releases

Release 4.0.0

• Goal 1: Simplify and standardize the process of authorization enforcement in CDAP.
• Goal 2: Support Authorization all across CDAP (Logs, Metrics, Stream Views, Metadata, Preferences, Kafka, Explore)

Release 4.1.0

• Goal 3: As a CDAP security admin, I will like users to access/view only the operation for which they have appropriate privileges. (Out of scope of 4.0)

Related Work

• Jira Legacy
  server Cask Community Issue Tracker serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-7454

  
  

• Jira Legacy
  server Cask Community Issue Tracker serverId 45b48dee-c8d6-34f0-9990-e6367dc2fe4b key CDAP-7455

• Standardize authorization enforcement across CDAP so we can use AuthEnforce annotation. For example, there are some handlers which do Authorization enforcement. Ideally, we will want them to happen in the admins.  

Future work

Enabling/Disabling action in UI and CLI

The future work involves taking advantage of authorization annotation to enable/disable actions in UI and CLI for an a user on the fly. Below we discuss an initial design to showcase that this is possible with authorization annotation.

We will add an a unique name called actionId to every annotation. This will serve as a map between UI and the backend so that for every action in UI we will know what privileges needs to be enforced and on which entity.

@AuthEnforce(

     entity("pid": EntityId)

     enforceOn(ApplicationId.class)

     privileges({ADMIN, EXECUTE})
 
     actionId("uniqueActionName")
)

PrivilegeInspector: When CDAP  Master starts privilege inspector can inspect all the authorization annotation to create a map of  privileges needed for a given action.

Querying Privileges: Once the action to required privilege map is build by the PrivilegeInspector this can be served to the  UI. We will introduce new endpoints which which will map every page in UI (like ns page, app page etc) which can be used to construct entityId on which authorization needs to be enforced and the UI will also pass a set of unique action names for which privileges needs to be checked.

For example, let's say a user is on a program page  (/ns/appid/program/checkactions)

/programs end point can be hit with  /program {ns, appid} {set of actions: start, stop}

Now the handler can use the map created by the privilegeInspector to know what privileges are needed for the requested action.

Authorization for listing entities through filter

Another enforcement type supported by  AuthorizationEnforcer is filtering. This is used in most of our listing APIs to list only the entities on which user has privilege. Supporting annotation for AuthorizationEnforcer.createFilter will be a good idea to take another step towards standardization in authorization enforcement across CDAP. But this is something which we will like to do in future due to time constraints of 4.0 and also because this is not needed to enable/disable action in UI/CLI.