...

Entity	Operation	Required Privileges	Resultant Privileges
Namespace	create	ADMIN (Instance)	ADMIN (Namespace)
	update	ADMIN (Namespace)
	list/get	READ (Instance)
	deleteget	READ (Namespace)
set preference	delete	ADMIN (Namespace)
	get set preference		Application	deployWRITE (Namespace)
	get preference	READ (Namespace)
	update	search	READ (Namespace)
deleteApplication			set preferencedeploy	WRITE (Namespace)	ADMIN (Application)
		get preferenceget	READ (Application)
	add metadata	list	READ (Namespace)	get metadata
	Programs	start/stop/debugupdate	ADMIN (Application)
set instances	delete	ADMIN (Application)
	set runtime argspreference	WRITE (Application)
	get instance preference	READ (Application)	get runtime args
		set preferenceadd metadata	ADMIN (Application)
	get preference metadata	READ (Application)
	get status
	get historyPrograms	start/stop/debug	EXECUTE (Program)
		add metadataset instances	ADMIN (Program)
	get metadata	list	READ (Namespace)	emit logs
		view logsset runtime args	EXECUTE (Program)
	emit metrics	get runtime args	READ (Program)		view metrics
	Streams	createget instances	READ (Program)
	update properties	set preference	ADMIN (Program)
delete			truncateget preference	READ (Program)
	enqueue asyncEnqueue batch	get status	READ (Program)
	get history	READ (Program)
read events			set preferencesadd metadata	ADMIN (Program)
	get preferences metadata	READ (Program)
add metadata	emit logs	WRITE (Program)
get metadata	view logs	READ (Program)
	emit metrics	WRITE (Program)
	view metrics	READ (Program)
Streams	Datasets	list	create	WRITE (Namespace)	ADMIN (Stream)
	get	update properties	ADMIN (Stream)
create			updatedelete	ADMIN (Stream)
	drop			executeAdmin (exists/truncate/upgradetruncate	ADMIN (Stream)
add metadata	enqueue asyncEnqueue		get metadatabatch	WRITE (Stream)
	emit metrics	get	READ (Stream)	view metrics
	Stream Views	createlist	READ (Namespace)
delete	read events	READ (Stream)
list	set preferences	ADMIN (Stream)
	get preferences	READ (Stream)
	add metadata	ADMIN (Stream)
	get metadata	READ (Stream)
		view lineage	READ (Stream)
		emit metrics	WRITE (Stream)

Namespace:

NamespaceHttpHandler
getAllNamespaces
getNamespace
updateNamespaceProperties
create
delete
deleteDatasets

App:

AppLifecycleHttpHandler
deploy
deploy
getAllApps
getAppInfo
deleteApp
deleteAllApps
updateApp

Stream:

StreamHandler
init
destroy
getInfo
create
enqueue
asyncEnqueue
batch
truncate
delete
setConfig

Metadata:

getAppMetadata
getProgramMetadata
getArtifactMetadata
getDatasetMetadata
getStreamMetadata
getViewMetadata
getAppProperties
getArtifactProperties
getProgramProperties
getDatasetProperties
getStreamProperties
getViewProperties
addAppProperties
addArtifactProperties
addProgramProperties
addDatasetProperties
addStreamProperties
addViewProperties
removeAppMetadata
removeArtifactMetadata
removeProgramMetadata
removeDatasetMetadata
removeStreamMetadata
removeViewMetadata
removeAppProperties
removeAppProperty
removeArtifactProperties
removeArtifactProperty
removeProgramProperties
removeProgramProperty
removeDatasetProperties
removeDatasetProperty
removeStreamProperties
removeViewProperties
removeStreamProperty
removeViewProperty
addAppTags
addArtifactTags
addProgramTags
addDatasetTags
addStreamTags
addViewTags
getAppTags
getArtifactTags
getProgramTags
getDatasetTags
getStreamTags
getViewTags
removeAppTags
removeAppTag
removeArtifactTags
removeArtifactTag
removeProgramTags
removeProgramTag
removeDatasetTags
removeDatasetTag
removeStreamTags
removeViewTags
removeStreamTag
removeViewTag
searchMetadata

Lineage:

datasetLineage
streamLineage
getAccessesForRun

Log:

...

view metrics	READ (Stream)
Datasets	list	READ (Namespace)
	get	READ (Dataset)
	create	WRITE (Namespace)	ADMIN (Dataset)
	update	ADMIN (Dataset)
	drop	ADMIN (Dataset)
	executeAdmin (exists/truncate/upgrade)	ADMIN (Dataset)
	add metadata	ADMIN (Dataset)
	get metadata	READ (Dataset)
	view lineage	READ (Dataset)
	emit metrics	WRITE (Dataset)
	view metrics	READ (Dataset)
Stream View	create	WRITE (Namespace) & ADMIN (Stream)	ADMIN (Stream View)
	delete	ADMIN (Stream View)
	list	READ (Namespace) & READ (Stream)
	get	READ (Stream View)
	add metadata	ADMIN (Stream View)
	get metadata	READ (Stream View)

Integration with Apache Sentry

...

How does CDAP get sentry-site.xml? Path provided via cConf?
Distinguishing Read/Write access is perhaps out of scope of 3.4, since we will need changes to Dataset Framework
Can access to all entities be authorized in one go? If so, how?
How does hierarchy work? e.g. write to stream requires READ perms on namespace + write perms on stream
In a secure/kerberos environment, what does it take to communicate with the Sentry Server?
In a secure/kerberos environment, what does it take to communicate with the Sentry Server?
Given that Sentry has a slightly data-engine-based schema, will we need some updates to the policy store to contain CDAP specific tables for storing CDAP Privileges? SENTRY_CDAP_PRIVILEGE and SENTRY_CDAP_PRIVILEGE_MAP tables?
What about instance-level authorization? Would users need to be authorized to a given CDAP instance as well, along with the namespace and entity?
Do we need EXECUTE operation just for Programs entity. Can we say that any user who has READ can run the program ?

Discussion Bhooshan & Rohit 02/17

CDAP Specific

External Auth Service: Sentry

ACL Management

Provide Authorization Hooks in CDAP
1. Intercept all HTTP calls
2. Thrift calls
3. Access to data from programs

Modules to implement
1. Binding
2. Model
3. Policy
4. E2E Tests

Should CDAP do ACL Management
1. CLI
2. HTTP Handlers
3. If we assume ACLs are set in Sentry through Sentry
  what if we switch to Dataset based store.

2. Authorization Checks

Code Block

title	Check

for a given user/group and type of access
	if allowed:
		perform operation
	else:
		throw AuthException

2. Figuring out how to interact with Sentry

- SentryGenericServiceClient
- How to know where Sentry is running?

3. We need an Authorization interface

Discussion with Gokul 02/08

Push down ACLs - No HBase support in Sentry
Custom datasets - how do you recognize read/writes
How do you distinguish between read/write
Sentry Integration - needs follow-ups
Performance (num RPC calls)
Sentry Persistent Storage - PolicyStoreProvider
Interactions with Auth system
Sentry web-app for UI may need customizations in Hue
How does switching between authorization enabled/disabled work

Out-of-scope User Stories (3.5 and beyond)

As a CDAP admin, I should be able to authorize reads/writes to datasets
As a CDAP admin, I should be able to authorize metadata changes to CDAP entities
As a CDAP system, I should be able to push down ACLs to storage providers
As a CDAP admin, I should be able to authorize reads/writes to custom datasets
As a CDAP system, I should be able to judge, document and improve the performance impact of authorization
As a CDAP authorization system, I should be able to interact with an external authentication system
As a CDAP admin, I should be able to use external UIs like Hue for ACL Management
As a CDAP admin, I should be able to see an audit log of all authorization-related changes in CDAP

...

Versions Compared

Old Version 59

New Version 60

Key

Integration with Apache Sentry

Discussion Bhooshan & Rohit 02/17

Discussion with Gokul 02/08

Out-of-scope User Stories (3.5 and beyond)

Page Comparison

Versions Compared

Old Version 59

New Version 60

Key

Integration with Apache Sentry

Discussion Bhooshan & Rohit 02/17

Discussion with Gokul 02/08

Out-of-scope User Stories (3.5 and beyond)