Page Comparison

...

Purpose

API

Body

Response

Routable

Comments

Approved?

Annotate business metadata for datasets

POST /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/properties

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

200: Successful

404: Dataset not found in specified namespace

Yes

New keys are added.
Existing keys are updated.

Annotate business metadata for apps

POST /v3/namespaces/{namespace-id}/apps/{app-id}/metadata/properties

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

200: Successful

404: App not found in specified namespace

Yes

New keys are added.
Existing keys are updated.

Annotate business metadata for programs

POST /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata/properties

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

200: Successful

404: Program not found in specified namespace

Yes

New keys are added.
Existing keys are updated.

Annotate business metadata for streams

POST /v3/namespaces/{namespace-id}/streams/{stream-id}/metadata/properties

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

200: Successful

404: Stream not found in specified namespace

Yes

New keys are added.
Existing keys are updated.

~~Record run-level dataset access metadata~~

Note: will be part of client call.

POST /v3/metadata/history

Code Block

{
  "programRun": {                                 "namespace" : "ns1",
      "app": "app1",
      "programType": "MAPREDUCE",
      "name": "pgm1",
      "runId": "r1"
  },
  "data": {
      "namespace": "ns1",
      "type": [STREAM|DATASET],
      "name": "d1"
  },  
  "accessType": "in",
  "metadata": {
    "user": {
      "key1": "value1",
      ...
    },
    "system": {
       "k1": "v1",
        ...
    }
  }
}

200: Successful

404: App/dataset not found in specified namespace (proper error message)

No

Update behavior TBD
How do you fit streams in here

Retrieve business metadata for datasets

GET /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/properties

N/A

200: Successful

404: Dataset not found in specified namespace

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

Yes

Retrieve business metadata for apps

GET /v3/namespaces/{namespace-id}/apps/{app-id}/metadata/properties

N/A

200: Successful

404: App not found in specified namespace

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

Yes

Retrieve business metadata for programs

GET /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata/properties

N/A

200: Successful

404: Program not found in specified namespace

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

Yes

Retrieve business metadata for streams

GET /v3/namespaces/{namespace-id}/stream/{stream-id}/metadata/properties

N/A

200: Successful

404: Stream not found in specified namespace

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

Yes

Retrieve system metadata for datasets

GET /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/properties/system

N/A

200: Successful

404: Dataset not found in specified namespace

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

Yes

Not supported in 3.2
Why is scope not a query param?.

Retrieve system metadata for apps

GET /v3/namespaces/{namespace-id}/apps/{app-id}/metadata/properties/system

N/A

200: Successful

404: App not found in specified namespace

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

Yes

Not supported in 3.2

Retrieve system metadata for programs

GET /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata/properties/system

N/A

200: Successful

404: Program not found in specified namespace

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

Yes

Not supported in 3.2

Retrieve system metadata for streams

GET /v3/namespaces/{namespace-id}/streams/{stream-id}/metadata/properties/system

N/A

200: Successful

404: Stream not found in specified namespace

Code Block
{ "key1" : "value1", "key2" : "value2", //... }

Yes

Not supported in 3.2

Delete all business metadata for datasets

DELETE  /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/properties

N/A

200: Successful

404: Dataset not found in specified namespace

Yes

Delete selected key from business metadata for datasets

DELETE  /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/properties/{key}

N/A

200: Successful

404: Dataset not found in specified namespace

Yes

Delete all business metadata for apps

DELETE  /v3/namespaces/{namespace-id}/apps/{app-id}/metadata/properties

200: Successful

404: App not found in specified namespace

Yes

Delete selected key from business metadata for apps

DELETE  /v3/namespaces/{namespace-id}/apps/{app-id}/metadata/properties/{key}

200: Successful

404: App not found in specified namespace

Yes

Delete all business metadata for programs

DELETE  /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata/properties

200: Successful

404: Program not found in specified namespace

Yes

Delete all business metadata for programs

DELETE  /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata/properties/{key}

200: Successful

404: Program not found in specified namespace

Yes

Delete all business metadata for streams

DELETE  /v3/namespaces/{namespace-id}/streams/{stream-id}/metadata/properties

200: Successful

404: Stream not found in specified namespace

Yes

Delete selected key from business metadata for streams

DELETE  /v3/namespaces/{namespace-id}/streams/{stream-id}/metadata/properties/{key}

200: Successful

404: Stream not found in specified namespace

Yes

Search Datasets containing business metadata

GET /v3/namespaces/{namespace-id}/metadata/search?query=term&target=dataset

N/A

200: Successful

Code Block
["dataset1", "dataset2"]

Yes

Only prefix search supported in 3.2.
Supported formats:
- Value Prefix
- Key:Value Prefix

Search Apps containing business metadata

GET /v3/namespaces/{namespace-id}/metadata/search?query=term&target=app

N/A

200: Successful

Code Block
["app1", "app2"]

Yes

Only prefix search supported in 3.2.
Supported formats:
- Value Prefix
- Key:Value Prefix

Search Programs containing business metadata

GET /v3/namespaces/{namespace-id}/metadata/search?query=term&target=program

N/A

200: Successful

Code Block
["app1", "app2"]

Yes

Only prefix search supported in 3.2.
Supported formats:
- Value Prefix
- Key:Value Prefix

Search Streams containing business metadata

GET /v3/namespaces/{namespace-id}/metadata/search?query=term&target=stream

N/A

200: Successful

Code Block
["stream1", "stream2"]

Yes

Only prefix search supported in 3.2.
Supported formats:
- Value Prefix
- Key:Value Prefix

View Dataset Lineage

GET /v3/namespaces/{namespace-id}/datasets/{dataset-id}/lineage?start=<start-ts>&end=<end-ts>&maxLevels=<max-levels>

N/A

200: Successful

Response TBD, but will contain a DAG representation

Yes

View Stream Lineage

GET /v3/namespaces/{namespace-id}/streams/{stream-id}/lineage?start=<start-ts>&end=<end-ts>&maxLevels=<max-levels>

N/A

200: Successful

Response TBD, but will contain a DAG representation

Yes

View Run Id Accesses

POSTGET /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/runs/{run-id}/metadata/accesses

N/A

200: Successful

Response Body TBD

Yes

TODO: Figure out a better name
May not be part of 3.2

View Dataset Lineage after specified stage

POST /v3/namespaces/{namespace-id}/datasets/{dataset-id}/lineage/next

TODO

Note: Query params of /lineage will become part of POST body.

Yes

Will not be supported in 3.2

View Stream Lineage after specified stage

POST /v3/namespaces/{namespace-id}/streams/{stream-id}/lineage/next

TODO

Note: Query params of /lineage will become part of POST body.

Yes

Will not be supported in 3.2

Add tags to a dataset

POST /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/tags

Code Block
["tag1", "tag2"]

200: Successful

404: Dataset not found in specified namespace

Yes

Add tags to an app

POST /v3/namespaces/{namespace-id}/apps/{app-id}/metadata/tags

Code Block
["tag1", "tag2"]

200: Successful

404: App not found in specified namespace

Yes

Add tags to a program

POST /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata/tags

Code Block
["tag1", "tag2"]

200: Successful

404: Program not found in specified namespace

Yes

Add tags to a stream

POST /v3/namespaces/{namespace-id}/streams/{stream-id}/metadata/tags

Code Block
["tag1", "tag2"]

200: Successful

404: Stream not found in specified namespace

Yes

Retrieve dataset tags

GET /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/tags

N/A

Code Block
["tag1", "tag2"]

Yes

Retrieve app tags

GET /v3/namespaces/{namespace-id}/apps/{app-id}/metadata/tags

N/A

Code Block
["tag1", "tag2"]

Yes

Retrieve program tags

GET /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata/tags

N/A

Code Block
["tag1", "tag2"]

Yes

Retrieve stream tags

GET /v3/namespaces/{namespace-id}/streams/{stream-id}/metadata/tags

N/A

Code Block
["tag1", "tag2"]

Yes

Remove all dataset tags

DELETE /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/tags

N/A

200: Successful

404: Dataset not found in specified namespace

Yes

Remove specified dataset tag

DELETE /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/tags/{tag}

N/A

200: Successful

404: Dataset not found in specified namespace

Yes

Remove all app tags

DELETE /v3/namespaces/{namespace-id}/apps/{app-id}/metadata/tags

N/A

200: Successful

404: App not found in specified namespace

Yes

Remove specified app tag

DELETE /v3/namespaces/{namespace-id}/apps/{app-id}/metadata/tags/{tag}

N/A

200: Successful

404: App not found in specified namespace

Yes

Remove all program tags

DELETE /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata/tags

N/A

200: Successful

404: Program not found in specified namespace

Yes

Remove specified program tag

DELETE /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata/tags/{tag}

N/A

200: Successful

404: Program not found in specified namespace

Yes

Remove all stream tags

DELETE /v3/namespaces/{namespace-id}/streams/{stream-id}/metadata/tags

200: Successful

404: Stream not found in specified namespace

Yes

Remove specified stream tag

DELETE /v3/namespaces/{namespace-id}/streams/{stream-id}/metadata/tags/{tag}

200: Successful

404: Stream not found in specified namespace

Yes

Remove all business metadata for a dataset

DELETE /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata

200: Successful

404: Dataset not found in specified namespace

Yes

Removes all properties and tags from a dataset. Will not happen in 3.2

Remove all business metadata for an app

DELETE /v3/namespaces/{namespace-id}/apps/{app-id}/metadata

200: Successful

404: App not found in specified namespace

Yes

Removes all properties and tags from an app. Will not happen in 3.2

Remove all business metadata for a program

DELETE /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata

200: Successful

404: Program not found in specified namespace

Yes

Removes all properties and tags from a program. Will not happen in 3.2

Remove all business metadata for a dataset

DELETE /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata

200: Successful

404: Dataset not found in specified namespace

Yes

Removes all properties and tags from a dataset. Will not happen in 3.2

Get all business metadata for a dataset

GET /v3/namespaces/{namespace-id}/datasets/{dataset-id}/metadata

200: Successful

404: Dataset not found in specified namespace

Yes

Retrieves all properties and tags for a dataset. Will not happen in 3.2

Get all business metadata for an app

GET /v3/namespaces/{namespace-id}/apps/{app-id}/metadata

200: Successful

404: App not found in specified namespace

Yes

Retrieves all properties and tags for an app. Will not happen in 3.2

Get all business metadata for a program

GET /v3/namespaces/{namespace-id}/apps/{app-id}/{program-type}/{program-id}/metadata

200: Successful

404: Program not found in specified namespace

Yes

Retrieves all properties and tags for a program. Will not happen in 3.2

Get all business metadata for a stream

GET /v3/namespaces/{namespace-id}/streams/{stream-id}/metadata

200: Successful

404: Stream not found in specified namespace

Yes

Retrieves all properties and tags for a stream. Will not happen in 3.2

...

Versions Compared

Old Version 95

New Version 96

Key