...
| Id | Description | Comments |
|---|---|---|
| U1 | As a user, I should be able to search Datasets containing the specified fields | List the kinds of queries that will be supported |
| U2 | As a CDAP system, I should be able to annotate CDAP entities with system metadata automatically | System metadata for each entity is listed below |
| U3 | As a user, I should be able to access and update CDAP metadata using the CDAP CLI | |
| U4 | As a developer, I should be able to access and update CDAP metadata using the CDAP Test Framework | |
| U5 | As a user, I should be able to search CDAP entities based on metadata using the CDAP UI | |
| U6 | As a user, I should be able to view the lineage of a CDAP dataset/stream in a specified time window using the CDAP UI |
New Metadata Search:
Metadata Storage and Search to support:
...
- Schema: {EmpName: String, EmpContact: {EmpTel: Integer, EmpAddr: String}} (Note: This is a nested schema)
Supported searches:- FieldName with FieldType: EmpName: String or EmpTel:Integer or EmpAddr: String
- FieldName: EmpName, EmpTel, EmpAddr
Storage:
Main Table: This table stores stores the metadata for the entity. It will be used when an user wants to get metadata of an entity. This table is not for searching.
| Key: Entity with key | Value: Value of Metadata |
|---|---|
| <Entity-Id><CodeName> | Alpha Tango Charlie |
| <Entity-Id><Tags> | {Tag1, Tag22} |
| <Entity-Id><Schema-Id> | {EmpName: String, EmpContact: {EmpTel: Integer, EmpAddr: String}} |
Index Table: This table will be used for searching and it will use IndexedTable.
| Key: Entity with index | Value: Index |
|---|---|
| <Entity-Id><CodeName: Alpha Tango Charlie> | CodeName: Alpha Tango Charlie |
| <Entity-Id><Codename: Alpha> | Codename: Alpha |
| <Entity-Id><Codename: Tango> | Codename: Tango |
| <Entity-Id><Codename: Charlie> | Codename: Charlie |
| <Entity-Id><Alpha Tango Charlie> | Alpha Tango Charlie |
| <Entity-Id><Alpha> | Alpha |
| <Entity-Id><Tango> | Tango |
| <Entity-Id><Charlie> | Charlie |
| <Entity-Id><Tags: Tag1> | Tags: Tag1 |
| <Entity-Id><Tags: Tag22> | Tags: Tag22 |
| <Entity-Id><Tag1> | Tag1 |
| <Entity-Id><Tag22> | Tag22 |
System Metadata
Kinds of system metadata:
...