Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Overview

This page covers the requirements, design and implementation of metadata and data discovery features in 3.3

High Level Requirements

  1. Metadata search
  2. Schema as metadata
  3. System metadata
  4. CLI, Test Framework Support for metadata
  5. UI for Metadata Search
  6. UI for Lineage
  7. UI for Adding/Updating metadata properties/tags
  8. Lineage based on Type of Dataset Access
  9. Monitoring/Logs for Metadata Service

Scope

User Stories

IdDescriptionRequirements FulfilledComments
U1As a user, I should be able have an efficient mechanism to search metadata for all CDAP entities More details after prototype
U2As a user, I should be able to search Datasets containing the specified fields List the kinds of queries that will be supported
U3As a CDAP system, I should be able to annotate CDAP entities with system metadata automatically 

List all the system tags that should be annotated

  • Kind of entity (dataset, app, program, program type, stream)?
  • artifact?

 

U4As a user, I should be able to access and update CDAP metadata using the CDAP CLI  
U5As a developer, I should be able to access and update CDAP metadata using the CDAP Test Framework  
U6As a user, I should be able to search CDAP entities based on metadata using the CDAP UI  
U7As a user, I should be able to view the lineage of a CDAP dataset/stream in a specified time window using the CDAP UI  
    

 

Design

 

 

  • No labels