Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Tracker Backend Design

Storing Audit Log

  • Goal: Read AuditLog messages from Kafka and write messages to Table dataset.
    • Reusing the MetadataConsumer flowlet from the Navigator App to handle reading messages from Kafka
    • New Flowlet (AuditLogPublisher) for writing Kafka messages to Dataset
      • Dataset is a Table class
      • Dataset key format: <namespace>-<type>-<name>-<messageTimeLong>
      • Dataset Columns: 
        • timestamp - Long - timestamp of the message generated
        • entityId - EntityId - the entity id that the message refers to. Only entity types with a namespace are supported.
        • user - String - the name of the user that generated the message. If the user blank, a default value of "unknown" is inserted.
        • actionType - String - The type of action that was taken. For more details, see: Audit information publishing
        • entityKind - String - The EntityType from the id, lowercase
        • entityName - String - The name of the Entity
        • metadata - AuditPayload - The change that was made, either a metadata change or an access. For all other types, the payload is empty

Reading Audit Log

  • Goal: Expose the AuditLog dataset as a REST API for consumption by the UI
    • REST API Design

      HTTP Request Type

      Endpoint:

      Request Params

      Response Status

      Response Body

      GET/namespaces/{namespace-id}/apps/Tracker/services/AuditLog/methods/auditlogtype - REQUIRED name - REQUIRED startTime - OPTIONAL - default is 0 endTime - OPTIONAL - default is Now() offset - OPTIONAL - default is 0 pageSize - OPTIONAL - default is 10

      200 returns the audit log entries requested

      500 error while searching

      {
      	totalResults: 1,
      	results: [{
      		time: 1457467029557,
      		entityId: {
      			namespace: "default",
      			application: "testCubeAdapter",
      			type: "Workflow",
      			program: "ETLWorkflow",
      			entity: "PROGRAM"
      		},
      		user: "unknown",
      		type: "METADATA_CHANGE",
      		payload: {
      			previous: {
      				SYSTEM: {
      					properties: { },
      					tags: [ ]
      				}
      			},
      			additions: {
      				SYSTEM: {
      					properties: { },
      					tags: [
      						"ETLMapReduce",
      						"Batch",
      						"Workflow",
      						"ETLWorkflow"
      					]
      				}
      			},
      			deletions: {
      				SYSTEM: {
      					properties: { },
      					tags: [ ]
      				}
      			}
      		}
      	}]
      	offset: 0
      }

 

 

  • No labels