Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Tracker Backend Design

Storing Audit Log

  • Goal: Read AuditLog messages from Kafka and write messages to Table dataset.
    • Reusing the MetadataConsumer flowlet from the Navigator App to handle reading messages from Kafka
      • Beacuse of this, the app requires a Kafka config in order to be installed
        • {
  "config": {
    "metadataKafkaConfig": {
      "brokerString": "<host>:<port>",
      "topic" : "audit"
    }
  }
}
    • New Flowlet (AuditLogPublisher) for writing Kafka messages to Dataset
      • Dataset is a Table class
      • Dataset key format: <namespace>-<type>-<name>-<messageTimeLong>
      • Dataset Columns: 
        • timestamp - Long - timestamp of the message generated
        • entityId - EntityId - the entity id that the message refers to. Only entity types with a namespace are supported.
        • user - String - the name of the user that generated the message. If the user blank, a default value of "unknown" is inserted.
        • actionType - String - The type of action that was taken. For more details, see: Audit information publishing
        • entityKind - String - The EntityType from the id, lowercase
        • entityName - String - The name of the Entity
        • metadata - AuditPayload - The change that was made, either a metadata change or an access. For all other types, the payload is empty

Reading Audit Log

  • Goal: Expose the AuditLog dataset as a REST API for consumption by the UI

    • REST API Design

      HTTP Request Type

      Endpoint:

      Request Params

      Response Status

      Response Body

      GET/namespaces/{namespace-id}/apps/Tracker/services/AuditLog/methods/auditlogtype - REQUIRED, name - REQUIRED, startTime - OPTIONAL - default is 0, endTime - OPTIONAL - default is Now(), offset - OPTIONAL - default is 0, pageSize - OPTIONAL - default is 10

      200 returns the audit log entries requested

      500 error while searching

      {
	totalResults: 1,
	results: [{
		time: 1457467029557,
		entityId: {
			namespace: "default",
			application: "testCubeAdapter",
			type: "Workflow",
			program: "ETLWorkflow",
			entity: "PROGRAM"
		},
		user: "unknown",
		type: "METADATA_CHANGE",
		payload: {
			previous: {
				SYSTEM: {
					properties: { },
					tags: [ ]
				}
			},
			additions: {
				SYSTEM: {
					properties: { },
					tags: [
						"ETLMapReduce",
						"Batch",
						"Workflow",
						"ETLWorkflow"
					]
				}
			},
			deletions: {
				SYSTEM: {
					properties: { },
					tags: [ ]
				}
			}
		}
	}]
	offset: 0
}

 

 

  • No labels