Audit Log
Storing Audit Log
- Goal: Read AuditLog messages from Kafka and write messages to Table dataset.
- Reusing the MetadataConsumer flowlet from the Navigator App to handle reading messages from Kafka
- Beacuse of this, the app requires a Kafka config in order to be installed
{ "config": { "metadataKafkaConfig": { "brokerString": "<host>:<port>", "topic" : "audit" } } }
- Beacuse of this, the app requires a Kafka config in order to be installed
- New Flowlet (AuditLogPublisher) for writing Kafka messages to Dataset
- Dataset is a Table class
- Dataset key format: <namespace>-<type>-<name>-<messageTimeLong>
- Dataset Columns:
- timestamp - Long - timestamp of the message generated
- entityId - EntityId - the entity id that the message refers to. Only entity types with a namespace are supported.
- user - String - the name of the user that generated the message. If the user blank, a default value of "unknown" is inserted.
- actionType - String - The type of action that was taken. For more details, see: Audit information publishing
- entityKind - String - The EntityType from the id, lowercase
- entityName - String - The name of the Entity
- metadata - AuditPayload - The change that was made, either a metadata change or an access. For all other types, the payload is empty
- Reusing the MetadataConsumer flowlet from the Navigator App to handle reading messages from Kafka
Reading Audit Log
- Goal: Expose the AuditLog dataset as a REST API for consumption by the UI
- Fields returned
- totalResults - the total number of results for the query
- offset - The starting offset of the first result
- results - An array of result records with a max length of pageSize
REST API Design
HTTP Request Type
Endpoint:
Request Params
Response Status
Response Body
GET /namespaces/{namespace-id}/apps/Tracker/services/AuditLog/methods/auditlog name is Required Description Default Value type yes The type of the entity to search for, e.g. dataset or stream. Any namespaced entity can be searched for. Possible values: application, artifact, dataset, dataset_module, dataset_type, flowlet, flowlet_queue, notification_feed, program, program_run, schedule, stream, stream_view name yes The name of the entity to search for startTime no The start time to search for. Accepts "now - 1d" syntax 0 endTime no The end time to search for. Accepts "now - 1d" syntax now offset no The offset to start the results at for paging 0 pageSize no The max number of results to return in the results 10 200 returns the audit log entries requested
500 error while searching
{ totalResults: 1, results: [{ time: 1457467029557, entityId: { namespace: "default", application: "testCubeAdapter", type: "Workflow", program: "ETLWorkflow", entity: "PROGRAM" }, user: "unknown", type: "METADATA_CHANGE", payload: { previous: { SYSTEM: { properties: { }, tags: [ ] } }, additions: { SYSTEM: { properties: { }, tags: [ "ETLMapReduce", "Batch", "Workflow", "ETLWorkflow" ] } }, deletions: { SYSTEM: { properties: { }, tags: [ ] } } } }], offset: 0 }
- Fields returned