Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
[
  /** Metadata change **/
  {
    "time": 1456956659469,
    "entityId": {
      "namespace": "ns1",
      "dataset": "ds1",
      "entity": "DATASET"
    },
    "user": "cdap",
    "type": "METADATA_CHANGE",
    "change": {
      "additions": [
        {
          "scope": "USER",
          "properties": {
            "key1": "value1"
          },
          "tags": [
            "tag1"
          ]
        }
      ],
      "deletions": [
        {
          "scope": "SYSTEM",
          "properties": {},
          "tags": [
            "tag2"
          ]
        }
      ]
    }
  },
  
  /** Dataset admin operation **/ 
  {
    "time": 1456956659470,
    "entityId": {
      "namespace": "ns1",
      "dataset": "ds1",
      "entity": "DATASET"
    },
    "user": "cdap",
    "type": "CREATE"
  },
  
  /** Dataset access **/
  {
    "time": 1456956659471,
    "entityId": {
      "namespace": "ns1",
      "dataset": "ds1",
      "entity": "DATASET"
    },
    "user": "cdap",
    "type": "ACCESS",
    "access": {
      "type": "READ",
      "entityId": {
        "namespace": "ns1",
        "application": "app1",
        "type": "Flow",
        "program": "flow1",
        "entity": "PROGRAM"
      }
    }
  }
]

 

ImplementationImplementation

The Audit log information will be published to CDAP Kafka server when `audit.publish.enabled` config parameter is set to true.

  • Dataset admin operations can be published by DatasetOpExecutor service.
  • Stream admin operations can be published by StreamAdmin class
  • Dataset and stream access information can be published by LineageWirterDatasetFramework (can be renamed to AuditingDatasetFramework).
  • Metadata changes can be published by DefaultMetadataStore class.

 

  • Note: Publishing of metadata updates to Kafka introduced by CDAP-3518 for Navigator integration is of a different format, and it will still continue to be published when enabled. It would be good to see if we can consolidate audit publishing and metadata publishing in future. There is not enough time to do that exercise for 3.4.