Skip to main content
Version: 24.3.x

Audit Logging Enterprise

For organizations subject to compliance and regulation where auditing is regularly required, Dremio offers full audit logging, wherein all user activities performed within Dremio are tracked and traceable via the audit.json file. Each time a user performs an action within Dremio, such as logging in or creating a view, the audit log captures the user's ID and username, object(s) affected, action performed, event type, SQL statements used, and more.

Audit logging is enabled by default and is available only to users with administrative rights at the System level.

Audit Log Location

  • For non-AWSE organizations, the log-file location may be configured via the dremio.log.path property in the dremio-env file. You can specify their location, size, and rotation schedule.
  • For organizations using AWSE, the log file is located within the same folder as queries.json.

Events and Actions Tracked

Dremio supports audit logging for the following objects (event types) and actions:

Event TypeActions
CONNECTIONLOGIN
ENGINE_CONFIGCREATE, UPDATE, DELETE
FOLDERCREATE, UPDATE, DELETE
PERSONAL_ACCESS_TOKENCREATE, DELETE
PHYSICAL_DATASETCREATE, UPDATE, DELETE
PRIVILEGEUPDATE, DELETE
QUEUECREATE, UPDATE, DELETE
REFLECTIONCREATE, UPDATE, DELETE
REPLICACREATE_STARTED, CREATE_COMPLETED, DELETE_STARTED, DELETE_COMPLETED
SOURCECREATE, UPDATE, DELETE
SPACECREATE, UPDATE, DELETE
SUPPORT_SETTINGRESET, SET
UDFCREATE, UPDATE, DELETE
USER_ACCOUNTCREATE, UPDATE, DELETE
VIRTUAL_DATASETCREATE, RENAME, UPDATE, DELETE

Audit Log Format

Audit logs include the following information:

  • timestamp - The time when the event was recorded.
  • userId - The ID value associated with the user's account.
  • userName - The username associated with the user account (which is typically used to log in).
  • status - The status of the action, typically used to indicate whether the event was approved or allowed.
  • eventType - The object or scope of the interaction that occurred.
  • action - The actual activity performed within the specified scope. This varies based on the eventType, but most often would be CREATE, DELETE, and UPDATE.
  • details - The data altered or created. This varies based on the eventType.

The following are examples of audit records Dremio would capture, and the information an audit entry would include depending on the event type.

Example: Authentication

The user dremio logged in on the Dremio application, so the following information would be added to the audit log:

Authentication log
{
"timestamp": "2021-11-23 16:30:53,400",
"userContext": {
"userId": "1",
"userName": "$dremio$"
},
"status": "OK",
"eventType": "AUTHENTICATION",
"action": "LOGIN",
"details": {
"userName": "dremio",
"userId": "",
"source": "FLIGHT"
}
}

Example: Reflection

The user dremio created a reflection called Raw Reflection (1) via Dremio's UI, so the following information would be added to the audit log:

Reflection log
{
"timestamp": "2021-11-22 10:06:38,432",
"userContext": {
"userId": "6ab04602-410b-4031-87ae-2d3d5f7dc",
"userName": "dremio"
},
"status": "OK",
"eventType": "REFLECTION",
"action": "CREATE",
"details": {
"reflectionId": "a5251b05-4873-4a9d-a008-303eeeeed",
"name": "Raw Reflection (1)",
"dataset": "7e3d4a8a-b92d-41ab-96dc-6a76a6248",
"type": "RAW",
"sortColumns": [
{
"name": "fare_amount"
}
],
"partitionColumns": [
{
"name": "passenger_count"
}
],
"distributionColumns": [],
"dimensions": [],
"measures": [],
"displayColumns": [
{
"name": "passenger_count"
},
{
"name": "pickup_datetime"
},
{
"name": "trip_distance_mi"
},
{
"name": "fare_amount"
},
{
"name": "tip_amount"
},
{
"name": "total_amount"
}
],
"partitiondistributionstrategy": "CONSOLIDATED",
"arrowCachingEnabled": false,
"targetDataset": ""
}
}

Example: View

The user dremio issued a SQL command to CREATE a view from Dremio's SQL Editor, so the following information would be added to the audit log:

View (virtual dataset) log
{
"timestamp": "2021-11-17 14:31:43,594",
"userContext": {
"userId": "4a3ea2fa-a3f6-4adb-8852-041a28cac",
"userName": "dremio"
},
"status": "OK",
"eventType": "VIRTUAL_DATASET",
"action": "CREATE",
"details": {
"id": "da08848d-d80d-4414-aaf4-40ce866ea",
"name": "a_employees",
"tag": "gwN/p7E5E1Q=",
"path": "myView.a_employees",
"sql": "SELECT \"firstname\", \"zipcode\"\nFROM \"local-msql\".\"dremio_db1\".\"Employees\"\nWHERE \"zipcode\" LIKE 'a%'",
"sqlContext": "myView",
"fields": [
{
"name": "firstname",
"dataType": "TEXT"
},
{
"name": "zipcode",
"dataType": "TEXT"
}
],
"oldName": "",
"oldPath": ""
}
}