Audit Logging Enterprise
For organizations subject to compliance and regulation where auditing is regularly required, Dremio offers full audit logging, wherein all user activities performed within Dremio are tracked and traceable via the audit.json
file. Each time a user performs an action within Dremio, such as logging in or creating a view, the audit log captures the user's ID and username, objects affected, action performed, event type, SQL statements used, and more.
Audit logging is enabled by default and is available only to users with administrative rights at the System level.
Audit Log Location
- The log-file location may be configured via the
dremio.log.path
property in thedremio-env
file. You can specify their location, size, and rotation schedule.
Events and Actions Tracked
Dremio supports audit logging for the following objects (event types) and actions:
Event Type | Actions |
---|---|
CONNECTION | LOGIN |
ENGINE_CONFIG | CREATE, UPDATE, DELETE, START, STOP, SCALE_UP, SCALE_DOWN |
FOLDER | CREATE, UPDATE, DELETE |
PERSONAL_ACCESS_TOKEN | CREATE, DELETE |
PHYSICAL_DATASET | CREATE, UPDATE, DELETE |
PRIVILEGE | UPDATE, DELETE |
QUEUE | CREATE, UPDATE, DELETE |
REFLECTION | CREATE, UPDATE, DELETE |
REPLICA | CREATE_STARTED, CREATE_COMPLETED, DELETE_STARTED, DELETE_COMPLETED |
SOURCE | CREATE, UPDATE, DELETE |
SPACE | CREATE, UPDATE, DELETE |
SUPPORT_SETTING | RESET, SET |
UDF | CREATE, UPDATE, DELETE |
USER_ACCOUNT | CREATE, UPDATE, DELETE |
VIRTUAL_DATASET | CREATE, RENAME, UPDATE, DELETE |
Audit Log Format
Audit logs include the following information:
timestamp
: The time when the event was recorded.userId
: The ID value associated with the user's account. The following values are placeholders that represent internal system users, which Dremio uses to log events before the user authenticates:1
and678cc92c-01ed-4db3-9a28-d1f871042d9f
.userName
: The username associated with the user account (which is typically used to log in).status
: The status of the action, typically used to indicate whether the event was approved or allowed.eventType
: The object or scope of the interaction that occurred.action
: The actual activity performed within the specified scope. This varies based on theeventType
, but most often would beCREATE
,DELETE
, andUPDATE
.details
: The data altered or created. This varies based on theeventType
.
Audit Log Examples
The following examples show the types of audit records that Dremio captures and the information included in an audit entry for each event type.
- Authentication
- Reflection
- View
User dremio
logged in on the Dremio application. The audit log would have the following information:
{
"timestamp": "2021-11-23 16:30:53,400",
"userContext": {
"userId": "1",
"userName": "$dremio$"
},
"status": "OK",
"eventType": "AUTHENTICATION",
"action": "LOGIN",
"details": {
"userName": "dremio",
"userId": "",
"source": "FLIGHT"
}
}
User dremio
created a reflection called Raw Reflection (1)
in the Dremio console. The audit log would have the following information:
{
"timestamp": "2021-11-22 10:06:38,432",
"userContext": {
"userId": "6ab04602-410b-4031-87ae-2d3d5f7dc",
"userName": "dremio"
},
"status": "OK",
"eventType": "REFLECTION",
"action": "CREATE",
"details": {
"reflectionId": "a5251b05-4873-4a9d-a008-303eeeeed",
"name": "Raw Reflection (1)",
"dataset": "7e3d4a8a-b92d-41ab-96dc-6a76a6248",
"type": "RAW",
"sortColumns": [
{
"name": "fare_amount"
}
],
"partitionColumns": [
{
"name": "passenger_count"
}
],
"distributionColumns": [],
"dimensions": [],
"measures": [],
"displayColumns": [
{
"name": "passenger_count"
},
{
"name": "pickup_datetime"
},
{
"name": "trip_distance_mi"
},
{
"name": "fare_amount"
},
{
"name": "tip_amount"
},
{
"name": "total_amount"
}
],
"partitiondistributionstrategy": "CONSOLIDATED",
"arrowCachingEnabled": false,
"targetDataset": ""
}
}
User dremio
issued a SQL command to CREATE
a view in Dremio's SQL Editor. The audit log would have the following information:
{
"timestamp": "2021-11-17 14:31:43,594",
"userContext": {
"userId": "4a3ea2fa-a3f6-4adb-8852-041a28cac",
"userName": "dremio"
},
"status": "OK",
"eventType": "VIRTUAL_DATASET",
"action": "CREATE",
"details": {
"id": "da08848d-d80d-4414-aaf4-40ce866ea",
"name": "a_employees",
"tag": "gwN/p7E5E1Q=",
"path": "myView.a_employees",
"sql": "SELECT \"firstname\", \"zipcode\"\nFROM \"local-msql\".\"dremio_db1\".\"Employees\"\nWHERE \"zipcode\" LIKE 'a%'",
"sqlContext": "myView",
"fields": [
{
"name": "firstname",
"dataType": "TEXT"
},
{
"name": "zipcode",
"dataType": "TEXT"
}
],
"oldName": "",
"oldPath": ""
}
}