Data Maintenance

Use the Data Maintenance API to manage and execute automatic maintenance tasks in Open Catalog, including OPTIMIZE and EXPIRE_SNAPSHOTS. Enabled tasks run automatically based on Dremio logic.

TaskInfo Object

{
  "id": "a75918ee-431b-4296-a2b3-298dff53ab12",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": true,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}

TaskInfo Attributes

id String (UUID)

UUID of this task.

---

type String

Type of maintenance to be run by this task.

• OPTIMIZE: Run OPTIMIZE on one or more tables.
• EXPIRE_SNAPSHOTS: Run VACUUM on one or more tables.

Example: OPTIMIZE

---

level String

The scope of the maintenance task, which must be TABLE to perform a maintenance task on one table.

Example: TABLE

---

sourceName String

The name of the Open Catalog.

Example: prod

---

isEnabled Boolean

Whether the task is enabled.

---

config Object

An object that contains the fully qualified name of the target table for the maintenance task.

Example:

{
  "tableId": "folder1.folder2.table1"
}

List All Tasks

Method and URL

GET /v0/projects/{project_id}/maintenance/tasks?filter

Parameters

project_id Path   String (UUID)

---

filter Query   String

A Common Expression Language (CEL) expression. For more information, see the intro to CEL.

Usable variables within the expression are:

• type: Optimization type. This can be OPTIMIZE or EXPIRE_SNAPSHOTS.
• level: Maintenance level, which must be TABLE.
• tableId: Table name preceded by zero or more namespaces (without a source name), with . as the separator.

Example: type=="OPTIMIZE"&&level=="TABLE"

---

maxResults Query   Integer

Maximum number of results to be returned by the server. The server may return fewer resources than this, but not more. Defaults to 10.

Example: maxResults=25

---

pageToken Query   String

Opaque pagination token. This token is returned by the server in a previous response and is used to request the next page of results. For example: 1->AAAAAQ, 2->AAAAAg, 3->AAAAAw, etc.

Example: pageToken=AAAAAQ

Example

Request

curl -X GET "https://api.dremio.cloud/v0/projects/$PROJECT_ID/maintenance/tasks?filter=type=="OPTIMIZE"&&level=="TABLE"&maxResults=10" \
-H "Authorization: Bearer $DREMIO_TOKEN" \
-H 'Content-Type: application/json'

Response

{
  "data": [
    {
      "id": "d510279d-5741-4feb-82b8-d6923c9ac6c8",
      "type": "OPTIMIZE",
      "level": "TABLE",
      "sourceName": "prod",
      "isEnabled": true,
      "config": {
        "tableId": "folder1.folder2.table1"
      }
    },
    {
      "id": "b0fb84d0-b2c9-455d-9013-3c3e417c46f0",
      "type": "OPTIMIZE",
      "level": "TABLE",
      "sourceName": "prod",
      "isEnabled": true,
      "config": {
        "tableId": "folder1.folder2.table2"
      }
    }
  ]
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Create a Maintenance Task

Method and URL

POST /v0/projects/{project_id}/maintenance/tasks

Parameters

project_id Path   String (UUID)

---

taskDefinition Body   Object

Represents a new task.

Example:

{
  "type": "OPTIMIZE",
  "isEnabled": true,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}

TaskDefinition Object Parameters

type String

Type of maintenance to run.

• OPTIMIZE: A table optimization task.
• EXPIRE_SNAPSHOTS: A table vacuum task.

---

isEnabled Boolean

Indicates whether the task is enabled.

---

config Body   Object

An object that contains a fully qualified object name in the indicated catalog as the target for the maintenance task.

Example:

{
  "tableId": "folder1.folder2.table1"
}

Example

Request

curl -X POST "https://api.dremio.cloud/v0/projects/$PROJECT_ID/maintenance/tasks" \
-H "Authorization: Bearer $DREMIO_TOKEN" \
-H 'Content-Type: application/json' \
--data-raw '{
  "type": "OPTIMIZE",
  "isEnabled": true,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}'

Response

{
  "id": "f526e534-f61c-43a5-9d35-51a8b0f75826",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": true,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Retrieve a Task by ID

Method and URL

GET /v0/projects/{project_id}/maintenance/tasks/{taskId}

Parameters

project_id Path   String (UUID)

---

taskId Path   String (UUID)

UUID of the task.

Example

Request

curl -X GET "https://api.dremio.cloud/v0/projects/$PROJECT_ID/maintenance/tasks/$TASK_ID" \
-H "Authorization: Bearer $DREMIO_TOKEN" \
-H 'Content-Type: application/json'

Response

{
  "id": "f526e534-f61c-43a5-9d35-51a8b0f75826",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": true,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Update a Task

Update the specified task, such as disabling it.

Method and URL

PUT /v0/projects/{project_id}/maintenance/tasks/{taskId}

Parameters

project_id Path   String (UUID)

---

taskId Path   String (UUID)

UUID of the task.

---

taskDefinition Body   Object

Represents an updated task specification.

Example:

{
  "type": "OPTIMIZE",
  "isEnabled": false,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}

Example

Request

curl -X PUT "https://api.dremio.cloud/v0/projects/$PROJECT_ID/maintenance/tasks/$TASK_ID" \
-H "Authorization: Bearer $DREMIO_TOKEN" \
-H 'Content-Type: application/json' \
--data-raw '{
  "type": "OPTIMIZE",
  "isEnabled": false,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}'

Response

{
  "id": "f526e534-f61c-43a5-9d35-51a8b0f75826",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": false,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Delete a Task

Method and URL

DELETE /v0/projects/{project_id}/maintenance/tasks/{taskId}

Parameters

project_id Path   String (UUID)

---

taskId Path   String (UUID)

UUID of the task.

Example

Request

curl -X DELETE "https://api.dremio.cloud/v0/projects/$PROJECT_ID/maintenance/tasks/$TASK_ID" \
-H "Authorization: Bearer $DREMIO_TOKEN" \
-H 'Content-Type: application/json'

Response

1

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error