Version: current [26.x]

Data Maintenance

Use the Data Maintenance API to manage and execute automatic maintenance tasks in the Open Catalog, including OPTIMIZE and EXPIRE_SNAPSHOTS. Enabled tasks run automatically based on Dremio logic.

TaskInfo Object

{
  "id": "a75918ee-431b-4296-a2b3-298dff53ab12",
  "type": "OPTIMIZE",
  "level": "CATALOG",
  "sourceName": "prod",
  "isEnabled": true,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}

TaskInfo Attributes

id String (UUID)

Task ID for this task.

Example: a75918ee-431b-4296-a2b3-298dff53ab12

type String

Type of maintenance to be run by this task.

OPTIMIZE: Run OPTIMIZE on one or more tables
EXPIRE_SNAPSHOTS: Run VACUUM EXPIRE SNAPSHOTS on one or more tables

Example: OPTIMIZE

level String

The scope of the maintenance task.

Supported values include:

CATALOG: Perform a maintenance task throughout the catalog
TABLE: Default, perform a maintenance task on one table

Example: CATALOG

sourceName String

The name of the Open Catalog.

Example: prod

isEnabled Boolean

Whether the task is enabled.

Example: true

config Object

Provides an object that contains the fully-qualified name of the target table of the maintenance task.

Example:

{
  "tableId": "folder1.folder2.table1" 
}

List Tasks

Lists tasks with filtering, paging, and max results.

Method and URL

GET /api/v3/catalog/{sourceId}/maintenance/tasks?filter

Parameters

sourceId Path String

Identifier of the Open Catalog hosting the maintenance tasks.

Example: 186665c9-2a6a-46b7-9a7d-8d3a867ad650

filter Query String

A Common Expression Language (CEL) expression. For more information, see the intro to CEL

Usable variables within the expression are:

type: Optimization type. This can be OPTIMIZE or EXPIRE_SNAPSHOTS.
level: Maintenance level. This can be TABLE or CATALOG.
tableId: Table name preceded by zero or more namespaces(without a source name), with . as the separator.

Example: type=="OPTIMIZE"&&level=="TABLE"

maxResults Query Integer

Maximum number of results to be returned by the server. The server may return fewer resources than this, but not more. Defaults to 10.

Example: maxResults=25

pageToken Query String

Opaque pagination token. This token is returned by the server in a previous response, and is used to request the next page of results. For example: 1->AAAAAQ, 2->AAAAAg, 3->AAAAAw, etc.

Example: pageToken=AAAAAQ

Example

Request

curl -X GET 'https://{hostname}/api/v3/catalog/186665c9-2a6a-46b7-9a7d-8d3a867ad650/maintenance/tasks?filter=type=="OPTIMIZE"&&level=="TABLE"&maxResults=10' \
--header 'Authorization: Bearer <dremioAccessToken>' \
--header 'Content-Type: application/json'

Response

{
  "data": [
  {
    "id": "d510279d-5741-4feb-82b8-d6923c9ac6c8",
    "type": "OPTIMIZE",
    "level": "TABLE",
    "sourceName": "prod",
    "isEnabled": true,
    "config": {
      "tableId": "folder1.folder2.table1"
    }
  },
  {
    "id": "b0fb84d0-b2c9-455d-9013-3c3e417c46f0",
    "type": "OPTIMIZE",
    "level": "TABLE",
    "sourceName": "prod",
    "isEnabled": true,
    "config": {
      "tableId": "folder1.folder2.table2"
    }
  }
]
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Create a Task

Creates a maintenance task.

Method and URL

POST /api/v3/catalog/{sourceId}/maintenance/tasks

Parameters

sourceId Path String

The sourceID of the Open Catalog peforming this task.

Example: 186665c9-2a6a-46b7-9a7d-8d3a867ad650

taskDefinition Body Object

Represents a new task.

Example:

{ 
   "type": "OPTIMIZE", 
   "isEnabled": true, 
   "config": { 
      "tableId": "folder1.folder2.table1" 
   } 
}

TaskDefinition Object Parameters

type String

Type of maintenance to run.

OPTIMIZE - A table optimization task
EXPIRE_SNAPSHOTS - A table vacuuum task

isEnabled Boolean

Indicates if the task is enabled.

Example: true

config Body Object

Provides an object that contains a fully-qualifed object name in the indicated catalog as as the target for the maintenance task.

Example:

{
  "tableId": "folder1.folder2.table1" 
}

Example

Request

curl -X POST 'https://{hostname}/api/v3/catalog/186665c9-2a6a-46b7-9a7d-8d3a867ad650/maintenance/tasks' \
--header 'Authorization: Bearer <dremioAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "type": "OPTIMIZE",
  "isEnabled": true,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
  }'

Response

{ 
  "id": "f526e534-f61c-43a5-9d35-51a8b0f75826",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": true,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Retrieve a Task by ID

Retrieve a task by specifying the task's ID.

Method and URL

GET /api/v3/catalog/{sourceId}/maintenance/tasks/{taskId}

Parameters

sourceId Path String

Unique identifier of the Open Catalog hosting the task.

Example: 186665c9-2a6a-46b7-9a7d-8d3a867ad650

taskId Path String (UUID)

Unique identifier of the task that you want to retrieve.

Example: 87049e43-8564-4ee7-8bb6-5bdaf5bd0959

Example

Request

curl -X GET 'https://{hostname}/api/v3/catalog/186665c9-2a6a-46b7-9a7d-8d3a867ad650/maintenance/tasks/f526e534-f61c-43a5-9d35-51a8b0f75826' \
--header 'Authorization: Bearer <dremioAccessToken>' \
>>>>>>> origin/rel/26.1
--header 'Content-Type: application/json'

Response

{
  "id": "f526e534-f61c-43a5-9d35-51a8b0f75826",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": true,
  "config": {
  "tableId": "folder1.folder2.table1"
  }
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Update a Task

Update the specified task, such as disabling it.

Method and URL

PUT /api/v3/catalog/{sourceId}/maintenance/tasks/{taskId}

Parameters

sourceId Path String

Unique identifier of the Open Catalog hosting the task.

Example: 186665c9-2a6a-46b7-9a7d-8d3a867ad650

taskId Path String (UUID)

Unique identifier of the task to update

Example: 87049e43-8564-4ee7-8bb6-5bdaf5bd0959

taskDefinition Body Object

Represents an updated task specification.

Example:

{ 
  "type": "OPTIMIZE", 
  "isEnabled": false, 
  "config": { 
    "tableId": "folder1.folder2.table1" 
  } 
}

Example

Request

curl -X PUT 'https://{hostname}/api/v3/catalog/186665c9-2a6a-46b7-9a7d-8d3a867ad650/maintenance/tasks/f526e534-f61c-43a5-9d35-51a8b0f75826' \
--header 'Authorization: Bearer <dremioAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw ' {
    "type": "OPTIMIZE",
    "isEnabled": false,
    "config": {
      "tableId": "folder1.folder2.table1"
    }
  }'

Response

{
  "id": "f526e534-f61c-43a5-9d35-51a8b0f75826",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": false,
  "config": {
    "tableId": "folder1.folder2.table1"
  }
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Delete a Task

Delete the specified task.

Method and URL

DELETE /api/v3/catalog/{sourceId}/maintenance/tasks/{taskId}

Parameters

sourceId Path String

Unique identifier of the source from where to retrieve the task.

Example: 186665c9-2a6a-46b7-9a7d-8d3a867ad650

taskId Path String (UUID)

Unique identifier of the task that you want to retrieve.

Example: 87049e43-8564-4ee7-8bb6-5bdaf5bd0959

Example

Request

curl -X DELETE 'https://{hostname}/api/v3/catalog/186665c9-2a6a-46b7-9a7d-8d3a867ad650/maintenance/tasks/f526e534-f61c-43a5-9d35-51a8b0f75826' \
--header 'Authorization: Bearer <dremioAccessToken>' \
--header 'Content-Type: application/json'

Response

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

TaskInfo Attributes​

List Tasks​

Parameters​

Example​

Response Status Codes​

Create a Task​

Parameters​

TaskDefinition Object Parameters​

Example​

Response Status Codes​

Retrieve a Task by ID​

Parameters​

Example​

Response Status Codes​

Update a Task​

Parameters​

Example​

Response Status Codes​

Delete a Task​

Parameters​

Example​

Response Status Codes​

TaskInfo Attributes

List Tasks

Parameters

Example

Response Status Codes

Create a Task

Parameters

TaskDefinition Object Parameters

Example

Response Status Codes

Retrieve a Task by ID

Parameters

Example

Response Status Codes

Update a Task

Parameters

Example

Response Status Codes

Delete a Task

Parameters

Example

Response Status Codes