Data Maintenance

Use the Data Maintenance API to manage and execute Auto Maintenance tasks, including OPTIMIZE and EXPIRE_SNAPSHOTS.

TaskInfo Object

{
  "id": "a75918ee-431b-4296-a2b3-298dff53ab12",
  "type": "OPTIMIZE",
  "level": "CATALOG",
  "sourceName": "prod",
  "isEnabled": true,
  "config": {
    "tableId": "sensor.central"
  }
}

TaskInfo Attributes

id String (UUID)

Server-assigned opaque task ID.

Example: a75918ee-431b-4296-a2b3-298dff53ab12

---

type String

Type of maintenance to be run by this task.

Enum: OPTIMIZE, EXPIRE_SNAPSHOTS

Each chosen type adds specific attributes to TaskInfo:

• OPTIMIZE - Adds attributes defined in OptimizedTaskInfo
• EXPIRE_SNAPSHOTS - Adds attributes defined in ExpireSnapshotsTaskInfo

Example: OPTIMIZE

---

level String

Scope of maintenance.

Enum: CATALOG, TABLE

Example: CATALOG

---

sourceName String

Source name.

Example: prod

---

isEnabled Boolean

Whether the task is enabled.

Example: true

OptimizedTaskInfo Attributes

These attributes expand TaskInfo when the type is OPTIMIZE.

config Object

Optimization config info.

Example: { "tableId": "sensor.central" }

Attributes of the OptimizeConfigInfo Object

tableId string

Fully qualified table ID.

Example: "sensor.central"

ExpireSnapshotsTaskInfo Attributes

These attributes expand TaskInfo when the type is EXPIRE_SNAPSHOTS.

config Object

Expire snapshots config info.

Example: { "tableId": "sensor.history" }

Attributes of the ExpireSnapshotsConfigInfo Object

tableId string

Fully qualified table ID.

Example: "sensor.history"

Listing Tasks

Lists tasks with optional filtering, paging, and max results.

Method and URL

GET /api/v3/catalog/{sourceId}/maintenance/tasks

Parameters

sourceId Path   String

Identifier of the source from where to retrieve the tasks.

Example: prod

---

filter Query   String

A Common Expression Language (CEL) expression. An intro to CEL can be found at https://github.com/google/cel-spec/blob/master/doc/intro.md.

Usable variables within the expression are:

• type: Optimization type. This can be OPTIMIZE or EXPIRE_SNAPSHOTS.
• level: Maintenance level. This can be TABLE or CATALOG.
• tableId: Table name preceded by zero or more namespaces(without a source name), with . as the separator.

Example: type=="OPTIMIZE"&&level=="TABLE"

---

maxResults Query   Integer

Maximum number of results to be returned by the server. The server may return fewer resources than this, but not more. Defaults to 10.

Example: maxResults=25

---

pageToken Query   String

Opaque pagination token. This token is returned by the server in a previous response, and is used to request the next page of results. For example: 1->AAAAAQ, 2->AAAAAg, 3->AAAAAw, etc.

Example: pageToken=AAAAAQ

Example Request

curl -X GET 'https://{hostname}/api/v3/catalog/prod/maintenance/tasks?filter=type=="OPTIMIZE"&&level=="TABLE"&maxResults=10' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json'

Example Response

{
  "data": [
    {
      "id": "d510279d-5741-4feb-82b8-d6923c9ac6c8",
      "type": "OPTIMIZE",
      "level": "TABLE",
      "sourceName": "prod",
      "isEnabled": true,
      "config": {
        "tableId": "sensor.central"
      }
    },
    {
      "id": "b0fb84d0-b2c9-455d-9013-3c3e417c46f0",
      "type": "OPTIMIZE",
      "level": "TABLE",
      "sourceName": "prod",
      "isEnabled": true,
      "config": {
        "tableId": "sensor.west"
      }
    }
  ]
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Creating a Task

Creates a task.

Method and URL

POST /api/v3/catalog/{sourceId}/maintenance/tasks

Parameters

sourceId Path   String

The identifier of the source.

Example: prod

---

taskDefinition Body   Object

Represents a new task.

Example: { "type": "OPTIMIZE", "isEnabled": true, "config": { "tableId": "sensor.central" } }

---

Parameters of the TaskDefinition Object

type String

Type of maintenance to run.

Enum: OPTIMIZE, EXPIRE_SNAPSHOTS

Each chosen type adds specific attributes to the TaskDefinition:

• OPTIMIZE - Adds attributes defined in OptimizedTaskDefinition
• EXPIRE_SNAPSHOTS - Adds attributes defined in ExpireSnapshotsTaskDefinition

Example: OPTIMIZE

---

isEnabled Boolean

Whether the task is enabled.

Example: true

Parameters of OptimizedTaskDefinition

These parameters expand TaskDefinition when the type is OPTIMIZE.

config Object

Optimization configuration.

Example: { "config": { "tableId": "sensor.central" } }

Parameters of the OptimizeConfigDefinition Object

tableId string

Fully qualified table ID.

Example: "sensor.central"

Parameters of ExpireSnapshotsTaskDefinition

These parameters expand TaskDefinition when the type is EXPIRE_SNAPSHOTS.

config Object

Snapshot expiration configuration.

Example: { "tableId": "sensor.history" }

Parameters of the ExpireSnapshotsConfigDefinition Object

tableId string

Fully qualified table ID.

Example: "sensor.history"

Example Request

curl -X POST 'https://{hostname}/api/v3/catalog/prod/maintenance/tasks' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
    "type": "OPTIMIZE",
    "isEnabled": true,
    "config": {
      "tableId": "sensor.central"
    }
  }'

Example Response

{
  "id": "f526e534-f61c-43a5-9d35-51a8b0f75826",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": true,
  "config": {
    "tableId": "sensor.central"
  }
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Retrieving a Task by ID

Retrieve a task by specifying the task's ID.

Method and URL

GET /api/v3/catalog/{sourceId}/maintenance/tasks/{taskId}

Parameters

sourceId Path   String

Unique identifier of the source from where to retrieve the task.

Example: prod

---

taskId Path   String (UUID)

Unique identifier of the task that you want to retrieve.

Example: 87049e43-8564-4ee7-8bb6-5bdaf5bd0959

---

Example Request

curl -X GET 'https://{hostname}/api/v3/catalog/prod/maintenance/tasks?filter=type=="OPTIMIZE"&&level=="TABLE"&maxResults=10' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json'

Example Response

{
  "id": "f526e534-f61c-43a5-9d35-51a8b0f75826",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": true,
  "config": {
    "tableId": "sensor.central"
  }
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Updating a Task

Update the specified task.

Method and URL

PUT /api/v3/catalog/{sourceId}/maintenance/tasks/{taskId}

Parameters

sourceId Path   String

Unique identifier of the source from where to retrieve the task.

Example: prod

---

taskId Path   String (UUID)

Unique identifier of the task that you want to retrieve.

Example: 87049e43-8564-4ee7-8bb6-5bdaf5bd0959

---

taskDefinition Body   Object

Represents an updated task specification.

Example: { "type": "OPTIMIZE", "isEnabled": false, "config": { "tableId": "sensor.central" } }

Example Request

curl -X PUT 'https://{hostname}/api/v3/catalog/prod/maintenance/tasks/f526e534-f61c-43a5-9d35-51a8b0f75826' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw ' {
    "type": "OPTIMIZE",
    "isEnabled": false,
    "config": {
      "tableId": "sensor.central"
    }
  }'

Example Response

{
  "id": "f526e534-f61c-43a5-9d35-51a8b0f75826",
  "type": "OPTIMIZE",
  "level": "TABLE",
  "sourceName": "prod",
  "isEnabled": false,
  "config": {
    "tableId": "sensor.central"
  }
}

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error

Deleting a Task

Delete the specified task.

Method and URL

DELETE /api/v3/catalog/{sourceId}/maintenance/tasks/{taskId}

Parameters

sourceId Path   String

Unique identifier of the source from where to retrieve the task.

Example: prod

---

taskId Path   String (UUID)

Unique identifier of the task that you want to retrieve.

Example: 87049e43-8564-4ee7-8bb6-5bdaf5bd0959

---

Example Request

curl -X DELETE 'https://{hostname}/api/v3/catalog/prod/maintenance/tasks/f526e534-f61c-43a5-9d35-51a8b0f75826' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json'

Example Response

1

Response Status Codes

200   OK

400   Bad Request

401   Unauthorized

403   Forbidden

404   Not Found

415   Unsupported Media Type

500   Internal Server Error