Engine

Dremio Arctic enables you to run optimization jobs to help you manage the accumulation of the data files that occurs through DML operations. This API allows you to add, update, and retrieve engines that run the optimization jobs.

Engine Object

{
  "credentials": {
    "type": "ACCESS_KEY",
    "accessKeyId": "8BDLDGT42KJ4JEXAMPLE"
  },
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/",
  "engineSize": "XX_SMALL_V1",
  "state": "ENABLED",
  "errorMessage": null
}

Engine Attributes

credentials

Object

Authentication credentials used by all jobs and schedules on the catalog.

Example { "type": "ACCESS_KEY", "accessKeyId": "8BDLDGT42KJ4JEXAMPLE", "secretAccessKey": "EXAMPLEtnFEMI/EXAMENG/bPxRfiCYEXAMPLEKEY" }

---

cloudId

String (UUID)

Unique identifier for the cloud in which to run jobs and schedules on the catalog.

Example bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9

---

logStorageLocation

String

Storage location for audit logs that Dremio engines create.

Example usrsdre-test-west1/usrsdre/

---

engineSize

String

Engine size used.

Enum XX_SMALL_V1, X_SMALL_V1, SMALL_V1, MEDIUM_V1, LARGE_V1, X_LARGE_V1, XX_LARGE_V1, XXX_LARGE_V1

Example XX_SMALL_V1

---

state

String

Status of the engine.

Enum CREATING, ENABLED, DISABLED, UPDATING, FAILED

Example ENABLED

---

errorMessage

String

For issues with the engine, a description of the problem.

Example null

credentials Object Attributes

type

String

Type of authentication credential used by all jobs and schedules on the catalog.

Enum ACCESS_KEY, IAM_ROLE

Example ACCESS_KEY

---

accessKeyId

String

For the ACCESS_KEY type, the access key ID used by all jobs and schedules on the catalog.

Example 8BDLDGT42KJ4JEXAMPLE

---

externalId

String

For the IAM_ROLE type, the external ID used by all jobs and schedules on the catalog.

Example 55637959

---

instanceProfileArn

String

For the IAM_ROLE type, the instance profile Amazon Resource Name (ARN) ID used by all jobs and schedules on the catalog.

Example arn:partition:service:region:account:resource

---

roleArn

String

For the IAM_ROLE type, the role Amazon Resource Name (ARN) used by all jobs and schedules on the catalog.

Example arn:partition:service:region:account:resource

Adding or Updating an Engine

Adds or modifies an engine for the specified Arctic catalog that will run optimization jobs.

Method and URL

POST /v0/arctic/catalogs/{catalogId}/engine

Parameters

catalogId

path

String (UUID)

Unique identifier for the cloud in which to run jobs and schedules on the catalog.

Example 1a234bc5-6789-46d7-b9cb-d236e195b34a

---

credentials

body

Object

Authentication credentials to use for all jobs and schedules on the catalog.

Example { "type": "ACCESS_KEY", "accessKeyId": "8BDLDGT42KJ4JEXAMPLE", "secretAccessKey": "EXAMPLEtnFEMI/EXAMENG/bPxRfiCYEXAMPLEKEY" }

---

cloudId

body

String (UUID)

Unique identifier for the cloud in which to run jobs and schedules on the catalog.

Example bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9

---

logStorageLocation

body

String

Storage location for audit logs that Dremio engines create.

Example usrsdre-test-west1/usrsdre/

---

engineSize

body

String

Engine size to use.

Enum XX_SMALL_V1, X_SMALL_V1, SMALL_V1, MEDIUM_V1, LARGE_V1, X_LARGE_V1, XX_LARGE_V1, XXX_LARGE_V1

Example XX_SMALL_V1

credentials Object Parameters

type

body

String

Type of authentication credential to use for all jobs and schedules on the catalog.

Enum ACCESS_KEY, IAM_ROLE

Example ACCESS_KEY

---

accessKeyId

body

String

For the ACCESS_KEY type, the access key ID to use for all jobs and schedules on the catalog.

Example 8BDLDGT42KJ4JEXAMPLE

---

secretAccessKey

body

String

For the ACCESS_KEY type, the secret access key to use for all jobs and schedules on the catalog.

Example 94yLpEH2fhX3CrfrgE93742d8YyDr89tNEXAMPLE

---

externalId

body

String

Optional

For the IAM_ROLE type, the external ID to use for all jobs and schedules on the catalog.

Example 55637959

---

instanceProfileArn

body

String

For the IAM_ROLE type, the instance profile Amazon Resource Name (ARN) ID to use for all jobs and schedules on the catalog.

Example arn:partition:service:region:account:resource

---

roleArn

body

String

For the IAM_ROLE type, the role Amazon Resource Name (ARN) to use for all jobs and schedules on the catalog.

Example arn:partition:service:region:account:resource

Example Request

curl -X POST 'https://api.dremio.cloud/v0/arctic/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/engine \
--header 'Authorization: Bearer <personal access token>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "credentials": {
    "type": "ACCESS_KEY",
    "accessKeyId": "8BDLDGT42KJ4JEXAMPLE",
    "secretAccessKey": "94yLpEH2fhX3CrfrgE93742d8YyDr89tNEXAMPLE"
  },
  "engineSize": "XX_SMALL_V1",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/"
}'

Example Response

{
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "credentials": {
    "accessKeyId": "8BDLDGT42KJ4JEXAMPLE",
    "type": "ACCESS_KEY"
  },
  "engineSize": "XX_SMALL_V1",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/",
  "state": "CREATING"
}

Responses

200

OK

400

Bad Request

404

Not Found

415

Unsupported Media Type

500

Internal Server Error

---

Retrieving an Engine

Retrieve the engine the Arctic catalog uses.

Method and URL

GET /v0/arctic/catalogs/{catalogId}/engine

Parameters

catalogId

path

String (UUID)

Unique identifier for the Arctic catalog whose engine you want to retrieve.

Example 1a234bc5-6789-46d7-b9cb-d236e195b34a

Example Request

curl -X GET 'https://api.dremio.cloud/v0/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/engine' \
--header 'Authorization: Bearer <personal access token>' \
--header 'Content-Type: application/json' 

Example Response

{
  "credentials": {
    "type": "ACCESS_KEY",
    "accessKeyId": "8BDLDGT42KJ4J5SNSL2Z"
  },
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/",
  "engineSize": "XX_SMALL_V1",
  "state": "ENABLED",
  "errorMessage": null
}

Responses

200

OK

204

No Content

404

Not Found

415

Unsupported Media Type

500

Internal Server Error

---