Engine

Dremio Arctic enables you to run optimization jobs to help you manage the accumulation of the data files that occurs through DML operations. This API allows you to add, update, and retrieve engines that run the optimization jobs.

Engine Object

{
  "credentials": {
    "type": "ACCESS_KEY",
    "accessKeyId": "8BDLDGT42KJ4JEXAMPLE"
  },
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/",
  "engineSize": "XX_SMALL_V1",
  "state": "ENABLED",
  "errorMessage": null
}

Engine Attributes

credentials Object

Authentication credentials used by all jobs and schedules on the catalog.

Example: {"type": "ACCESS_KEY","accessKeyId": "8BDLDGT42KJ4JEXAMPLE","secretAccessKey": "EXAMPLEtnFEMI/EXAMENG/bPxRfiCYEXAMPLEKEY"}

cloudId String (UUID)

Unique identifier for the cloud in which to run jobs and schedules on the catalog.

Example: bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9

logStorageLocation String

Storage location for audit logs that Dremio engines create.

Example: usrsdre-test-west1/usrsdre/

engineSize String

Engine size used.

Enum: XX_SMALL_V1, X_SMALL_V1, SMALL_V1, MEDIUM_V1, LARGE_V1, X_LARGE_V1, XX_LARGE_V1, XXX_LARGE_V1

Example: XX_SMALL_V1

state String

Status of the engine.

Enum: CREATING, ENABLED, DISABLED, UPDATING, FAILED

Example: ENABLED

errorMessage String

For issues with the engine, a description of the problem.

Example: null

Attributes of the `credentials` Object

type String

Type of authentication credential used by all jobs and schedules on the catalog.

Enum: ACCESS_KEY, IAM_ROLE

Example: ACCESS_KEY

accessKeyId String

For the ACCESS_KEY type, the access key ID used by all jobs and schedules on the catalog.

Example: 8BDLDGT42KJ4JEXAMPLE

externalId String

For the IAM_ROLE type, the external ID used by all jobs and schedules on the catalog.

Example: 55637959

instanceProfileArn String

For the IAM_ROLE type, the instance profile Amazon Resource Name (ARN) ID used by all jobs and schedules on the catalog.

Example: arn:partition:service:region:account:resource

roleArn String

For the IAM_ROLE type, the role Amazon Resource Name (ARN) used by all jobs and schedules on the catalog.

Example: arn:partition:service:region:account:resource

Adding or Updating an Engine

Adds or modifies an engine for the specified Arctic catalog that will run optimization jobs.

Method and URL

POST /v0/arctic/catalogs/{catalogId}/engine

Parameters

catalogId Path String (UUID)

Unique identifier for the cloud in which to run jobs and schedules on the catalog.

Example: 1a234bc5-6789-46d7-b9cb-d236e195b34a

credentials Body Object

Authentication credentials to use for all jobs and schedules on the catalog.

Example: {"type": "ACCESS_KEY","accessKeyId": "8BDLDGT42KJ4JEXAMPLE","secretAccessKey": "EXAMPLEtnFEMI/EXAMENG/bPxRfiCYEXAMPLEKEY"}

cloudId Body String (UUID)

Unique identifier for the cloud in which to run jobs and schedules on the catalog.

Example: bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9

logStorageLocation Body String

Storage location for audit logs that Dremio engines create.

Example: usrsdre-test-west1/usrsdre/

engineSize Body String

Engine size to use.

Enum: XX_SMALL_V1, X_SMALL_V1, SMALL_V1, MEDIUM_V1, LARGE_V1, X_LARGE_V1, XX_LARGE_V1, XXX_LARGE_V1

Example: XX_SMALL_V1

Parameters of the `credentials` Object

type Body String

Type of authentication credential to use for all jobs and schedules on the catalog.

Enum: ACCESS_KEY, IAM_ROLE

Example: ACCESS_KEY

accessKeyId Body String

For the ACCESS_KEY type, the access key ID to use for all jobs and schedules on the catalog.

Example: 8BDLDGT42KJ4JEXAMPLE

secretAccessKey Body String

For the ACCESS_KEY type, the secret access key to use for all jobs and schedules on the catalog.

Example: 94yLpEH2fhX3CrfrgE93742d8YyDr89tNEXAMPLE

externalId Body String Optional

For the IAM_ROLE type, the external ID to use for all jobs and schedules on the catalog.

Example: 55637959

instanceProfileArn Body String

For the IAM_ROLE type, the instance profile Amazon Resource Name (ARN) ID to use for all jobs and schedules on the catalog.

Example: arn:partition:service:region:account:resource

roleArn Body String

For the IAM_ROLE type, the role Amazon Resource Name (ARN) to use for all jobs and schedules on the catalog.

Example: arn:partition:service:region:account:resource

Example Request

curl -X POST 'https://api.dremio.cloud/v0/arctic/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/engine \
--header 'Authorization: Bearer <personal access token>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "credentials": {
    "type": "ACCESS_KEY",
    "accessKeyId": "8BDLDGT42KJ4JEXAMPLE",
    "secretAccessKey": "94yLpEH2fhX3CrfrgE93742d8YyDr89tNEXAMPLE"
  },
  "engineSize": "XX_SMALL_V1",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/"
}'

Example Response

{
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "credentials": {
    "accessKeyId": "8BDLDGT42KJ4JEXAMPLE",
    "type": "ACCESS_KEY"
  },
  "engineSize": "XX_SMALL_V1",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/",
  "state": "CREATING"
}

Response Status Codes

200 OK

400 Bad Request

404 Not Found

415 Unsupported Media Type

500 Internal Server Error

Retrieving an Engine

Retrieve the engine the Arctic catalog uses.

Method and URL

GET /v0/arctic/catalogs/{catalogId}/engine

Parameters

catalogId Path String (UUID)

Unique identifier for the Arctic catalog whose engine you want to retrieve.

Example: 1a234bc5-6789-46d7-b9cb-d236e195b34a

Example Request

curl -X GET 'https://api.dremio.cloud/v0/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/engine' \
--header 'Authorization: Bearer <personal access token>' \
--header 'Content-Type: application/json' 

Example Response

{
  "credentials": {
    "type": "ACCESS_KEY",
    "accessKeyId": "8BDLDGT42KJ4J5SNSL2Z"
  },
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/",
  "engineSize": "XX_SMALL_V1",
  "state": "ENABLED",
  "errorMessage": null
}

Response Status Codes

200 OK

204 No Content

404 Not Found

415 Unsupported Media Type

500 Internal Server Error

Engine Attributes​

Attributes of the credentials Object​

Adding or Updating an Engine​

Parameters​

Parameters of the credentials Object​

Response Status Codes​

Retrieving an Engine​

Parameters​

Response Status Codes​

Engine Attributes

Attributes of the `credentials` Object

Adding or Updating an Engine

Parameters

Parameters of the `credentials` Object

Response Status Codes

Retrieving an Engine

Parameters

Response Status Codes