Dremio Cloud
Dremio Software
 
On this page

    Engine preview

    Dremio Arctic enables you to run optimization jobs to help you manage the accumulation of the data files that occurs through DML operations. This API allows you to add, update, and retrieve engines that run the optimization jobs.

    Engine Object 
    {
  "credentials": {
    "type": "ACCESS_KEY",
    "accessKeyId": "8BDLDGT42KJ4JEXAMPLE"
  },
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/",
  "engineSize": "XX_SMALL_V1",
  "state": "ENABLED",
  "errorMessage": null
}

    Engine Attributes

    credentials

    Object

    Authentication credentials used by all jobs and schedules on the catalog.

    Example { "type": "ACCESS_KEY", "accessKeyId": "8BDLDGT42KJ4JEXAMPLE", "secretAccessKey": "EXAMPLEtnFEMI/EXAMENG/bPxRfiCYEXAMPLEKEY" }

    cloudId

    String (UUID)

    Unique identifier for the cloud in which to run jobs and schedules on the catalog.

    Example bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9

    logStorageLocation

    String

    Storage location for audit logs that Dremio engines create.

    Example usrsdre-test-west1/usrsdre/

    engineSize

    String

    Engine size used.

    Enum XX_SMALL_V1 , X_SMALL_V1 , SMALL_V1 , MEDIUM_V1 , LARGE_V1 , X_LARGE_V1 , XX_LARGE_V1 , XXX_LARGE_V1

    Example XX_SMALL_V1

    state

    String

    Status of the engine.

    Enum CREATING , ENABLED , DISABLED , UPDATING , FAILED

    Example ENABLED

    errorMessage

    String

    For issues with the engine, a description of the problem.

    Example null

    credentials Object Attributes

    type

    String

    Type of authentication credential used by all jobs and schedules on the catalog.

    Enum ACCESS_KEY , IAM_ROLE

    Example ACCESS_KEY

    accessKeyId

    String

    For the ACCESS_KEY type, the access key ID used by all jobs and schedules on the catalog.

    Example 8BDLDGT42KJ4JEXAMPLE

    externalId

    String

    For the IAM_ROLE type, the external ID used by all jobs and schedules on the catalog.

    Example 55637959

    instanceProfileArn

    String

    For the IAM_ROLE type, the instance profile Amazon Resource Name (ARN) ID used by all jobs and schedules on the catalog.

    Example arn:partition:service:region:account:resource

    roleArn

    String

    For the IAM_ROLE type, the role Amazon Resource Name (ARN) used by all jobs and schedules on the catalog.

    Example arn:partition:service:region:account:resource

    Adding or Updating an Engine

    Adds or modifies an engine for the specified Arctic catalog that will run optimization jobs.

    Method and URL 
    POST /v0/arctic/catalogs/{catalogId}/engine

    Parameters

    catalogId

    path

    String (UUID)

    Unique identifier for the cloud in which to run jobs and schedules on the catalog.

    Example 1a234bc5-6789-46d7-b9cb-d236e195b34a

    credentials

    body

    Object

    Authentication credentials to use for all jobs and schedules on the catalog.

    Example { "type": "ACCESS_KEY", "accessKeyId": "8BDLDGT42KJ4JEXAMPLE", "secretAccessKey": "EXAMPLEtnFEMI/EXAMENG/bPxRfiCYEXAMPLEKEY" }

    cloudId

    body

    String (UUID)

    Unique identifier for the cloud in which to run jobs and schedules on the catalog.

    Example bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9

    logStorageLocation

    String

    Storage location for audit logs that Dremio engines create.

    Example usrsdre-test-west1/usrsdre/

    engineSize

    String

    Engine size to use.

    Enum XX_SMALL_V1 , X_SMALL_V1 , SMALL_V1 , MEDIUM_V1 , LARGE_V1 , X_LARGE_V1 , XX_LARGE_V1 , XXX_LARGE_V1

    Example XX_SMALL_V1

    credentials Object Parameters

    type

    String

    Type of authentication credential to use for all jobs and schedules on the catalog.

    Enum ACCESS_KEY , IAM_ROLE

    Example ACCESS_KEY

    accessKeyId

    String

    For the ACCESS_KEY type, the access key ID to use for all jobs and schedules on the catalog.

    Example 8BDLDGT42KJ4JEXAMPLE

    secretAccessKey

    String

    For the ACCESS_KEY type, the secret access key to use for all jobs and schedules on the catalog.

    Example 94yLpEH2fhX3CrfrgE93742d8YyDr89tNEXAMPLE

    externalId

    String

    Optional

    For the IAM_ROLE type, the external ID to use for all jobs and schedules on the catalog.

    Example 55637959

    instanceProfileArn

    String

    For the IAM_ROLE type, the instance profile Amazon Resource Name (ARN) ID to use for all jobs and schedules on the catalog.

    Example arn:partition:service:region:account:resource

    roleArn

    String

    For the IAM_ROLE type, the role Amazon Resource Name (ARN) to use for all jobs and schedules on the catalog.

    Example arn:partition:service:region:account:resource


    Example Request 
    curl -X POST 'https://api.dremio.cloud/v0/arctic/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/engine \
--header 'Authorization: Bearer <personal access token>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "credentials": {
    "type": "ACCESS_KEY",
    "accessKeyId": "8BDLDGT42KJ4JEXAMPLE",
    "secretAccessKey": "94yLpEH2fhX3CrfrgE93742d8YyDr89tNEXAMPLE"
  },
  "engineSize": "XX_SMALL_V1",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/"
}'
    Example Response 
    {
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "credentials": {
    "accessKeyId": "8BDLDGT42KJ4JEXAMPLE",
    "type": "ACCESS_KEY"
  },
  "engineSize": "XX_SMALL_V1",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/",
  "state": "CREATING"
}

    Responses

    200

    OK

    400

    Bad Request

    404

    Not Found

    415

    Unsupported Media Type

    500

    Internal Server Error

    Retrieving an Engine

    Retrieve the engine the Arctic catalog uses.

    Method and URL 
    GET /v0/arctic/catalogs/{catalogId}/engine

    Parameters

    catalogId

    path

    String (UUID)

    Unique identifier for the Arctic catalog whose engine you want to retrieve.

    Example 1a234bc5-6789-46d7-b9cb-d236e195b34a

    Example Request 
    curl -X GET 'https://api.dremio.cloud/v0/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/engine' \
--header 'Authorization: Bearer <personal access token>' \
--header 'Content-Type: application/json'
    Example Response 
    {
  "credentials": {
    "type": "ACCESS_KEY",
    "accessKeyId": "8BDLDGT42KJ4J5SNSL2Z"
  },
  "cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
  "logStorageLocation": "usrsdre-test-west1/usrsdre/",
  "engineSize": "XX_SMALL_V1",
  "state": "ENABLED",
  "errorMessage": null
}

    Responses

    200

    OK

    204

    No Content

    404

    Not Found

    415

    Unsupported Media Type

    500

    Internal Server Error