On this page

    Engines

    An engine represents a Dremio Cloud entity that manages compute resources. Each engine has one or more replicas that are created for executing queries. An engine replica is a logical group of executors defined by the engine size. This API allows you to list existing engines and add, modify, enable/disable, and delete engines.

    Engine Object
        "id": "e1cb20ff-a234-46f1-9f97-550e2bb5310f",
        "name": "firstEngine",
        "size": "XX_SMALL_V1",
        "activeReplicas": 0,
        "minReplicas": 0,
        "maxReplicas": 10,
        "autoStopDelaySeconds": 300,
        "queueTimeLimitSeconds": 300,
        "runtimeLimitSeconds": 0,
        "drainTimeLimitSeconds": 0,
        "state": "ENABLED",
        "cloudTags": [],
        "queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
        "statusChangedAt": "Thu Aug 19 14:39:40 UTC 2021",
        "description": "",
        "maxConcurrency": 4
    

    Engine Attributes

    id

    String (UUID)

    The user-defined UUID to identify an engine.

    Example e1cb20ff-a234-46f1-9f97-550e2bb5310f


    name

    String

    The user-defined name for an engine.

    Example firstEngine


    size

    String

    The size of the engine. Each engine replica that is created will be of this size.

    Enum XX_SMALL_V1 , X_SMALL_V1 , SMALL_V1 , MEDIUM_V1 , LARGE_V1 , X_LARGE_V1 , XX_LARGE_V1 , XXX_LARGE_V1

    Example SMALL_V1


    activeReplicas

    integer

    The number of engine replicas that are currently active.

    Example 0


    minReplicas

    integer

    The minimum number of engine replicas that will be enabled at any given time.

    Example 0


    maxReplicas

    integer

    The maximum number of engine replicas that will be enabled at any given time.

    Example 10


    autoStopDelaySeconds

    integer

    The time (in seconds) that auto stop is delayed.

    Example 300


    queueTimeLimitSeconds

    integer

    The maximum amount of time (in seconds) that a query will wait in the engine’s queue before it is canceled.

    Note:

    The typical time to start a new replica is two minutes (120 seconds), so we recommend setting this to more than 120 seconds. Changing this setting does not affect queries that are either currently running or already in the queue.

    Example 300


    runtimeLimitSeconds

    integer

    The maximum amount of time (in seconds) that a query can run before being terminated.

    Example 0


    drainTimeLimitSeconds

    integer

    The maximum amount of time (in seconds) that an engine replica will continue to run after the engine is resized, disabled, or deleted before it is terminated and the running queries terminate.

    Example 0


    state

    String

    The current state of the engine.

    Enum DELETING , DISABLED , DISABLING , ENABLED , ENABLING , INVALID

    Example ENABLED


    cloudTags

    object

    AWS tags.

    Example {"key": "dremio", "value": "test"}


    queriedAt

    String

    The date and time that the engine was last used to execute a query.

    Example Thu Jan 01 00:00:00 UTC 2021


    statusChangedAt

    String

    The date and time (in UTC time) that the state of the engine changed.

    Example Thu Aug 19 14:39:40 UTC 2021


    description

    String

    The description for the engine.

    Example engine for sales workloads


    maxConcurrency

    integer

    The maximum number of concurrent queries that an engine replica can run.

    Example 4


    cloudTags

    key

    String

    The key identifier for the tag.

    Example dremio


    value

    String

    The value of the tag.

    Example test


    Listing All Engines

    Returns a list of all the engines in the specified project and the metadata for each engine.

    Listing All Engines
    GET /v0/projects/{project-id}/engines
    

    Parameters

    project-id

    path

    String (UUID)

    The user-defined UUID for the project you want to list engines for.

    Example Request
    curl -X GET 'https://api.dremio.cloud/v0/projects/02d36975-73eb-47ed-9bb5-de73060380f6/engines' \
    -H 'Authorization: Bearer <personal access token>' \
    -H 'Content-Type: application/json'
    
    Example Response
    {
    [
      {
        "id": "e1cb20ff-a234-46f1-9f97-550e2bb5310f",
        "name": "firstEngine",
        "size": "XX_SMALL_V1",
        "activeReplicas": 0,
        "minReplicas": 0,
        "maxReplicas": 10,
        "autoStopDelaySeconds": 300,
        "queueTimeLimitSeconds": 300,
        "runtimeLimitSeconds": 0,
        "drainTimeLimitSeconds": 0,
        "state": "ENABLED",
        "cloudTags": [],
        "queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
        "statusChangedAt": "Thu Aug 19 14:39:40 UTC 2021",
        "description": "",
        "maxConcurrency": 4
      },
      {
        "id": "a99eb46a-8121-4b79-b2ed-917e07f96092",
        "name": "preview",
        "size": "XX_SMALL_V1",
        "activeReplicas": 0,
        "minReplicas": 0,
        "maxReplicas": 1,
        "autoStopDelaySeconds": 180,
        "queueTimeLimitSeconds": 300,
        "runtimeLimitSeconds": 0,
        "drainTimeLimitSeconds": 0,
        "state": "ENABLED",
        "cloudTags": [],
        "queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
        "statusChangedAt": "Fri Aug 20 03:58:54 UTC 2021",
        "description": "",
        "maxConcurrency": 4
      }
    ]
    

    Responses

    200

    OK

    400

    Bad Request

    401

    Unauthorized


    Adding an Engine

    Add an engine to the cloud account that is associated with the specified project.

    Adding an Engine
    POST /v0/projects/{project-id}/engines
    

    Parameters

    Engine

    project-id

    path

    String (UUID)

    The user-defined UUID to identify the execution engine.


    name

    query

    String

    The user-defined name for an engine.


    id

    query

    string (UUID)

    The user-defined UUID to identify an engine.


    size

    query

    String

    The size of the engine that you want to create. Each engine replica that is created will be of this size.

    Enum XX_SMALL_V1 , X_SMALL_V1 , SMALL_V1 , MEDIUM_V1 , LARGE_V1 , X_LARGE_V1 , XX_LARGE_V1 , XXX_LARGE_V1


    minReplicas

    query

    integer

    The minimum number of engine replicas that will be enabled at any given time.


    maxReplicas

    query

    integer

    The maximum number of engine replicas that will be enabled at any given time.


    autoStopDelaySeconds

    query

    integer

    The time (in seconds) that auto stop is delayed.


    queueTimeLimitSeconds

    query

    integer

    The maximum amount of time (in seconds) that a query will wait in the engine’s queue before it is canceled. This should not be set to less than two minutes (120 seconds), the typical time it takes to start a new replica. Changing this setting does not affect queries that are currently running or in the queue.


    runtimeLimitSeconds

    query

    integer

    The maximum amount of time (in seconds) that a query can run before being terminated.


    drainTimeLimitSeconds

    query

    integer

    The maximum amount of time (in seconds) that an engine replica will continue to run after the engine is resized, disabled, or deleted before it is terminated and the running queries terminate.


    cloudTags

    query

    object

    Optional

    AWS tags.


    description

    query

    String

    Optional

    The description for the engine.


    maxConcurrency

    query

    integer

    The maximum number of concurrent queries that an engine replica can handle.


    requestId

    query

    string (UUID)

    The user-defined idempotency key, which is a unique value generated by the user that the server uses to register consequent retries of the same request. You can generate the request ID by using a UUID generator tool. For more information, see idempotent requests .

    cloudTags

    key

    query

    String

    The key identifier for the tag.


    value

    query

    String

    The value of the tag.

    Example Request
    curl -X POST 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines' \
    -H 'Authorization: Bearer <personal access token>' \
    -H "Content-Type: application/json" \
    -H "Accept: application/json" \
    -d '{
        "name": "newengine",
        "size": "XX_SMALL_V1",
        "minReplicas": 0,
        "maxReplicas": 1,
        "autoStopDelaySeconds": 7200,
        "queueTimeLimitSeconds": 300,
        "runtimeLimitSeconds": 0,
        "drainTimeLimitSeconds": 1800,
        "cloudTags": [
            {
            "key": "tag1",
            "value": "value1"
            }
          ],
          "description": "engine description",
          "maxConcurrency": 2,
          "requestId": "d9904076-01db-11ec-9a03-0242ac130003"
          }'
    
    Example Response
    {
      "id": "168acd56-16a0-47df-91f8-5487a8fccfc4"
    }
    

    Responses

    200

    OK

    400

    Bad Request

    401

    Unauthorized

    405

    Method Not Allowed

    500

    Internal Server Error


    Retrieving an Engine

    Get the metadata for an existing engine by providing the project and engine IDs.

    Retrieving an Engine
    GET /v0/projects/{project-id}/engines/{id}
    

    Parameters

    project-id

    path

    string (UUID)

    The user-defined UUID to identify the execution engine.


    id

    path

    String

    The ID of the engine for which you want to retrieve metadata.

    Example Request
    curl -X GET 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4' \
    -H 'Authorization: Bearer <personal access token>' \
    -H 'Content-Type: application/json'
    
    Example Response
    {
      "id": "168acd56-16a0-47df-91f8-5487a8fccfc4",
      "name": "newengine",
      "size": "XX_SMALL_V1",
      "activeReplicas": 0,
      "minReplicas": 0,
      "maxReplicas": 1,
      "autoStopDelaySeconds": 7200,
      "queueTimeLimitSeconds": 300,
      "runtimeLimitSeconds": 0,
      "drainTimeLimitSeconds": 1800,
      "state": "ENABLED",
      "cloudTags": [
        {
          "key": "tag1",
          "value": "value1"
        }
      ],
      "queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
      "statusChangedAt": "Fri Aug 20 17:29:18 UTC 2021",
      "description": "engine description",
      "maxConcurrency": 2
    }
    

    Responses

    200

    OK

    400

    Engine does not exist

    401

    Unauthorized

    500

    Internal Server Error


    Modifying an Engine

    Modify the engine by providing the project and engine IDs. Parameters that are not provided will be left unchanged. The engine name cannot be modified.

    Modifying an Engine
    PUT /v0/projects/{project-id}/engines/{id}
    

    Parameters

    project-id

    path

    string (UUID)

    The user-defined UUID to identify the execution engine.


    id

    path

    String (UUID)

    The ID of the engine for which you want to retrieve metadata.


    size

    query

    String

    Optional

    The size of the engine that you want to change to. Each engine replica that is modified will be of this size.

    Enum XX_SMALL_V1 , X_SMALL_V1 , SMALL_V1 , MEDIUM_V1 , LARGE_V1 , X_LARGE_V1 , XX_LARGE_V1 , XXX_LARGE_V1


    minReplicas

    query

    integer

    Optional

    The minimum number of engine replicas that will be enabled at any given time.


    maxReplicas

    query

    integer

    Optional

    The maximum number of engine replicas that will be enabled at any given time.


    autoStopDelaySeconds

    query

    integer

    Optional

    The time (in seconds) that auto stop is delayed.


    queueTimeLimitSeconds

    query

    integer

    Optional

    The maximum amount of time (in seconds) that a query will wait in the engine’s queue before it is canceled. This should not be set to less than two minutes (120 seconds), the typical time it takes to start a new replica. Changing this setting does not affect queries that are either currently running or already in the queue.


    runtimeLimitSeconds

    query

    integer

    Optional

    The maximum amount of time (in seconds) that a query can run before being terminated.


    drainTimeLimitSeconds

    query

    integer

    Optional

    The maximum amount of time (in seconds) that an engine replica will continue to run after the engine is resized, disabled, or deleted before it is terminated and the running queries terminate.


    cloudTags

    query

    object

    Optional

    AWS tags.


    description

    query

    String

    Optional

    The description for the engine you are modifying.


    maxConcurrency

    query

    integer

    Optional

    The maximum number of concurrent queries that an engine replica can run.

    cloudTags

    key

    query

    String

    Optional

    The key identifier for the tag.


    value

    query

    String

    Optional

    The value of the tag.

    Example Request
    curl -X PUT 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4' \
    -H 'Authorization: Bearer <personal access token>' \
    -H 'Content-Type: application/json' \
    -d '{
          "description": "modified engine description",
          "size": "XX_SMALL_V1",
          "autoStopDelaySeconds": 7200,
          "drainTimeLimitSeconds": 1800,
          "maxConcurrency": 2,
          "maxReplicas": 1,
          "minReplicas": 0,
          "queueTimeLimitSeconds": 300,
          "runtimeLimitSeconds": 0,
          "cloudTags":  [ ]
          }'
    
    Example Response
    No response
    

    Responses

    204

    No Content

    400

    Bad Request

    401

    Unauthorized

    500

    Internal Server Error


    Enabling an Engine

    Enable an engine.

    Enabling an Engine
    PUT /v0/projects/{project-id}/engines/{id}/enable
    

    Parameters

    project-id

    path

    string (UUID)

    The user-defined UUID to identify the execution engine.


    id

    path

    String

    The ID of the engine that you want to enable.


    Example Request
    curl -X PUT 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4/enable' \
    -H 'Authorization: Bearer <personal access token>' \
    -H 'Content-Type: application/json'
    
    Example Response
    No response
    

    Responses

    204

    No Content

    401

    Unauthorized

    500

    Internal Server Error


    Disabling an Engine

    Disable an engine.

    Disabling an Engine
    PUT /v0/projects/{project-id}/engines/{id}/disable
    

    Parameters

    project-id

    path

    string (UUID)

    User defined UUID to identify the execution engine.


    id

    path

    String

    The ID of the engine that you want to disable.


    Example Request
    curl -X PUT 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4/disable' \
    -H 'Authorization: Bearer <personal access token>' \
    -H 'Content-Type: application/json'
    
    Example Response
    No response
    

    Responses

    204

    No Content

    401

    Unauthorized

    500

    Internal Server Error


    Deleting an Engine

    Delete an engine by providing the project and engine ID.

    Deleting an Engine
    DELETE /v0/projects/{project-id}/engines/{id}
    

    Parameters

    project-id

    path

    string (UUID)

    The user-defined UUID to identify the execution engine.


    id

    path

    String

    The ID of the engine that you want to delete.


    Example Request
    curl -X DELETE 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4' \
    -H 'Authorization: Bearer <personal access token>' \
    -H 'Content-Type: application/json'
    
    Example Response
    No response
    

    Responses

    204

    No Content

    401

    Unauthorized

    500

    Internal Server Error