Skip to main content

Engines

An engine represents a Dremio Cloud entity that manages compute resources. Each engine has one or more replicas that are created for executing queries. An engine replica is a logical group of executors defined by the engine size. This API allows you to list existing engines and add, modify, enable/disable, and delete engines.

Engines Object
    "id": "e1cb20ff-a234-46f1-9f97-550e2bb5310f",
"name": "firstEngine",
"size": "XX_SMALL_V1",
"activeReplicas": 0,
"minReplicas": 0,
"maxReplicas": 10,
"autoStopDelaySeconds": 300,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 0,
"state": "ENABLED",
"cloudTags": [],
"queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
"statusChangedAt": "Thu Aug 19 14:39:40 UTC 2021",
"description": "",
"maxConcurrency": 4

Engines Attributes

id

String (UUID)

Unique identifier of an engine.

Example e1cb20ff-a234-46f1-9f97-550e2bb5310f


name

String

The user-defined name for an engine.

Example firstEngine


size

String

The size of the engine. Each engine replica that is created will be of this size.

Enum XX_SMALL_V1, X_SMALL_V1, SMALL_V1, MEDIUM_V1, LARGE_V1, X_LARGE_V1, XX_LARGE_V1, XXX_LARGE_V1

Example SMALL_V1


activeReplicas

integer

The number of engine replicas that are currently active.

Example 0


minReplicas

integer

The minimum number of engine replicas that will be enabled at any given time.

Example 0


maxReplicas

integer

The maximum number of engine replicas that will be enabled at any given time.

Example 10


autoStopDelaySeconds

integer

The time (in seconds) that auto stop is delayed.

Example 300


queueTimeLimitSeconds

integer

The maximum amount of time (in seconds) that a query will wait in the engine’s queue before it is canceled.

Note: The typical time to start a new replica is two minutes (120 seconds), so we recommend setting this to more than 120 seconds. Changing this setting does not affect queries that are either currently running or already in the queue.

Example 300


runtimeLimitSeconds

integer

The maximum amount of time (in seconds) that a query can run before being terminated.

Example 0


drainTimeLimitSeconds

integer

The maximum amount of time (in seconds) that an engine replica will continue to run after the engine is resized, disabled, or deleted before it is terminated and the running queries terminate.

Example 0


state

String

The current state of the engine.

Enum DELETING, DISABLED, DISABLING, ENABLED, ENABLING, INVALID

Example ENABLED


cloudTags

object

AWS tags.

Example {"key": "dremio", "value": "test"}


queriedAt

String

The date and time that the engine was last used to execute a query.

Example Thu Jan 01 00:00:00 UTC 2021


statusChangedAt

String

The date and time (in UTC time) that the state of the engine changed.

Example Thu Aug 19 14:39:40 UTC 2021


description

String

The description for the engine.

Example engine for sales workloads


maxConcurrency

integer

The maximum number of concurrent queries that an engine replica can run.

Example 4


cloudTags

key

String

The key identifier for the tag.

Example dremio


value

String

The value of the tag.

Example test


Listing All Engines

Returns a list of all the engines in the specified project and the metadata for each engine.

Listing All Engines
GET /v0/projects/{project-id}/engines

Parameters

project-id

path

String (UUID)

Unique identifier of the project you want to list engines for.

Example Request
curl -X GET 'https://api.dremio.cloud/v0/projects/02d36975-73eb-47ed-9bb5-de73060380f6/engines' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
Example Response
{
[
{
"id": "e1cb20ff-a234-46f1-9f97-550e2bb5310f",
"name": "firstEngine",
"size": "XX_SMALL_V1",
"activeReplicas": 0,
"minReplicas": 0,
"maxReplicas": 10,
"autoStopDelaySeconds": 300,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 0,
"state": "ENABLED",
"cloudTags": [],
"queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
"statusChangedAt": "Thu Aug 19 14:39:40 UTC 2021",
"description": "",
"maxConcurrency": 4
},
{
"id": "a99eb46a-8121-4b79-b2ed-917e07f96092",
"name": "preview",
"size": "XX_SMALL_V1",
"activeReplicas": 0,
"minReplicas": 0,
"maxReplicas": 1,
"autoStopDelaySeconds": 180,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 0,
"state": "ENABLED",
"cloudTags": [],
"queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
"statusChangedAt": "Fri Aug 20 03:58:54 UTC 2021",
"description": "",
"maxConcurrency": 4
}
]

Responses

200

OK

400

Bad Request

401

Unauthorized


Adding an Engine

Add an engine to the cloud account that is associated with the specified project.

Adding an Engine
POST /v0/projects/{project-id}/engines

Parameters

Engine

project-id

path

String (UUID)

Unique identifier of the execution engine.


name

body

String

The user-defined name for an engine.


id

body

string (UUID)

Unique identifier of an engine.


size

body

String

The size of the engine that you want to create. Each engine replica that is created will be of this size.

Enum XX_SMALL_V1, X_SMALL_V1, SMALL_V1, MEDIUM_V1, LARGE_V1, X_LARGE_V1, XX_LARGE_V1, XXX_LARGE_V1

minReplicas

body

integer

The minimum number of engine replicas that will be enabled at any given time.


maxReplicas

body

integer

The maximum number of engine replicas that will be enabled at any given time.


autoStopDelaySeconds

body

integer

The time (in seconds) that auto stop is delayed.


queueTimeLimitSeconds

body

integer

The maximum amount of time (in seconds) that a query will wait in the engine’s queue before it is canceled. This should not be set to less than two minutes (120 seconds), the typical time it takes to start a new replica. Changing this setting does not affect queries that are currently running or in the queue.


runtimeLimitSeconds

body

integer

The maximum amount of time (in seconds) that a query can run before being terminated.


drainTimeLimitSeconds

body

integer

The maximum amount of time (in seconds) that an engine replica will continue to run after the engine is resized, disabled, or deleted before it is terminated and the running queries terminate.


cloudTags

body

object

Optional

AWS tags.


description

body

String

Optional

The description for the engine.


maxConcurrency

body

integer

The maximum number of concurrent queries that an engine replica can handle.


requestId

body

string (UUID)

The user-defined idempotency key, which is a unique value generated by the user that the server uses to register consequent retries of the same request. Generate the request ID by using a UUID generator tool.

For more information, read idempotent requests.

cloudTags

key

body

String

The key identifier for the tag.


value

body

String

The value of the tag.

Example Request
curl -X POST 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines' \
-H 'Authorization: Bearer <personal access token>' \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-d '{
"name": "newengine",
"size": "XX_SMALL_V1",
"minReplicas": 0,
"maxReplicas": 1,
"autoStopDelaySeconds": 7200,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 1800,
"cloudTags": [
{
"key": "tag1",
"value": "value1"
}
],
"description": "engine description",
"maxConcurrency": 2,
"requestId": "d9904076-01db-11ec-9a03-0242ac130003"
}'
Example Response
{
"id": "168acd56-16a0-47df-91f8-5487a8fccfc4"
}

Responses

200

OK

400

Bad Request

401

Unauthorized

405

Method Not Allowed

500

Internal Server Error


Retrieving an Engine

Get the metadata for an existing engine by providing the project and engine IDs.

Retrieving an Engine
GET /v0/projects/{project-id}/engines/{id}

Parameters

project-id

path

string (UUID)

Unique identifier of the execution engine.


id

path

String

The ID of the engine for which you want to retrieve metadata.

Example Request
curl -X GET 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
Example Response
{
"id": "168acd56-16a0-47df-91f8-5487a8fccfc4",
"name": "newengine",
"size": "XX_SMALL_V1",
"activeReplicas": 0,
"minReplicas": 0,
"maxReplicas": 1,
"autoStopDelaySeconds": 7200,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 1800,
"state": "ENABLED",
"cloudTags": [
{
"key": "tag1",
"value": "value1"
}
],
"queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
"statusChangedAt": "Fri Aug 20 17:29:18 UTC 2021",
"description": "engine description",
"maxConcurrency": 2
}

Responses

200

OK

400

Engine does not exist

401

Unauthorized

500

Internal Server Error


Modifying an Engine

Modify the engine by providing the project and engine IDs. Parameters that are not provided will be left unchanged. The engine name cannot be modified.

Modifying an Engine
PUT /v0/projects/{project-id}/engines/{id}

Parameters

project-id

path

string (UUID)

Unique identifier of the execution engine.


id

path

String (UUID)

The ID of the engine for which you want to retrieve metadata.


size

body

String

Optional

The size of the engine that you want to change to. Each engine replica that is modified will be of this size.

Enum XX_SMALL_V1, X_SMALL_V1, SMALL_V1, MEDIUM_V1, LARGE_V1, X_LARGE_V1, XX_LARGE_V1, XXX_LARGE_V1

minReplicas

body

integer

Optional

The minimum number of engine replicas that will be enabled at any given time.


maxReplicas

body

integer

Optional

The maximum number of engine replicas that will be enabled at any given time.


autoStopDelaySeconds

body

integer

Optional

The time (in seconds) that auto stop is delayed.


queueTimeLimitSeconds

body

integer

Optional

The maximum amount of time (in seconds) that a query will wait in the engine’s queue before it is canceled. This should not be set to less than two minutes (120 seconds), the typical time it takes to start a new replica. Changing this setting does not affect queries that are either currently running or already in the queue.


runtimeLimitSeconds

body

integer

Optional

The maximum amount of time (in seconds) that a query can run before being terminated.


drainTimeLimitSeconds

body

integer

Optional

The maximum amount of time (in seconds) that an engine replica will continue to run after the engine is resized, disabled, or deleted before it is terminated and the running queries terminate.


cloudTags

body

object

Optional

AWS tags.


description

body

String

Optional

The description for the engine you are modifying.


maxConcurrency

body

integer

Optional

The maximum number of concurrent queries that an engine replica can run.

cloudTags

key

body

String

Optional

The key identifier for the tag.


value

body

String

Optional

The value of the tag.

Example Request
curl -X PUT 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json' \
-d '{
"description": "modified engine description",
"size": "XX_SMALL_V1",
"autoStopDelaySeconds": 7200,
"drainTimeLimitSeconds": 1800,
"maxConcurrency": 2,
"maxReplicas": 1,
"minReplicas": 0,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"cloudTags": [ ]
}'
Example Response
No response

Responses

204

No Content

400

Bad Request

401

Unauthorized

500

Internal Server Error


Enabling an Engine

Enable an engine.

Enabling an Engine
PUT /v0/projects/{project-id}/engines/{id}/enable

Parameters

project-id

path

string (UUID)

Unique identifier of the execution engine.


id

path

String

The ID of the engine that you want to enable.


Example Request
curl -X PUT 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4/enable' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
Example Response
No response

Responses

204

No Content

401

Unauthorized

500

Internal Server Error


Disabling an Engine

Disable an engine.

Disabling an Engine
PUT /v0/projects/{project-id}/engines/{id}/disable

Parameters

project-id

path

string (UUID)

Unique identifier of the execution engine.


id

path

String

The ID of the engine that you want to disable.


Example Request
curl -X PUT 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4/disable' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
Example Response
No response

Responses

204

No Content

401

Unauthorized

500

Internal Server Error


Deleting an Engine

Delete an engine by providing the project and engine ID.

Deleting an Engine
DELETE /v0/projects/{project-id}/engines/{id}

Parameters

project-id

path

string (UUID)

Unique identifier of the execution engine.


id

path

String

The ID of the engine that you want to delete.


Example Request
curl -X DELETE 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
Example Response
No response

Responses

204

No Content

401

Unauthorized

500

Internal Server Error