Engines
An engine represents a Dremio Cloud entity that manages compute resources. Each engine has one or more replicas that are created for executing queries. An engine replica is a logical group of executors defined by the engine size. This API allows you to list existing engines and add, modify, enable/disable, and delete engines.
Engine Object "id": "e1cb20ff-a234-46f1-9f97-550e2bb5310f",
"name": "firstEngine",
"size": "XX_SMALL_V1",
"activeReplicas": 0,
"minReplicas": 0,
"maxReplicas": 10,
"autoStopDelaySeconds": 300,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 0,
"state": "ENABLED",
"cloudTags": [],
"queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
"statusChangedAt": "Thu Aug 19 14:39:40 UTC 2021",
"description": "",
"maxConcurrency": 4
Engine Attributes
id
String (UUID)
Unique identifier of an engine.
Example
e1cb20ff-a234-46f1-9f97-550e2bb5310f
name
String
The user-defined name for an engine.
Example
firstEngine
size
String
The size of the engine. Each engine replica that is created will be of this size.
Enum
XX_SMALL_V1
,
X_SMALL_V1
,
SMALL_V1
,
MEDIUM_V1
,
LARGE_V1
,
X_LARGE_V1
,
XX_LARGE_V1
,
XXX_LARGE_V1
Example
SMALL_V1
activeReplicas
integer
The number of engine replicas that are currently active.
Example 0
minReplicas
integer
The minimum number of engine replicas that will be enabled at any given time.
Example 0
maxReplicas
integer
The maximum number of engine replicas that will be enabled at any given time.
Example 10
autoStopDelaySeconds
integer
The time (in seconds) that auto stop is delayed.
Example 300
queueTimeLimitSeconds
integer
The maximum amount of time (in seconds) that a query will wait in the engine’s queue before it is canceled.
Note:
The typical time to start a new replica is two minutes (120 seconds), so we recommend setting this to more than 120 seconds. Changing this setting does not affect queries that are either currently running or already in the queue.
Example 300
runtimeLimitSeconds
integer
The maximum amount of time (in seconds) that a query can run before being terminated.
Example 0
drainTimeLimitSeconds
integer
The maximum amount of time (in seconds) that an engine replica will continue to run after the engine is resized, disabled, or deleted before it is terminated and the running queries terminate.
Example 0
state
String
The current state of the engine.
Enum
DELETING
,
DISABLED
,
DISABLING
,
ENABLED
,
ENABLING
,
INVALID
Example
ENABLED
object
AWS tags.
Example {"key": "dremio", "value": "test"}
queriedAt
String
The date and time that the engine was last used to execute a query.
Example
Thu Jan 01 00:00:00 UTC 2021
statusChangedAt
String
The date and time (in UTC time) that the state of the engine changed.
Example
Thu Aug 19 14:39:40 UTC 2021
description
String
The description for the engine.
Example
engine for sales workloads
maxConcurrency
integer
The maximum number of concurrent queries that an engine replica can run.
Example 4
cloudTags
key
String
The key identifier for the tag.
Example
dremio
value
String
The value of the tag.
Example
test
Listing All Engines
Returns a list of all the engines in the specified project and the metadata for each engine.
Listing All EnginesGET /v0/projects/{project-id}/engines
Parameters
project-id
path
String (UUID)
Unique identifier of the project you want to list engines for.
curl -X GET 'https://api.dremio.cloud/v0/projects/02d36975-73eb-47ed-9bb5-de73060380f6/engines' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
{
[
{
"id": "e1cb20ff-a234-46f1-9f97-550e2bb5310f",
"name": "firstEngine",
"size": "XX_SMALL_V1",
"activeReplicas": 0,
"minReplicas": 0,
"maxReplicas": 10,
"autoStopDelaySeconds": 300,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 0,
"state": "ENABLED",
"cloudTags": [],
"queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
"statusChangedAt": "Thu Aug 19 14:39:40 UTC 2021",
"description": "",
"maxConcurrency": 4
},
{
"id": "a99eb46a-8121-4b79-b2ed-917e07f96092",
"name": "preview",
"size": "XX_SMALL_V1",
"activeReplicas": 0,
"minReplicas": 0,
"maxReplicas": 1,
"autoStopDelaySeconds": 180,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 0,
"state": "ENABLED",
"cloudTags": [],
"queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
"statusChangedAt": "Fri Aug 20 03:58:54 UTC 2021",
"description": "",
"maxConcurrency": 4
}
]
Responses
200
OK
400
Bad Request
401
Unauthorized
Adding an Engine
Add an engine to the cloud account that is associated with the specified project.
Adding an EnginePOST /v0/projects/{project-id}/engines
Parameters
Engine
project-id
path
String (UUID)
Unique identifier of the execution engine.
name
query
String
The user-defined name for an engine.
id
query
string (UUID)
Unique identifier of an engine.
size
query
String
The size of the engine that you want to create. Each engine replica that is created will be of this size.
Enum
XX_SMALL_V1
,
X_SMALL_V1
,
SMALL_V1
,
MEDIUM_V1
,
LARGE_V1
,
X_LARGE_V1
,
XX_LARGE_V1
,
XXX_LARGE_V1
minReplicas
query
integer
The minimum number of engine replicas that will be enabled at any given time.
maxReplicas
query
integer
The maximum number of engine replicas that will be enabled at any given time.
autoStopDelaySeconds
query
integer
The time (in seconds) that auto stop is delayed.
queueTimeLimitSeconds
query
integer
The maximum amount of time (in seconds) that a query will wait in the engine’s queue before it is canceled. This should not be set to less than two minutes (120 seconds), the typical time it takes to start a new replica. Changing this setting does not affect queries that are currently running or in the queue.
runtimeLimitSeconds
query
integer
The maximum amount of time (in seconds) that a query can run before being terminated.
drainTimeLimitSeconds
query
integer
The maximum amount of time (in seconds) that an engine replica will continue to run after the engine is resized, disabled, or deleted before it is terminated and the running queries terminate.
query
object
Optional
AWS tags.
description
query
String
Optional
The description for the engine.
maxConcurrency
query
integer
The maximum number of concurrent queries that an engine replica can handle.
requestId
query
string (UUID)
The user-defined idempotency key, which is a unique value generated by the user that the server uses to register consequent retries of the same request. Generate the request ID by using a UUID generator tool. For more information, read idempotent requests.
cloudTags
key
query
String
The key identifier for the tag.
value
query
String
The value of the tag.
curl -X POST 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines' \
-H 'Authorization: Bearer <personal access token>' \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-d '{
"name": "newengine",
"size": "XX_SMALL_V1",
"minReplicas": 0,
"maxReplicas": 1,
"autoStopDelaySeconds": 7200,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 1800,
"cloudTags": [
{
"key": "tag1",
"value": "value1"
}
],
"description": "engine description",
"maxConcurrency": 2,
"requestId": "d9904076-01db-11ec-9a03-0242ac130003"
}'
{
"id": "168acd56-16a0-47df-91f8-5487a8fccfc4"
}
Responses
200
OK
400
Bad Request
401
Unauthorized
405
Method Not Allowed
500
Internal Server Error
Retrieving an Engine
Get the metadata for an existing engine by providing the project and engine IDs.
Retrieving an EngineGET /v0/projects/{project-id}/engines/{id}
Parameters
project-id
path
string (UUID)
Unique identifier of the execution engine.
id
path
String
The ID of the engine for which you want to retrieve metadata.
curl -X GET 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
{
"id": "168acd56-16a0-47df-91f8-5487a8fccfc4",
"name": "newengine",
"size": "XX_SMALL_V1",
"activeReplicas": 0,
"minReplicas": 0,
"maxReplicas": 1,
"autoStopDelaySeconds": 7200,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"drainTimeLimitSeconds": 1800,
"state": "ENABLED",
"cloudTags": [
{
"key": "tag1",
"value": "value1"
}
],
"queriedAt": "Thu Jan 01 00:00:00 UTC 2021",
"statusChangedAt": "Fri Aug 20 17:29:18 UTC 2021",
"description": "engine description",
"maxConcurrency": 2
}
Responses
200
OK
400
Engine does not exist
401
Unauthorized
500
Internal Server Error
Modifying an Engine
Modify the engine by providing the project and engine IDs. Parameters that are not provided will be left unchanged. The engine name cannot be modified.
Modifying an EnginePUT /v0/projects/{project-id}/engines/{id}
Parameters
project-id
path
string (UUID)
Unique identifier of the execution engine.
id
path
String (UUID)
The ID of the engine for which you want to retrieve metadata.
size
query
String
Optional
The size of the engine that you want to change to. Each engine replica that is modified will be of this size.
Enum
XX_SMALL_V1
,
X_SMALL_V1
,
SMALL_V1
,
MEDIUM_V1
,
LARGE_V1
,
X_LARGE_V1
,
XX_LARGE_V1
,
XXX_LARGE_V1
minReplicas
query
integer
Optional
The minimum number of engine replicas that will be enabled at any given time.
maxReplicas
query
integer
Optional
The maximum number of engine replicas that will be enabled at any given time.
autoStopDelaySeconds
query
integer
Optional
The time (in seconds) that auto stop is delayed.
queueTimeLimitSeconds
query
integer
Optional
The maximum amount of time (in seconds) that a query will wait in the engine’s queue before it is canceled. This should not be set to less than two minutes (120 seconds), the typical time it takes to start a new replica. Changing this setting does not affect queries that are either currently running or already in the queue.
runtimeLimitSeconds
query
integer
Optional
The maximum amount of time (in seconds) that a query can run before being terminated.
drainTimeLimitSeconds
query
integer
Optional
The maximum amount of time (in seconds) that an engine replica will continue to run after the engine is resized, disabled, or deleted before it is terminated and the running queries terminate.
query
object
Optional
AWS tags.
description
query
String
Optional
The description for the engine you are modifying.
maxConcurrency
query
integer
Optional
The maximum number of concurrent queries that an engine replica can run.
cloudTags
key
query
String
Optional
The key identifier for the tag.
value
query
String
Optional
The value of the tag.
curl -X PUT 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json' \
-d '{
"description": "modified engine description",
"size": "XX_SMALL_V1",
"autoStopDelaySeconds": 7200,
"drainTimeLimitSeconds": 1800,
"maxConcurrency": 2,
"maxReplicas": 1,
"minReplicas": 0,
"queueTimeLimitSeconds": 300,
"runtimeLimitSeconds": 0,
"cloudTags": [ ]
}'
No response
Responses
204
No Content
400
Bad Request
401
Unauthorized
500
Internal Server Error
Enabling an Engine
Enable an engine.
Enabling an EnginePUT /v0/projects/{project-id}/engines/{id}/enable
Parameters
project-id
path
string (UUID)
Unique identifier of the execution engine.
id
path
String
The ID of the engine that you want to enable.
curl -X PUT 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4/enable' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
No response
Responses
204
No Content
401
Unauthorized
500
Internal Server Error
Disabling an Engine
Disable an engine.
Disabling an EnginePUT /v0/projects/{project-id}/engines/{id}/disable
Parameters
project-id
path
string (UUID)
Unique identifier of the execution engine.
id
path
String
The ID of the engine that you want to disable.
curl -X PUT 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4/disable' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
No response
Responses
204
No Content
401
Unauthorized
500
Internal Server Error
Deleting an Engine
Delete an engine by providing the project and engine ID.
Deleting an EngineDELETE /v0/projects/{project-id}/engines/{id}
Parameters
project-id
path
string (UUID)
Unique identifier of the execution engine.
id
path
String
The ID of the engine that you want to delete.
curl -X DELETE 'https://api.dremio.cloud/v0/projects/c683b988-aa93-439c-a9ac-0d7f72b53d23/engines/168acd56-16a0-47df-91f8-5487a8fccfc4' \
-H 'Authorization: Bearer <personal access token>' \
-H 'Content-Type: application/json'
No response
Responses
204
No Content
401
Unauthorized
500
Internal Server Error