Engine
Dremio Arctic enables you to run optimization jobs to help you manage the accumulation of the data files that occurs through DML operations. This API allows you to add, update, and retrieve engines that run the optimization jobs.
Engine Object{
"credentials": {
"type": "ACCESS_KEY",
"accessKeyId": "8BDLDGT42KJ4JEXAMPLE"
},
"cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
"logStorageLocation": "usrsdre-test-west1/usrsdre/",
"engineSize": "XX_SMALL_V1",
"state": "ENABLED",
"errorMessage": null
}
Engine Attributes
Object
Authentication credentials used by all jobs and schedules on the catalog.
Example { "type": "ACCESS_KEY", "accessKeyId": "8BDLDGT42KJ4JEXAMPLE", "secretAccessKey": "EXAMPLEtnFEMI/EXAMENG/bPxRfiCYEXAMPLEKEY" }
cloudId
String (UUID)
Unique identifier for the cloud in which to run jobs and schedules on the catalog.
Example bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9
logStorageLocation
String
Storage location for audit logs that Dremio engines create.
Example usrsdre-test-west1/usrsdre/
engineSize
String
Engine size used.
EnumXX_SMALL_V1, X_SMALL_V1, SMALL_V1, MEDIUM_V1, LARGE_V1, X_LARGE_V1, XX_LARGE_V1, XXX_LARGE_V1
Example XX_SMALL_V1
state
String
Status of the engine.
EnumCREATING, ENABLED, DISABLED, UPDATING, FAILED
Example ENABLED
errorMessage
String
For issues with the engine, a description of the problem.
Example null
credentials
Object Attributes
type
String
Type of authentication credential used by all jobs and schedules on the catalog.
EnumACCESS_KEY, IAM_ROLE
Example ACCESS_KEY
accessKeyId
String
For the ACCESS_KEY type, the access key ID used by all jobs and schedules on the catalog.
Example 8BDLDGT42KJ4JEXAMPLE
externalId
String
For the IAM_ROLE type, the external ID used by all jobs and schedules on the catalog.
Example 55637959
instanceProfileArn
String
For the IAM_ROLE type, the instance profile Amazon Resource Name (ARN) ID used by all jobs and schedules on the catalog.
Example arn:partition:service:region:account:resource
roleArn
String
For the IAM_ROLE type, the role Amazon Resource Name (ARN) used by all jobs and schedules on the catalog.
Example arn:partition:service:region:account:resource
Adding or Updating an Engine
Adds or modifies an engine for the specified Arctic catalog that will run optimization jobs.
Method and URLPOST /v0/arctic/catalogs/{catalogId}/engine
Parameters
catalogId
path
String (UUID)
Unique identifier for the cloud in which to run jobs and schedules on the catalog.
Example 1a234bc5-6789-46d7-b9cb-d236e195b34a
body
Object
Authentication credentials to use for all jobs and schedules on the catalog.
Example { "type": "ACCESS_KEY", "accessKeyId": "8BDLDGT42KJ4JEXAMPLE", "secretAccessKey": "EXAMPLEtnFEMI/EXAMENG/bPxRfiCYEXAMPLEKEY" }
cloudId
body
String (UUID)
Unique identifier for the cloud in which to run jobs and schedules on the catalog.
Example bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9
logStorageLocation
body
String
Storage location for audit logs that Dremio engines create.
Example usrsdre-test-west1/usrsdre/
engineSize
body
String
Engine size to use.
EnumXX_SMALL_V1, X_SMALL_V1, SMALL_V1, MEDIUM_V1, LARGE_V1, X_LARGE_V1, XX_LARGE_V1, XXX_LARGE_V1
Example XX_SMALL_V1
credentials
Object Parameters
type
body
String
Type of authentication credential to use for all jobs and schedules on the catalog.
EnumACCESS_KEY, IAM_ROLE
Example ACCESS_KEY
accessKeyId
body
String
For the ACCESS_KEY type, the access key ID to use for all jobs and schedules on the catalog.
Example 8BDLDGT42KJ4JEXAMPLE
secretAccessKey
body
String
For the ACCESS_KEY type, the secret access key to use for all jobs and schedules on the catalog.
Example 94yLpEH2fhX3CrfrgE93742d8YyDr89tNEXAMPLE
externalId
body
String
Optional
For the IAM_ROLE type, the external ID to use for all jobs and schedules on the catalog.
Example 55637959
instanceProfileArn
body
String
For the IAM_ROLE type, the instance profile Amazon Resource Name (ARN) ID to use for all jobs and schedules on the catalog.
Example arn:partition:service:region:account:resource
roleArn
body
String
For the IAM_ROLE type, the role Amazon Resource Name (ARN) to use for all jobs and schedules on the catalog.
Example arn:partition:service:region:account:resource
Example Request
curl -X POST 'https://api.dremio.cloud/v0/arctic/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/engine \
--header 'Authorization: Bearer <personal access token>' \
--header 'Content-Type: application/json' \
--data-raw '{
"cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
"credentials": {
"type": "ACCESS_KEY",
"accessKeyId": "8BDLDGT42KJ4JEXAMPLE",
"secretAccessKey": "94yLpEH2fhX3CrfrgE93742d8YyDr89tNEXAMPLE"
},
"engineSize": "XX_SMALL_V1",
"logStorageLocation": "usrsdre-test-west1/usrsdre/"
}'
{
"cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
"credentials": {
"accessKeyId": "8BDLDGT42KJ4JEXAMPLE",
"type": "ACCESS_KEY"
},
"engineSize": "XX_SMALL_V1",
"logStorageLocation": "usrsdre-test-west1/usrsdre/",
"state": "CREATING"
}
Responses
200
OK
400
Bad Request
404
Not Found
415
Unsupported Media Type
500
Internal Server Error
Retrieving an Engine
Retrieve the engine the Arctic catalog uses.
Method and URLGET /v0/arctic/catalogs/{catalogId}/engine
Parameters
catalogId
path
String (UUID)
Unique identifier for the Arctic catalog whose engine you want to retrieve.
Example 1a234bc5-6789-46d7-b9cb-d236e195b34a
curl -X GET 'https://api.dremio.cloud/v0/catalogs/1a234bc5-6789-46d7-b9cb-d236e195b34a/engine' \
--header 'Authorization: Bearer <personal access token>' \
--header 'Content-Type: application/json'
{
"credentials": {
"type": "ACCESS_KEY",
"accessKeyId": "8BDLDGT42KJ4J5SNSL2Z"
},
"cloudId": "bc5787c6-e9bf-4159-a1b0-93d3cc6f19e9",
"logStorageLocation": "usrsdre-test-west1/usrsdre/",
"engineSize": "XX_SMALL_V1",
"state": "ENABLED",
"errorMessage": null
}
Responses
200
OK
204
No Content
404
Not Found
415
Unsupported Media Type
500
Internal Server Error