On this page

    AWS Edition Engine REST API

    Dremio administrators use the AWS Edition REST API to create, update, and delete engines for Dremio AWS Edition. Before performing any of the following procedures, first generate an API token as described in Authentication and open a Dremio project.

    Listing Engines for a Project

    Use the GET http://{DREMIO_ORIGIN}/apiv2/provision/clusters endpoint to retrieve a list of all engines in a Dremio project, as shown in the following example:

    curl -X GET 'http://18.237.234.109:9047/apiv2/provision/clusters' -H 'Accept: application/json' -H 'Content-Type:application/json' -H 'Authorization:_dremioapikey'

    Dremio returns a json document in the HTTP response body after a successful request:

    {
       "clusterList":[
          {
             "id":"82a37f06-ec34-4eaf-b30c-354837795b0c",
             "tag":"ryFMjQ2KjjA=",
             "currentState":"STOPPED",
             "stateChangeTime":1611183365149,
             "clusterType":"EC2",
             "name":"default",
             "dynamicConfig":{
                "containerCount":2
             },
             "containers":{
                "runningList":[           
                ],
                "decommissioningCount":0,
                "provisioningCount":0,
                "pendingCount":0,
                "disconnectedList":[
                   
                ],
                "pendingList":[
                   
                ],
                "decommissioningList":[
                   
                ]
             },
             "desiredState":"STOPPED",
             "awsProps":{
                "vpc":"vpc-007bd9db0278fdf7f",
                "subnetId":"my-subnet",
                "useClusterPlacementGroup":true,
                "disablePublicIp":false,
                "sshKeyName":"mykey",
                "instanceType":"m5d.8xlarge",
                "connectionProps":{
                   "authMode":"AUTO",
                   "region":"us-west-2"
                },
                "awsTags":[
                   
                ]
             },
             "shutdownInterval":7200000,
             "isAllowAutoStart":true,
             "isAllowAutoStop":true
          }
       ]
    }
    

    Retrieving Information about an Engine

    Use the GET http://{DREMIO_ORIGIN}/apiv2/provision/cluster/{CLUSTER-ID} endpoint to retrieve detailed information about a specific engine, as shown in the following example:

    curl -X GET 'http://18.237.234.109:9047/apiv2/provision/cluster/82a37f06-ec34-4eaf-b30c-354837795b0c'
       -H 'Accept: application/json' -H 'Content-Type:application/json' -H 'Authorization:_dremioapikey'
    

    Dremio returns a json document in the HTTP response body after a successful request:

    {
       "id":"82a37f06-ec34-4eaf-b30c-354837795b0c",
       "tag":"ryFMjQ2KjjA=",
       "currentState":"STOPPED",
       "stateChangeTime":1611183365149,
       "clusterType":"EC2",
       "name":"default",
       "dynamicConfig":{
          "containerCount":2
       },
       "containers":{
          "runningList":[
             
          ],
          "decommissioningCount":0,
          "provisioningCount":0,
          "pendingCount":0,
          "disconnectedList":[
             
          ],
          "pendingList":[
             
          ],
          "decommissioningList":[
             
          ]
       },
       "desiredState":"STOPPED",
       "awsProps":{
          "vpc":"vpc-007bd9db0278fdf7f",
          "subnetId":"subnet-09daa36b480c4d9b7",
          "useClusterPlacementGroup":true,
          "disablePublicIp":false,
          "sshKeyName":"my-key",
          "instanceType":"m5d.8xlarge",
          "connectionProps":{
             "authMode":"AUTO",
             "region":"us-west-2"
          },
          "awsTags":[
             
          ]
       },
       "shutdownInterval":7200000,
       "isAllowAutoStart":true,
       "isAllowAutoStop":true
    }
    

    Retrieving AWS Engine Defaults

    Use the GET http://{DREMIO_ORIGIN}:9047/aws/defaults endpoint to retreive the default values for an engine. Many of these values are required in the HTTP request body for creating and updating engines.

    curl -X GET 'http://18.237.234.109:9047/apiv2/provision/aws/defaults
       -H Accept: application/json' -H 'Authorization:_dremioapikey'
    

    Dremio returns a json document such as the following in the HTTP response body:

    {
      "vpc": "my-vpc",
      "subnetId": "my-subnet",
      "useClusterPlacementGroup": true,
      "disablePublicIp": false,
      "sshKeyName": "my-key",
      "instanceType": "m5d.8xlarge",
      "connectionProps": {
        "authMode": "AUTO",
        "region": "us-west-2"
      },
      "awsTags": []
    

    Creating an Engine

    Use the POST http://{DREMIO_ORIGIN}/provision/cluster/{CLUSTER-ID} endpoint to create an engine.

    curl -X POST 'http://18.237.234.109:9047/apiv2/provision/cluster'
       -H 'Accept: application/json' -H 'Content-Type:application/json'
       -H 'Authorization:_dremioapikey' --data '
    {
       "clusterType": "EC2",
       "name": "name",
       "isAllowAutoStop": true,
       "isAllowAutoStart": true,
       "shutdownInterval": 600000,
       "dynamicConfig": {
          "containerCount": 1
       },
       "awsProps": {
          "vpc": "myvpc",
          "subnetId": "mysubnet",
          "useClusterPlacementGroup": true,
          "disablePublicIp": false,
          "sshKeyName": "my-key",
          "instanceType": "m5d.2xlarge",
          "connectionProps": {
             "authMode": "AUTO",
             "region": "us-west-2"
          }
       }
    }'
    

    Dremio returns a json document in the HTTP response body after a successful request:

    {
        "id": "8872b8b2-1f68-44b9-84ce-9c50e06431e0",
        "tag": "VBuslFKCxtg=",
        "currentState": "STARTING",
        "stateChangeTime": 1611968046413,
        "clusterType": "EC2",
        "name": "name",
        "dynamicConfig": {
            "containerCount": 1
        },
        "containers": {
            "runningList": [],
            "decommissioningCount": 0,
            "provisioningCount": 0,
            "pendingCount": 1,
            "disconnectedList": [],
            "pendingList": [
                {
                    "containerId": "i-02d94fb6b2cde3cf8",
                    "containerPropertyList": [
                        {
                            "key": "instanceId",
                            "value": "i-02d94fb6b2cde3cf8"
                        }
                    ]
                }
            ],
            "decommissioningList": []
        },
        "desiredState": "RUNNING",
        "awsProps": {
            "vpc": "vpc-849000e1",
            "subnetId": "subnet-f8d476a1",
            "useClusterPlacementGroup": true,
            "disablePublicIp": false,
            "sshKeyName": "my-key",
            "instanceType": "m5d.2xlarge",
            "connectionProps": {
                "authMode": "AUTO",
                "region": "us-west-2"
            },
            "awsTags": []
        },
        "shutdownInterval": 600000,
        "isAllowAutoStart": true,
        "isAllowAutoStop": true
    }
    

    Updating Engine Configuration

    Use the PUT http://{DREMIO_ORIGIN}:9047/provision/cluster/{CLUSTER-ID} endpoint to update the configuration of an engine in a cluster. The request must contain the following HTTP request body, as shown in the following example. The request body must contain all of the following parameters.

    {
       "id":"{CLUSTER-ID}",
       "tag":"TAG",
       "desiredState":"[RUNNING,STOPPED]",
       "clusterType":"EC2",
       "dynamicConfig":{
          "containerCount":"{CONTAINER-COUNT}"
       },
       "awsProps":{
          "vpc":"{VPC-ID}",
          "subnetId":"{SUBNET-ID}",
          "sshKeyName":"SSH-KEYNAME",
          "connectionProps":{
             "authMode":"AUTO"
          }
       }
    }
    

    Use the GET http://{DREMIO_ORIGIN}:9047/aws/defaults endpoint to view all parameters. The following example updates the shutdown interval from 600000 to 120000:

    curl -X PUT 'http://{DREMIO_ORIGIN:9047/apiv2/provision/cluster/{CLUSTER-ID}' -H 'Accept: application/json' -H 'Content-Type:application/json' -H 'Authorization:_dremio97g0qje3cgjt8vmi77ri6lwtk8' --data '
    {
      "id": "cc76d7ae-5476-44b8-8316-26743327662g",
      "tag": "ih6o2hxO2AI=",
      "currentState": "STOPPED",
      "stateChangeTime": 1611691702661,
      "clusterType": "EC2",
      "name": "js",
      "dynamicConfig": {
        "containerCount": 1
      },
      "containers": {
        "runningList": [],
        "decommissioningCount": 0,
        "provisioningCount": 0,
        "pendingCount": 0,
        "disconnectedList": [],
        "pendingList": [],
        "decommissioningList": []
      },
      "desiredState": "STOPPED",
      "awsProps": {
        "vpc": "my-vpc",
        "subnetId": "my-subnet",
        "useClusterPlacementGroup": true,
        "disablePublicIp": false,
        "sshKeyName": "my-key",
        "instanceType": "m5d.8xlarge",
        "connectionProps": {
          "authMode": "AUTO",
          "region": "us-west-2"
        },
        "awsTags": []
      },
      "shutdownInterval": 120000,
      "isAllowAutoStart": true,
      "isAllowAutoStop": true
    }'
    

    Dremio returns a json document in the HTTP response body after a successful request:

    {
      "id": "cc76d7ae-5476-44b8-8316-267433276562g",
      "tag": "7IjCt36qTgo=",
      "currentState": "STARTING",
      "stateChangeTime": 1611711836819,
      "clusterType": "EC2",
      "name": "js",
      "dynamicConfig": {
        "containerCount": 1
      },
      "containers": {
        "runningList": [],
        "decommissioningCount": 0,
        "provisioningCount": 0,
        "pendingCount": 1,
        "disconnectedList": [],
        "pendingList": [
          {
            "containerId": "i-0df3e31ca87cc31b6",
            "containerPropertyList": [
              {
                "key": "instanceId",
                "value": "i-0df3e31ca87cc31b6"
              }
            ]
          }
        ],
        "decommissioningList": []
      },
      "desiredState": "RUNNING",
      "awsProps": {
        "vpc": "vpc-007bd9db0278fdf7",
        "subnetId": "my-subnet",
        "useClusterPlacementGroup": true,
        "disablePublicIp": false,
        "sshKeyName": "my-key",
        "instanceType": "m5d.8xlarge",
        "connectionProps": {
          "authMode": "AUTO",
          "region": "us-west-2"
        },
        "awsTags": []
      },
      "shutdownInterval": 120000,
      "isAllowAutoStart": true,
      "isAllowAutoStop": true
    }
    

    Starting and Stopping an Engine

    Use the PUT http://{DREMIO_ORIGIN}:9047/provision/cluster/{CLUSTER-ID} endpoint to stop and restart an engine. To stop an engine, the HTTP request body must set the value of desiredState to STOPPED. To restart the engine, set the value of desiredState to RUNNING, as shown in the following example:

    curl -X PUT 'http://{DREMIO_ORIGIN}/provision/cluster/93a37f06-ec34-4eaf-b30c-354837795b0c'
       -H 'Accept: application/json" -H 'Content-Type:application/json' 
       -H 'Authorization:_dremio{API-TOKEN}' --data '
    {
       "id":"93a37f06-ec34-4eaf-b30c-354837795c1d",
       "tag":"mytag",
       "desiredState":"RUNNING",
       "clusterType":"EC2",
       "dynamicConfig":{
          "containerCount":2
       },
       "awsProps":{
          "vpc":"my-vpc",
          "subnetId":"my-subnet",
          "sshKeyName":"my-key",
          "connectionProps":{
             "authMode":"AUTO"
          }
       }
    }
    '
    

    Dremio returns a json document with the following structure after a successful request:

    {
      "id": "02cc318f-7807-4466-88c6-0e3355db848f",
      "tag": "mcB9qWZPiuc=",
      "currentState": "STOPPING",
      "stateChangeTime": 1611709681032,
      "clusterType": "EC2",
      "name": "default",
      "dynamicConfig": {
        "containerCount": 1
      },
      "containers": {
        "runningList": [
          {
            "containerId": "i-0172dd52ce7c9db95",
            "containerPropertyList": [
              {
                "key": "instanceId",
                "value": "i-0172dd52ce7c9db95"
              },
              {
                "key": "host",
                "value": "ip-174-57-188-194.us-west-2.compute.internal"
              },
              {
                "key": "privateIp",
                "value": "174-57-188-194"
              },
              {
                "key": "publicIp",
                "value": "43.322.63.57"
              },
              {
                "key": "launchTime",
                "value": "2021-01-27T00:08:16Z"
              }
            ]
          }
        ],
        "decommissioningCount": 0,
        "provisioningCount": 0,
        "pendingCount": 0,
        "disconnectedList": [],
        "pendingList": [],
        "decommissioningList": []
      },
      "desiredState": "RUNNING",
      "awsProps": {
        "vpc": "my-vpc",
        "subnetId": "subnet-09daa36b480c4d9b7",
        "useClusterPlacementGroup": true,
        "disablePublicIp": false,
        "sshKeyName": "my-key",
        "instanceType": "m5d.2xlarge",
        "connectionProps": {
          "authMode": "AUTO",
          "region": "us-west-2"
        },
        "awsTags": []
      },
      "shutdownInterval": 300000,
      "isAllowAutoStart": true,
      "isAllowAutoStop": true
    }
    

    Deleting an Engine

    Use the DELETE 'http://{DREMIO_ORIGIN}/provision/cluster/{CLUSTER-ID} endpoint with no HTTP request body to delete an engine, as shown in the following example:

    curl -X DELETE 'http://{DREMIO_ORIGIN}/provision/cluster/93a37f06-ec34-4eaf-b30c-354837795b0c'
    

    Dremio returns 1 after a successful request.