Dremio Cloud
Dremio Software
 
On this page

    Managing Clouds

    A cloud represents a virtual environment (AWS) in which Dremio Cloud engines run and where the project metadata store is created and maintained. A cloud object is associated with a single AWS region in your AWS account. For information on supported regions and availability zones, see the Supported Regions page. A single cloud can be associated with many projects.

    Viewing All Clouds

    To view the clouds that are in a Dremio Cloud organization:

    1. Click the Organization This is the icon that represents the Organization. icon in the side navigation bar.
    2. Click the Settings This is the icon that represents the Organization settings. icon at the top of the Organization page.
    3. Select Clouds in the organization settings sidebar.

    Adding a Cloud Manually

    Follow the steps below to add a new cloud to your organization manually:

    1. In the top-right corner of the Clouds page, click the Add Cloud button.

    2. For Cloud Name, specify a name for your cloud that is unique within your organization.

    3. For Region, enter the region associated with the cloud account you are connecting to Dremio Cloud. For more information, see supported AWS regions.

    4. In the Compute Settings section, you give Dremio Cloud access to your cloud. Follow these steps:

      a. Copy the policy JSON to your clipboard. This JSON lists the tasks that Dremio Cloud needs permission to carry out on your cloud. You use the policy JSON in the AWS Console in step 5.

      b. In the Compute Credentials field, select the security method that you want to use to give Dremio Cloud the permission it needs.

    5. Log in to the AWS Console, and then follow either of these steps:

    6. In the Dremio Cloud Add Cloud dialog, follow either of these steps:

      • If you selected Access Key in step 4, paste into the appropriate fields the AWS access key ID and AWS secret access key that you downloaded after creating an IAM User in step 5.

      • If you selected IAM Role in step 4, enter the Cross-Account Role ARN of the IAM Role you created in step 5.

    7. Click Test to verify that Dremio Cloud can access the new cloud.

    8. In the AWS Console, create a security group that allows engines to connect to Dremio Cloud over TLS.

    9. In the Dremio Cloud Add Cloud dialog, specify in the Security Group field the ID of the security group that you created in step 8.

    10. In the Subnets field, specify the subnet ID associated with your Amazon Virtual Private Cloud (Amazon VPC). If you want to use more than one subnet, click Add Subnet once for each additional subnet, and then specify the ID of one subnet per added field. When Dremio Cloud needs to start up query engines, it searches the subnets for resources it can use to spin up required instances. Adding more subnets increases the chances of finding resources, should one or more of the availability zones associated with a subnet run out of instances required by Dremio Cloud.

      Guidelines for adding subnets:

      • Ensure that subnet IDs are unique across all of the availability zones within an Amazon VPC.

      • Ensure that each subnet that you specify belongs to a separate availability zone; for example, if you specify subnet A and subnet B, they cannot both be in availability zone C, but must be in separate availability zones.

      • Specify only private subnets or only public subnets; mixing private and public subnets is not supported.

    11. Click OK.

    Adding a Cloud with an AWS CloudFormation Template (CFT)

    Follow the steps below to add a new cloud to your organization using a CFT:

    1. In the top-right corner of the Clouds page, click the Add Cloud button.

    2. Click the link to create a cloud using the CFT.

    3. In the Cloud Name field, specify a name for your cloud that is unique within your organization.

    4. For Region, enter the region associated with the cloud account you are connecting to Dremio Cloud. For more information, see supported AWS regions.

    5. Click Launch CloudFormation Template, which will open the AWS Console in a new browser tab for next steps. If not already authenticated, sign in to your AWS account on this page.

    6. On the Quick create stack page in the AWS Console, for Stack name, enter a name to identify the stack. This name must be unique to your account and cannot include underscores.

    7. For VPC, select a VPC in which the compute resources (EC2 instances) will be created.

    8. For Subnets, select one or more subnets within the VPC. You can select multiple subnets across different availability zones.

    9. Select I acknowledge that AWS CloudFormation might create IAM resources.

    10. Click Create stack.

      Stack creation can take up to four minutes. When stack creation is complete, the required cloud resources have been configured successfully and you can return to the browser tab running Dremio Cloud to continue.

    Editing a Cloud

    When you edit a cloud, you can change these settings:

    • The name of the cloud
    • The AWS access key ID and AWS secret access key, if you want to create a new AWS IAM user for Dremio Cloud to use when accessing the cloud
    • The ID of the security group
    • The VPC Endpoint ID (for AWS PrivateLink configurations)

    To edit a cloud:

    1. On the Clouds page, hover over the row of the cloud and click the Edit This is the icon that represents the Edit settings. icon that appears next to the cloud. The Edit Cloud dialog is opened.

    2. (Optional) In the Cloud Name field, specify a different name that is unique within your Dremio Cloud organization.

    3. (Optional) If you want to create a new AWS IAM user for Dremio Cloud to use, follow these steps:

      a. Log in to the AWS Console.

      b. Create an AWS IAM user and download the access key ID and secret access key.

      c. In the Dremio Cloud Edit Cloud dialog, select **Access Key in the Compute Credentials field.

      d. Paste into the appropriate fields the AWS access key ID and AWS secret access key that you downloaded after creating an IAM User.

      e. Click Test to verify that Dremio Cloud can access the cloud.

    4. (Optional) If you want to use a different security group, follow these steps:

      a. From the AWS Console, create a security group that allows engines to connect to Dremio Cloud over TLS.

      b. In the Dremio Cloud Edit Cloud dialog, for Security Group, specify the ID of the new security group.

    5. (Optional) If you are using an AWS PrivateLink configuration:

      a. From the AWS Console, create a VPC Endpoint and copy the endpoint ID, which directs traffic from a private network to Dremio Cloud.

      b. Returning to the Dremio Edit Cloud dialog, enter the VPC Endpoint ID you created in the previous step above.

    6. Click OK.

    Deleting a Cloud

    To delete a cloud from a Dremio Cloud organization:

    warning:

    To delete a cloud you must first delete or select a new cloud for the engines that are associated with this cloud.

    1. On the Clouds page, hover over the row of the cloud and click the Delete This is the icon that represents the Delete settings. icon that appears next to the cloud.
    2. Confirm that you want to delete the cloud.