On this page

    Configuring Cloud Resources Manually

    If you are configuring your first Sonar project and would like to configure cloud resources manually, then navigate to the Dremio Cloud homepage and click Add Sonar Project on the Sonar card.

    note:

    Before you begin, review the Prerequisites for Configuring Cloud Resources.

    Adding a Project

    note:

    If you have already completed the Create project page in the Add Sonar Project dialog, proceed to Configure Resources: Set up Storage Access.

    In the Add Sonar Project dialog:

    1. On the Create project page, for Project Name, enter a name to identify this project. This name can be changed after project creation.
    2. For AWS Region, select the region in which Dremio will create and manage engines used to execute queries. For a list of supported regions, reference Supported Regions.
    3. For AWS Tags, add the AWS tags to identify the compute resources created for this project.
    4. Click Next.
    5. On the Configure resources page, click the link at the bottom to create the AWS resources and grant Dremio permissions manually. Follow the steps in the next section to configure cloud resources.

    Set Up Storage Access

    In the Set Up Storage Access section of the Configure resources page, configure the S3 bucket that will be used as the metadata store for your project.

    When you configure storage settings, you grant Dremio Cloud read and write permissions to the S3 bucket.

    To configure storage settings:

    1. Create an Amazon S3 bucket. For steps on how to create an S3 bucket in the AWS Console, see AWS: Creating a Dremio Cloud Project Store.

    2. For Project Store, paste the path of the S3 bucket. This fills in the PROJECT-STORE-BUCKET-NAME parameter in the polcy JSON. The rest of the policy JSON sets the minimum policy requirement to allow Dremio Cloud read and write access to the project store.

    3. To grant Dremio read and write permissions to the project store, follow either of these two sets of steps:

      • To grant the permissions by means of an IAM user:

        a. For Authentication Method, select Access Key.

        b. Copy the policy JSON from the dialog and paste it into a location that you can access later. To understand what permissions the policy JSON grants, see Policy JSON: Setting Up Storage Access.

        c. In the AWS Console, create an IAM user with an access key.

        d. For AWS Access Key ID, paste the Access Key ID from the .csv file you downloaded when you created the user.

        e. For AWS Secret Access Key, paste the AWS Secret Access Key from the .csv file you downloaded when you created the user.

      • To grant the permissions by means of a cross-account IAM role:

        a. For Authentication Method, select IAM Role.

        b. Copy the policy JSON from the dialog and paste it into a location that you can access later.

        c. Copy this JSON snippet and paste it and paste it into a location that you can access later.

        Trust Policy for IAM Role
          {
            "Version": "2012-10-17",
            "Statement": [
              {
                "Effect": "Allow",
                "Principal": {
                  "Service": "ec2.amazonaws.com"
                },
                "Action": "sts:AssumeRole"
              },
              {
                "Effect": "Allow",
                "Principal": {
                  "AWS": "arn:aws:iam::<trust account ID>:root"
                },
                "Action": "sts:AssumeRole",
                "Condition": {
                  "StringEquals": {
                    "sts:ExternalId": "<external ID>"
                  }
                }
              }
            ]
          }
        

        d. Copy the Trust Account ID and the External ID and paste them into the JSON that you copied in the previous step.

        d. In the AWS Console, create a cross-account IAM role.

        e. For Role ARN, paste the cross-account role ARN that you copied when you created the role.

        f. For Instance Profile ARN, paste the instance profile ARN that you copied when you created the role.

    Set up Compute Settings

    In the Set Up Compute Settings section of the Configure resources page, specify the credentials for that will be used to create and manage compute engines in your Amazon VPC.

    When you configure compute settings, you grant Dremio Cloud permissions to create and manage compute instances for Dremio engines. As with the storage settings, you can grant the permissions by means of an IAM user or an IAM role.

    note:

    You can use the same IAM user that you created for granting access to storage settings by adding an additional JSON policy to the existing IAM user. For steps on how to attach new permission policies to an existing IAM user, see setting up AWS permissions.

    Permissions are granted only on resources that are tagged as dremio_managed.

    1. Copy the content of the Policy JSON text box and paste it in a location that you can access later. To understand the permissions the policy JSON grants, see Policy JSON: Set up Compute Access.
    2. To grant Dremio Cloud permissions to create and manage compute instances for Dremio engines, follow either of these two sets of steps:
    • To grant the permissions by means of an IAM user:

      a. For Deploy to AWS using, select Access Key.

      b. Create an IAM user with an access key.

      c. For AWS Access Key ID, paste the Access Key ID from the .csv file you downloaded when you created the user.

      d. For AWS Secret Access Key, paste the AWS Secret Access Key from the .csv file you downloaded when you created the user.

    • To grant the permissions by means of a cross-account IAM role:

      a. For Deploy to AWS using, select IAM Role.

      b. Create a cross-account IAM role.

      c. For Cross-Account Role ARN, paste the Role ARN that you copied when you created the role.

    Set up Network Settings

    In the Network Settings section of the Cloud Connect page, specify a security group for your VPC to use with Dremio Cloud, and add an outbound rule that allows compute engines to connect to Dremio Cloud’s control plane by using TLS.

    1. Create a security group

      For steps on how to create a security group for your VPC, see create an AWS security group.

    2. In the list of security groups, select the one you created.

    3. Verify that the inbound and outbound rules are correct.

    4. Copy the security group ID and paste it in a location that you can retrieve it from in a later step.

    5. Navigate to the VPC Dashboard in the AWS Console.

    6. Locate the subnets you want to use for your Dremio Cloud account and copy their IDs. Paste them in a location that you can retrieve them from in a later step. Guidelines for choosing subnets are located here.

    7. To use AWS PrivateLink, copy the VPC endpoint ID and paste it in a location that you can retrieve it from in a later step. If your organization does not already have a VPC endpoint, follow these steps to create one.

    8. Return to the Cloud Connect page in the sign-up process for Dremio Cloud.

    9. For Subnet, paste the IDs of the subnets that you want to use.

    10. For Security Group, paste the ID of the security group that you created.

    11. (Optional) For Dremio VPC Endpoint ID, paste the VPC endpoint ID if you are using AWS PrivateLink.