On this page

    Prerequisites in AWS and Permissions Required by Dremio

    Before you sign up, you need to ensure that you meet a few prerequisites.

    As part of signing up, you grant a number of permissions to Dremio on your Amazon Virtual Private Cloud (Amazon VPC).

    Prerequisites

    • An AWS account and region that you want to use with Dremio Cloud. The regions that Dremio Cloud supports are listed here.

    • An Amazon Virtual Private Cloud (Amazon VPC) and subnets in which to give Dremio Cloud permissions on storage, compute, and network resources.

    • Your AWS user account must have permission to run a CloudFormation template that creates the security group, the s3 bucket for the project, and IAM roles or IAM users.

    • Outbound connectivity from your Amazon VPC and subnets is required to allow query engines to communicate with Dremio Cloud. Engines establish a connection with the Dremio Cloud control plane using port 443 (HTTPS) outbound to the internet. No open ports are required in your Amazon VPC, and neither subnets for incoming connections nor engines require public IP addresses.

      To verify outbound connectivity from your subnets, run the following command from an EC2 instance within each subnet:

      curl -v https://gw.dremio.cloud
      

    Permissions

    While signing up, you link your Amazon Virtual Private Cloud (Amazon VPC) to your project, granting Dremio Cloud these permissions through IAM roles or IAM users:

    • Permissions on an S3 bucket in your Amazon VPC for storing metadata and views for the project. The permissions are:

      • Allow Dremio to enumerate S3 buckets within the account.
      • Allow Dremio read and write access to the Project Store bucket used to store housekeeping information such as metadata and reflections.
      • Allow Dremio to determine the region, list content and add tags on the Project Store bucket.
      • Allow Dremio read access to sample datasets used to get users started easily on the platform without connecting their own data.
    • Permissions to create and manage compute engines in your Amazon VPC. The permissions are:

      • Allow Dremio to terminate instances with the dremio_managed tag.
      • Require the dremio_managed tag for instances/volumes when creating instances.
      • Allow creating instances without the dremio_managed tag on resources other than instances/volumes.
      • Allow Dremio to create tags on instances and volumes only upon the initial creation of an instance.
      • Allow Dremio to create tags on placement groups (PG) upon the initial creation of a PG.
      • Allow Dremio to create tags on a launch template (LT) upon the initial creation of a LT.
      • Allow Dremio to create tags on a fleet upon the initial creation of the fleet.
      • Allow Dremio to create fleet only when including the dremio_managed tag.
      • Allow Dremio to create fleet with other resources without the dremio_managed tag.
      • Allow Dremio to delete fleets with the dremio_managed tag only.
      • Allow Dremio to create a launch template.
      • Allow Dremio to delete launch templates with the dremio_managed tag only.
      • Allow Dremio to describe fleets with the dremio_managed tag.
      • Allow Dremio to delete placement groups with the dremio_managed tag only.
      • Allow Dremio to create a placement group.
      • Allow Dremio to enumerate resources in the account.

    You also create a security group for your Amazon VPC to use with Dremio Cloud, and add an outbound rule that allows compute engines to connect to Dremio Cloud’s control plane by using TLS.

    Policy Template to Grant Access to the Project Store

    The following policy template is the minimum policy requirement to allow read and write access to the project store. Replace BUCKET-NAME with the S3 bucket you want to use as the Dremio Cloud project store:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "s3:ListBucket",
            "s3:ListAllMyBuckets"
          ],
          "Resource": "*"
        },
        {
          "Effect": "Allow",
          "Action": [
            "s3:DeleteObject",
            "s3:GetObject",
            "s3:PutObject"
          ],
          "Resource": [
            "arn:aws:s3:::BUCKET-NAME/*"
          ]
        },
        {
          "Effect": "Allow",
          "Action": [
            "s3:GetBucketLocation",
            "s3:ListBucket",
            "s3:PutBucketTagging"
          ],
          "Resource": [
            "arn:aws:s3:::BUCKET-NAME"
          ]
        },
        {
          "Effect": "Allow",
          "Action": [
            "s3:ListBucket",
            "s3:GetBucketLocation",
            "s3:GetObject"
          ],
          "Resource": [
            "arn:aws:s3:::ap-southwest-1.examples.dremio.com",
            "arn:aws:s3:::eu-west-1.examples.dremio.com",
            "arn:aws:s3:::us-east-1.examples.dremio.com",
            "arn:aws:s3:::us-west-1.examples.dremio.com",
            "arn:aws:s3:::us-west-2.examples.dremio.com"
          ]
        }
      ]
    }
    

    Policy for Enabling Dremio Cloud to Manage Engines

    The following policy enables Dremio Cloud to create and manage engines in your AWS account:

    {
      "Version": "2012-10-17",
      "Statement": [
        # Allow Dremio to terminate instances with the "dremio_managed" tag.
        {
          "Effect": "Allow",
          "Action": "ec2:TerminateInstances",
          "Resource": "arn:aws:ec2:*:*:instance/*",
          "Condition": {
            "StringEquals": {
              "ec2:ResourceTag/dremio_managed": "true"
            }
          }
        },
        # Require the "dremio_managed" tag for instances/volumes when creating instances.
        {
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "Resource": [
            "arn:aws:ec2:*:*:volume/*",
            "arn:aws:ec2:*:*:instance/*"
          ],
          "Condition": {
            "StringEquals": {
              "aws:RequestTag/dremio_managed": "true"
            }
          }
        },
        # Allow creating instances without the "dremio_managed" tag on resources other than instances/volumes.
        {
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "Resource": [
            "arn:aws:ec2:*:*:launch-template/*",
            "arn:aws:ec2:*:*:fleet/*",
            "arn:aws:ec2:*::image/*",
            "arn:aws:ec2:*:*:network-interface/*",
            "arn:aws:ec2:*:*:security-group/*",
            "arn:aws:ec2:*:*:subnet/*",
            "arn:aws:ec2:*:*:placement-group/*"
          ]
        },
        # Allow Dremio to create tags on instances/volumes only upon the initial creation of an instance.
        {
          "Effect": "Allow",
          "Action": "ec2:CreateTags",
          "Resource": [
            "arn:aws:ec2:*:*:instance/*",
            "arn:aws:ec2:*:*:volume/*"
          ],
          "Condition": {
            "StringEquals": {
              "ec2:CreateAction": "RunInstances"
            }
          }
        },
        # Allow Dremio to create tags on placement groups (PG) upon the initial creation of a PG.
        {
          "Effect": "Allow",
          "Action": "ec2:CreateTags",
          "Resource": "arn:aws:ec2:*:*:placement-group/*",
          "Condition": {
            "StringEquals": {
              "ec2:CreateAction": "CreatePlacementGroup"
            }
          }
        },
        # Allow Dremio to create tags on a launch template (LT) upon the initial creation of a LT.
        {
          "Effect": "Allow",
          "Action": "ec2:CreateTags",
          "Resource": "arn:aws:ec2:*:*:launch-template/*",
          "Condition": {
            "StringEquals": {
              "ec2:CreateAction": "CreateLaunchTemplate"
            }
          }
        },
        # Allow Dremio to create tags on a fleet upon the initial creation of the fleet.
        {
          "Effect": "Allow",
          "Action": "ec2:CreateTags",
          "Resource": "arn:aws:ec2:*:*:fleet/*",
          "Condition": {
            "StringEquals": {
              "ec2:CreateAction": "CreateFleet"
            }
          }
        },
        # Allow Dremio to create fleet only when including the "dremio_managed" tag.
        {
          "Effect": "Allow",
          "Action": "ec2:CreateFleet",
          "Resource": "arn:aws:ec2:*:*:fleet/*",
          "Condition": {
            "StringEquals": {
              "aws:RequestTag/dremio_managed": "true"
            }
          }
        },
        # Allow Dremio to create fleet with other resources without the "dremio_managed" tag.
        {
          "Effect": "Allow",
          "Action": "ec2:CreateFleet",
          "Resource": [
            "arn:aws:ec2:*:*:instance/*",
            "arn:aws:ec2:*:*:image/*",
            "arn:aws:ec2:*:*:launch-template/*",
            "arn:aws:ec2:*:*:network-interface/*",
            "arn:aws:ec2:*:*:placement-group/*",
            "arn:aws:ec2:*:*:security-group/*",
            "arn:aws:ec2:*:*:subnet/*"
          ]
        },
        # Only allow Dremio to delete fleets with the "dremio_managed" tag.
        {
          "Effect": "Allow",
          "Action": "ec2:DeleteFleets",
          "Resource": "arn:aws:ec2:*:*:fleet/*",
          "Condition": {
            "StringEquals": {
              "ec2:ResourceTag/dremio_managed": "true"
            }
          }
        },
        # Allow Dremio to create a launch template.
        {
          "Effect": "Allow",
          "Action": "ec2:CreateLaunchTemplate",
          "Resource": "arn:aws:ec2:*:*:launch-template/*"
        },
        # Only allow Dremio to delete launch templates with the "dremio_managed" tag.
        {
          "Effect": "Allow",
          "Action": "ec2:DeleteLaunchTemplate",
          "Resource": "arn:aws:ec2:*:*:launch-template/*",
          "Condition": {
            "StringEquals": {
              "ec2:ResourceTag/dremio_managed": "true"
            }
          }
        },
        # Allow Dremio to describe fleets with the "dremio_managed" tag.
        {
          "Effect": "Allow",
          "Action": "ec2:DescribeFleets",
          "Resource": "arn:aws:ec2:*:*:fleet/*",
          "Condition": {
            "StringEquals": {
              "ec2:ResourceTag/dremio_managed": "true"
            }
          }
        },
        # Only allow Dremio to delete placement groups with the "dremio_managed" tag.
        {
          "Effect": "Allow",
          "Action": "ec2:DeletePlacementGroup",
          "Resource": "arn:aws:ec2:*:*:placement-group/*",
          "Condition": {
            "StringEquals": {
              "ec2:ResourceTag/dremio_managed": "true"
            }
          }
        },
        # Allow Dremio to create a placement group.
        {
          "Effect": "Allow",
          "Action": "ec2:CreatePlacementGroup",
          "Resource": "arn:aws:ec2:*:*:placement-group/*"
        },
        # Allow Dremio to enumerate resources in the account.
        {
          "Effect": "Allow",
          "Action": [
            "ec2:DescribeImages",
            "ec2:DescribeLaunchTemplateVersions",
            "ec2:DescribeLaunchTemplates",
            "ec2:DescribeVpcs",
            "ec2:DescribeSubnets",
            "ec2:DescribeTags",
            "ec2:DescribeAvailabilityZones",
            "ec2:DescribeInstances",
            "ec2:DescribeInstanceStatus",
            "ec2:DescribeNetworkInterfaces",
            "ec2:DescribeNetworkInterfaceAttribute",
            "ec2:DescribePlacementGroups",
            "ec2:DescribeSecurityGroups",
            "ec2:DescribeVpcEndpoints",
            "ec2:DescribeVolumes"
          ],
          "Resource": "*"
        },
        # This section is necessary only if you are using a storage role.
        {
          "Effect": "Allow",
          "Action": [
            "iam:PassRole",
            "sts:AssumeRole"
          ],
          "Resource": [
            "<Storage Role ARN>"
          ]
        }
      ]
    }