Dremio Community

Configuring AWS IAM Permissions and IAM Roles

Prerequisites

  • AWS Account
  • Permissions to create IAM Policies and IAM Roles

If you do not use AWS CloudFormation to deploy Dremio AWS Edition, you must manually create an IAM role with a policy that allows Dremio to access your cluster and attach it to an EC2 instance. In addition, you must add the following resource tag to your EC2 instance:

Key: dremio_managed

Value: true

Follow these steps to create the IAM role

Step 1: Log in to your AWS account and navigate to the IAM service. Click Policies in the left navigation menu, then click Create policy.

Configure IAM

Step 2: Click “JSON” and paste the following JSON policy, then click Review policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ec2:DeleteVolume",
      "Resource": "arn:aws:ec2:*:*:volume/*",
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:AttachVolume",
        "ec2:DetachVolume",
        "ec2:ReplaceIamInstanceProfileAssociation",
        "ec2:TerminateInstances"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ec2:*:*:volume/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:DeleteSnapshot",
      "Resource": "arn:aws:ec2:*::snapshot/*",
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "elasticfilesystem:CreateMountTarget",
        "elasticfilesystem:DeleteFileSystem",
        "elasticfilesystem:DeleteMountTarget"
      ],
      "Resource": "arn:aws:elasticfilesystem:*:*:file-system/*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:CreateTags",
      "Resource": "arn:aws:ec2:*:*:volume/*",
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "elasticfilesystem:CreateFileSystem",
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:CreateVolume",
      "Resource": "arn:aws:ec2:*:*:volume/*",
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": [
        "arn:aws:ec2:*:*:volume/*",
        "arn:aws:ec2:*:*:instance/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": [
        "arn:aws:ec2:*::image/*",
        "arn:aws:ec2:*:*:network-interface/*",
        "arn:aws:ec2:*:*:security-group/*",
        "arn:aws:ec2:*:*:subnet/*",
        "arn:aws:ec2:*:*:key-pair/*",
        "arn:aws:ec2:*:*:placement-group/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "ec2:CreateSnapshot",
      "Resource": "arn:aws:ec2:*::snapshot/*",
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:CreateSnapshot",
      "Resource": "arn:aws:ec2:*:*:volume/*",
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/dremio_managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:CreateTags",
      "Resource": "arn:aws:ec2:*:*:volume/*",
      "Condition": {
        "StringEquals": {
          "ec2:CreateAction": "CreateVolume"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:CreateTags",
      "Resource": "arn:aws:ec2:*::snapshot/*",
      "Condition": {
        "StringEquals": {
          "ec2:CreateAction": "CreateSnapshot"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:CreateTags",
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ec2:*:*:volume/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:CreateAction": "RunInstances"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:CreateTags",
      "Resource": "arn:aws:ec2:*:*:placement-group/*",
      "Condition": {
        "StringEquals": {
          "ec2:CreateAction": "CreatePlacementGroup"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface",
        "ec2:DeleteNetworkInterface",
        "ec2:CreatePlacementGroup",
        "ec2:DeletePlacementGroup"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeIamInstanceProfileAssociations",
        "ec2:DescribeImages",
        "ec2:DescribeInstances",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DescribeNetworkInterfaceAttribute",
        "ec2:DescribePlacementGroups",
        "ec2:DescribeSnapshots",
        "ec2:DescribeSubnets",
        "ec2:DescribeTags",
        "ec2:DescribeVolumes"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "elasticfilesystem:DescribeFileSystems",
        "elasticfilesystem:DescribeMountTargets",
        "elasticfilesystem:DescribeMountTargetSecurityGroups"
      ],
      "Resource": "arn:aws:elasticfilesystem:*:*:file-system/*"
    },
    {
      "Effect": "Allow",
      "Action": "iam:GetInstanceProfile",
      "Resource": "arn:aws:iam::*:instance-profile/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:GetPolicy",
        "iam:GetPolicyVersion"
      ],
      "Resource": "arn:aws:iam::*:policy/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:GetRolePolicy",
        "iam:ListAttachedRolePolicies",
        "iam:ListRolePolicies"
      ],
      "Resource": "arn:aws:iam::*:role/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:SimulateCustomPolicy",
        "s3:HeadBucket",
        "s3:ListAllMyBuckets"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::dremio-me-*/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:GetBucketLocation",
        "s3:ListBucket",
        "s3:PutBucketTagging"
      ],
      "Resource": "arn:aws:s3:::dremio-me-*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::ap-southwest-1.examples.dremio.com",
        "arn:aws:s3:::eu-west-1.examples.dremio.com",
        "arn:aws:s3:::us-east-1.examples.dremio.com",
        "arn:aws:s3:::us-west-1.examples.dremio.com",
        "arn:aws:s3:::us-west-2.examples.dremio.com"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": [
        "arn:aws:s3:::ap-southwest-1.examples.dremio.com/*",
        "arn:aws:s3:::eu-west-1.examples.dremio.com/*",
        "arn:aws:s3:::us-east-1.examples.dremio.com/*",
        "arn:aws:s3:::us-west-1.examples.dremio.com/*",
        "arn:aws:s3:::us-west-2.examples.dremio.com/*"
      ]
    }
  ]
}
Configure IAM

Step 3: Add a name to the policy and click Create policy.

Configure IAM

Step 4: Click Roles in the left navigation menu, then click Create role.

Configure IAM

Step 5: Select AWS service and choose EC2 from the use cases, then click Next.

Configure IAM

Step 6: In the Filter policies tab, enter the name of the policy you just created, select the policy, then click Next.

Configure IAM

Step 7: (Optional) Add tags and click Next.

Give a name to your role and hit “Create role”.

Configure IAM

Step 8: Now that the role is created, the last step is to add permissions for the Get/Pass role. Click “Add inline policy”.

Configure IAM

Step 9: Click on the “JSON” tab and paste the following JSON.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:PassRole"
            ],
             "Resource": "arn:aws:iam::***:role/*role-id-here*"
        }
    ]
}

Note Replace the *** wildcards in resource ID with the region/account-id for the ARN. See Amazon Resource Names for more information. Replace *role-id-here* with the resource id for your new IAM role.

Configure IAM

Step 10. Add a name to the inline policy and click “Create policy”.

Configure IAM

You have successfully created the IAM role.

Configure IAM

Troubleshooting

If you see any of the following error messages, this means that either you are missing this action in your policy, or is not properly configured. Please verify if the resource/conditions are configured properly according to the policy described above:

  1. “You are not authorized to perform the following action(s): iam:GetRole, ec2:DeleteVolume. Please add these permissions to your IAM instance profile to continue.”
  2. “You are not authorized to perform this action: Service:Action” e.g. “You are not authorized to perform this action: ec2: DeleteVolume”
  3. “If you are using the Dremio recommended IAM policy, it looks like you have not added the dremio_managed tag to this instance, which is required for that IAM policy to work. Please add the tag “dremio_managed = true” to this instance.”