Deploying Dremio on AWS
note:
AWS Cloudformation templates were previously provided for testing and evaluation purposes only. They have been deprecated and will be replaced by an upcoming AWS Marketplace listing. Existing Cloudformation deployments will continue to function however it is recommended that the Marketplace listing is used for testing and evaluation once it is available.
Prerequisite
- AWS account
- AWS Keypair
- Permission to create security groups
- (Optional) VPC and subnet for non-default VPC and subnet
Note:
Dremio for an AWS deployment creates coordinator and executor nodes that are associated with the selected cluster size.
Step 1: Click on Dremio’s AWS Cloudformation Link
This link is associated with Dremio’s AWS Cloudformation template: Evaluate via AWS Cloudformation
Step 2: Log in to AWS
Log in using your AWS account (if you’re not already logged in).
Step 3: Configure the Dremio Deployment
From CloudFormation > Stacks, create a stack for the Dremio deployment.
- From the Select Template page, click Next.
- Specify the stack details and click Next.
- Specify the stack name (default: myDemio)
- Specify the parameters for the Dremio cluster and the AWS resource configuration.
- (Optional) Configure the stack options and click Next.
- Review your configuration amd click Create.
Parameters
| Parameter | Description |
|---|---|
| Cluster Size* | Pick a size based on your needs. |
| Deploy to VPC | VPC for the Dremio cluster deployment. |
| Deploy to Subnet | Subnet for the Dremio cluster deployment. The subnet must be in the selected VPC. |
| Dremio download URL | (Optional) A public accessible URL to a Dremio RPM installer. Default: <empty> (latest CE release) |
| AWS keypair | The AWS keypair used for the Dremio cluster. |
Cluster Size parameter
Based on the cluster size, the following machine types and number of executor nodes are implemented. Note that the cost of your AWS deployment depends on the selected cluster size.
| Cluster size | Coordinator VM Size | Executor VM Size | No. of Executors |
|---|---|---|---|
| X-Small | m5.2xlarge | r5d.4xlarge | 1 |
| Small | m5.2xlarge | r5d.4xlarge | 5 |
| Medium | m5.4xlarge | r5d.4xlarge | 10 |
| Large | m5.4xlarge | r5d.4xlarge | 25 |
| X-Large | m5.4xlarge | r5d.4xlarge | 50 |
Step 4: Connect to Dremio
From CloudFormation:
- Select your Stack.
- Verify that your deployment is complete from Stack info.
- Once your deployment is complete, select the Outputs tab and click on the Dremio UI link to launch Dremio.
- Register with Dremio and login.
Getting Started
For more information, see Dremio Tutorials & Resources