Foo

DEPRECATED

AWS Cloudformation templates were previously provided for testing and evaluation purposes only. They have been deprecated and will be replaced by an upcoming AWS Marketplace listing. Existing Cloudformation deployments will continue to function however it is recommended that the Marketplace listing is used for testing and evaluation once it is available.

Prerequisite

  • AWS account
  • AWS Keypair
  • Permission to create security groups
  • (Optional) VPC and subnet for non-default VPC and subnet

Note

Dremio for an AWS deployment creates coordinator and executor nodes that are associated with the selected cluster size.

This link is associated with Dremio’s AWS Cloudformation template: Evaluate via AWS Cloudformation

Step 2: Log in to AWS

Log in using your AWS account (if you’re not already logged in).

Step 3: Configure the Dremio Deployment

From CloudFormation > Stacks, create a stack for the Dremio deployment.

  1. From the Select Template page, click Next.

  2. Specify the stack details and click Next.

    • Specify the stack name (default: myDemio)
    • Specify the parameters for the Dremio cluster and the AWS resource configuration.
  3. (Optional) Configure the stack options and click Next.

  4. Review your configuration amd click Create.

    AWS Stack Details

Parameters

Parameter Description
Cluster Size* Pick a size based on your needs.
Deploy to VPC VPC for the Dremio cluster deployment.
Deploy to Subnet Subnet for the Dremio cluster deployment. The subnet must be in the selected VPC.
Dremio download URL (Optional) A public accessible URL to a Dremio RPM installer. Default: <empty> (latest CE release)
AWS keypair The AWS keypair used for the Dremio cluster.

Cluster Size parameter

Based on the cluster size, the following machine types and number of executor nodes are implemented. Note that the cost of your AWS deployment depends on the selected cluster size.

Cluster size Coordinator VM Size Executor VM Size No. of Executors
X-Small m5.2xlarge r5d.4xlarge 1
Small m5.2xlarge r5d.4xlarge 5
Medium m5.4xlarge r5d.4xlarge 10
Large m5.4xlarge r5d.4xlarge 25
X-Large m5.4xlarge r5d.4xlarge 50

Step 4: Connect to Dremio

From CloudFormation:

  1. Select your Stack.
  2. Verify that your deployment is complete from Stack info.
  3. Once your deployment is complete, select the Outputs tab and click on the Dremio UI link to launch Dremio.
    AWS Launch URL
  4. Register with Dremio and login.

Start an Existing Project

Prerequisites

Note

Starting a Project requires permission to use the IAM role associated with the project.

Step 1: Launch Dremio AWS Edition through the AWS Marketplace

Step 2:

Once you have launched Dremio from the Marketplace, if you have existing Projects, you will land on the Project List page. The Project List page shows all of your existing Projects. In the example below,there are 3 “Open” or running Projects (identifed by the green icons), and 2 “closed” or non-running Projects (identified by the gray icons). For the closed Project that you want to open, hover under the “Action” column at the far and you will bring up actions that can be done to that Project. Click on the “Launch” icon to open your Project.

Projet List
  1. Click Create and Launch
  2. Dremio creates resources required for the project within AWS and launches the project, throughout the process messages are displayed describing each action. After the project is created the webpage automatically trasitions to the Project’s sign-up page.
    Launch Project

Step 3: Complete account signup

  1. Accept the license agreement by clicking I Accept after reading the agreement
  1. Create the first user account Create Project

Step 4: Finished

The Dremio Project is complete and ready to be used. For more information, see Dremio Tutorials & Resources

Create New Project Parameters

Configuration parameters to customize when creating a new Project:

Parameter Description
Project Name The name for the Project
Initial Engine Size The number of nodes for the default engine. Can be changed after creating the project. Options are:
  • Small (2 nodes)
  • Medium (4 nodes)
  • Large (8 nodes)
  • XLarge (16 nodes)
  • 2XLarge (32 nodes)
  • 3XLarge (64 nodes)
  • Custom - Customizable size between 1 and 1000 nodes
Enable Automatic Backups If daily backups should be created automatically
Engine Node Type (optional) The EC2 Instance type for nodes in the default engine. Can be changed after creating the project. Options are:
  • Standard (m5d.8xlarge) - General Purpose (32c/128gb)
  • High Memory (r5d.4xlarge) - Higher memory per node (16c/128gb)
  • High CPU (c5d.18xlarge) - Higher compute per node (72c/144gb)
  • High Cache (i3.4xlarge) - Larger local storage per node (16c/122gb)
IAM Role (optional) The IAM role the project should assume. If left blank will use the IAM Role from the AWS Marketplace launch.

See Configuring Project Permisions for additional information on configuring IAM Permissions for a Project