Skip to main content

AWS PrivateLink

For establishing outbound connectivity to the Dremio control plane, you can use AWS PrivateLink. PrivateLink easily connects services like Dremio Cloud across varying accounts and VPC configurations for a simplified network architecture.

If you choose to use PrivateLink, there are two ways to set up the configuration:

  • Configuring with a CloudFormation template (CFT)
  • Configuring cloud resources manually

Configuring with the CFT​

For ease of configuration, we recommend using a CFT, because the CFT will create a security group and set up PrivateLink for you.

A security group acts as a virtual firewall to control the traffic that is allowed to and from your resources, ensuring that only traffic from Dremio Cloud reaches the resources that you have allocated for your Dremio Cloud organization. For additional information, see Control traffic to your AWS resources using security groups.

The CFT can be used at the time of onboarding. To use a CFT, choose to Launch CloudFormation Template when adding a Sonar project and configuring project resources. See the annotated onboarding CFT for a breakdown of all resources and permissions, which includes creating a cloud and a project.

After onboarding, you can use the annotated cloud creation CFT to create subsequent clouds in your project and set up PrivateLink during the process. See Adding a Cloud for the steps.

Configuring Cloud Resources Manually​

You will need a VPC endpoint ID for PrivateLink when configuring cloud resources manually. VPC endpoints are virtual resources used to facilitate secure connections between a VPC and Dremio Cloud. By creating endpoints to serve as authorized traffic destinations, you can prevent the exposure of your traffic outside your VPC and its associated services.

To obtain a VPC endpoint ID:

  1. Go to the AWS Management Console and sign in with your credentials.
  2. Navigate to Services > Networking & Content Delivery > VPC.
  3. Select Endpoints from the side navigation bar.
  4. On the Endpoints page, copy the VPC endpoint ID and paste it in a location where you can retrieve it when completing the Set Up Network Access section of the manual process.

If your organization does not already have a VPC endpoint, follow these steps for creating a VPC endpoint.

Next Steps​

After configuring Dremio Cloud to work with your VPC via PrivateLink, you should add a source with which to begin managing your data.

Troubleshooting​

I'm receiving the following error: private-dns-enabled cannot be set because there is already a conflicting DNS domain for "X" in the VPC "Y".​

If you encountered the above error while creating an endpoint for Dremio, this means that the Dremio-provided service name you used is already in use. Check any existing endpoints to ensure whether the same service name is already in use. Should no other endpoint exist with the same service name, contact Dremio's Support team for additional assistance.

My Dremio Cloud Engines are displaying an error about how they cannot access the VPC endpoint.​

This error typically appears from the Engines page when access to the VPC endpoint is disrupted or broken off completely. Most often, this happens when the endpoint used by Dremio is either deleted or the security group is altered incorrectly. Verify that the VPC endpoint still exists and the security group is properly configured.

My VPC endpoint was deleted.​

If your VPC endpoint was somehow deleted, it cannot be restored. You must manually recreate a VPC endpoint. Using the new VPC endpoint ID, you can either update your cloud configuration or create a new cloud configuration.