Managing Projects

A project isolates compute, data and other resources needed by a team for data analysis. An organization may contain multiple projects. A project must be linked to a single cloud account. Your first project was created and linked to your cloud account as part of the sign-up process.

Each project requires access to an S3 bucket for storing various types of data, including:

• The data for reflections that are created in the project
• File uploads to home spaces
• The default path for new tables that are used for data and manifests for datasets
• All of the tables that store records of events and other historical data

Each project in your organization contains a preview engine. The preview engine is always on by default, which incurs AWS EC2 costs. You can manually disable this engine, or any other engine, under Project Settings > Engines.

Deactivated Projects

Dremio displays deactivated/inactive projects in the project selector in the side navigation bar and on the Projects page.

• An inactive project will be activated automatically when any user tries to access it through the Dremio app or via an API call.

• You can activate a project manually from the Projects page.

• It takes approximately five minutes to activate a project.

Archived Projects

Dremio displays archived projects only on the Projects page, and they can only be viewed by admin users.

• An archived project will not be restored automatically if a user tries to access it.

• An archived project can only be restored manually by a user with OWNERSHIP privileges on the project or users in the ADMIN role.

• It takes approximately five minutes to restore an archived project.

Viewing All Projects

To view the list of all projects:

1. Click the Dremio Services icon in the side navigation bar and select Sonar Projects.

   The Sonar Projects page displays information about all projects in your organization.

   The following table describes project statuses:

   Status	Description
   Creating	Dremio is creating the project.
   Active	The project is active and can be accessed immediately.
   Inactive	The project has been deactivated. Dremio will activate the project if any user tries to access it, or you can activate from the ellipsis menu (…) to the right.
   Deactivating	Dremio is deactivating the project.
   Activating	Dremio is activating the project.
   Archiving	Dremio is archiving the project.
   Archived	The project has been archived.
   Restoring	Dremio is restoring an archived project.

Adding a Project

Before you add a project, ensure that there is an S3 bucket in your AWS account that you can give the project access to. Dremio Cloud uses the bucket as the metadata store for your project. If you need to create an S3 bucket, follow the steps in AWS: Create a Dremio Cloud Project Store.

To add a project:

1. In the top-right corner of the Sonar Projects page, click the Add Project button.

2. For Project Name, specify a name that is unique within the organization.

3. For Cloud, select the cloud in which to create the project store and compute engines for running queries against datasets in the project.

4. Complete one of the following options:

   a. Click Launch CloudFormation Template, which will navigate you to the AWS Console. For next steps, see Adding a Project Using the CFT.

   b. If you would like to create the cloud resources and Sonar project manually, click the link at the bottom of the Add Project dialog. For next steps, see Adding a Project Manually.

Adding a Project with an AWS CloudFormation Template (CFT)

Follow the steps on the Quick Create Stack page in the AWS Console:

1. For Stack name, enter a name to identify the stack. This name must be unique to your account and cannot include underscores.

2. (Optional) For Project Store, a unique name has been generated for the S3 bucket that will be created to store metadata. You can specify a different name here but it must be a unique name across AWS.

3. (Optional) For Encryption Type, SSE-S3 is selected by default to encrypt the project store S3 bucket. You can select SSE-KMS (AWS Managed Key) or SSE-KMS (Customer Managed Key) if additional encryption is required.

4. (Optional) If you select SSE-KMS (Customer Managed Key), specify the KMS Key ARN.

5. Select I acknowledge that AWS CloudFormation might create IAM resources.

6. Click Create stack.

Stack creation can take up to four minutes. When stack creation is complete, the required cloud resources have been configured successfully.

Adding a Project Manually

A Storage Access section will appear in the Add Project dialog. Follow the steps below for completing this section:

1. For Project Store, paste the path of the Amazon S3 bucket. Doing so sets the path in the Resource section of the policy JSON. The rest of the policy JSON sets the minimum policy requirement to allow Dremio Cloud read and write access to the project store.

2. Copy the content of the Policy JSON text box.

3. To grant Dremio Cloud read and write permissions to the project store, follow either of these two sets of steps:

• To grant the permissions by means of an IAM user:

  a. For Project Data Credentials, select Access Key.

  b. Create an IAM user with an access key.

  c. For AWS Access Key ID, paste the Access Key ID from the .csv file you downloaded when you created the user.

  d. For AWS Secret Access Key, paste the AWS Secret Access Key from the .csv file you downloaded when you created the user.

• To grant the permissions by means of a cross-account IAM role:

  a. For Project Data Credentials, select IAM Role.

  b. Copy this JSON snippet and paste it into a location that you can access for the rest of this procedure.

  Policy JSON

      {
        "Effect": "Allow",
        "Principal": {
          "AWS": "arn:aws:iam::<trust account ID>:root"
        },
        "Action": "sts:AssumeRole",
        "Condition": {
          "StringEquals": {
            "sts:ExternalId": "<external ID>"
          }
        }
      }
  

  c. Copy the trust account ID and the external ID and paste them into the JSON that you copied in the previous step.

  d. Create a cross-account IAM role in the AWS Console.

  e. For AWS Cross-Account Role ARN, paste the cross-account role ARN that you copied when you created the role.

  f. For AWS Instance Profile ARN, paste the instance profile ARN that you copied when you created the role.

4. Click Add. A new project has been manually added to your organization.

Activating a Project

You can activate an inactive project on the Projects page, or by clicking the project in the project selector.

To activate a project from the Projects page:

1. Click the Dremio Services icon in the side navigation bar and select Sonar Projects.

2. Click the ellipsis menu to the far right of the inactive project, and then click Activate Project.

   The project status will change to Activating while the project is activated. You can access the project after the status changes to Active.

To activate a project from the project selector:

1. Hover over the project selector in the side navigation bar.

2. Click the name of an inactive project.

   Dremio displays a page that tells you the project had been deactivated and it will be activated in a few minutes. When the project has been activated, Dremio displays the project home page in the Datasets view.

Archiving a Project (Project Owners and Admin Users Only)

1. Click the Dremio Services icon in the side navigation bar and select Sonar Projects.

2. Click the ellipsis menu to the far right of an active or inactive project, and then click Archive Project.

   The project status will change to Archiving while the project is archived. When archiving is complete, the status changes to Archived.

Restoring an Archived Project (Project Owners and Admin Users Only)

1. Click the Dremio Services icon in the side navigation bar and select Sonar Projects.

2. Click the ellipsis menu to the far right of an archived project and select Restore Project.

   The project status will change to Restoring while the project is restored. You can access the project after the status changes to Active.

Setting the Default Project

When your data consumers connect to Dremio Cloud from BI tools, they need to connect to the projects in which their datasets are located. They can either connect to the default project or they can select a different project to connect to.

If an organization administrator does not set this value, Dremio Cloud automatically sets the default project to the oldest project in your organization.

You can change the default project at any time.

To specify the default project for your organization:

1. Click the Organization icon in the side navigation bar.

2. Click the Settings icon at the top of the Organization page.

3. Select General Information in the organization settings sidebar.

4. In the Default Project field, select the project that you want data consumers to connect to by default through their BI tools.

5. Click Save.

Obtaining the ID of a Project

A BI client application might require the ID of a project as part of the information for creating a connection to Dremio Cloud. You can obtain the ID from the General Information page of a project’s settings.

To obtain a project ID:

1. Click the Project Settings icon in the side navigation bar.

2. Select General Information in the project settings sidebar.

3. Copy the value in the Project ID field.

Editing a Project

The name is the only editable attribute of a project.

To edit the name of a project:

1. Click the Project Settings icon in the side navigation bar.

2. Select General Information in the project settings sidebar.

3. In the Project Name field, change the name and click Save.

Enabling Autocomplete

To help you construct SQL statements more quickly, you can enable autocomplete. Enabling autocomplete can provide suggestions for SQL keywords, catalog objects, and functions while you are constructing SQL statements. This feature can be enabled or disabled for all your users within the project settings, and once the setting is toggled, you must refresh your browser to see the changes.

To enable autocomplete for a project:

1. Click the Project Settings icon in the side navigation bar.

2. Select SQL in the project settings sidebar.

3. Toggle the autocomplete setting to enable.

Deleting a Project

If you delete the default project for your organization, Dremio Cloud automatically selects the oldest project to be the new default. You can override this default. See Setting the Default Project for an Organization.

To delete a project:

1. Click the Project Settings icon in the side navigation bar.

2. Select General Information in the project settings sidebar.

3. Click Delete Project.

4. Confirm that you want to delete the project.