On this page

    Managing Projects

    A project isolates compute, data and other resources needed by a team for data analysis. An organization may contain multiple projects. A project must be linked to a single cloud account. Your first project was created and linked to your cloud account as part of the sign-up process.

    Each project requires access to an S3 bucket for storing various types of data, including:

    • The data for reflections that are created in the project
    • File uploads to home spaces
    • The default path for new tables that are used for data and manifests for datasets
    • All of the tables that store records of events and other historical data

    Each project in your organization contains a preview engine. The preview engine is always on by default, which incurs AWS EC2 costs. You can manually disable this engine, or any other engine, under Project Settings > Engines.

    note:

    To help avoid unnecessary costs, Dremio automatically deactivates any project that has not been accessed in the last 15 days. Dremio sends a courtesy email to project owners three days prior to deactivation.

    In addition, users with OWNERSHIP privileges or users in the ADMIN role can archive a project. Inactive and archived projects do not consume any EC2 resources and do not incur any costs in your AWS account.

    Deactivated Projects

    Dremio displays deactivated/inactive projects in the project selector in the side navigation bar and on the Projects page.

    • An inactive project will be activated automatically when any user tries to access it through the Dremio app or via an API call.

    • You can activate a project manually from the Projects page.

    • It takes approximately five minutes to activate a project.

    Archived Projects

    Dremio displays archived projects only on the Projects page, and they can only be viewed by admin users.

    • An archived project will not be restored automatically if a user tries to access it.

    • An archived project can only be restored manually by a user with OWNERSHIP privileges on the project or users in the ADMIN role.

    • It takes approximately five minutes to restore an archived project.

    Viewing All Projects

    To view the list of all projects:

    1. Click the Settings (gear) icon in the side navigation bar, then click Organization Settings.

    2. Click Projects in the navigation panel on the left.

      The Projects page displays information about all projects in your organization.

      The following table describes project statuses:

      StatusDescription
      CreatingDremio is creating the project.
      ActiveThe project is active and can be accessed immediately.
      InactiveThe project has been deactivated. Dremio will activate the project if any user tries to access it, or you can activate from the ellipsis menu (…) to the right.
      DeactivatingDremio is deactivating the project.
      ActivatingDremio is activating the project.
      ArchivingDremio is archiving the project.
      ArchivedThe project has been archived.
      RestoringDremio is restoring an archived project.

    Adding a Project

    Before you add a project, ensure that there is an S3 bucket in your AWS account that you can give the project access to. Dremio Cloud uses the bucket as the metadata store for your project. If you need to create an S3 bucket, follow the steps in AWS: Create a Dremio Cloud Project Store.

    warning:

    Once you create the project, you cannot move it to a different cloud.

    To add a new project:

    1. In the top-right corner of the Projects page, click Add Project.

    2. In the Project Name field, specify a name that is unique within the organization.

    3. In the Cloud field, select the cloud in which to create the project store and compute engines for running queries against datasets in the project.

    4. In the Project Store field, paste the path of the Amazon S3 bucket. Doing so sets the path in the Resource section of the policy JSON. The rest of the policy JSON sets the minimum policy requirement to allow Dremio Cloud read and write access to the project store.

    5. Copy the content of the Policy JSON text box.

    6. To grant Dremio Cloud read and write permissions to the project store, follow either of these two sets of steps:

      • To grant the permissions by means of an IAM user:

        a. In the Project Data Credentials field, select Access Key.

        b. Create an IAM user with an access key.

        c. In the AWS Access Key ID field, paste the Access Key ID from the .csv file you downloaded when you created the user.

        d. In the AWS Secret Access Key field, paste the AWS Secret Access Key from the .csv file you downloaded when you created the user.

      • To grant the permissions by means of a cross-account IAM role:

        a. In the Project Data Credentials field, select IAM Role.

        b. Copy this JSON snippet and paste it into a location that you can access for the rest of this procedure.

        Policy JSON
            {
              "Effect": "Allow",
              "Principal": {
                "AWS": "arn:aws:iam::<trust account ID>:root"
              },
              "Action": "sts:AssumeRole",
              "Condition": {
                "StringEquals": {
                  "sts:ExternalId": "<external ID>"
                }
              }
            }
        

        c. Copy the trust account ID and the external ID and paste them into the JSON that you copied in the previous step.

        d. Create a cross-account IAM role in the AWS Console.

        e. In the Role ARN field, paste the cross-account role ARN that you copied when you created the role.

        f. In the Instance Profile ARN field, paste the instance profile ARN that you copied when you created the role.

    7. Click Save.

    Activating a Project

    You can activate an inactive project on the Projects page, or by clicking the project in the project selector.

    To activate a project from the Projects page:

    1. Click the Settings (gear) icon in the side navigation bar, then click Organization Settings.

    2. Click Projects in the navigation panel on the left.

    3. Click the ellipsis menu to the far right of the inactive project, and then click Activate Project.

      The project status will change to Activating while the project is activated. You can access the project after the status changes to Active.

    To activate a project from the project selector:

    1. Hover over the project selector in the side navigation bar.

    2. Click the name of an inactive project.

      Dremio displays a page that tells you the project had been deactivated and it will be activated in a few minutes. When the project has been activated, Dremio displays the project home page in the Datasets view.

    Archiving a Project (Project Owners and Admin Users Only)

    note:

    You cannot archive an Arctic project or the default project.

    1. Click the Settings (gear) icon in the side navigation bar, then click Organization Settings.

    2. Click Projects in the navigation panel on the left.

    3. Click the ellipsis menu to the far right of an active or inactive project, and then click Archive Project.

      The project status will change to Archiving while the project is archived. When archiving is complete, the status changes to Archived.

    Restoring an Archived Project (Project Owners and Admin Users Only)

    1. Click the Settings (gear) icon in the side navigation bar, then click Organization Settings.

    2. Click Projects in the navigation panel on the left.

    3. Click the ellipsis menu to the far right of an archived project, and then click Restore Project.

      The project status will change to Restoring while the project is restored. You can access the project after the status changes to Active.

    Setting the Default Project

    When your data consumers connect to Dremio Cloud from BI tools, they need to connect to the projects in which their datasets are located. They can either connect to the default project or they can select a different project to connect to.

    If an organization administrator does not set this value, Dremio Cloud automatically sets the default project to the oldest project in your organization.

    You can change the default project at any time.

    note:

    Data consumers who do not have access to the default project must select a project ID for an alternative project when they connect to Dremio Cloud from their BI tools.

    To specify the default project for your organization:

    1. Click the Settings (gear) icon in the side navigation bar, then click Organization Settings.

    2. Click General Information in the navigation panel on the left.

    3. In the Default Project field, select the project that you want data consumers to connect to by default through their BI tools.

    Obtaining the ID of a Project

    A BI client application might require the ID of a project as part of the information for creating a connection to Dremio Cloud. You can obtain the ID from the General Information page of a project’s settings.

    To obtain a project ID:

    1. Click the Settings (gear) icon in the side navigation bar, then click Project Settings.

    2. Click General Information in the navigation panel on the left.

    3. Copy the value in the Project ID field.

    Editing a Project

    The name is the only editable attribute of a project.

    To edit the name of a project:

    1. On the Projects page, hover over the row of the project that you want to edit and click the edit (pencil) button.

      The Edit Project dialog is opened.

    2. In the Edit Project dialog, change the name that is in the Project Name field.

    Enabling Autocomplete

    To help you construct SQL statements more quickly, you can enable autocomplete. Enabling autocomplete can provide suggestions for SQL keywords, catalog objects, and functions while you are constructing SQL statements. This feature can be enabled or disabled for all your users within the project settings, and once the setting is toggled, you must refresh your browser to see the changes.

    To enable autocomplete for a project:

    1. Click the Settings (gear) icon in the side navigation bar, then click Project Settings.

    2. Click SQL in the navigation panel on the left.

    3. Toggle the autocomplete setting to enable.

    Deleting a Project

    If you delete the default project for your organization, Dremio Cloud automatically selects the oldest project to be the new default. You can override this default. See Setting the Default Project for an Organization.

    note:

    Every organization must contain at least one project. If your organization contains only the project that you want to delete, then you cannot delete the project until another project is created in the organization.

    To delete a project:

    1. On the Projects page, hover over the row of the project that you want to delete and click the delete (trash can) button.

    2. Confirm that you want to delete the project.