Skip to main content
Version: current [25.0.x]

Back up Dremio

Dremio metadata and user uploaded files can be backed up. The backup does not include the contents of the distributed cache such as acceleration cache, downloaded files and query results.

Requirements​

  • The Dremio daemon is running.
  • The backup command is run on the coordinator node.
  • The backup path is writable by the user running Dremio daemon.
note

You can only restore a backup using the same version of Dremio that was used to create the backup.

Syntax​

Backup syntax
<dremio_home>/bin/dremio-admin backup
    -u <DREMIO_ADMIN_USER> \
    -p <DREMIO_ADMIN_PASS> \
    -d <BACKUP_PATH>

Options​

To obtain a list of backup options on the command line:

Get a list of backup options
./dremio-admin backup -h
Backup options command output
    -a, --accept-all
      accept all ssl certificates
      Default: false
  * -d, --backupdir
      backup directory path. for example, /mnt/dremio/backups or 
      hdfs://$namenode:8020/dremio/backups 
    -h, --help
      show usage
    -i, --include-profiles
      include profiles in backup.
      Default: false
    -j, --json
      Use a json formatted backup (default binary).
      Default: false
    -l, --local-attach
        Attach locally to Dremio JVM to authenticate user.
        Not compatible with user/password options. 
    -p, --password
      password
    -u, --user
      username (admin)
note

If the --password option is specified without a value, the utility prompts for one interactively.

Example: Basic Backup​

This following example backs up Dremio to the /tmp/backup directory:

Back up to /tmp/backup
/dremio-admin backup -u dremio -p dremio123 -d /tmp/backup

A confirmation message is displayed when the backup is complete.

Example: Backup with local-attach mode​

The following example backs up Dremio to the /tmp/backup directory in local attach mode.

Back up to /tmp/backup in local attach mode
./dremio-admin backup -d /tmp/backup -l

output message
The output message in server.out.

Output for backup to /tmp/backup in local attach mode
Backup created at /tmp/dremio_backup_2017-02-23_18.25, dremio tables 1400, uploaded files 150

Backing Up Dremio Step-by-Step​

  1. Ensure that the Dremio daemon is running, on the master node:

  2. On the master node, run the following command:

    Backup command
     <dremio_home>/bin/dremio-admin backup \
      -u <DREMIO_ADMIN_USER> \
      -p <DREMIO_ADMIN_PASS> \
      -d <BACKUP_PATH>

  3. Backup is complete! Look for the confirmation message:

    Backup confirmation message
    Backup created at /tmp/dremio_backup_2017-02-23_18.25, dremio tables 1400, uploaded files 150

Backing Up Dremio with AWS​

For administrators with configurations using AWS, you have the option to automatically back up your Dremio project on a daily basis. With the option for this enabled, Dremio will regularly initiate snapshots of the EBS volume, which stores all of the project metadata.

However, in some scenarios, you may need to perform a manual backup of the project, such as when you're preparing to upgrade Dremio and a arecent snapshot of the project is advised in case of a rollback.

Locating the EBS Volume​

  1. From the AWS console, open the EC2 service and from the left-hand menu navigate to Elastic Block Store > Volumes.
  2. From the volumes listing, filter by the tag dremio_project_name : <your_project_name>, replacing the text after the colon with your project's name. You should find one volume for your project.
  3. Select the volume and open the Tags tab. Copy of all the tags and their values onto a note, as this will be needed in the next section.

Creating a Snapshot of the Volume​

  1. Select the checkbox to the left of the volume name.

  2. Click the Actions button above the filter bar.

  3. Click Create Snapshot.

  4. In the snapshot's configuration window, add a useful description as well as the tags and values you noted in the last step of the previous section. You may also want to add a tag identifying the volume as a manual (rather than automatic) backup. While the tags may be added later, we recommend including them now for tracking purposes.

    note

    Including these tags is required, as these are what new Dremio deployments use to discover the correct snapshot.

  5. Click Create Snapshot.

Success is indicated by a message stating Create Snapshot Request Succeeded.

If you need to restore from this backup, see the sectionRestoring Dremio from an AWS Backup. If you want to create a clone from this backup, see the section Cloning a Dremio Stack using Backup/Restore.