On this page

    Backup Dremio

    Dremio metadata and user uploaded files can be backed up. The backup does not include the contents of the distributed cache such as acceleration cache, downloaded files and query results.

    Requirements

    • The Dremio daemon is running.
    • The backup command is run on the coordinator node.
    • The backup path is writable by the user running Dremio daemon.

    note:

    Restoring Dremio: A backup can only be restored using the same version of Dremio that the backup was created on.

    Syntax

    <dremio_home>/bin/dremio-admin backup
        -u <DREMIO_ADMIN_USER> \
        -p <DREMIO_ADMIN_PASS> \
        -d <BACKUP_PATH>
    

    Options

    To obtain a list of backup options on the command line:

    ./dremio-admin backup -h
    

    Backup options:

        -a, --accept-all
          accept all ssl certificates
          Default: false
      * -d, --backupdir
          backup directory path. for example, /mnt/dremio/backups or 
          hdfs://$namenode:8020/dremio/backups 
        -h, --help
          show usage
        -i, --include-profiles
          include profiles in backup. Available as of Dremio 4.1.4
          Default: false
        -j, --json
          Use a json formatted backup (default binary). Available as of Dremio 4.1.4
          Default: false
        -l, --local-attach
            Attach locally to Dremio JVM to authenticate user. Available as of Dremio 3.3
            Not compatible with user/password options. 
        -p, --password
          password
        -u, --user
          username (admin)
    

    note:

    If the --password flag is specified without a value, the utility prompts for one interactively.

    Example: Basic Backup

    This following example backs up Dremio to the /tmp/backup directory:

    /dremio-admin backup -u dremio -p dremio123 -d /tmp/backup
    

    A confirmation message is displayed when the backup is complete.

    Example: Backup with local-attach mode

    The following example backs up Dremio to the /tmp/backup directory in local attach mode.

    note:

    The --local-attach option is available as of Dremio 3.3

    usage

    ./dremio-admin backup -d /tmp/backup -l
    

    output message
    The output message in server.out.

    Backup created at /tmp/dremio_backup_2017-02-23_18.25, dremio tables 1400, uploaded files 150
    

    Backing Up Dremio Step-by-Step

    1. Ensure that the Dremio daemon is running, on the master node:
    2. On the master node, run the following command:
       <dremio_home>/bin/dremio-admin backup \
        -u <DREMIO_ADMIN_USER> \
        -p <DREMIO_ADMIN_PASS> \
        -d <BACKUP_PATH>
      
    3. Backup is complete! Look for the confirmation message:
    Backup created at /tmp/dremio_backup_2017-02-23_18.25, dremio tables 1400, uploaded files 150
    

    Backing Up Dremio with AWS

    For administrators with configurations using AWS, you have the option to automatically back up your Dremio project on a daily basis. With the option for this enabled, Dremio will regularly initiate snapshots of the EBS volume, which stores all of the project metadata.

    However, in some scenarios, you may need to perform a manual backup of the project, such as when you’re preparing to upgrade Dremio and a arecent snapshot of the project is advised in case of a rollback.

    Locating the EBS Volume

    1. From the AWS console, open the EC2 service and from the left-hand menu navigate to Elasticsearch Block Store > Volumes.

    2. From the volumes listing, filter by the tag dremio_project_name : <your_project_name>, replacing the text after the colon with your project’s name. You should find one volume for your project.

    3. Select the volume and open the Tags tab. Copy of all the tags and their values onto a note, as this will be needed in the next section.

    Creating a Snapshot of the Volume

    1. Select the checkbox to the left of the volume name.

    2. Click the Actions button above the filter bar.

    3. Click Create Snapshot.

    4. In the snapshot’s configuration window, add a useful description as well as the tags and values you noted in the last step of the previous section. You may also want to add a tag identifying the volume as a manual (rather than automatic) backup. While the tags may be added later, we recommend including them now for tracking purposes.

    note:

    Including these tags is required, as these are what new Dremio deployments use to discover the correct snapshot.

    1. Click Create Snapshot.

    Success is indicated by a message stating Create Snapshot Request Succeeded.

    If you need to restore from this backup, see the sectionRestoring Dremio from an AWS Backup. If you want to create a clone from this backup, see the section Cloning a Dremio Stack using Backup/Restore.