Restore Dremio

This topic describes the Dremio CLI command, dremio-admin restore, and general information about restoring Dremio from a backup.

[info] This utility is only available on Mac and Linux.

Dremio metadata and user uploaded files can be backed up and restored. Doing a restore does not restore the contents of the distributed cache such as acceleration cache, downloaded files and query results.

Requirements

• All cluster nodes are completely shut down. See Startup/Shutdown for more information.
• This command is run on the master node.

[warning] Important

A backup can only be restored using the same version of Dremio that the backup was created on.

Syntax

<dremio_home>/bin/dremio-admin restore -d <BACKUP_PATH> [additional options]

Options

To obtain a list of restore options on the command line:

./dremio-admin restore -h

Restore options:

  * -d, --backupdir
      backup directory path. for example, /mnt/dremio/backups or 
      hdfs://$namenode:8020/dremio/backups 
    -h, --help
      show usage
    -r, --restore
      restore dremio metadata
      Default: false
    -v, --verify
      verify backup contents
      Default: false

Example

./dremio-admin restore -d /tmp/dremio_backup

Restoring Dremio Step-by-step

The following are step-by-step instructions for restoring Dremio from a backup.

1. Make sure all cluster nodes are shutdown.

2. On the master node, create a copy of <DREMIO_LOCAL_DATA_PATH>. (e.g. /data/dremio/ depending on your setup)

3. On the master node, delete contents of <DREMIO_LOCAL_DATA_PATH> and then create an empty directory called db readable and writable by the user running restore tool and Dremio daemon under <DREMIO_LOCAL_DATA_PATH>.

4. On the master node, run the following command located under <DREMIO_HOME>/bin/ to verify backup where -v option verifies backup contents.

     $ ./dremio-admin restore 
     -d <BACKUP_FOLDER_PATH>
     -v
   

5. If above step is successful, run the following command located under <DREMIO_HOME>/bin/ where -r option initiates a restore.

     $ ./dremio-admin restore 
     -d <BACKUP_FOLDER_PATH>
     -r
   

6. Look for the confirmation message. For example:

    ...
    Restored from backup at /tmp/dremio_backup_2017-02-23_18.25, dremio tables 14, uploaded files 1
   

Troubleshooting

Dremio on Edge Nodes

Problem
When Dremio is running on a edge node (Hadoop client installed) and a dremio-admin restore -v or -r is performed, by default, it looks at HDFS and comes back with file does not exist. The folder/file obviously does not exist is Hadoop.

Restore fails with the following stack:

Error Message: java.io.FileNotFoundException: File /tmp/dremiobackup does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:901)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:112)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:961)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:958)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:958)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1537)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1580)
at com.dremio.dac.util.BackupRestoreUtil.scanInfoFiles(BackupRestoreUtil.java:191)
at com.dremio.dac.util.BackupRestoreUtil.validateBackupDir(BackupRestoreUtil.java:230)
at com.dremio.dac.cmd.Restore.main(Restore.java:81)
verify failed java.io.FileNotFoundException: File /tmp/dremiobackup does not exist

Workaround
Use file:/// to direct to local. For example, use the following command instead:

./dremio-admin restore -d file:///tmp/dremiobackup/dremio_backup_2019-04-22_20.30 -r

For More Information

• Creating a Backup for getting started with backups.