Tarball Install and Upgrade
This topic describes how to install and upgrade Dremio using a Tarball package.
Installing Dremio
Perform these steps on each node of your Dremio cluster:
-
Create a
Create dremio groupdremio
group.sudo groupadd -r dremio
-
Create a
Create dremio userdremio
user.sudo useradd -r -g dremio -d /var/lib/dremio -s /sbin/nologin dremio
-
Create Dremio directories.
Create dremio directoriessudo mkdir /opt/dremio sudo mkdir /var/run/dremio && sudo chown dremio:dremio /var/run/dremio sudo mkdir /var/log/dremio && sudo chown dremio:dremio /var/log/dremio sudo mkdir /var/lib/dremio && sudo chown dremio:dremio /var/lib/dremio
-
Unpack Dremio into the /opt/dremio directory.
Unpack dremio into /opt/dremiosudo tar xvf dremio-<VERSION>.tar.gz -C /opt/dremio --strip-components=1
-
Create a symbolic link for the configuration folder.
Create symbolic link for configuration foldersudo ln -s /opt/dremio/conf /etc/dremio
-
(Optional) Set up your service configuration (SystemD setup).
a. Copy the unit file for the
Copy and move unit file for dremio daemondremio
daemon from<DREMIO_HOME>/share/dremio.service
to/etc/systemd/system/dremio.service
.sudo cp <DREMIO_HOME>/share/dremio.service /etc/systemd/system/dremio.service
b. Restart
Restart systemdsystemd
.sudo systemctl daemon-reload
c. (Optional) Enable
Enable dremio to start at boot (optional)dremio
to start at boot.sudo systemctl enable dremio
-
Configure Dremio. See Dremio Configuration for more information.
Sample Service Configuration
The following snippet shows the configuration in the <DREMIO_HOME>/share/dremio.service
file:
#
# Installation is assumed to be under /opt/dremio
#
[Unit]
Description=Dremio Daemon Server
Documentation=https://docs.dremio.com
Wants=network-online.target
After=network-online.target
[Service]
User=dremio
Group=dremio
RuntimeDirectory=dremio
ExecStart=/opt/dremio/bin/dremio start-fg
Restart=on-failure
# Set maximum number of file descriptors Dremio can open.
# Dremio requires a large number to work correctly.
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
Upgrading Dremio
This topic provides upgrade instructions for Tarball-based installations.
Before you begin, do the following:
- While Dremio is still running, create a backup of your Dremio cluster.
- If HDFS/MapR-FS is configured, perform a snapshot of your distributed store directories.
To upgrade your Dremio installation:
-
Ensure that your existing Dremio cluster is backed up. See the Create a backup section.
-
Shut down all Dremio provisioned nodes. For example:
<DREMIO-HOME>/bin/dremio stop
- See the Shutting Down with YARN Deployments section.
- See the Shutting Down with non-YARN Deployments section.
- See the Shutdown Commands section.
-
On each Dremio node in your cluster, move your old Dremio installation to another location. For example:
Move Dremio installationmv /opt/dremio /opt/dremio_bak
-
Download and install the new Dremio version.
- For YARN deployments, install on each coordinator node in your Dremio cluster.
- For non-YARN deployments, install on all nodes in your Dremio cluster.
Perform the following steps on each Dremio node depending on your deployment (YARN vs non-YARN):
Note
Important: Be sure the create the new directories with the same path as the original installation. In particular, the /data directory and associated data files must be in the same path as the original installation.-
Create a new Dremio directory. For example:
Create new dremio directorysudo mkdir /opt/dremio
-
Unpack Dremio into the /opt/dremio directory. For example:
Unpack dremio into /opt/dremiosudo tar xvf dremio-<version>.tar.gz -C /opt/dremio --strip-components=1
-
Create a symbolic link for the configuration folder. For example:
Create symbolic link for configuratin foldersudo ln -s /opt/dremio/conf /etc/dremio
-
Copy the /conf directory and associated configuration files from your backup-copy location (from Step# 3) to the new installation location. As a result, the default files from the fresh install are overwritten. For example:
Copy and move /conf directory and filescp /opt/dremio_bak/conf/* /opt/dremio/conf
- For YARN deployments, perform this step on each coordinator node in your Dremio cluster.
- For non-YARN deployments, perform this step on all nodes in your Dremio cluster.
-
On the coordinator node, copy the /data directory and associated data files from your backup-copy location (from Step# 3) to the new installation location. For example:
Copy and move /data directory and files on coordinator nodecp -r /opt/dremio_bak/data /opt/dremio/data
Note
Important: The new /data directory and files must be in the same directory path as the original installation. A different directory path results in the loss of all reflections because they are accessed by their full path.If your original data directory is in a location different from /opt/dremio/data, then this step can be skipped.
-
On the coordinator node, execute the upgrade script as the user running the Dremio Daemon service. For example, as “dremio”:
Execute upgrade script on coordinator node<DREMIO-HOME>/bin/dremio-admin upgrade
-
Start up all Dremio provisioned nodes in the Dremio cluster. For example:
Start Dremio provisioned nodes<DREMIO-HOME>/bin/dremio start
- See the Starting Up with YARN Deployments section.
- See the Starting Up with non-YARN Deployments section.
- See the Startup Commands section.
-
Backup your new Dremio cluster. See Create a backup for more information.