This topic describes how to install and upgrade Dremio using a Tarball package.
Perform these steps on each node of your Dremio cluster:
Create a dremio
group.
sudo groupadd -r dremio
Create a dremio
user.
sudo useradd -r -g dremio -d /var/lib/dremio -s /sbin/nologin dremio
Create Dremio directories.
sudo mkdir /opt/dremio
sudo mkdir /var/run/dremio && sudo chown dremio:dremio /var/run/dremio
sudo mkdir /var/log/dremio && sudo chown dremio:dremio /var/log/dremio
sudo mkdir /var/lib/dremio && sudo chown dremio:dremio /var/lib/dremio
Unpack Dremio into the /opt/dremio directory.
sudo tar xvf dremio-<VERSION>.tar.gz -C /opt/dremio --strip-components=1
Create a symbolic link for the configuration folder.
sudo ln -s /opt/dremio/conf /etc/dremio
(Optional) Set up your service configuration (SystemD setup).
a. Copy the unit file for the dremio
daemon from <DREMIO_HOME>/share/dremio.service
to /etc/systemd/system/dremio.service
.
sudo cp <DREMIO_HOME>/share/dremio.service /etc/systemd/system/dremio.service
b. Restart systemd
.
sudo systemctl daemon-reload
c. (Optional) Enable dremio
to start at boot.
sudo systemctl enable dremio
Configure Dremio. See Dremio Configuration for more information.
The following snippet shows the configuration in the <DREMIO_HOME>/share/dremio.service
file:
#
# Installation is assumed to be under /opt/dremio
#
[Unit]
Description=Dremio Daemon Server
Documentation=https://docs.dremio.com
Wants=network-online.target
After=network-online.target
[Service]
User=dremio
Group=dremio
RuntimeDirectory=dremio
ExecStart=/opt/dremio/bin/dremio start-fg
Restart=on-failure
# Set maximum number of file descriptors Dremio can open.
# Dremio requires a large number to work correctly.
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
This topic provides upgrade instructions for Tarball-based installations.
Before you begin, do the following:
To upgrade your Dremio installation:
Ensure that your existing Dremio cluster is backed up. See the Create a backup section.
Shut down all Dremio provisioned nodes. For example: <DREMIO-HOME>/bin/dremio stop
On each Dremio node in your cluster, move your old Dremio installation to another location. For example:
mv /opt/dremio /opt/dremio_bak
Download and install the new Dremio version.
Perform the following steps on each Dremio node depending on your deployment (YARN vs non-YARN):
Important
Be sure the create the new directories with the same path as the original installation. In particular, the /data directory and associated data files must be in the same path as the original installation.
sudo mkdir /opt/dremio
sudo tar xvf dremio-<version>.tar.gz -C /opt/dremio --strip-components=1
sudo ln -s /opt/dremio/conf /etc/dremio
Copy the /conf directory and associated configuration files
from your backup-copy location (from Step# 3) to the new installation location.
As a result, the default files from the fresh install are overwritten. For example:
cp /opt/dremio_bak/conf/* /opt/dremio/conf
On the coordinator node, copy the /data directory and associated data files
from your backup-copy location (from Step# 3) to the new installation location. For example:
cp -r /opt/dremio_bak/data /opt/dremio/data
Important
The new /data directory and files must be in the same directory path as the original installation. A different directory path results in the loss of all reflections because they are accessed by their full path.If your original data directory is in a location different from /opt/dremio/data, then this step can be skipped.
On the coordinator node, execute the upgrade script as the user running the Dremio Daemon service.
For example, as “dremio”:
dremio-<version>/bin/dremio-admin upgrade
Start up all Dremio provisioned nodes in the Dremio cluster. For example: <DREMIO-HOME>/bin/dremio start
.
Backup your new Dremio cluster. See Create a backup for more information.