Tarball Install and Upgrade
This topic describes how to install and upgrade Dremio using a Tarball package.
Installing Dremio
Perform these steps on each node of your Dremio cluster:
Create a
dremio
group.sudo groupadd -r dremio
Create a
dremio
user.sudo useradd -r -g dremio -d /var/lib/dremio -s /sbin/nologin dremio
Create Dremio directories.
sudo mkdir /opt/dremio sudo mkdir /var/run/dremio && sudo chown dremio:dremio /var/run/dremio sudo mkdir /var/log/dremio && sudo chown dremio:dremio /var/log/dremio sudo mkdir /var/lib/dremio && sudo chown dremio:dremio /var/lib/dremio
Unpack Dremio into the /opt/dremio directory.
sudo tar xvf dremio-<VERSION>.tar.gz -C /opt/dremio --strip-components=1
Create a symbolic link for the configuration folder.
sudo ln -s /opt/dremio/conf /etc/dremio
(Optional) Set up your service configuration (SystemD setup).
a. Copy the unit file for the
dremio
daemon from<DREMIO_HOME>/share/dremio.service
to/etc/systemd/system/dremio.service
.sudo cp <DREMIO_HOME>/share/dremio.service /etc/systemd/system/dremio.service
b. Restart
systemd
.sudo systemctl daemon-reload
c. (Optional) Enable
dremio
to start at boot.sudo systemctl enable dremio
Configure Dremio. See Dremio Configuration for more information.
Sample Service Configuration
The following snippet shows the configuration in the <DREMIO_HOME>/share/dremio.service
file:
#
# Installation is assumed to be under /opt/dremio
#
[Unit]
Description=Dremio Daemon Server
Documentation=https://docs.dremio.com
Wants=network-online.target
After=network-online.target
[Service]
User=dremio
Group=dremio
RuntimeDirectory=dremio
ExecStart=/opt/dremio/bin/dremio start-fg
Restart=on-failure
# Set maximum number of file descriptors Dremio can open.
# Dremio requires a large number to work correctly.
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
Upgrading Dremio
This topic provides upgrade instructions for Tarball-based installations.
Before you begin, do the following:
- While Dremio is still running, create a backup of your Dremio cluster.
- If HDFS/MapR-FS is configured, perform a snapshot of your distributed store directories.
To upgrade your Dremio installation:
Ensure that your existing Dremio cluster is backed up. See the Create a backup section.
Shut down all Dremio provisioned nodes. For example:
<DREMIO-HOME>/bin/dremio stop
- See the Shutting Down with YARN Deployments section.
- See the Shutting Down with non-YARN Deployments section.
- See the Shutdown Commands section.
On each Dremio node in your cluster, move your old Dremio installation to another location. For example:
mv /opt/dremio /opt/dremio_bak
Download and install the new Dremio version.
- For YARN deployments, install on each coordinator node in your Dremio cluster.
- For non-YARN deployments, install on all nodes in your Dremio cluster.
Perform the following steps on each Dremio node depending on your deployment (YARN vs non-YARN):
Note
Important: Be sure the create the new directories with the same path as the original installation. In particular, the /data directory and associated data files must be in the same path as the original installation.- Create a new Dremio directory. For example:
sudo mkdir /opt/dremio
- Unpack Dremio into the /opt/dremio directory. For example:
sudo tar xvf dremio-<version>.tar.gz -C /opt/dremio --strip-components=1
- Create a symbolic link for the configuration folder. For example:
sudo ln -s /opt/dremio/conf /etc/dremio
Copy the /conf directory and associated configuration files from your backup-copy location (from Step# 3) to the new installation location. As a result, the default files from the fresh install are overwritten. For example:
cp /opt/dremio_bak/conf/* /opt/dremio/conf
- For YARN deployments, perform this step on each coordinator node in your Dremio cluster.
- For non-YARN deployments, perform this step on all nodes in your Dremio cluster.
On the coordinator node, copy the /data directory and associated data files from your backup-copy location (from Step# 3) to the new installation location. For example:
cp -r /opt/dremio_bak/data /opt/dremio/data
Note
Important: The new /data directory and files must be in the same directory path as the original installation. A different directory path results in the loss of all reflections because they are accessed by their full path.If your original data directory is in a location different from /opt/dremio/data, then this step can be skipped.
On the coordinator node, execute the upgrade script as the user running the Dremio Daemon service. For example, as “dremio”:
<DREMIO-HOME>/bin/dremio-admin upgrade
Start up all Dremio provisioned nodes in the Dremio cluster. For example:
<DREMIO-HOME>/bin/dremio start
.- See the Starting Up with YARN Deployments section.
- See the Starting Up with non-YARN Deployments section.
- See the Startup Commands section.
Backup your new Dremio cluster. See Create a backup for more information.