On this page

    Tarball Install and Upgrade

    This topic describes how to install and upgrade Dremio using a Tarball package.

    Installing Dremio

    Perform these steps on each node of your Dremio cluster:

    1. Create a dremio group.

      Create dremio group
      sudo groupadd -r dremio
      
    2. Create a dremio user.

      Create dremio user
      sudo useradd -r -g dremio -d /var/lib/dremio -s /sbin/nologin dremio
      
    3. Create Dremio directories.

      Create dremio directories
      sudo mkdir /opt/dremio
      sudo mkdir /var/run/dremio && sudo chown dremio:dremio /var/run/dremio
      sudo mkdir /var/log/dremio && sudo chown dremio:dremio /var/log/dremio
      sudo mkdir /var/lib/dremio && sudo chown dremio:dremio /var/lib/dremio
      
    4. Unpack Dremio into the /opt/dremio directory.

      Unpack dremio into /opt/dremio
      sudo tar xvf dremio-<VERSION>.tar.gz -C /opt/dremio --strip-components=1
      
    5. Create a symbolic link for the configuration folder.

      Create symbolic link for configuration folder
      sudo ln -s /opt/dremio/conf /etc/dremio
      
    6. (Optional) Set up your service configuration (SystemD setup).

      a. Copy the unit file for the dremio daemon from <DREMIO_HOME>/share/dremio.service to /etc/systemd/system/dremio.service.

      Copy and move unit file for dremio daemon
      sudo cp <DREMIO_HOME>/share/dremio.service /etc/systemd/system/dremio.service
      

      b. Restart systemd.

      Restart systemd
      sudo systemctl daemon-reload
      

      c. (Optional) Enable dremio to start at boot.

      Enable dremio to start at boot (optional)
      sudo systemctl enable dremio
      
    7. Configure Dremio. See Dremio Configuration for more information.

    Sample Service Configuration

    The following snippet shows the configuration in the <DREMIO_HOME>/share/dremio.service file:

    Sample service configuration
        #
        # Installation is assumed to be under /opt/dremio
        #
        [Unit]
        Description=Dremio Daemon Server
        Documentation=https://docs.dremio.com
        Wants=network-online.target
        After=network-online.target
        
        [Service]
        User=dremio
        Group=dremio
        RuntimeDirectory=dremio
        ExecStart=/opt/dremio/bin/dremio start-fg
        Restart=on-failure
        # Set maximum number of file descriptors Dremio can open.
        # Dremio requires a large number to work correctly.
        LimitNOFILE=65536
        
        
        [Install]
        WantedBy=multi-user.target
    

    Upgrading Dremio

    This topic provides upgrade instructions for Tarball-based installations.

    Before you begin, do the following:

    To upgrade your Dremio installation:

    1. Ensure that your existing Dremio cluster is backed up. See the Create a backup section.

    2. Shut down all Dremio provisioned nodes. For example: <DREMIO-HOME>/bin/dremio stop

    3. On each Dremio node in your cluster, move your old Dremio installation to another location. For example:

      Move Dremio installation
      mv /opt/dremio /opt/dremio_bak
      
    4. Download and install the new Dremio version.

      • For YARN deployments, install on each coordinator node in your Dremio cluster.
      • For non-YARN deployments, install on all nodes in your Dremio cluster.

      Perform the following steps on each Dremio node depending on your deployment (YARN vs non-YARN):

      Note
      Important: Be sure the create the new directories with the same path as the original installation. In particular, the /data directory and associated data files must be in the same path as the original installation.

      1. Create a new Dremio directory. For example:

        Create new dremio directory
        sudo mkdir /opt/dremio
        
      2. Unpack Dremio into the /opt/dremio directory. For example:

        Unpack dremio into /opt/dremio
        sudo tar xvf dremio-<version>.tar.gz -C /opt/dremio --strip-components=1
        
      3. Create a symbolic link for the configuration folder. For example:

        Create symbolic link for configuratin folder
        sudo ln -s /opt/dremio/conf /etc/dremio
        
    5. Copy the /conf directory and associated configuration files from your backup-copy location (from Step# 3) to the new installation location. As a result, the default files from the fresh install are overwritten. For example:

      Copy and move /conf directory and files
      cp /opt/dremio_bak/conf/* /opt/dremio/conf
      
      • For YARN deployments, perform this step on each coordinator node in your Dremio cluster.
      • For non-YARN deployments, perform this step on all nodes in your Dremio cluster.
    6. On the coordinator node, copy the /data directory and associated data files from your backup-copy location (from Step# 3) to the new installation location. For example:

      Copy and move /data directory and files on coordinator node
      cp -r /opt/dremio_bak/data /opt/dremio/data
      

      Note
      Important: The new /data directory and files must be in the same directory path as the original installation. A different directory path results in the loss of all reflections because they are accessed by their full path.

      If your original data directory is in a location different from /opt/dremio/data, then this step can be skipped.

    7. On the coordinator node, execute the upgrade script as the user running the Dremio Daemon service. For example, as “dremio”:

      Execute upgrade script on coordinator node
      <DREMIO-HOME>/bin/dremio-admin upgrade
      
    8. Start up all Dremio provisioned nodes in the Dremio cluster. For example:

      Start Dremio provisioned nodes
      <DREMIO-HOME>/bin/dremio start
      
    9. Backup your new Dremio cluster. See Create a backup for more information.