Getting Started with YARN

Dremio's YARN quickstart provides details about how to install a Dremio YARN deployment using an RPM or Tarball.

tip

Review the System Requirements before you get started.

Install the Coordinator Node

You can install the coordinator using an RPM or Tarball.

Installing via RPM

1. Download the RPM from Dremio Downloads.

2. Open a terminal and go to the directory where you saved the RPM.

3. Install using yum.

   sudo yum install dremio-community-LATEST.noarch.rpm

4. Start Dremio.

   sudo service dremio start
   note

   Optionally, when running Dremio on a single laptop/machine that may experience network or IP changes, the following line can be added to dremio.conf under /etc/dremio/ to allow Dremio to keep working: registration.publish-host: "localhost"

Installing via Tarball

1. Download the Tarball from Dremio Downloads.

2. Open a terminal and go to the directory where you saved the Tarball.

3. Extract the Tarball.

   tar -xf dremio-community-LATEST.tar.gz

4. Navigate to the bin folder.

   cd dremio-comunity-<version>/bin

5. Start Dremio

   ./dremio start

Set up YARN

1. Open the Dremio console at http://localhost:9047.

2. Click Settings in the left sidebar, click Engines, and then click Add Engine.

   

3. Configure the available options according to the following:

   • Hadoop Engine: Select your preferred engine.

   • This is a secure engine: This option can be left as is.

   • Engine Name: Enter a name for the engine.

   • Resource Manager: Provide the Resource manager address of the Hadoop environment.

   • NameNode: Provide the NameNode address of the Hadoop environment.

   • Spill Directories: Provide a location for Dremio to spill operators like Aggs and Sorts.

   • Queue: Specify the Yarn queue where Dremio containers will be created.

   • Workers: The number of Dremio executors (containers) to be created on the Hadoop data nodes.

   • Cores per Worker: Number of CPUs per Dremio executor (container).

   • Memory per worker: Memory per Dremio executor (container).

   • Additional Properties: Specify any additional custom JVM arguments to be passed.

   • Cloud Cache: Location to write the Dremio Cloud Cache file.

4. Click Add.

5. Add sources to your deployment and you are ready to run queries.