System Requirements
Operating System
Linux Distributions
Dremio supports the following distributions and versions of Linux:
- RHEL and CentOS 6.7+ and 7.3+
- SLES 12 SP2+
- Ubuntu 14.04+
- Debian 7+
Server or Instance Hardware
Dremio is designed to run on clusters of 1-1000 servers (on premise) or instances (cloud). You can get started by running Dremio on a standard laptop, but when you're ready to evaluate the performance or scalability of the system, or deploy it in production, it's important to utilize adequate hardware.
| Dremio Node Role | Hardware Required |
|---|---|
| Coordinators | 8 CPU cores recommended 16GB RAM recommended |
| Executors | 4 CPU core minimum (16 cores recommended) 16GB RAM minimum (128GB recommended) Note: Even if you have a machine with 64GB of RAM, only 16GB is used by default. To change this setting, modify the DREMIO_MAX_DIRECT_MEMORY_SIZE_MB property in the dremio-env file and restart the executor node(s). |
Server System
Dremio can be deployed on Linux, Mac and Windows. For production deployments, Linux is the recommended operating system.
An RPM package is available for RHEL 6+ and CentOS 6+. A Tarball is available for other operating systems.
To install Dremio, the following access privileges are required:
sshandscpaccessrootorsudoprivileges
Java Development Kit
Dremio requires that Java SE 8 (also known as JDK 1.8) be installed. Supported distributions are OpenJDK and Oracle JDK. Other versions are currently not supported.
Dremio utilizes the Java compiler (javac) for runtime code generation.
You can check to see if your operating system has Java installed (and which version) with this command:
$ java -version
You can download the latest version of Java SE from the Oracle website.
Network
There should be a low-latency, high-bandwidth network connection between Dremio and the data sources.
The following ports must be open:
| Purpose | Port | From | To |
|---|---|---|---|
| UI (HTTPS) | 9047 | Corporate network (end users) | Coordinators |
| ODBC/JDBC clients (e.g., Tableau, Power BI) | 31010 | Corporate network (end users) | Coordinators |
| ZooKeeper (internal) | 2181 | Other Dremio nodes (coordinators and executors) | Coordinators |
| Inter-node communication | 45678 | Other Dremio nodes | Executors |
| Data source reads | Varies | All Dremio nodes | Data source nodes |
Web Sockets
Dremio uses Web Sockets. If you encounter the following error message: "Your Internet connection may be offline, or WebSockets to Dremio are being blocked.", ensure that your environment allows WebSocket communication.
Performance
A 10 GbE network is recommended when connecting to large data sources that hold terabytes or petabytes of data.
In particular, for maximum performance, it is recommended to use a 10 GbE network between coordinators and executors, executors and executors, and executors and data sources.
Best Practices
For Unix/Linux operating systems, increase your open file limit for users (this impacts the Dremio processes) to 65536.
If you have a machine with a large amount of RAM (for example, 64GB), increase the Dremio RAM configuration setting from the default (16GB). To increase Dremio RAM:
- Modify the
DREMIO_MAX_DIRECT_MEMORY_SIZE_MBproperty in the dremio-env file.
WARNING: Be sure to leave at least 1-2 GB of memory for the operating system. - Restart the executor node(s).
- Modify the
[warning] Warning: For the
DREMIO_MAX_DIRECT_MEMORY_SIZE_MBallocation, be sure to leave at least 1-2 GB of memory for the OS.
- To verify the memory assigned to the nodes (heap and direct), run the following query:
SELECT * FROM sys.memory