Installing and Upgrading
Installation and upgrading instructions are specific to either RPM or Tarball packages.
For information about installing or upgrading, see:
12.0.0 Upgrade Notes
Enabling Legacy Authentication Mode for Flight Server Endpoint
By default, Dremio 12.0.0 enables arrow.flight.auth2
authentication mode. However, the endpoint continues to support the legacy authentication mode from earlier Dremio versions. Dremio administraters may configure the Dremio 12.0.0 server endpoint to use the legacy authentication mode for backward compatibility with earlier Flight client applications.
To enable legacy authentication mode:
- Add a
services.flight.auth.mode
statement to your dremio.conf configuration file with a value oflegacy.arrow.flight.auth
:
services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: true,
flight: {
enabled: true,
port: 32010,
auth.mode: "legacy.arrow.flight.auth"
}
}
- Restart Dremio.
4.7 Upgrade Notes
Mapping Oracle Data Types
Dremio administrators must notify Dremio developers using Oracle data sources to update their application code before running against Dremio 4.7.0. Dremio now maps the DATE
type in Oracle data sources to the TIMESTAMP
Dremio data type. Prior to version 4.7.0, Dremio mapped DATE
to the Dremio DATE
data type. Dremio developers must update their application logic to use the TIMESTAMP
Dremio data type when mapping data types.
4.3 Upgrade Notes
Before upgrading to 4.3 from a version of Dremio prior to 3.3.1, you must first do the following:
Upgrade to any version of Dremio from 3.3.1 to 4.1.x.
Verify that Dremio has finished migrating ACLs.
Search
server.log
for the following message:... More ACLs to move: false
. The message typically appears in the log file after one day.Upgrade to Dremio 4.3.
4.2 Upgrade Notes
Before upgrading to 4.2 from a version of Dremio prior to 3.3.1, you must first do the following:
Upgrade to any version of Dremio from 3.3.1 to 4.1.x.
Verify that Dremio has finished migrating ACLs.
Search
server.log
for the following message:... More ACLs to move: false
. The message typically appears in the log file after one day.Upgrade to Dremio 4.2.
Hive Configuration
Please note that Hive source configuration using XML files such as core-site.xml and hive-site.xml are no longer inherited from the Dremio configuration directory. Please refer to Hive Configuration for instructions on where to write these files.
4.0 Upgrade Notes
Processing Time
The upgrade process may take a prolonged amount of time depending on:
- Length of the refresh cycle for reflections.
- Use of decimals for relational sources since decimal usage impacts metadata.
Decimal Mapping Behavior
When you upgrade, decimal columns in RDBMS sources show as decimal in Dremio.
Amazon S3 Distributed Store
The following exception may occur when using Amazon S3 as a distributed store and an EC2 Metadata Authentication mechanism.
Exception
java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3a.SharedInstanceProfileCredentialsProvider not found
See Configuring Distributed Storage for more information.
Dremio 2.0.3 or lower
Upgrading from Dremio 2.0.3 or lower to Dremio 4.0 is not supported.
- Upgrade to Dremio 3.x first.
- Then, upgrade to 4.0.
3.3 Upgrade Notes
- The upgrade process may take a prolonged amount of time depending on the length of the refresh cycle for reflections.
3.2 Upgrade Notes
Upgrading from Dremio 1.x to 3.2 is not supported.
To upgrade to Dremio 3.2, do so incrementally:
- Upgrade from Dremio 1.x to 3.1 first.
- Upgrade from Dremio 3.1 to 3.2.
3.0 Upgrade Notes
When starting Dremio (after upgrading to 3.0), most reflections enqueue a refresh job (typically, this happens only once).
This refresh occurs even if the reflection’s refresh interval isn’t due or the refection has “never refresh” set. Thereafter, if refresh intervals were set, then all reflections resume their usual refresh cycle. Monitor your reflection to ensure that the refresh completes successfully.Enhanced Connector Framework
With this release, Dremio is utilizing improved relational connectors with the SQL Server and Postgres connectors. This provides enhanced performance and extensive pushdown capabilities. This logic also considers UDTs (User Defined Types) to be unknown types and skips them.When upgrading from the legacy version of these connectors to the new version has the potential of breaking reports if a UDT is in the report.
If your deployment has SSL enabled for the web server using custom certificates, you must add the following property to the dremio.conf file on all coordinators before starting Dremio after upgrade:
services.coordinator.web.ssl.auto-certificate.enabled: false
See the Web Server Encryption section in Using Wire Encryption for more information.