On this page

    4.1 Release Notes

    What’s New

    Multiple AWS Clusters

    In AWS deployments, Dremio supports the ability to provision multiple separate execution clusters from a single Dremio coordinator node, dynamically schedule execution clusters to run idependently at different times and automatically start and stop based on workload requirements at runtime.

    4.1.1 Release Notes

    Enhancements in 4.1.1

    Mathematical Functions

    Added the option to provide a seed value to the RANDOM() function, enables users to generate consistent results within queries that use RANDOM() for testing and verification purposes. See Math Functions for additional details.

    Fixed Issues in 4.1.1

    Running the ALTER PDS command with different case for the table name removed the table’s metadata
    Resolved by supporting mixed case in table names in the ALTER PDS command.

    Selecting all columns in a JDBC query but in a different order caused the query to fail
    If all columns for a table were selected in a JDBC query the request was processed as SELECT * even if the columns were in a different order, which failed schema consistency checks. Resolved by explicitly using column names when provided.

    C3 not enabled by default for Reflections stored on Cloud Storage
    Refection Distribued Storage enables C3 by default when using Cloud Storage.

    Substitution with many reflections on leaf nodes may take long time to complete in planning
    Resolved by enhancing reflection substitution logic performance by reducing unneeded candidates.

    Queries with reflections can timeout during query planning
    Resolved by improving the implication check used by filters during acceleration planning

    Aggregate reflection substitution gives wrong results when querying with DATE_ADD
    Resolved by fixing reflection substitution logic with the DATE_ADD function.

    REVERSE function on Hive tables with ORC files fails with IndexOutOfBoundsException
    Resolved by fixing REVERSE function processing in Hive tables

    Operations to view reflection status, either through sys.reflections or the Admin UI can hang
    Resolved by always retreiving reflection status from the metadata store

    Improved memory default settings for servers with larger memory capacity
    Default memory setttings for coordinator and exector nodes were updated if not specified, refer to Configuring Memory for the updated default settings.

    CLOB/BLOB Data Types in Oracle not supported
    Added support for the CLOB and BLOB Data Types in Oracle and support for unknown types in other relational sources.

    Incorrect Partitions selected when filtering based on Decimal values
    Resolved by fixing an issue with partion selection with Decimal values which could result in missing valid partitions.

    Query fails with java.lang.IndexOutOfBoundsException
    Resolved by improving spilling in the HashAgg Operator

    4.1.2 Release Notes

    Fixed Issues in 4.1.2

    Attempts to read data from Azure Data Lake Storage fail with “DATA_READ ERROR: java.util.concurrent.TimeoutException”
    Resolved by correcting issue with Azure SDK and asynchronous reads

    No error shown if invalid credentials are provided for an Azure Storage source with a storageV2 type
    Resolved by ensuring credential errors are reported correctly with storageV2 types

    Executor nodes fail to start in YARN environments with “ERROR - java.lang.RuntimeException: Unable to initialize Initializer LdapBootstrapAdminUserInitializer”
    Resolved by correcting startup exception error related to LDAP and YARN

    4.1.3 Release Notes

    Fixed Issues in 4.1.3

    Unable to connect to Azure sources when using the Azure Active Directory authentication method
    Resolved by correcting an authentication issue in date format handling

    4.1.4 Release Notes

    Enhancements in 4.1.4

    Backup is now considerably faster
    Backups are now written in binary mode versus in JSON which is considerly faster plus backups are now multithreaded to speed up the overall process. Additionally a new option was added to include profiles in the backup. See Backups for details.

    Fixed Issues in 4.1.4

    In clause optimization not happening for NOT IN queries
    Resolved by preserving the IN/NOT IN form during the optimization phase rather than being converted to an OR

    Upgraded the Jackson library version to 2.10.2

    Memory leak in spoolingBatchBuffer
    Resolved by fixing the leak in an internal data structure

    Unexpected error occurred when doing filter in UI when preview result set has zero rows
    Resolved by not displaying the chart if the table doesn’t have any rows

    Reflection matching failing due to java.lang.ArrayIndexOutOfBoundsException
    Resolved by fixing a bug in an internal data structure

    Unable to use WANdisco with Hive and Dremio 4.0
    Resolved by adding the ability to add custom dependencies to the Hive plugin bundles. Dremio loads Hive related classes in separate classloaders in Dremio 4.0 and later, so dependencies for Hive now need to be placed in the following locations:

    • CE - Hive 2 - dremio\plugins\connectors\hive2.d
    • CE - Hive 3 - dremio\plugins\connectors\hive3.d
    • EE - Hive 2 - dremio\plugins\connectors\hive2-ee.d
    • EE - Hive 3 - dremio\plugins\connectors\hive3-ee.d

    Advanced reflection view on a dataset doesn’t show the reflections in Safari
    Resolved to ensure that the view works with the Safari browser

    Unable to add Elasticsearch 6.7 on Dremio 4.1
    Resolved by fixing adding some of the Elastic sources needed to support the latest version of Jersey

    Tableau closes sometimes with “RuntimeAssert: We have a Disconnect exception, but DataSourceException::Name was not set!"
    Resolved by adding upgrading to Avatica to 1.12 to support the Connection.isValid implementation

    Unable to push down TRUNCATE on decimal columns
    Resolved by supporting the push down of Truncate on decimal columns

    For SQL Server, errors of expression type uniqueidentifier is invalid for COLLATE clause
    Resolved by not adding a COLLATE clause in SQL queries for columns of type uniqueidentifier.

    4.1.6 Release Notes

    Known Issues in 4.1.6

    Certificate errors when SSL/TLS is enabled with the Web UI
    If certificates are used to enable SSL/TLS with the Web UI, please hold off upgrading to Dremio 4.1.6 until resolved

    Fixed Issues in 4.1.6

    Orphaned YARN containers cause stability issues
    Improved the watchdog process that monitors YARN containers to ensure orphaned processes are removed. YARN will automatically start another container with Dremio.

    REGEXP_LIKE() is not pushed down
    Added pushdown support for REGEXP_LIKE with Oracle, PostgreSQL and Teradata

    After several hours or days if nodes are left blacklisted queries begin to fail with an error: IllegalStateException: Failed to find receiver for sender (71)
    Resolved by sending the current list of executors and excluding blacklisted nodes.

    OrcRawRecordMerger uses excessive heap memory when there are too many delta files with an ORC ACID table
    Caused by too many Hive ORC deltas, improved the error message to recommend running Hive compaction on the table to correct the issue.

    Results vary depending on where LIKE operator is being used in WHERE clause
    Resolved by preserving all non-simple conditions in SimpleFilterFinder

    Locally attached dremio-admin backup does not function on OSX
    Resolved by adding appropriate Security Context for dremio-admin backup in local-attach mode

    BlackDuckScan identified CVEs related to Apache Thrift 0.9.3
    Resolved by upgrading version of libthrift

    Some queries take several seconds to plan and subseconds to execute
    Resolved by improving the performance of Pre-Logical Filter Pushdown

    TRUNC on Redshift is not pushed down
    Added pushdown support for TRUNC with Redshift

    Decimal values with out of range precision or scale are not shown in Dremio
    Resolved by truncating values if they are beyond the supported range and displaying within Dremio.

    S3 Gov Cloud connection not working
    Added support for S3 STS authentication without compat mode

    Unable to run CTAS on S3 with the error Creating buckets is not supported
    Resolved by including the configured basePath in the path resolution

    Removed in 4.1.6

    • MapR (Community Edition)

    4.1.7 Release Notes

    Fixed Issues in 4.1.7

    Certificate errors when SSL/TLS is enabled with the Web UI
    Resolved keystore issues with multiple certificates or wildcard characters

    High planning time for queries caused by too many partition values
    Resolved by fixing long partition pruning times with multiple large partition chunks

    4.1.8 Release Notes

    Fixed Issues in 4.1.8

    RDBMS Queries with multiple joined tables with duplicate column names can result in ambiguous column references
    Resolved by ensuring distinct columns within SQL of pushdown queries.

    Some file errors don’t expose the path associated with the error
    Add the file path for these errors to the query profile.

    Attempting to query Parquet files with conflicting complex nested types would sometimes fail
    Resolved by skipping better handling change in schema

    Failures sometimes occurred when working with certain correlated queries on Postgres
    Changed conditions to avoid failure

    Difficulty determining state of system when are queries are cancelled at resource exhaustion
    Additional context information is now shown when such failures occur.

    Errors when trying to filter or flatten nested data that included decimals in nested fields
    Addressed by handling nested decimal fields within more processing operations.

    Slow metadata operations when connecting RDBMS systems that fail to declare scale and/or precision for Decimal columns
    Updated handling to coerce queries and data results to common scale/precisions according to Dremio decimal rules.

    Memory leak associated with adding/removing Azure sources large numbers of times
    Resolved by ensuring all resources are released when Azure sources are deleted.

    Hive external table pointing to ADLS does not work on 4.0.x
    Resolved by fixing classloader when accessing HIVE ADLS table