Dremio Cloud
Dremio Software
 
On this page

    4.9 Release Notes

    What’s New

    Runtime Filtering - Non-partitioned columns

    Expanded Runtime filtering to support predicates on non-partitioned columns.

    Other Enhancements

    • Dremio provides a heap monitor for coordinator nodes to protect from heap outages. The heap monitor cancels queries that are using the maximum heap size. The feature is enabled by default.

    • Improved query planning to significantly reduce logical planning times for queries with a large number of data reflections, multi-level joins, or aggregations.

    • Improved the performance of queries when caching reflections.

    • AWS Edition supports smaller EC2 instance types for Engines making it easier to use Dremio for trial and evalutation purposes.

    Fixed Issues in 4.9.0

    UI does not allow to view all spaces when moving a VDS
    Fixed the issue by displaying the scrollbar.

    Users cannot select the dataset after previewing the schema in global search results
    Fixed the global search results UI.

    Some reflections were marked cannot accelerate in the UI and does not work
    Resolved by returning the correct type for count function during projectable aggregate conversion.

    Dremio throws a ClassCastException when a date type is used in the where clause of a query
    Fixed by internally catching the ClassCastException.

    UI displays a Something went wrong error when clicking on the graph of a VDS
    Fixed by correcting the overlapped VDS condition in the UI.

    Dremio returns the 0001-01-01 as 0000-12-13 date field on ORC tables
    Fixed by not pushing down filters for date columns.

    Incremental refresh on aggregate reflection does not aggregate
    Fixed by adding the rollup aggregate during incremental refresh.

    User encountered “This query cannot be planned - possible due to use of an unsupported feature” while executing a query
    Fixed the error message, now reads: “Query contains too many correlated subqueries”.

    User encountered reflection materialization in RUNNING state that never completes
    Fixed deleted reflection on coordinator restart.

    Job API returns fewer results when using a particular offset
    Fixed by reading multiple arrow files based on offset and limit values.

    Metadata refresh on source stops refreshing the rest of the datasets when one dataset’s number of files exceeds dremio.store.dfx.max_file limit
    Fixed by issuing a warning message when a dataset exceeds dremio.store.dfx.max_file limit and continue refresh the remaining datasets.

    User encounters ArrayIndexOutOfBoundsException when selecting a particular column in a query
    Fixed the Dremio query planner issue.

    Dremio returns a wrong record count when reflection is used in some queries
    Fixed the issue by reworking materialization partition names.

    Job status shows query still running while query is completed
    Resolved the issue by correctly terminating job status.

    dremio-admin restore fails with NPE when restoring from a faulty backup
    Fixed an issue that prevents restoration of job_info files with empty dataSetPaths.

    User encountered UnsupportedOperationException when querying parquet files stored on ADLS
    Fixed the issue by not using an union type as a partition column.

    User encountered sql2rel.RelDecorrelator.decorrelateRel when running a query
    Fixed the issue by removing multiple constant groups by keys in aggregate before decorrelation during query planning.

    User encountered query timeout due to memory issue
    Resolved the issue by freeing up jmodel and ClassGenerator objects after code generation.

    User encountered Cannot convert RexNode to equivalent Dremio expression during VDS creation
    Fixed the issue by changing order of query planning for certain cases.

    Inaccurate reported pendingTime or metadataRetrivalTime in queries.json
    Fixed the issue by correcting the reported time.

    Dremio engine stopped but executors still show as running
    Fixed the issue by fixing the node listener logic.

    Reflection creation succeed but reflection marked cannot accelerate in UI and does not work
    Fixed the issue by using the correct type for count conversion.

    User experienced slowness in Dremio performance
    Resolved by releasing CodeGenerator object early to reduce heap usage

    User encountered ClassCastException during query execution
    Resolved the issue by catching the exception to prevent query from failing.

    Long running JDBC queries incorrectly marked as Failed instead of Cancelled if user cancels them
    Fixed the issue by correcting the query status.

    Failed to access dataset in source with FileNotFoundException
    Updated the error message to be invalid metadata (For example, the file was deleted.)

    Failed to perform alter PDS FORGET METADATA | REFRESH METADATA with PATH NOT FOUND error
    Fixed by relaxing some unnecessary validation checks.

    Job status for a failed query mismatched between left and right panel on the job page UI in scale-out-coordinator setup
    Fixed by properly handling the job status update.

    Failed to access home space due to change of the username which is case sensitive
    Workaround is to delete the user’s home space using the CLI tool: ./dremio-admin delete-user-homespace -u

    Gandiva cache is not saving all project expressions
    Fixed by caching all expressions (direct or eval).

    UI failed to show all spaces when there is a long list of spaces or try to move a vds from one space to another
    Fixed the UI scroll height to view all spaces.

    Dremio helm chart v2 failed due to wrong type for executor.extraInitContainers
    Fixed the context for extraInitContainers when templating the user provided value.

    CVE-2020-13692: Vulnerabilities reported against PostgreSQL JDBC driver, version version 42.2.5
    Resolved by updating the reference to the PostgreSQL JDBC driver to version 42.2.17.

    4.9.1 Release Notes

    What’s New

    Preview: Arrow Flight

    Dremio 4.9.1 offers a new Arrow Flight endpoint for Arrow Flight connections. The feature is enabled by default on port 32010. Arrow Flight enables high speed data transfer compared to ODBC/JDBC connections by utilizing the Apache Arrow format to avoid serializing and deserializing data.

    Dremio provides sample Flight client applications at Dremio Hub. See the Arrow Flight documentation for more information about Arrow Flight.

    Known Issues in 4.9.1

    The Arrow Flight endpoint returns a buffer_index out of range error for union-typed columns.

    Fixed Issues in 4.9.1

    Fails to connect to PostgreSQL servers with GSSAPI authentication enabled.
    Resolved by updating the reference to the PostgreSQL JDBC driver to version 42.2.18.

    Wait times of 2 to 3 seconds for HDFS threads
    Fixed by extracting file size and mtime while retrieving Hive Parquet splits.

    4.9.3 Release Notes (Enterprise Edition Only)

    Dremio recommends that users install Dremio 4.9.3 when upgrading to 4.9.x to ensure that new permissions are set correctly. The 4.9.0 and 4.9.1 releases of Dremio Enterprise Edition are no longer available.

    Important:
    After upgrading to Dremio 4.9.3+ from 4.9.0 or 4.9.1, Dremio recommends that administrators reset permissions for datasets, folders, spaces, and sources that were set using the user interface in 4.9.0 or 4.9.1.

    Fixed Issues in 4.9.3

    When Dremio users select the Can Edit permission in the user interface, Dremio doesn’t automatically select the Can Query permission, as well
    Resolved by automatically selecting and locking the Can Query permission when a Dremio user selects the Can Edit permission.