Dremio Cloud
Dremio Software
 
On this page

    4.8 Release Notes

    What’s New

    Runtime Filtering

    Runtime filtering improves query performance by dynamically applying dimension table filters to joined fact tables.

    Default Reflections

    Default Reflections increase query performance by reducing planning time for queries using complex virtual dataset definitions. To use Default Reflections, create a raw reflection on all columns in a virtual dataset. Default Reflections will use the raw reflection rather than considering all permutations of underlying tables and views, which can accelerate SQL planning when using complex virtual datasets.

    Non-Public IP (AWS Edition)

    Dremio supports deployments without an associated public IP.

    Custom Projects (AWS Edition)

    Configuration files in the Dremio shared EFS directory are copied locally and used by executor nodes. Dremio administrators can make additional configurations by editing a script in the shared directory that Dremio executor runs on startup.

    Changed Behavior

    • Dremio no longer allows users to drop tables that have child tables. Dremio users should move child tables before attempting to drop a parent table.

    • Dremio filters out invalid options, such as store.plugin.check_state, during system startup.

    • Dremio does not automatically expand virtual datases when a Raw Relection exists on all columns.

    Other Enhancements

    • This release provides Dremio administrators two new options to use the dremio-admin clean command to remove orphaned jobs:

      dremio-admin clean -p
dremio-admin clean --delete-orphan-profiles

      Dremio alters the virtual datasets related to orphaned jobs.

    • The Spaces, Data Lakes, and External Sources lists on the Datasets page display the number of data sources in that category. The External Sources list is closed by default.

    • Storage plugins start faster because Dremio uses access() rather than list() for health checks.

    Fixed Issues in 4.8.0

    Users can’t access virtual datasets created by another user from an external query
    Resolved by running all table functions in virtual datasets as the creator of the dataset.

    Dremio throws a NullPointerException when cleaning old jobs
    Fixed by adding two new options to the dremio-admin clean command. See Other Enhancements above for more information.

    Engine fails to restart after closing a Dremio project and deleting the CloudFormation stack in AWS Edition
    Set the default engine to use the coordinator node’s AWS security group.

    Queries with a WHERE clause fail when run against a virtual dataset with the FLATTEN SQL function in its lineage
    Resolved by preventing filters with the FLATTEN SQL function.

    Inconsistent query results with vectorized joins when runtime filtering is enabled
    Fixed by setting the validity bit from right for VECTORIZED_BIGINT mode.

    DROP_TABLE does not delete child schemas of a parent table
    Fixed by preventing Dremio users from dropping tables with child tables. Dremio users should move child tables before attempting to drop a parent table.

    Users on MacOS using Chrome and Firefox browsers accidentally trigger the brower back function and lose changes to virtual datasets
    Resolved by preventing accidental scrolling in browsers on MacOS.

    Save button unusable when enabling raw Data Reflections on a data source
    Fixed race condition in Reflection creation UI.

    “unable to find bucket” error when trying to open a Hive table
    Resolved by listing contents of buckets rather than using the bucket’s ACL to evaluate user’s access.

    Failures to create a new data source with the name of an existing data source do not explain issue
    Updated the error message to include recommended next step and a stack trace.

    Query results contain “PhysicalOperatorSetupException: Can not parallelize fragment” error
    Fixed by converting the input of WriterPrel to match the distribution traits.

    Failure to save a virtual dataset does not notify user of the name conflict
    Updated the error message to inform the user of the name conflict.

    Dremio cancelled job because query consumed system capacity during query planning
    Resolved with new FilterSetOpTransposeRule constructor.

    Query performance affected by underparallelization
    Fixed by removing rowadjust after logical planning.

    Dremio displays incorrect result count in query results against a .csv file
    Fixed a bug in the text reader that discards the last byte of a chunk.

    IndexOutOfBoundsException when refreshing metadata for parquet-formatted files
    Resolved by handling zero row group empty parquet files.

    Inconsistent number of records returned by COUNT(*) and COUNT(DISTINCT [field name] givenfield, ...) aggregate functions
    Fixed by using a vectorized orc reader when partition and table schemas do not match.

    Dremio fails to read ORC tables using complex types
    Resolved by copying struct vector with repeating null values.

    Mongo does not support not equal and equal null in nested fields
    Added support for LIKE, != NULL, and = NULL in nested fields. Dremio also fixed erroneous NOT LIKE filters. Dremio does not push down NOT LIKE since it is not supported before MongoDB version 4.0+.

    Dremio doesn’t inform the user that they are using an unsupported Java Virtual Machine
    Dremio displays an error message such as the following when attempting to start with an unsupported Java Virtual Machine: Java 8 is required to run Dremio. Java 11 is being used.

    4.8.1 Release Notes

    Fixed Issues in 4.8.1

    /api/v3/catalog/{id} REST API exposes access credentials for data sources to non-administrative users
    Resolved a bug in code for clearing credentials.

    UI indicates that Dremio user “cannot accelerate” a Reflection
    Fixed by using correct type for count conversion.