On this page

    4.6.1 Release Notes


    • New advanced option for AWS S3 data sources, Enable file status check, and new property for metadata storage in dremio.conf:
    debug: { 
      dist.s3_file_status_check.enabled: enabled

    These options control whether Dremio verifies that a file exists in AWS S3 and the distributed storage for a data source, respectively. Both are enabled by default. If users notice failed LOAD MATERIALIZATION or DROP TABLE data acceleration jobs when using AWS S3 for distributed storage, disable dist.s3_file_status_check.enabled in dremio.conf and disable the Enable file status check advanced option on the data source.

    • New metric, NUM_COLUMNS_TRIMMED, reports the number of trimmed columns in Parquet-formatted files.

    Fixed Issues in 4.6.1

    Validation error with java.io.FileNotFoundException when refreshing a Data Reflection
    Fixed by disabling the new Enable file status check advanced option for AWS S3 data source and disabling the debug.dist.s3_file_status_check.enabled property in dremio.conf.

    Queries on Dremio metadata containing a WHERE clause with both LIKE and OR operators return incorrect results
    Fixed by correctly pushing down OR query filter.

    Executor nodes fail with ForemanException
    Fixed by removing unnecessary columns and rowgroups from footers of Parquet files.

    When asynchronous access is disabled, Dremio is unable to gather medata from the footers of Parquet files
    Fixed by reverting to a known working parquet footer.

    Dremio crashes with java.io.FileNotFoundException
    Fixed issue with data consistency during refreshes of Data Reflections for AWS S3 data sources.

    Inconsistent job status reported in job profile and job details
    Fixed by asynchronously handling completion events from executor nodes.

    Superfluous columns are not trimmed while scanning Data Reflections
    Fixed by adding a handler method.