4.6 Release Notes

What's New

AWS Edition

• Dremio offers a new hourly paid listing in AWS Marketplace.

• Dremio displays a checkmark in a new Enterprise column on the Projects page for projects that were last run with Dremio Enterprise features enabled.

Other Enhancements

• Dremio users can launch Microsoft Power BI by clicking an icon on the Client toolbar on the Dremio Datasets page. The Client toolbar allows users to launch Tableau and Power BI from Dremio. Dremio administrators can configure which clients appear on the toolbar for a Dremio project in the Client Tools pane of the Support Settings page and with a new REST API endpoint.

• Dremio supports MongoDB 4.2.

• The Power BI connector is now GA.

• Dremio supports AWS Glue Catalog as a data source and automatically synchronizes all databases and tables in the Glue Data Catalog that reside in AWS S3. Dremio supports the following data formats for AWS Glue:

  • Parquet
  • ORC
  • Delimited text files (CSV/TSV)

• A new user interface for the Elastic Engines page:

  • On the Admin page under Workload Management, Provisioning is now Engines.
  • The Engines page displays a list of all available engines for the project. Dremio administrators can add and remove workers from an engine, as well as edit and delete engines.
  • The Engine Details page contains a Queues tab, which displays the queue memory limit and the job memory limit per node.

• A new CONVERT_TIMEZONE SQL function that converts a timestamp to a specified timezone. Two new system tables, sys.timezone_names and sys.timezone_abbrevs, list Dremio-supported timezone names and abbreviations. The function accepts three parameters:

  • sourceTimezone (optional)
  • destinationTimezone
  • timestamp

  If the sourceTimezone parameter is not present, Dremio assumes the timestamp provided in the third parameter is in UTC format.

  The sourceTimezone and destinationTimezone parameters accept any of the following values:

  • timezone name from the sys.timezone_names
  • timezone abbreviation from sys.timezone_abbrevs
  • offset, such as +02:00
    

• Upgraded to the latest version of Apache Arrow, 0.17.1.

• Sorting now utilizes lz4 compression for improved performance.

• A new environment variable in dremio-env, DREMIO_GC_OPTS, that specifies which Java garbage collector Dremio uses. Dremio supports the Garbage First Garbage Collector (XX:+UseG1GC) and the Parallel Collector (-XX:+UseParallelGC). By default, Dremio uses G1 GC.

  Dremio detects if a conflict exists between the garbage collection configuration specified with the new DREMIO_GC_OPTS environment variable and the Set Up YARN dialog. If garbage collection is configured for YARN, Dremio uses that value in the event of a conflict. If the garbage collection is not configured for YARN, Dremio uses the value specified by DREMIO_GC_OPTS.

• Improved error messages for the JDBC Connector.

Changed Behavior

• Users without permissions on a data source can no longer create a table from query results (CTAS). Dremio now checks for user permissions on a data source for the following queries:

  • createTableAs
  • createEmptyTable
  • createNewTable
  • createView
  • updateView
  • dropView
  • dropTable
  • truncateTable
  • addColumns
  • dropColumn
  • changeColumn

• The Qlik Sense client application again appears on the Analyze With menu on the Datasets page, but no longer appears on the Client toolbar.

• The TO_TIMESTAMP SQL function supports the timezone abbreviations in the new sys.timezone_abbrevs system table.

• Data Reflections configured for incremental refreshes only perform a full refresh after changes to the schema that impact the Data Reflection. Changes to the schema that do not impact the Data Reflection no longer inititate a full refresh.

• The Power BI Connector prioritizes Dremio.pqx rather than Dremio.mez when both files are installed. Use of .mez files is deprecated.

Fixed Issues in 4.6.0

NOT IN clause sometimes returns NULL records
Fixed an issue that caused the NOT IN clause to return NULL records when the clause contains five or more values.

Sorting performance may be impacted by compression and small batch sizes
Fixed by disabling an unncessary flush.

Dremio fails to restart after upgrading from Dremio 4.1.8 to 4.5 on AWS Edition
Fixed by removing previously configured elastic engines in Dremio during the upgrade.

JDBC storage plugin fails while setting up SQL queries
Fixed a bug that occurred when aliases are used in the ORDER BY SQL clause.

Unable to load credentials on Amazon EKS deployments
Fixed by retrying when AWS SDK returns an SdkClientException.

Unable to delete a Dremio user with special characters in the username
Fixed an issue with escape characters.

Unable to restore a standalone Dremio deployment using AWS S3 for distributed storage to another standalone deployment also configured for distributed storage with S3
Fixed misleading error message during restore operation.

Tableau Server on Linux cannot connect to a Dremio deployment when SSL/TLS is enabled
Fixed by upgrading the SSL driver to version 1.5.0.1001 in the Dremio ODBC driver for Linux.

JDBC storage plugin sometimes fails while setting up queries on Postgres
Fixed issue that added superfluous collations.

Power BI Connector generates an error when using text filters with non-English characters
Fixed by overriding SQLGetTypeInfo to return only one type for SQL_WVARCHAR and SQL_WCHAR.

UnsupportedOperationException while trying to read HDFS datasources in Parquet format
Fixed issue with case sensitivity in column names of HDFS datasources in Parquet format.