Dremio offers a new hourly paid listing in AWS Marketplace.
Dremio displays a checkmark in a new Enterprise
column on the Projects
page for projects that were last run with Dremio Enterprise features enabled.
Dremio users can launch Microsoft Power BI by clicking an icon on the Client toolbar on the Dremio Datasets
page. The Client toolbar allows users to launch Tableau and Power BI from Dremio. Dremio administrators can configure which clients appear on the toolbar for a Dremio project in the Client Tools
pane of the Support Settings
page and with a new REST API endpoint.
Dremio supports MongoDB 4.2.
The Power BI connector is now GA.
Dremio supports AWS Glue Catalog as a data source and automatically synchronizes all databases and tables in the Glue Data Catalog that reside in AWS S3. Dremio supports the following data formats for AWS Glue:
A new user interface for the Elastic Engines
page:
Admin
page under Workload Management
, Provisioning
is now Engines
.Engines
page displays a list of all available engines for the project. Dremio administrators can add and remove workers from an engine, as well as edit and delete engines.Engine Details
page contains a Queues
tab, which displays the queue memory limit and the job memory limit per node.A new CONVERT_TIMEZONE
SQL function that converts a timestamp to a specified timezone. Two new system tables, sys.timezone_names
and sys.timezone_abbrevs
, list Dremio-supported timezone names and abbreviations. The function accepts three parameters:
sourceTimezone
(optional)destinationTimezone
timestamp
If the sourceTimezone
parameter is not present, Dremio assumes the timestamp provided in the third parameter is in UTC format.
The sourceTimezone
and destinationTimezone
parameters accept any of the following values:
sys.timezone_names
sys.timezone_abbrevs
+02:00
Upgraded to the latest version of Apache Arrow, 0.17.1.
Sorting now utilizes lz4 compression for improved performance.
A new environment variable in dremio-env
, DREMIO_GC_OPTS
, that specifies which Java garbage collector Dremio uses. Dremio supports the Garbage First Garbage Collector (XX:+UseG1GC
) and the Parallel Collector (-XX:+UseParallelGC
). By default, Dremio uses G1 GC.
Dremio detects if a conflict exists between the garbage collection configuration specified with the new DREMIO_GC_OPTS
environment variable and the Set Up YARN
dialog. If garbage collection is configured for YARN, Dremio uses that value in the event of a conflict. If the garbage collection is not configured for YARN, Dremio uses the value specified by DREMIO_GC_OPTS
.
Improved error messages for the JDBC Connector.
Users without permissions on a data source can no longer create a table from query results (CTAS). Dremio now checks for user permissions on a data source for the following queries:
createTableAs
createEmptyTable
createNewTable
createView
updateView
dropView
dropTable
truncateTable
addColumns
dropColumn
changeColumn
The Qlik Sense client application again appears on the Analyze With
menu on the Datasets
page, but no longer appears on the Client toolbar.
The TO_TIMESTAMP
SQL function supports the timezone abbreviations in the new sys.timezone_abbrevs
system table.
Data Reflections configured for incremental refreshes only perform a full refresh after changes to the schema that impact the Data Reflection. Changes to the schema that do not impact the Data Reflection no longer inititate a full refresh.
The Power BI Connector prioritizes Dremio.pqx
rather than Dremio.mez
when both files are installed. Use of .mez
files is deprecated.
NOT IN
clause sometimes returns NULL records
Fixed an issue that caused the NOT IN
clause to return NULL records when the clause contains five or more values.
Sorting performance may be impacted by compression and small batch sizes
Fixed by disabling an unncessary flush.
Dremio fails to restart after upgrading from Dremio 4.1.8 to 4.5 on AWS Edition
Fixed by removing previously configured elastic engines in Dremio during the upgrade.
JDBC storage plugin fails while setting up SQL queries
Fixed a bug that occurred when aliases are used in the ORDER BY
SQL clause.
Unable to load credentials on Amazon EKS deployments
Fixed by retrying when AWS SDK returns an SdkClientException
.
Unable to delete a Dremio user with special characters in the username
Fixed an issue with escape characters.
Unable to restore a standalone Dremio deployment using AWS S3 for distributed storage to another standalone deployment also configured for distributed storage with S3
Fixed misleading error message during restore operation.
Tableau Server on Linux cannot connect to a Dremio deployment when SSL/TLS is enabled
Fixed by upgrading the SSL driver to version 1.5.0.1001 in the Dremio ODBC driver for Linux.
JDBC storage plugin sometimes fails while setting up queries on Postgres
Fixed issue that added superfluous collations.
Power BI Connector generates an error when using text filters with non-English characters
Fixed by overriding SQLGetTypeInfo
to return only one type for SQL_WVARCHAR
and SQL_WCHAR
.
UnsupportedOperationException
while trying to read HDFS datasources in Parquet format
Fixed issue with case sensitivity in column names of HDFS datasources in Parquet format.
Enable file status check
, and new property for metadata storage in dremio.conf
:debug: {
dist.s3_file_status_check.enabled: enabled
}
These options control whether Dremio verifies that a file exists in AWS S3 and the distributed storage for a data source, respectively. Both are enabled by default. If users notice failed LOAD MATERIALIZATION
or DROP TABLE
data acceleration jobs when using AWS S3 for distributed storage, disable dist.s3_file_status_check.enabled
in dremio.conf
and disable the Enable file status check
advanced option on the data source.
NUM_COLUMNS_TRIMMED
, reports the number of trimmed columns in Parquet-formatted files.Validation error with java.io.FileNotFoundException
when refreshing a Data Reflection
Fixed by disabling the new Enable file status check
advanced option for AWS S3 data source and disabling the debug.dist.s3_file_status_check.enabled
property in dremio.conf
.
Queries on Dremio metadata containing a WHERE clause with both LIKE
and OR
operators return incorrect results
Fixed by correctly pushing down OR
query filter.
Executor nodes fail with ForemanException
Fixed by removing unnecessary columns and rowgroups from footers of Parquet files.
When asynchronous access is disabled, Dremio is unable to gather medata from the footers of Parquet files
Fixed by reverting to a known working parquet footer.
Dremio crashes with java.io.FileNotFoundException
Fixed issue with data consistency during refreshes of Data Reflections for AWS S3 data sources.
Inconsistent job status reported in job profile and job details
Fixed by asynchronously handling completion events from executor nodes.
Superfluous columns are not trimmed while scanning Data Reflections
Fixed by adding a handler method.
Queries fail with CONNECTION ERROR: Error setting up remote fragment execution
Fixed issue where parallelizer picked executor nodes outside of the selected engine.