4.5 Release Notes
What's New
AWS Edition
Streamlined user experience with Dremio Projects with AWS CloudFormation.
Support for starting and stopping coordinator nodes.
Other Enhancements
Tableau SDK Connector with single sign-on.
AWS Redshift GA.
MongoDB data sources support MongoDB 4.0.
Jobs progress through the following states for each query:
- Pending
- Metadata Retrieval
- Planning
- Queued
- Engine Start
- Execution Planning
- Starting
- Running
Jobs will finish with one of the following states:
- Completed
- Canceled
- Failed
Efficient retrieval of metadata from large catalogs.
Changed Behavior
MongoDB Data Source
Dremio no longer pushes down aggregation pipelines to MongoDB.
System Requirement
Dremio requires a new ephemeral network port for inter-node communications in addition to port 45678. To assign a static port number to the conduit port, assign a value to services.conduit.port
in the Dremio configuration file, dremio.conf
. If TLS is enabled, Dremio applies the same configuration to communications of the new conduit port. To use a different configuration or to enable TLS for only the new conduit port, specify all values for services.conduit.ssl
in dremio.conf
.
Fixed Issues in 4.5.0
Dremio 4.2.x and 4.3.x unable to access data nodes on Hadoop clusters prior to 3.0
Fixed by updating the version of the Hadoop client.
JDBC Connector taking too long to retrieve query results
Fixed by adding an END_OF_STREAM_MESSAGE
message that signals when a queue is empty.
Queries do not complete after refreshing the metadata for a table in Hive
Fixed by removing a race condition.
Slow sorts in some circumstances when micro-spilling is disabled and batch sizes are especially small
Addressed by resizing record batches during spilling despite uneven incoming batches.
IllegalStateException: Inconsistent row count error while joining two Hive data sets using complex data in Parquet format
Fixed by handling non-existing Parquet columns.
Executor nodes in AWS Edition don't automatically terminate if the AWS API is unavailable
Added capability to terminate executor nodes using Linux commands.
TOTAL_SPILLED_DATA_SIZE is 0 after an external sort with micro-spilling enabled
Fixed by updating the metric for total spill size.
JDBC driver leaves connection open with Looker data sources
Fixed by more robustly cleaning up resources after a connection failure.
Standby coordinator node consumes more memory than allocated
Fixed memory error in standby coordinator node.
Job API sometimes returns fewer records than specified with the LIMIT
statement
Fixed a bug that occurs when the offset is equal to or greater than the batch size.
Schema learning fails with 'New schema found. Please re-attempt the query' error message
Fixed by only pushing down projections that are simple column references.