Skip to main content

Changelog

This changelog provides a detailed record of the ongoing updates and enhancements we have made to improve your Dremio Cloud experience.

September 21, 2023

What's New
APP

Dremio Arctic and all of its related features are no longer in preview mode.

DX-82613

APP

In the Dremio console, ownership in Sonar and Arctic is now listed separately from other privileges, at the top of the Privileges page, and the procedure for transferring ownership is streamlined. For more information, see Transferring Organization Ownership and Transferring Ownership.

DX-69442

APP

Added keyboard shortcuts in the SQL Runner for showing or hiding the Text-to-SQL panel and for triggering Text-to-SQL. For more information, see Keyboard Shortcuts.

DX-67771

APP

The Record Count column has been moved next to the Current Footprint column in the Project Settings > Reflections table.

DX-60641

APP

Added support for the il-central-1: Israel (Tel Aviv) region to the AWS Glue source.

DX-69481

APP

Dremio Cloud provides more helpful information in the error message if an invalid tag or branch name is supplied.

DX-69034

APP

You can now leave the Database Name field blank in a PostgresSQL source.

DX-67472

APP

Reduced the number of S3 lookups required for Arctic DML and DDL operations to improve performance in query planning.

DX-66247

APP

Reduced the amount of heap memory used by the query plan cache.

DX-69868

Issues Fixed
API

Updated the Source API to prevent sending secret values in clear text.

DX-44436

APP

Setting query_label() as an engine rule was resulting in an exception error.

DX-71949

APP

In some cases, default raw reflection matching was not working as expected for users not assigned to the ADMIN role.

DX-70893

APP

After searching for and selecting a username or role in catalog privileges, the search string was not automatically being cleared.

DX-68966

APP

The Tableau and Power BI buttons were visible in the SQL Runner for unsaved queries.

DX-69867

SQL

Saving a query as a view without having run the query was resulting in an error if no engine replicas were active.

DX-71680

SQL

Fixed an issue with COL_LIKE() when input and pattern contained the % character.

DX-69931

SQL

LIKE was not being highlighted as a reserved keyword in the SQL editor.

DX-69755

SQL

Fixed an issue that was causing the use of GRANT ALL on a project to fail with an "invalid project privilege" error.

DX-65571

SQL

OPTIMIZE and VACUUM queries on tables with reflections were being evaluated for reflection matching, causing an error.

DX-64997

SQL

Fixed an issue with filter pushdowns that was causing some preview queries to fail even though the same query was successful when using Run.

DX-56539

Known Issues
Known Issues

Due to metadata caching, it may take up to five minutes to reflect revoked privileges on objects in a Sonar project, including Arctic catalogs.

Known Issues

Users with the organization-level MANAGE GRANTS privilege who have not been assigned the ADMIN role are not able to assign privileges to users or roles unless they have been explicitly assigned the CREATE USER or CREATE ROLE privilege. This issue will be fixed in a future release so that users with MANAGE GRANTS for the organization can assign privileges without needing ADMIN or CREATE USER/CREATE ROLE.

DX-62435

September 12, 2023

Issues Fixed
SQL

Fixed an issue that was causing the RST_STREAM closed stream error when processing large result sets via JDBC or ODBC.

DX-64679

September 8, 2023

Issues Fixed
SQL

Some queries were failing with a Failure getting source error.

DX-71093

SQL

Queries against views and tables in an Arctic catalog were not showing up on the Jobs page. This issue only affected organizations that use the default Arctic catalog instead of spaces.

DX-67652

September 6, 2023

What's New
APP

The following regions are now supported in the AWS Glue source:

  • ap-south-2: Asia Pacific (Hyderabad)
  • ap-southeast-3: Asia Pacific (Jakarta)
  • ap-southeast-4: Asia Pacific (Melbourne)
  • eu-south-2: EU (Spain)
  • eu-central-2: EU (Zurich)
  • me-central-1: Middle East (UAE)
    DX-69347
SQL

Added a new table function, SYS.RECOMMEND_REFLECTIONS, that recommends aggregation reflections to accelerate existing SQL queries. For more information, see Reflection Recommendations.

DX-61857

SQL

This release adds support for the following SQL functions:

Issues Fixed
APP

The Status of some failed queries was being reported as RUNNING instead of FAILED in the Jobs Overview page.

DX-69767

APP

Updated com.google.guava:guava to 32.1.1-jre to address CVE-2023-2976 in Dremio's internal Iceberg fork.

DX-68888

APP

Updated validation settings to ensure that only privileged users could view Acceleration Settings on the Project Settings > Reflections page.

DX-68877

SQL

Fixed an issue that was causing inconsistent query results when ARRAY_CONTAINS was used with nullability checks.

DX-69920

SQL

Plans for queries containing CONVERT_FROM could not be cached.

DX-66675

SQL

To address a CONCURRENT_MODIFICATION error seen in concurrent metadata refresh queries on Parquet tables, if the query is submitted by scheduler, failures are ignored. If the query is submitted by users, the failed query is retried until it succeeds.

DX-69749

August 28, 2023

What's New
APP

When viewing catalog or folder contents on the Datasets page, the "More" menu (...) for tables and views now contains a Delete option, allowing users with appropriate privileges to delete a table or view.

DX-68290

Issues Fixed
APP

For some failed queries, status in the Job profile was being reported as RUNNING instead of FAILED.

DX-69767

APP

When creating a new project, the Arctic catalog name was not being validated prior to launching the CloudFormation template.

DX-69621

APP

The Open Results link on the Jobs overview page was not working as expected for queries that were run from edit mode on the Dataset page.

DX-69601

APP

When saving a view, the items in the "Save View As" dialog were not sorted in the same way as on the Dataset page.

DX-68432

SQL

For organizations created prior to August 17, 2023, granting or revoking table or view privileges for the first time via SQL was successful, but an error was produced on the Organization Settings > Privileges page.

DX-69645

SQL

If a dataset name or the name of a parent folder contained a space or ampersand (&) character, clicking on the dataset would populate the SQL Runner with a truncated SELECT statement.

DX-66366

August 17, 2023

What's New
APP

As of this release, each Sonar project in new Dremio Cloud organizations will come deployed with an Arctic catalog, which will support data management capabilities (folders, tables, etc.) for the project. This primary Arctic catalog replaces your home space.

DX-63250

APP

You can now use Role-Based Access Control (RBAC) privileges to control which roles and users can read, write, and manage tables and views in Arctic catalogs.

DX-42564

APP

Dremio Sonar now supports the same SQL syntax as Spark when working with Arctic/Nessie sources.

DX-60108

APP

Dremio automatically optimizes incremental reflection file size to improve reflection performance.

DX-61779

APP

In this release, you must explicitly create folders (namespaces) before creating tables or views in them.

DX-67970

APP

The Usage page under Organization Settings now includes usage data for Arctic catalogs.

DX-66291

APP

Dremio will avoid a full data scan for simple aggregations on partition columns, reading the manifest metadata instead, which improves performance for queries on very large tables.

DX-66029

SQL

After DML operations against unpartitioned Iceberg tables, Dremio now compacts the data files written by the DML operation to improve future read performance.

DX-58884

Issues Fixed
APP

Updated the Snowflake connector to fix intermittent issues when adding Snowflake as a source.

DX-67959

APP

If you optimized a table in Dremio Arctic (Optimize Once) and then viewed the dataset settings for another table, the Optimize Once button remained disabled unless you refreshed the page.

DX-67808

APP

Logged in users were getting redirected to the login page instead of the create password page when clicking on an invite that had not yet been accepted.

DX-67398

APP

When running a job multiple times, the status and job link for the last attempt are now displayed as expected.

DX-66716

APP

Increased concurrency limits to avoid errors when concurrent inserts into the same table were being sent from different streams.

DX-61285

SQL

Fixed an issue that could cause a memory leak when querying an Iceberg table with positional deletes.

DX-68497

SQL

At times, the DAY() function was returning either integer or timestamp, depending on how the query was written.

DX-67793

SQL

Fixed an issue that was causing an error when running OPTIMIZE TABLE on a table with reflections.

DX-67861

SQL

In some cases, the IF EXISTS option for DROP BRANCH and DROP TAG was being ignored.

DX-67553

SQL

Top-level CASE statements intended to return a Boolean were not being rewritten correctly, resulting in an error for some SQL Server queries.

DX-67527

SQL

Some SQL Server queries with nested CASE statements were failing with invalid SQL comparison syntaxes.

DX-67183

SQL

Fixed an issue with the LEFT() SQL function on Oracle sources for queries with dates.

DX-67036

SQL

Some date subtraction queries were not getting pushed down for Oracle sources.

DX-65396

July 27, 2023

Issues Fixed
SQL
Fixed an issue that was resulting in incorrect results when a view containing a correlated sub-query was used more than once in the same query.
DX-67662
SQL
Some reflections on views based on MySQL sources were failing with timeout errors.
DX-61201

July 24, 2023

What's New
APP

This release provides performance improvements in the Jobs listing page, and any user with sufficient privileges can now view reflection jobs in the table.

DX-64196

APP

The details panel displaying Wiki content is now available inside a folder on the main branch in an Arctic source.

DX-67453

APP

This release adds support for a new connector that allows querying data from Apache Druid. For more information, see Apache Druid.

DX-46607

SQL

You can now drag and drop a table from your home space into the Text-to-SQL panel in the SQL Runner.

DX-67329

SQL

We have made some improvements in the efficiency of the queries suggested when using the Text-to-SQL feature.

DX-66337

Issues Fixed
API
The Engines REST API was producing an error instead of generating an empty list if the user did not have access to any engines.
DX-65193
APP
If you optimized a table in Dremio Arctic (Optimize Once) and then viewed the dataset settings for another table, the Optimize Once button remained disabled unless you refreshed the page.
DX-67808
APP
If the preview engine had been disabled, it can now be enabled again.
DX-67282
APP
Unknown organizations were showing up when signing in to Dremio Cloud from client applications.
DX-67273
APP
A new external ID was being generated every time a cloud entity was edited, which was breaking the IAM role trust relationship. In such cases, users had to temporarily switch to IAM access key credentials with the same permissions as the IAM role.
DX-61964
APP
When parsing CSV, empty lines at the top of the CSV are now ignored, allowing column names to be extracted from lines other than the first
DX-60797
SQL
Fixed an issue that was allowing non-privileged users to alter or view reflections via SQL.
DX-67672

July 7, 2023

What's New
General Updates

Dremio Cloud has new onboarding flow that walks you through creating your first project and gives you an overview of Dremio features. You can hide the tutorials if you like, and show them again from the Help menu.

June 29, 2023

What's New
APP

This release enables new privileges for Arctic catalog-level role-based access control: COMMIT, CREATE BRANCH, CREATE TAG, MANAGE GRANTS, OWNERSHIP, USAGE, and WRITE SETTINGS.

Breaking Change: With this change, non-privileged users (e.g., users who are not catalog or organization owners) will not be able to access Arctic catalogs and their underlying datasets. Catalog or organization owners will need to explicitly grant these privileges to users and roles on the Arctic catalog, as described in Privileges.

APP

The metadata card is now displayed for datasets in Arctic catalogs. For more information, see Viewing the Metadata of an Arctic Dataset.

APP

You can now build reflections on versioned Arctic tables and views. For more information, see Using Arctic Branches with Reflections.

APP

The "Data-as-code" source category has been renamed "Arctic Catalogs."

APP

You can now add the SampleDB source to any project using the Add Data Source dialog.

APP

The "Profile" tab on the Jobs detail page has been renamed "Visual Profile."

APP

You can now add wiki content and labels to Dremio Arctic catalogs that are available in your Sonar project (main branch only).

SQL

Autocomplete has been improved to provide better and faster context-aware suggestions for SQL keywords, catalog objects, and functions while you are constructing SQL statements.

SQL

When dragging a dataset into the SQL editor, the AT BRANCH | TAG | COMMIT <reference> will be appended to the query.

SQL

Dremio now supports LIST literals using the ARRAY keyword. For more information, see Data Types.

SQL

The schema for the sys.organization.usage table has been changed to support Arctic catalog usage. The project_id column has been split into entity_id and entity_type so you can quickly determine if usage belongs to a Sonar project or an Arctic catalog. For more information, see SYS.ORGANIZATION.USAGE.

Issues Fixed
APP

Added validation to ensure that you cannot create an Engine Routing rule with the same name as an existing rule.

APP

The Preview engine cannot be disabled or deleted because it is used to run metadata refresh queries for relational sources and other background activities.

APP

Fixed a security issue that could, in some cases, allow users to bypass email verification for new accounts.

APP

Improved the message that is displayed when an Engine Routing rule rejects a query or if the query does not match any rules.

June 14, 2023

What's New
APP

Dremio's Text-to-SQL is now available in Preview! Check out Analyzing Data with Text-to-SQL for an example.

DX-63998

June 7, 2023

What's New
API

You can now use the REST API to cancel a Dremio Arctic optimization job. For more information, see Canceling a Job.

DX-63324
APP

You can now cancel a Dremio Arctic optimization job from the Catalog Settings > Configuration page. For more information, see Canceling a Job.

DX-65852
APP

The Project Settings page now includes an option to manage preferences for the Dremio console. The SQL settings option is no longer available, and the Autocomplete and Copy or Download Results settings are now available under Preferences. For more information, see Dremio Preferences.

DX-65852
APP

You can now configure the default behavior in the Dremio console when clicking on a dataset to either open the SQL Runner with a default SELECT statement on the dataset or open the Datasets page. For more information, see Querying a Dataset on Click.

DX-65852
APP

The metadata card for datasets now includes links to launch a BI tool. For more information about the metadata card, see Viewing the Metadata of a Dataset.

APP

This release includes changes that provide improved performance for planning times for queries on Dremio Arctic views.

DX-66218
APP

The name and type of a dataset's source is now displayed when viewing the dataset in the lineage graph.

DX-65153
APP

Added more security around DML permission checks to ensure that users cannot access data to which they should not have access.

DX-64746
Issues Fixed
APP

After modifying and saving the details of a user and selecting another user to edit, the details of the previously modified user were being displayed very briefly in the user detail fields.

DX-66513
APP

Some reflection refreshes that were accelerated by default raw reflections were not being matched in user queries.

DX-66212
APP

Users who had not been assigned to the ADMIN role could see the Add Space button when viewing all spaces.

DX-65186
APP

In some cases, invalid partition stats were causing the row count estimate for a table to come back as 1.

DX-64986
APP

In the dataset details panel, wiki content was being truncated if it was too long. You can now scroll horizontally on wiki details.

DX-64091
APP

At times, during background metadata refresh, some folders on Amazon S3 sources that had been formatted as tables were losing their formatting. This was causing the PLAN ERROR: UnsupportedOperationException error for external dashboards that used views or reflections built on those tables.

DX-63415
APP

Empty, disabled reflections were being created if you edited an existing reflection.

DX-63356
APP

When using AWS PrivateLink and handling large amounts of data, system table queries could exceed the default timeout, resulting in a FlightRuntimeException error.

DX-63087
APP

Optimization jobs were initially being displayed at the bottom of the Jobs list, but they will now be included in the list according to start time.

DX-62569
APP

When promoting some datasets, quoted fields were not being treated as quoted when white space was included before it.

DX-60798
SQL

Queries on History tables were failing intermittently.

DX-65439
SQL

GRANT commands on catalog entities were failing with "Role/User \<ID> not found" if existing user or role grantees were no longer present in the system.

DX-65364
SQL

In some cases, when clicking the Functions button in the SQL Runner, the list of functions was empty.

DX-66706
SQL

When opening a versioned view in a new tab, the Dremio console was displaying an error instead of loading the dataset.

DX-65857
SQL

Right-clicking on a dataset and selecting "Open in New Tab" was opening a new browser tab, but the SQL Runner was empty.

DX-65754

June 1, 2023

What's New
APP

When you add the first Sonar project to your Dremio Cloud organization, a new sample source called "SampleDB" is now included in the project. To get started with Dremio and learn how to query data across multiple sources and more, check out Getting Started with Dremio Sonar.

May 24, 2023

Issues Fixed
SQL

Right-clicking on a dataset and selecting "Open in New Tab" was opening a new browser tab, but the SQL Runner was empty.

DX-65754
SQL

Queries on sys.project.history tables were failing intermittently.

DX-65439
SQL

When clicking a dataset to query it, the "Unsaved Changes" dialog was being displayed in the SQL Runner after changes had already been saved or when no changes had been made.

DX-64579
SQL

When using AWS PrivateLink and handling large amounts of data, system table queries could exceed the default timeout, resulting in a FlightRuntimeException error.

DX-63087

May 9, 2023

What's New
APP

The metadata card for datasets now includes a link to the lineage graph. For more information about the metadata card, see Viewing the Metadata of a Dataset.

APP
The Organization Settings icon is now located below the Dremio logo in the side navigation bar rather than in the header of the organization landing page.
SQL

For Apache Iceberg tables, you can now use the EXPIRE SNAPSHOTS option in the VACUUM TABLE SQL command to remove old snapshots and files associated with these snapshots that are no longer needed. For more information, see Expiring Snapshots of Apache Iceberg Tables and VACUUM TABLE.

SQL

Dremio Cloud now allows you to rewrite manifest files and optimize select partitions for Apache Iceberg tables:

  • Manifest files can be rewritten based on a size criteria, enabling the optimization of the metadata, which helps reduce query planning time. For more information, see Optimizing Tables.

  • You can optimize selected partitions using the partition filter clause. For information about using this SQL command, see OPTIMIZE TABLE.

SQL

This release includes two new SQL functions:

  • ST_GEOHASH returns the corresponding geohash for given latitude and longitude coordinates (see ST_GEOHASH).

  • ST_FROMGEOHASH returns the latitude and longitude coordinates of the center of a given geohash (see ST_FROMGEOHASH).

SQL

In this Dremio Cloud release, you can enable or disable the options to download or copy query results from the SQL Runner by navigating to Project Settings > SQL. For more information, see Result Set Actions.

SQL

Clicking on a dataset now takes you to the SQL Runner with a SELECT * query on the dataset. To edit a view, you can click the pencil icon that appears on the far side of the view’s row, to the right of the number of jobs, or in the metadata card.

SQL
You can now save up to 1,000 SQL scripts per user in the SQL Runner. The previous limit was 100.
Issues Fixed
APP

Dremio was taking a long time to list views in a space and tables in a source, but performance was good when listing the same in the SQL Runner's Data panel.

DX-60863

April 11, 2023

What's New
APP

In this release, you can now use an AWS CloudFormation template to add a Sonar project. For more information, see Adding a Project.

SQL

You can now rewrite manifest files and optimize select partitions for Iceberg tables in Dremio.

  • Manifest files can be rewritten based on a size criteria, enabling the optimization of the metadata, which helps reduce query planning time. For more information, see Optimizing Tables.

  • You can optimize selected partitions using the partition filter clause. For information about using this SQL command, see Optimize Table.

SQL

When available, version context is now included with table references in the query profile to indicate more clearly which version of a table is used in a given query.

DX-61828
SQL

When viewing a SQL script, the script ID is included in the page URL, allowing you to copy and share the URL with another user in your organization. The user must have VIEW privileges on the script that you are sharing.

SQL

You can now save scripts that contain up to 250,000 characters.

DX-59839
SQL

In this release, you can expand or collapse individual phases in the raw profile view.

DX-63270
Issues Fixed
APP

The Add Project button is no longer displayed on the Sonar Projects page if the logged in user does not have the privilege to create a project.

DX-63519
APP
Hovering over certain bars in the usage chart was causing an unknown error to occur.
DX-63516

March 11, 2023

Issues Fixed
APP

Fixed an issue where duplicate field names in a view could result in a "Some virtual datasets were out of date and have been corrected" error.

DX-63350

March 10, 2023

What's New
API

Arctic-related APIs to manage data optimizations native to Arctic are now available. The APIs available to optimize Arctic tables and views use the catalogs endpoint. These APIs enable you to configure engines, manage schedules, and run optimization jobs. For more information, see Arctic.

APP

Dremio Arctic now provides an automated jobs service that optimizes the storage of Iceberg tables to ensure queries are run efficiently. This jobs automation helps you manage the accumulation of the data files that occurs through DML operations. For more information, see Automatic Table Optimization.

DX-56659
APP

You can now manage organization-level privileges for Dremio Cloud under Organization Settings > Privileges. For more information, see Privileges.

DX-59514
APP

The metadata card for datasets has been updated to indicate whether or not a dataset has reflections, when the dataset was created and last updated, and the dataset owner. For more information, see Viewing the Metadata of a Dataset.

APP

In this release, you can now use an AWS CloudFormation template to add a cloud to your organization. For more information, see Adding a Cloud with an AWS CloudFormation Template.

Issues Fixed
APP

After signing up for Dremio Test Drive, users were getting directed to the organization home page instead of the Datasets page in the Test Drive project.

DX-62921
APP

The commit history for a Dremio Arctic view or table was showing entries from other views and tables.

DX-62778
APP

At times, when clicking for details from the commit history in Dremio Arctic, the details page was displaying a 404 error.

DX-62774
APP

Some reflections were getting dropped even though they were configured to never expire.

DX-62771
SQL

Fixed an issue with view delegation that was causing queries run by users not assigned the ADMIN role to not be accelerated, and query profiles contained substitution errors.

DX-62697
SQL

Fixed an issue that was preventing users from saving the modified SQL for a table as a view in Dremio Arctic without first running the query.

DX-62626
SQL

Trying to update the SQL for a saved view in Dremio Arctic was producing a generic Something went wrong error.

DX-62337
SQL

On occasion, the Format Query option was unavailable in the SQL editor.

DX-61451
SQL

In some cases, XML responses from AWS Glue were not being handled properly and causing queries to fail.

DX-60928

February 17, 2023

Issues Fixed
APP

Fixed an issue that was causing problems with AWS PrivateLink connectivity to the Dremio Cloud control plane.

DX-62584

February 7, 2023

What's New
SQL

In this release, when executing queries in the SQL Runner, Dremio displays the states of the running job in the query results area until the query results are displayed.

DX-55603
Issues Fixed
APP

At times, when trying to add an Arctic catalog using the name of a catalog that had been previously deleted, Dremio was producing an error and the catalog was not created.

DX-61111
SQL

The Sources and Scripts lists on the SQL Runner page were not scrollable when too many items were available.

DX-61886
SQL

Query results were not being displayed after opening and running a versioned view.

DX-61771
SQL

Fixed an issue that was causing the same heading to be displayed for each column section in the Review changes dialog after modifying a view configured with a reflection and clicking the Show more link on the Reflections tab.

DX-61564
SQL

When viewing a table or view on the Datasets page, the Catalog tab has been renamed Details.

DX-61502
SQL

When running queries against an Arctic source using the Arrow Flight SQL JDBC driver, the queries were failing with a Tenant context should not be null error.

DX-60417
SQL

In some cases, queries against text tables with evolved schemas were failing with an ArrayIndexOutOfBoundsException error.

DX-58969

January 20, 2023

Issues Fixed
SQL

Some queries using multiple CONVERT_FROM functions on different JSON data type columns were failing to read with an Unable to find the referenced field error.

DX-61434

January 19, 2023

What's New
SQL

This release includes a new SQL function, COL_LIKE, which tests whether an expression column matches a pattern column. For more information, see COL_LIKE.

DX-60811
Issues Fixed
APP

In some cases, listing Arctic datasets from Sonar would result in a generic Something went wrong error.

DX-61410
APP

At times, when trying to add an Arctic catalog using the name of a catalog that had been previously deleted, Dremio was producing an error and the catalog was not created.

DX-61111
SQL

When running queries against an Arctic source using the Arrow Flight SQL JDBC driver, the queries were failing with a Tenant context should not be null error.

DX-60417

January 10, 2023

What's New
APP

You can now view or modify dataset details without having to go to the Datasets page and click on the Details tab. Wherever the icon for a dataset is displayed in Dremio, you can hover over the icon to display the metadata card, which now displays label information as well as a link to Open Details. The details link opens the metadata details panel on the right side of the browser, letting you add or remove labels and view or edit the Wiki associated with the dataset. For more information, see Viewing the Metadata of a Dataset.

APP

In this release, a new source connector allows you to query data from a Dremio software cluster. For more information, see Connecting to a Dremio Software Cluster.

APP

This release adds support for a new connector that allows querying data from Snowflake data warehouses. For more information, see Snowflake.

SQL

This release supports the use of BROADCAST hints in queries to distribute data across all execution engines. Hints must be entered as /*+ <hint> */, which is standard across data warehouses. Prior to this release, text enclosed in /* */ was treated as as a comment. Dremio will continue to treat text enclosed in /* */ as a comment unless the first character is +. For more information, see Distributing Data Evenly Across Executor Nodes During Joins.

DX-56572

SQL

You can now hide the SQL editor if you want to focus the page on the results table. If you hide the editor and navigate to a different page, the editor will remain hidden when you return to the SQL runner. For more information, see SQL Editor.

SQL

In the SQL editor, you can now format your SQL using the Format SQL shortcut (Cmd + Shift + f or Ctrl + Shift + f). As long as the current syntax is valid, the SQL formatter applies a conventional style to your query by aligning commands for readability. For more information, see SQL Editor.

Issues Fixed
APP

In some cases, clicking on a specific role for details under Organization Settings > Roles was generating a 404/page not found error.

DX-60666
APP

If you entered a non-matching value when filtering for users on the Jobs page, the contents of the User filter was empty the next time it was opened.

DX-60196
APP

In some cases, with the Arrow Flight SQL ODBC driver, users were getting an error when testing the connection to Microsoft Excel in the ODBC Administrator on Windows.

DX-60176
APP

Details about access times for some catalog operations were not included in query profiles.

DX-60175
SQL

Some queries executed against an AWS Glue source were failing because the source appeared to be unavailable.

DX-59723
SQL

When opening a reflection to view details under Settings > Reflections, an error indicating that the reflection did not exist could be displayed, even though the reflection was valid.

DX-59291
SQL

The cursor and selected text were misaligned in the SQL Runner in some browsers running on Windows.

DX-60648
SQL

Fixed an issue that was causing queries failed if certain expression splits contained CAST AS UNION.

DX-60607
SQL

In the SQL runner, some of the actions text under query results was not displaying correctly when using Safari.

DX-60553
SQL

When using search to filter all scripts in the SQL runner, clearing the contents of the search box was generating an unexpected error.

DX-60546
SQL

Made some improvements in approximation of double value to nearest decimal when filtering on a table with double type, but files contain decimal values.

DX-60338

Previous Releases

December 21, 2022

Issues Fixed
SQL

Fixed an issue that was causing misalignment of the cursor and selected text in the SQL Runner in some browsers running on Windows.

DX-60648

December 19, 2022

Issues Fixed
SQL

Fixed an issue that was causing the non-highlighted portions of a query or commented lines in a query to be cleared from the SQL editor when running or previewing the query.

DX-60468
SQL

Details about access times for some catalog operations were not included in query profiles.

DX-60175

December 14, 2022

What's New
SQL

You can now optimize Iceberg tables to maximize the speed and efficiency of data retrieval. Rewrite data files using a compaction process to combine small files into larger files or split large files to reduce metadata overhead and runtime file open costs. For more information, see Optimizing tables.

SQL

You can now roll back to a previous state of an Iceberg table using either a snapshot ID or a timestamp reference. For more information, see Rolling back tables.

SQL

This release includes a new SQL function, ARRAY_CONTAINS, which returns whether a list contains a given value. For more information, see ARRAY_CONTAINS.

SQL

The LIKE operator can now take optional keywords ANY, ALL,or SOME to allow matching against a list of patterns. For more information, see LIKE.

SQL

Breaking Change: If you mix implicit and explicit joins, only the last of the implicitly joined tables can be in the ON clause. Otherwise you will receive a "Table not found" error. For example,

select *
from
NAS2."customer.parquet" c,
NAS2."nation.parquet" n
left join
NAS2."orders.parquet" as o
on c.c_custkey = o.o_custkey

results in the error Table 'c' not found. The solution is to replace the comma with an explicit cross join like this:

select *
from
NAS2."customer.parquet" c
cross join
NAS2."nation.parquet" n
left join
NAS2."orders.parquet" as o
on c.c_custkey = o.o_custkey

This is functionally equivalent since implicit joins implement a cross product of the two tables.

Issues Fixed
APP

Added a copy button to all instances of a Commit ID in Arctic Catalogs.

DX-59587
APP

In Dremio Arctic, all Commit IDs have been updated to be 8 characters long.

DX-59586
APP

The button to add an Arctic Catalog to a Sonar project was not visible if an organization did not contain any projects.

DX-59533
APP

When trying to use an invalid name for a branch in Dremio Arctic, details are now provided about allowed characters and supported naming conventions.

DX-58843
APP

The author of a commit in Dremio Arctic was being displayed as the author's internal ID instead of his or her username.

DX-53696
APP

For some users, when trying to log in to Dremio Cloud using Okta, the login would fail and return users to the login prompt.

DX-60042
APP

When loading some views in BI tools to create a visualization, more fields were being displayed than actual columns in the view. If any of the excess fields were used in the visualization, an error was produced.

DX-58864
APP

For tables created from a folder of files, the jobs count on the Datasets page was incorrect as it always showed 0.

DX-58604
APP

Improved the error message that is displayed when trying to create a new folder with a duplicate name in the same location.

DX-58464
APP

If you clicked on a job to view details, your position on the page was reset when clicking the Back button or the Jobs link on the page header. Your position on the main Jobs page is now maintained in these scenarios.

DX-54834
SQL

In some cases, with the Arrow Flight SQL ODBC driver, users were getting an error when testing the connection to Microsoft Excel in the ODBC Administrator on Windows.

DX-60176
SQL

Clicking Discard in the SQL Runner would not clear the query context if you hadn't first run a query.

DX-59516
SQL

When copying a view definition and pasting into the SQL editor, the pasted SQL was incorrect because newlines were not being retained.

DX-59275
SQL

Fixed an issue that was causing accelerated queries to fail after a project was reactivated or restored.

DX-57197
SQL

Removed duplicate user query alternative plans during reflection matching to reduce logical planning times.

DX-57063
SQL

Fixed an issue that was causing COALESCE queries containing NULLIF calls to not get pushed down to Oracle.

DX-55270

November 30, 2022

Issues Fixed
SQL
DX-56538

If a query contained CONVERT_FROM() on a large json literal string, the query was failing with an OutOfMemoryException error.

SQL
DX-59062

Running SHOW SCHEMAS was failing in when used with Arrow Flight, causing failures with Apache Superset.

SQL
DX-56449

Fixed an issue that was causing some queries to fail with a NullPointerException in com.dremio.exec.store.iceberg.DremioInputFile.newStream().

November 17, 2022

What's New
SQL
DX-56250

This release includes performance improvements for incremental metadata refreshes on partitioned Parquet tables.

Issues Fixed
General Updates
DX-55844

This release contains a number of fixes to address potential security issues.

APP
DX-58719

Fixed an issue that was causing a tooltip to fail to appear when trying to execute a query using a stopped engine.

APP
DX-58471

Navigating to a page URL with an invalid project ID was failing to produce a 404, as expected.

APP
DX-58392

When working in an Arctic Catalog source, the branch was getting reset to main if you clicked the History button to go to the Commits page.

APP
DX-57774

For tables created from a folder of files, the jobs count on the Datasets page was incorrect as it always showed 0.

APP
DX-55680

Fixed an issue that was causing the status of a cancelled job to show as RUNNING or PLANNING.

APP
DX-54834

If you clicked on a job to view details, your position on the page was reset when clicking the Back button or the Jobs link on the page header. Your position on the main Jobs page is now maintained in these scenarios.

APP
DX-54194

When a materialization took too long to deserialize, the job updating the materialization cache entry could hang and block all reflection refreshes.

SQL
DX-58926

INFORMATION_SCHEMA.COLUMNS queries on Arctic Catalogs were failing if Dremio was unable to access the Iceberg metadata file.

SQL
DX-57123

Fixed a bug that was creating a race condition, causing REFRESH REFLECTION and REFRESH DATASET jobs to hang when reading Iceberg metadata.

SQL
DX-57025

Fixed an issue that was causing Oracle pushdowns to fail on JOIN queries that used the TO_DATE function.

November 7, 2022

What's New
APP

For Dremio Arctic, you can now add the Arctic catalog as a source within either the Arctic service or the Sonar service. For more information, see Connecting an Arctic Source to Sonar.

APP

For Dremio Arctic catalogs that are available in your Sonar project, you can now access tables and views that are on the main branch. For more information, see Creating a Live Connection to a Dataset from Dremio Cloud.

November 4, 2022

Issues Fixed
APP
DX-57361

Nested views based on tags in Arctic were not resolving properly.

APP
DX-57121

When working in an Arctic source, if the branch name included a space in its name, Dremio was producing a generic "Unexpected error occurred" when creating a branch.

APP
DX-57019

If you created an empty folder in an Arctic source, you couldn't delete it using the Remove Folder action.

APP
DX-56544

Fixed an issue that was taking users from a page under Settings to an invalid page in the SQL Runner.

APP
DX-56442

If you selected some text in the Dremio app, the Copy Selection popup was appearing in the upper-right corner of the screen instead of next to the selected text.

APP
DX-55599

On the Jobs page, you can now right-click on a job's row and use a new popup to view the job details, open the job details in another tab, or copy the job ID.

APP
DX-55002

On catalog entities, ownership granted to a role was not being inherited by users in that role.

APP
DX-54560

Fields requiring paths to an S3 bucket are now pre-populated with the s3:// prefix, so you only have to include the remainder of the path.

APP
DX-54194

When a materialization took too long to deserialize, the job updating the materialization cache entry could hang and block all reflection refreshes.

SQL
DX-57312

In the SQL Runner, if you expanded an object in the Data panel while the object was empty, new items created in the object would not show up until you refreshed the page.

SQL
DX-56625

If you highlighted a single statement in the SQL Runner and used the keyboard shortcut to run it, Dremio was running the entire query instead of only what had been highlighted.

SQL
DX-55000

The starting_ts and starting_ts_epoch columns have been added to sys.project.history.jobs table. These columns contain the date/time and time since epoch when the job entered the STARTING state.

October 21, 2022

Issues Fixed
APP
DX-58023

Fixed an issue that was preventing any users added after the first user from seeing or querying the Dremio sample dataset in the first Sonar project.

October 19, 2022

What's New
General Updates
DX-54577

We have continued to streamline and simplify the sign-up and onboarding flow. All you need to do is choose your authentication method and provide a name for your organization. After Dremio finishes creating all of the required cloud resources and your first project, you'll end up on the new landing page, where you can manage organization settings or dive deeper into Dremio Sonar or Dremio Arctic.

APP

Dremio Cloud has a new home page for your organization where you can quickly access organization settings, Sonar Projects, and Arctic Catalogs. For more information, see Navigating Dremio Cloud.

APP

Dremio Arctic now supports the Arctic Catalog, an Iceberg catalog that enables you to list and manage Iceberg tables and views. An Arctic catalog consists of one or more branches, and a branch contains zero or more folders, tables and views. For more information, see Overview of Dremio Arctic and Managing Catalogs.

APP

By default, your first project now contains the Dremio sample data source, with the NYC Taxi trip data promoted to a table and ready for your first query. From the organization landing page, click Add Sonar Project on the Sonar card to add your first project, then go to Getting Started with Dremio Sonar to see how quickly you can run your first query.

APP

To help you better manage cloud costs, Dremio Cloud's preview engine is now configured to stay disabled until you need it. Regular tasks like metadata refresh will run less often by default, and those jobs are bundled to consolidate execution time.

note

If the preview engine is not running and you want to add a new database source, Dremio will ask you to confirm that you want to start the preview engine before proceeding.

DX-54138
SQL
DX-46373

Dremio now supports wide tables. See Datasets for more information and limitations.

Issues Fixed
General Updates
DX-54176, DX-54174, DX-54214

This release includes a number of fixes that resolve potential security issues.

General Updates
DX-56334

Updated the PostgreSQL JDBC Driver to version 42.4.1 to address CVE-2022-31197.

General Updates
DX-56143

Updated org.eclipse.jetty:jetty-io to version 9.4.47 to address potential security issues.

API
DX-57026

When provisioning users, groups, and roles from Azure AD to Dremio Cloud, roles were not being assigned to added users. The issue was caused by a case mismatch in SCIM API calls. To address this issue in Dremio Cloud, SCIM APIs are now more lenient regarding the case-sensitivity of JSON keys and sub-types.

API
DX-32370

Frequent, consecutive requests to the Job API endpoint to retrieve a Job's status could result in an UNKNOWN StatusRuntimeException error.

APP
DX-57282

At times, when using Dremio's Azure AD integration, users would see threads blocked on a call to Dremio's internal Azure AD cache.

APP
DX-56891

Disabling the preview engine was causing connection issues for database sources.

APP
DX-56244

Fixed an issue that was causing REFRESH REFLECTION and REFRESH DATASET jobs to hang when reading Iceberg metadata.

APP
DX-55748

Under certain conditions, such as during Dremio Cloud maintenance windows, reflection jobs or dataset metadata jobs may stop refreshing.

APP
DX-54194

When a materialization took too long to deserialize, the job updating the materialization cache entry could hang and block all reflection refreshes.

SQL
DX-56674

Some queries were failing with a message indicating an error that could not be retried, caused by a sort operation that exceeded the maximum amount of memory.

SQL
DX-56196

BRANCH was not getting highlighted as a pre-reserved keyword in the SQL editor.

SQL
DX-56085

For some queries on views using OR or IN, unnecessary subquery scans were not removed, causing inconsistent results.

SQL
DX-55002

On catalog entities, ownership granted to a role was not being inherited by users in that role.

SQL
DX-54273

Fixed an issue that was causing the error GandivaException: Failed to make LLVM module due to Function double abs(double) not supported yet for certain case expressions used as input arguments.

SQL
DX-52840

On occasion, projecting complex data types would result in a Schema change exception.

October 18, 2022

Issues Fixed
API
DX-57026

When provisioning users, groups, and roles from Azure AD to Dremio Cloud, roles were not being assigned to added users. The issue was caused by a case mismatch in SCIM API calls. To address this issue, SCIM APIs are now more lenient regarding the case-sensitivity of JSON keys and sub-types.

September 28, 2022

What's New
General Updates
DX-54322

To help avoid unnecessary costs, Dremio automatically deactivates any project that has not been accessed in the last 15 days. Dremio sends a courtesy email to project owners three days prior to deactivation. In addition, users with OWNERSHIP privileges or users in the ADMIN role can archive a project. Inactive and archived projects do not consume any EC2 resources and do not incur any costs in your AWS account. See Managing Projects for more information.

Issues Fixed
General Updates
DX-56556

Fixed an issue that was causing REFRESH REFLECTION and REFRESH DATASET jobs to hang when reading Iceberg metadata.

APP
DX-55748

Under certain conditions, such as during Dremio Cloud maintenance windows, reflection jobs or dataset metadata jobs could stop refreshing.

September 19, 2022

Issues Fixed
General Updates
DX-55125

When trying to sign up for Dremio Cloud using the Google, Microsoft, or GitHub options, the page was stuck in a loading state for several minutes and eventually closed with an error.

September 8, 2022

What's New
APP
DX-54322

We've made a number of changes to continue our ongoing UI improvements (e.g., better tooltips, improved error messages, etc.).

Issues Fixed
APP
DX-55125

Fixed an issue that could cause the Dremio app to hang when changing the current Dremio edition.

APP
DX-54835

In some cases, frequent scale up and scale down events were occurring on the Preview engine, and more replicas than the maximum were in use.

APP
DX-53563

When a dataset was using too many tags to display in a single line on the Datasets page, there was no way to expand the list to see all tags. In these cases, Dremio now displays an ellipsis button (...) to expand the list of tags.

APP
DX-52818

If the Preview engine was stopped (i.e., minimum replicas set to zero) and you clicked on a view from the Datasets page, no preview was displayed but Dremio started the Preview engine. The screen was blocked by a popup dialog indicating that the Preview engine was starting.

APP
DX-55520

Tags and Wiki content were being hidden for any promoted dataset. In this release they are only hidden for datasets belonging to Data-as-code sources.

SQL
DX-49518

In some cases, if users had saved a large number of SQL Runner scripts and tried to save another, Dremio could take several seconds to create the script and refresh the page.

August 22, 2022

What's New
APP
DX-54322

We've made a number of changes to continue our ongoing UI improvements (e.g., better tooltips, improved error messages, etc.).

Issues Fixed
APP
DX-54468

Changes made to the columns displayed on the Jobs page, or the order of the columns, were not being saved after leaving the page.

SQL

For some queries, when viewing the details of a job that was opened from the Jobs page, clicking the Open Results link in the upper-right corner of the page resulted in a dataset tmp.UNTITLED version ... error on the Datasets page.

SQL
DX-53827

When looking at the query Graph for a view and clicking the query icon of the parent table, the Dataset Settings from the results were showing settings for the view instead of the table.

SQL
DX-53740

In rare cases, an issue in the planning phase could result in the same query returning different results depending on the query context.

SQL
DX-53358

After changing the engine configuration, some queries were failing with an IndexOutOfBoundsException error.

SQL
DX-52553

When a query that used a reflection was executed multiple times, some of the jobs used the reflection and some did not.

SQL
DX-52443

Dremio was failing to parse queries on a view when the query originated from a JDBC/ODBC client that had the quoting connection property set to a non-default value.

SQL
DX-50906

Some queries that used the FLATTEN() function were showing results for a Preview, but no data was returned when using Run.

SQL
DX-54938

In some scenarios, invalid metadata about partition statistics was leading to inaccurate rowcount estimates for tables. The result was slower than expected query execution or out of memory issues. For each table included in a query where this behavior appears, perform an ALTER TABLE <table-name> FORGET METADATA, then re-promote the resulting file or folder to a new table. This will ensure that the table is created with the correct partition statistics.

SQL
DX-54059

VIEW was not being highlighted as a pre-reserved keyword in the SQL editor.

August 9, 2022

What's New
General Updates
DX-41179

When signing up for Dremio Cloud and creating a new organization, the CloudFormation template now supports AWS Key Management Service for encrypting the project store. After launching the template in the final step of the sign-up wizard (Step 3 of 3 - Configure AWS resources), you can select any of the following for Encryption Type: SSE-S3 (default), SSE-KMS (AWS Managed Key), SSE-KMS (Customer Managed Key)

Note:
If you select SSE-KMS (AWS Managed Key), AWS will create and manage the key for you. If you select SSE-KMS (Customer Managed Key), you will need to provide the KMS Key ARN of the key you have already created.

APP
DX-49082

For Dremio Arctic, we’ve added support for views, enabling you to create, replace, show, and drop views in your data source. Views created in Arctic have the notion of versions and are Iceberg views (see 0.14 Iceberg View Spec for the experimental implementation).

APP
DX-50054

Operators in the Job profile now include applicable dataset names and an icon to indicate that the operator used a reflection.

SQL
DX-48583

In addition to sharing with users, you can now share scripts saved in the SQL Runner with a role (see Adding Privileges).

SQL
DX-49988

Improved the error message that is displayed when trying to run DML commands that are not supported on views saved from Iceberg tables.

Issues Fixed
APP
DX-52174

When creating a personal access token, pressing the Enter key while the cursor was in the Lifetime field was incrementing the count by one instead of submitting the form.

APP
DX-52181

When hovering over any of the bars on the Usage chart, an "o" character was being displayed after the date.

APP
DX-47205

Some engine scaling events were being associated with the wrong engine ID.

SQL
DX-53038

Fixed an issue that was causing failures in Microsoft SQL Server queries that contained a boolean filter set to true.

SQL
DX-49941

INSERT, MERGE, UPDATE, TRUNCATE, and DELETE queries in the SQL Runner were failing with an Invalid path error when using a partial key/path.

SQL
DX-47568

CONVERT_FROM queries were returning errors if they included an argument that was an empty binary string. This issue has been fixed, and such queries have been optimized for memory utilization.

SQL
DX-48273

If issues were encountered when running queries against a view, Dremio was returning an error that was unhelpful. The error returned now includes the root cause and identifies the specific view requiring attention.

July 21, 2022

What's New
General Updates

We've made additional enhancements to simplify the signup flow, such as removing unrelated or unknown organizations and ensuring that you return to the correct organization when resuming sign-up.

General Updates

If you close the cookie consent or cookie preferences dialog without accepting functional cookies, we'll disable functional cookies for you and won't show the cookie consent dialog again.

General Updates

We've made some improvements to correctly validate inputs when you add a project and select the IAM Role option for Project Data Credentials.

General Updates

Based on feedback from our users, Dremio no longer creates an engine named First Engine when signing up.

APP

Dremio now supports reflections on the sys.project.history.jobs and sys.project.history.events system tables.

APP

Changes to Cloud and Edition are now included in the audit log, and the events are available in the default project for an organization.

APP

When a job has completed successfully and doesn't have a profile, the Profile page now contains more information about why the profile/execution details are not available.

June 29, 2022

What's New
General Updates

Based on feedback from our customers, we have made several enhancements to the sign-up experience. We have added more helpful information and additional details about various options and fields that need to be filled in, provided more feedback about progress, and new users are now logged in directly to their new organization when sign-up is complete.

SQL

When you open a dataset in the SQL editor, Dremio will no longer generate a preview of the dataset by default. You will need to click Preview or Run to view the dataset.

June 16, 2022

What's New
General Updates
DX-49693

Added a close button to the cookie consent dialog on the Dremio Cloud and Tableau login pages.

APP

Dremio now supports row-access and column-masking policies for row and column controls over user query access to sensitive tables, views, and columns. This allows administrators to dynamically exclude or mask private data at the column and row levels prior to query execution and without physically altering the original values.

APP
DX-47063

Extra spaces will now be trimmed from fields to avoid errors when adding external sources.

APP
DX-48013

Added a plus button to the upper-right corner of the page for spaces and Nessie sources that allows users to quickly upload a file or add a new folder, table, or view.

APP
DX-49126

The option to enable Arrow caching in advanced reflection settings has been removed because Arrow caching is not supported with unlimited splits.

SQL

This release adds to existing Iceberg DML capabilities allows users to run DELETE, UPDATE, MERGE, and TRUNCATE statements against Iceberg tables. See SQL Commands for more information.

SQL

You can now create SQL scalar user-defined functions (UDFs), which are callable routines that make it easier for you to write and reuse SQL logic across queries. UDFs let you extend the capabilities of Dremio SQL, provide a layer of abstraction to simplify query construction, encapsulate business logic, and support row and column policies for access control.

SQL

Added support for internal schema using SQL commands, which lets the user override the data type of a column instead of using the type that Dremio automatically detected.

SQL

The DATEDIFF and ADD_MONTHS Hive functions are supported in queries.

SQL
DX-48213

Script owners in the SQL Runner are now indicated with a small orange flag next their username. Script owners cannot be removed or have their privileges changed.

SQL
DX-49374

The option to save a script will be disabled if the user already has 100 scripts, which is the maximum per user, and the Dremio app now provides a tooltip to provide information why an additional script cannot be saved.

SQL
DX-49641

When adding or modifying SQL Runner script privileges, the View privilege is enabled automatically if any of the other privileges are enabled.

Issues Fixed
API
DX-48967

The created_by column in the sys.organization.roles table was showing LOCAL for users created via the SCIM API.

APP
DX-44599

If you tried to access the Execution tab for a job that had failed or was in progress, an unknown "Profile Fragment is Empty" error would be displayed. Now Dremio provides more useful information about the job status.

APP
DX-47521

The metadata_retrieval_ts, engine_start_ts, metadata_retrieval_epoch, and engine_start_epoch fields in the sys.project.history.jobs table were empty.

APP
DX-48970

The created_by column was empty for predefined roles (admin, local) in the sys.organization.roles table.

APP
DX-49822

When creating a personal access token, the value in the Lifetime field was displayed as NaN when using the up and down arrows to change the value.

APP
DX-50085

Tree items on the Join page were expanding one level too far, to the column level.

APP
DX-50221

Fixed an issue that was preventing users from scrolling to items at the bottom of the Data panel in the SQL Runner if items near the top had been expanded.

SQL
DX-48201

A DROP TABLE query may have appeared to run successfully, even if the table being dropped didn't exist.

SQL
DX-48356

If a query ended with a comment followed by whitespace, the comment was being treated as a query.

SQL
DX-49849

When query results were truncated to optimize performance, the notification about truncated results was showing up inconsistently.

May 24, 2022

What's New
General Updates
DX-49546

This release adds support for Okta's custom authorization server issuer URL: https://${yourOktaDomain}/oauth2/${authorizationServerId}. Previously, only https://${yourOktaDomain} was supported.

APP

The Columns button on the Jobs page has been replaced with the Manage Columns button, and you can change the order of displayed columns using drag and drop.

APP
DX-44234

Dremio now displays the status of replica scale down events as Pending, Completed, or Failed to more accurately represent replica state.

APP
DX-49374

The option to save a script will be disabled if you already have 100 scripts, which is the maximum per user, and the Dremio app now provides a tooltip to provide information why an additional script cannot be saved.

SQL

In this release, Dremio supports changing the partition spec of an Iceberg table via the ALTER command.

SQL

Along with ROW and ARRAY keywords, you can now use STRUCT and LIST keywords to represent complex data types.

SQL

In this release, you can share scripts with others in your organization by adding users and assigning privileges to View, Modify, Manage Grants, or Delete. For more information, see Adding Privileges.

SQL
DX-48955

The default save options for an untitled script are Save Script and Save Script as. When editing an active script, the Save Script option is only available if you have MODIFY privileges.

Issues Fixed
General Updates
DX-49089

Fixed an issue that was causing users to be directed to the Users page when clicking on the email signup link.

General Updates
DX-49090

After signing up, Dremio was directing users to the login page instead of taking them to their organization.

General Updates
DX-49348

In some cases, the Google, Microsoft, and GitHub options were not working as expected during signup.

APP
DX-47419

When Arrow caching was enabled, some query runs may have shown inconsistent results.

APP
DX-45709

When viewing the details of a failed query from the Jobs page, if the query had been cancelled because it exceeded the currently defined memory limits, a generic/unknown No value present error could pop up with the details page.

APP
DX-48269

Fixed an issue that was preventing users from downloading profiles.

APP
DX-48778, DX-42370

An unknown error was being generated when attempting to remove a reflection from the Acceleration dialog and saving the change, and the error would continue to be displayed.

APP
DX-49048

The Default CTAS Format setting was being displayed under General settings for a source instead of Advanced Options.

SQL
DX-47423

On rare occasions, a query could hang for a long time, especially when multiple queries were run while the engine was starting up.

SQL
DX-45561

The first query issued on a new data source or running on a newly created engine or preview engine was taking slightly longer to run than subsequent queries.

SQL
DX-49147

Comma separators had been enabled in some fields where they should not have been enabled, such as Port.

SQL
DX-47953

The results table in the SQL Editor was displaying multiple space characters in the query as single spaces in the results.

SQL
DX-48356

If a query ended with a comment followed by whitespace, the comment was being treated as a query.

SQL
DX-48957

Trying to add a field to a single-line query ending in a semicolon was resulting in a Failure parsing the query error when applying the change.

SQL
DX-49026

The option to cancel a running query was not available in the SQL Runner.

April 21, 2022

What's New
General Updates
DX-47148

Made some updates to cookie handling for the Tableau login flow to ensure that users are directed to the right page when signing in.

General Updates
DX-47816

If a user tries to sign in after being removed from an organization, they will automatically be directed to the signup page instead of the login page for the organization.

DX-40702, DOCS-3
APP

Starring Catalog Items: This release of Dremio allows you to star frequently used catalog items, such as views, tables, spaces, and sources, which adds them to your Starred list for easier access. The Starred list holds up to 25 entities at a time, and items remain on the list if you open a new browser or clear the cache. See Starring a Dataset for more information.

DX-44853
APP

This release includes various updates to icons and other elements of the Dremio app to enhance usability and overall design.

DX-39981
SQL

Multi-line SQL: You can now execute multiple SQL statements in the SQL Editor. Statements run sequentially in the order they appear. Each statement will generate a unique Job ID, and the results of each statement will be displayed in a separate tab in the result area. See Querying Your Data for more information.

DX-40701
SQL

SQL Autocomplete: Autocomplete is now available in the SQL Editor. When enabled, autocomplete lets you view and insert possible completions in the editor using the mouse or the arrow keys with Tab or Enter. Autocomplete can provide suggestions for SQL keywords, catalog objects, and functions while you are constructing SQL statements. Suggestions depend on the current context. The autocomplete feature can be enabled or disabled for all users using the Project Settings.

DX-39681
SQL
You can now use commits as a reference to query tables.
DX-46880
SQL
When a user switches between scripts, the active query and its history will be cleared.
Issues Fixed
General Updates
DX-46216

When creating the first project, the wizard was allowing users to proceed without selecting an AWS Region.

General Updates
DX-46551

The organization name was not being displayed on the login page for Tableau users who were returning to the Dremio app.

General Updates
DX-46951

Users were not seeing all organizations with which their email address was associated on the organization page, even though all cookies were enabled.

DX-47530 **TO VERIFY**
API

Email addresses are now URL encoded when making API calls for Tableau logins to avoid issues with emails containing special characters.

DX-44039
APP

Some requests to create a new engine were failing with a 502 or 500 error.

DX-43177
APP

Microsoft Azure Synapse Analytics was showing up as an available external source, though it is not currently supported.

DX-46371
APP

Fixed an issue that was causing the following error when trying to open a view in the Dataset page: Some virtual datasets are out of date and need to be manually updated.

DX-46562
APP

When viewing All External Sources in the Datasets view, Nessie repositories were included in the list of external sources.

DX-46563 **TO VERIFY**
APP

Previously configured storage credentials were not being preserved when editing a Nessie source.

DX-47280
APP

If Power BI and Tableau were disabled, they were still available under the Analyze menu.

DX-47417
APP

Engine replicas were being shut down in the middle of query execution if the corresponding engine was modified or disabled while the query was running.

DX-47486
APP

When there were too many projects and engines to display on a single page under Usage settings, users could not scroll down to see the full list.

DX-47614
APP

The name of the view currently open in the Dataset page was changing to New Query (edited) after running a filter in the SQL editor.

DX-47779If
APP

If Wiki content on the catalog page was too long, the left navigation bar was not resizing as the page was scrolled down.

DX-47824
APP

Fixed an issue where the icon for switching between Light Mode and Dark Mode would not display properly when in Light Mode on Job Details page.

DX-48312
APP

In some cases, not all DCU consumption details were showing up under Billing > Transactions.

DX-45691
SQL

During periods of high load, when a large number of concurrent queries were running, users may have seen jobs failing with UNKNOWN or MongoWriteException errors.

DX-44921
SQL

Queries that failed due to a recoverable exception, such as a schema change or concurrent metadata refresh for internal refresh queries, are now listed on the Jobs page as Running instead of Failed.

DX-44195 **TO VERIFY**
SQL

When viewing job details, from the Jobs page or the Run link in the SQL Runner, the status of some jobs was incorrect in the case of multiple query attempts.

DX-45027
SQL

ASC and DESC were not being highlighted as reserved keywords in the SQL Editor.

DX-46810
SQL

The CREATE TABLE command was not working when used with IF NOT EXISTS.

DX-47728
SQL

Clicking the plus icon (+) next to a column name in the data panel was adding the full column path to the SQL Editor instead of only the column name.

March 21, 2022

What's New
DX-44895
General Updates
The links to Dremio's Privacy Policy and Terms of Service have been updated on the signup and login pages.
APP

Dremio Cloud now offers powerful SQL database-like functionality on Apache Iceberg tables through industry-standard SQL commands. For more information, see SQL Commands.

DX-43923
APP

If you want to change to Dremio's Standard edition but have Enterprise features configured, those features will be indicated along with links to the configuration pages that control them.

DX-42323
APP

The design (color, border style, etc.) of some UI elements in Dremio have been updated to match the design of the rest of the app.

DX-45678
APP

The peak memory usage shown in the operator profile has been updated to show the maximum of memory reserved and memory used.

DX-45755
APP

The Nessie version used in Dremio Arctic has been upgraded to 0.21.2.

DX-45102
SQL

In the Query Visualizer, the Processing Time label has been changed to Runtime, and Memory is now Total Memory.

Issues Fixed
DX-43525
General Updates

When clicking Remove on the organization selection form, users were getting directed back to the signup page.

DX-45013
APP

At times, using the project switcher to change the current project to an Arctic project was generating an unknown error.

DX-44276
APP

Multiple background metadata refresh jobs were getting triggered at the same time for a dataset, causing some jobs to fail.

DX-41342
APP

Fixed an issue that was displaying a brief message about queue assignment rules when navigating to the Engine Routing page from the Reflections page under Project Settings.

DX-44164
APP

Fixed an issue with metadata refresh that could result in incorrect results or query exceptions due to an expected row count mismatch.

DX-44778
APP

If a login method was disabled but that same method was stored in a cookie, users could still use that method to log in.

DX-44268
APP

If a user was active in one or more organizations, they were unable to sign up for a new organization.

DX-46135
APP

In some cases, engines were failing to start in a new Dremio Cloud organization.

DX-45800
APP

In some cases, when viewing usage data for the current month and there was no data available for the current month, users were seeing an unknown error.

DX-45755
APP

Merging in an Arctic project was failing if there were more than 20 commits on the source branch.

DX-44856
APP

The Nessie endpoint was not being passed when a Nessie source was added.

DX-45650
APP

Usage across all engines was not getting aggregated correctly, so total usage data was incorrect.

DX-45028
APP

Users were able to enter custom input when selecting columns to add Measures to a Reflection, but custom input should not have been enabled and was generating an unknown error.

DX-45033
APP

A table cannot be selected as the destination for saving a view, but users were able to select a table when saving a view, and the view was not saved.

DX-44850
APP

When a user was deleted, the notification confirming the action was showing the deleted user as undefined instead of as the actual user ID.

DX-42536
APP

When users tried to add a tag to a dataset's catalog, a Tags are not saved error was produced even though the tag was still applied.

DX-45096
SQL

A query with not in was not returning the correct results for Hadoop and Hive datasets.

DX-45658
SQL

Clicking Discard in the SQL Runner was not clearing the query contents.

DX-44237
SQL

Some queries against ORC tables in a Glue source were failing with the following error: Hadoop native library is required for Hive ORC data, but is not loaded

March 7, 2022

Issues Fixed
General Updates
DX-45800

In some cases, when viewing usage data for the current month and there was no data available for the current month, users were seeing an unknown error.

General Updates
DX-44165

If ALTER privileges on a space were revoked for a user, the user could still access the space but they did not have any access to the views or tables in the space.

General Updates
DX-43598

Ownership of a view could be successfully granted to an inactive user.

March 1, 2022

What's New
General Updates

A link to full terms of the invoice/billing agreement is now available in the Add Billing Account dialog.

General Updates

Various updates have been made to improve query execution as well as general app performance.

API

The state of a project created using the /projects API will now return as either CREATING or ACTIVE.

API

The requestId attribute is now optional when creating a project with the /projects API.

API

The following ruleSet attributes have been renamed in the Engine Rules (/rules) API: ruleName -> name, ruleCondition -> condition, ruleAction -> action, outputValue -> engineName, PLACE -> ROUTE (action attribute). Additionally, the following attributes have been removed, outputId, ruleUniqueId, and version.

APP

When editing an engine, the Last Replica Auto-Stop field under Advanced Configuration requires a minimum value of 60 seconds.

APP

The SQL Runner now allows you to save your SQL as a script. See Querying Your Data for more information.

APP

Changed a generic internal server error message that was displaying when trying to cancel a query preview that had finished. The message now indicates that the completed job cannot be cancelled.

APP

Standard Edition users can no longer create roles, enterprise identity providers, or external token providers, and Dremio now provides tooltips to indicate that these are Enterprise features.

APP

Disabled the Delete All button under Account Settings > Personal Access Tokens if there are no tokens.

APP

Engine scaling events (adding or removing replicas) are now captured in the Dremio Cloud audit log.

SQL

We've made some changes to the way engine status is displayed in the SQL Runner to make job and engine state easier to understand.

SQL

Users can now expand (|) or collapse (|) the catalog explorer panel when working in the SQL Editor or SQL Runner.

SQL

Added the ability to push down DATE_ADD and DATE_SUB functions to RDBMS sources.

SQL

Added the ability to push down DATE_ADD and DATE_SUB functions to RDBMS sources.

SQL

The queue_name column has been removed from the jobs runtime system table (SYS.PROJECT.JOBS) because queues are not supported in Dremio Cloud.

SQL

In SYS.PROJECT.VIEWS and SYS.PROJECT."TABLES", if an owner of a view or table has been deleted, owner_id now shows up as $deleted$ instead of the invalid user ID.

Issues Fixed
General Updates

Fixed an issue that was causing errors when running the CloudFormation template during the automatic cloud connect portion of the signup process.

General Updates

During signup, user input in the Organization and Project fields was not getting validated, resulting in a generic error when trying to submit.

General Updates

If password authentication was disabled after a user had started the signup process from the invite link or by entering their email address on the login page, the user could still log in with a password.

General Updates

Fixed an issue that was redirecting users to the login screen when trying to remove an organization from the organization selection screen.

General Updates

If a user was active in one or more organizations, they were unable to sign up for a new organization.

General Updates

Continuing the signup process on an incomplete organization was resulting in a CloudFormation template error.

API

The Billing transactions API was not filtering results properly for a billing account.

APP

The Reflections tab was not showing up under Project Settings for local or PUBLIC users.

APP

Tooltips to provide details about engine states were not always displaying when expected.

APP

If a user's login session had expired and they tried to visit a page in the app, an unexpected error was showing up instead of taking the user directly to the login page.

APP

The tooltip to indicate the status of an engine that was starting up was not displaying unless users hovered over the engine dropdown.

APP

When browsing the Dremio app with Edge, scrollbars were not showing up and users had to zoom out to view certain portions of the screen.

APP

Not all users were showing up when searching in the User filter on the Jobs page.

APP

The Upload File button on the Datasets page was available all the time, regardless of whether or not it had been disabled.

APP

The Tableau and PowerBI buttons were not showing up or remaining hidden as expected, and they are now enabled all the time in the SQL Editor.

APP

If password authentication was disabled after a user had started the signup process from the invite link or by entering their email address on the login page, the user could still log in with a password.

APP

When adding a tag to a dataset's catalog, a Tags are not saved error was displayed even though the tag was still applied.

APP

After removing a view from a space, the total number of views in the space was not getting updated in the Spaces panel.

APP

In some cases, if a Parquet file in a Delta Lake table had many row groups, COUNT(*) queries were failing due to a divide by 0 exception.

SQL

Even though ownership of a folder or table had been granted to a role, users belonging to that role were unable to access the folder or table.

SQL

Ownership (owner_id in SYS.ORGANIZATION.USERS) of invited users was not being set to the ID of the user who invited them.

SQL

IN filters were not working after applying an internal schema on a varchar file.

SQL

Long paths to a dataset were running into the tab names above the SQL Editor.

SQL

Fixed an issue that was causing some reflections to fail with a NessieReferenceConflictException error.

February 15, 2022

What's New
General Updates

Audit logging is now available in Dremio Cloud. See Audit Logging and SYS.PROJECT.HISTORY.EVENTS for more information.

General Updates

Email verification is now required during user signup when using username/password authentication.

General Updates

If a user had not completed the onboarding process during signup, the incomplete organization was showing up on the login page and there was no way to remove the incomplete organization. The option to remove such organizations is now available.

General Updates

We've simplified and streamlined our login flow by using cookies. If a user has accepted the use of all cookies, the organization selection and login screens will be skipped if the user still has an active login session. Additionally, the main login screen will be skipped if the user has logged in previously from the current browser (even if the login session has expired).

API

An engine is no longer created when adding a project with the /projects API. Users can create engines independent of projects.

APP

Dremio Cloud Admins now have access to Billing, where they can change between Dremio Standard or Enterprise editions, create and modify a billing account, and view detailed transaction data for DCU usage.

APP

Apache Iceberg is now a supported format for tables.

APP

Copyright information is now displayed directly in the help menu, and the About Dremio option has been removed. Edition details can now be found in the Billing section under Organization Settings.

APP

Each OAuth application listed under Organization Settings now includes the application ID.

APP

IP addresses are now used instead of hostnames to track replicas, meaning that DNS hostnames no longer need to be enabled in the VPC. Additionally, the Engines detail page uses Host IP instead of Hostname and lists replica IP addresses.

APP

Our enhanced CloudFormation Template provides updated IAM policies with narrowed permission scopes to enhance customer security.

APP

The is_member condition for building Engine Rules was not working for internal roles.

APP

The support portal link on the Help menu is only available to Admin users in organizations using the Enterprise edition of Dremio Cloud.

APP

We've added new engine states to provide more granular details about engine status. For more information, see Managing Engines.

APP

Users with the PUBLIC role can now view the OAuth Application page under Organization Settings.

SQL

Admins can grant or revoke the CONFIGURE SECURITY privilege to a user or role. Users who have been granted this privilege (directly or via role) will have full access to Authentication, External Token Providers, BI Applications, and OAuth Applications under Organization Settings.

SQL

Runtime system tables for tables and views are now available in Dremio Cloud. See SYS.PROJECT."TABLES" and SYS.PROJECT.VIEWS for more information.

SQL

Runtime and historical jobs tables are now accessible as sys.project.jobs and sys.project.history.jobs.

Issues Fixed
API

Users were not able to modify any engine attributes via the Engines API unless they specified all available attributes. All attributes are optional now.

APP

Clicking the help link in the Add OAuth Application dialog was resulting in a 404/page not found error.

APP

Fixed a Job <job-id> output doesn’t exist error message to provide more information about why the job could not be retrieved and how the user can address the issue.

APP

Fixed an issue that was causing an unknown error when searching on the Engines page.

APP

Fixed an issue that was preventing users from disabling SSO for Power BI by simply clearing the Enable checkbox and saving — the Save button was not available in this scenario.

APP

If a non-admin user had ownership granted for a view, the view was not showing up for the user.

APP

We've fixed an issue that would occasionally result in lost connectivity to a source.

APP

When adding a new project, the Cloud used for the latest project is selected by default.

APP

When hovering over a specific entry in the Usage graph, Dremio now shows details that include the date and DCUs used on that day.

APP

When reordering an engine on the Engine Routing page, both an error and a success message were being displayed.

APP

When viewing usage details for a specific project, changing the usage period was taking users back to the All Projects view.

November 2021

Issues Fixed
APP

The bug that prevented you from assigning the Administrator role to users has been resolved.

APP

The logos for social login options have been fixed to correctly match with the text.

APP

Errors messages for issues with granting and revoking privileges and not having priviliges to perform an action have been enhanced to be specific and clear.

APP

Enhanced error message to include parsing message when the engine rule being added is syntactically incorrect (there was an UNKNOWN error being returned previously).

October 2021

What's New
General Updates

Added support for the eu-west-2 (London) region.

General Updates

Fixed a deadlock with processing engine scaling events, which prevented engines from scaling up and shutting down.

General Updates

In addition to admin users, users with the OWNERSHIP or MANAGE GRANTS privileges on an object can now transfer ownership to a new user.

Issues Fixed
SQL

Improved error messages when a user attempts to run ALTER USER with a PASSWORD clause (this action is not supported through SQL).

SQL

You can now successfully add a user as an admin on the Administrators page.

September 2021

What's New
General Updates

EU Control Plane: We now support the eu-central-1 (Frankfurt) and eu-west-1 (Ireland) AWS regions for the control plane, allowing for better latency and performance with EU-based data planes. For additional information on EU endpoints, see the Supported Regions page.

General Updates

The PUBLIC role no longer has privileges to modify, disable, or delete engines.

General Updates

Added a Default Project setting, which tools like Tableau and Power BI default to so you don't have to specify a project ID to connect to Dremio.

API

EU Control Plane: The API endpoint for the control plane is https://api.eu.dremio.cloud.

API

Personal access tokens are supported in the authorization header.

APP

EU Control Plane: The App endpoint for the control plane is https://app.eu.dremio.cloud.

APP

The login page of https://app.dremio.cloud displayed the deleted organizations. Now, the login page only displays the active organizations.

Issues Fixed
API

When an engine was created with a duplicate name through a POST operation, a 500 Internal Server Error response code was returned. The error handling has been corrected to return a 400 Bad Request response code when an engine is created with a duplicate name.

APP

Previously deleted organizations were displayed on the login page. This has been fixed so that only active organizations are listed on the log in page.

APP

The Documentation link in the Help menu now points to https://docs-beta.dremio.com.

APP

On the SQL Editor page, the search tab in the Datasets pop out has been removed because it the search function doesn't currently work. It will be added back when the search issues have been resolved.

SQL

Querying sys.privileges to retrieve the granted privileges did not return privileges on the current organization and the current project. Organization and project-level privileges and the associated organization and project are now returned correctly when querying sys.privileges.

Client
A fix has been implemented to prevent a new replica from launching when client applications that use JDBC went offline after the preview engine had been disabled/enabled. The periodic offline of JDBC sources was caused by the replica name being cached, but further updates were left untracked.