On this page

    Changelog

    We continue to add new features and fix issues to enhance Dremio Cloud. This changelog provides you with a record of those changes.

    April 21, 2022

    What's New
    General Updates

    Made some updates to cookie handling for the Tableau login flow to ensure that users are directed to the right page when signing in.

    General Updates

    If a user tries to sign in after being removed from an organization, they will automatically be directed to the signup page instead of the login page for the organization.

    APP

    Starring Catalog Items: This release of Dremio allows you to star frequently used catalog items, such as views, tables, spaces, and sources, which adds them to your Starred list for easier access. The Starred list holds up to 25 entities at a time, and items remain on the list if you open a new browser or clear the cache. See Starring a Dataset for more information.

    APP

    This release includes various updates to icons and other elements of the Dremio app to enhance usability and overall design.

    SQL

    Multi-line SQL: You can now execute multiple SQL statements in the SQL Editor. Statements run sequentially in the order they appear. Each statement will generate a unique Job ID, and the results of each statement will be displayed in a separate tab in the result area. See Querying Your Data for more information.

    SQL

    SQL Autocomplete: Autocomplete is now available in the SQL Editor. When enabled, autocomplete lets you view and insert possible completions in the editor using the mouse or the arrow keys with Tab or Enter. Autocomplete can provide suggestions for SQL keywords, catalog objects, and functions while you are constructing SQL statements. Suggestions depend on the current context. The autocomplete feature can be enabled or disabled for all users using the Project Settings.

    SQL
    You can now use commits as a reference to query tables.
    SQL
    When a user switches between scripts, the active query and its history will be cleared.
    Issues Fixed
    General Updates

    When creating the first project, the wizard was allowing users to proceed without selecting an AWS Region.

    General Updates

    The organization name was not being displayed on the login page for Tableau users who were returning to the Dremio app.

    General Updates

    Users were not seeing all organizations with which their email address was associated on the organization page, even though all cookies were enabled.

    APP

    Some requests to create a new engine were failing with a 502 or 500 error.

    APP

    Microsoft Azure Synapse Analytics was showing up as an available external source, though it is not currently supported.

    APP

    Fixed an issue that was causing the following error when trying to open a view in the Dataset page: Some virtual datasets are out of date and need to be manually updated.

    APP

    When viewing All External Sources in the Datasets view, Nessie repositories were included in the list of external sources.

    APP

    Previously configured storage credentials were not being preserved when editing a Nessie source.

    APP

    If Power BI and Tableau were disabled, they were still available under the Analyze menu.

    APP

    Engine replicas were being shut down in the middle of query execution if the corresponding engine was modified or disabled while the query was running.

    APP

    When there were too many projects and engines to display on a single page under Usage settings, users could not scroll down to see the full list.

    APP

    Email addresses are now URL encoded when making API calls for Tableau logins to avoid issues with emails containing special characters.

    APP

    The name of the view currently open in the Dataset page was changing to New Query (edited) after running a filter in the SQL editor.

    APP

    If Wiki content on the catalog page was too long, the left navigation bar was not resizing as the page was scrolled down.

    APP

    Fixed an issue where the icon for switching between Light Mode and Dark Mode would not display properly when in Light Mode on Job Details page.

    APP

    In some cases, not all DCU consumption details were showing up under Billing > Transactions.

    SQL

    During periods of high load, when a large number of concurrent queries were running, users may have seen jobs failing with UNKNOWN or MongoWriteException errors.

    SQL

    Queries that failed due to a recoverable exception, such as a schema change or concurrent metadata refresh for internal refresh queries, are now listed on the Jobs page as Running instead of Failed.

    SQL

    When viewing job details, from the Jobs page or the Run link in the SQL Runner, the status of some jobs was incorrect in the case of multiple query attempts.

    SQL

    ASC and DESC were not being highlighted as reserved keywords in the SQL Editor.

    SQL

    The CREATE TABLE command was not working when used with IF NOT EXISTS.

    SQL
    Clicking the plus icon (+) next to a column name in the data panel was adding the full column path to the SQL Editor instead of only the column name.
    Known Issues
    Known Issues

    Sorting by Role or Status on the settings page for Organization > Users is not working as expected and has been disabled.

    Known Issues

    When viewing the details of a failed query from the Jobs page, if the query had been cancelled because it exceeded the currently defined memory limits, a generic/unknown No value present may pop up with the details page.

    Known Issues

    If you try to access the Execution tab for a job and the information is unavailable, Dremio will render new content related to the status of the information. If a job has failed, Dremio will report that the information is unavailable. If the job can still be cancelled, Dremio will report that the information is not yet available.

    Known Issues

    The first query issued on a new data source or running on a newly created engine or preview engine will take slightly longer to run than subsequent queries.

    Known Issues

    When Arrow caching is enabled, some query runs may show inconsistent results. To avoid this issue you can disable Arrow caching.

    Known Issues

    On rare occasions, a query can hang for a long time, especially when multiple queries are run while the engine is starting up. To work around this issue, you can run the query again or restart the engine.

    Known Issues

    A DROP TABLE query may appear to run successfully, even if the table being dropped doesn’t exist.

    Known Issues

    In some cases, the dataset jobs count in the folder view can be empty, despite the dataset having jobs associated with it. Use the Jobs page to see the accurate list of jobs.

    March 21, 2022

    What's New
    General Updates
    The links to Dremio's Privacy Policy and Terms of Service have been updated on the sign-up and login pages.
    APP

    Dremio Cloud now offers powerful SQL database-like functionality on Apache Iceberg tables through industry-standard SQL commands. For more information, see SQL Commands for Apache Iceberg Tables.

    APP

    If you want to change to Dremio’s Standard edition but have Enterprise features configured, those features will be indicated along with links to the configuration pages that control them.

    APP

    The design (color, border style, etc.) of some UI elements in Dremio have been updated to match the design of the rest of the app.

    APP

    The peak memory usage shown in the operator profile has been updated to show the maximum of memory reserved and memory used.

    APP

    The Nessie version used in Dremio Arctic has been upgraded to 0.21.2.

    SQL

    In the Query Visualizer, the Processing Time label has been changed to Runtime, and Memory is now Total Memory.

    Issues Fixed
    General Updates

    When clicking Remove on the organization selection form, users were getting directed back to the sign up page.

    APP

    At times, using the project switcher to change the current project to an Arctic project was generating an unknown error.

    APP

    Multiple background metadata refresh jobs were getting triggered at the same time for a dataset, causing some jobs to fail.

    APP

    Fixed an issue that was displaying a brief message about queue assignment rules when navigating to the Engine Routing page from the Reflections page under Project Settings.

    APP

    Fixed an issue with metadata refresh that could result in incorrect results or query exceptions due to an expected row count mismatch.

    APP

    If a login method was disabled but that same method was stored in a cookie, users could still use that method to log in.

    APP

    If a user was active in one or more organizations, they were unable to sign up for a new organization.

    APP

    In some cases, engines were failing to start in a new Dremio Cloud organization.

    APP

    In some cases, when viewing usage data for the current month and there was no data available for the current month, users were seeing an unknown error.

    APP

    Merging in an Arctic project was failing if there were more than 20 commits on the source branch.

    APP

    The Nessie endpoint was not being passed when a Nessie source was added.

    APP

    Usage across all engines was not getting aggregated correctly, so total usage data was incorrect.

    APP

    Users were able to enter custom input when selecting columns to add Measures to a Reflection, but custom input should not have been enabled and was generating an unknown error.

    APP

    A table cannot be selected as the destination for saving a view, but users were able to select a table when saving a view, and the view was not saved.

    APP

    When a user was deleted, the notification confirming the action was showing the deleted user as undefined instead of as the actual user ID.

    APP

    When users tried to add a tag to a dataset’s catalog, a Tags are not saved error was produced even though the tag was still applied.

    SQL

    A query with not in was not returning the correct results for Hadoop and Hive datasets.

    SQL

    Clicking Discard in the SQL Runner was not clearing the query contents.

    SQL

    Some queries against ORC tables in a Glue source were failing with the following error: Hadoop native library is required for Hive ORC data, but is not loaded

    March 7, 2022

    Issues Fixed
    This update fixes the following known issues in Dremio Cloud:

    General Updates
    In some cases, when viewing usage data for the current month and there was no data available for the current month, users were seeing an unknown error.
    General Updates
    If `ALTER` privileges on a space were revoked for a user, the user could still access the space but they did not have any access to the views or tables in the space.
    General Updates
    Ownership of a view could be successfully granted to an inactive user.

    March 1, 2022

    What's New
    General Updates

    A link to full terms of the invoice/billing agreement is now available in the Add Billing Account dialog.

    General Updates

    Various updates have been made to improve query execution as well as general app performance.

    APP

    When editing an engine, the Last Replica Auto-Stop field under Advanced Configuration requires a minimum value of 60 seconds.

    APP

    The SQL Runner now allows you to save your SQL as a script. See Querying Your Data for more information.

    APP

    Changed a generic internal server error message that was displaying when trying to cancel a query preview that had finished. The message now indicates that the completed job cannot be cancelled.

    APP

    Standard Edition users can no longer create roles, enterprise identity providers, or external token providers, and Dremio now provides tooltips to indicate that these are Enterprise features.

    APP

    Disabled the Delete All button under Account Settings > Personal Access Tokens if there are no tokens.

    APP

    Engine scaling events (adding or removing replicas) are now captured in the Dremio Cloud audit log.

    SQL

    We’ve made some changes to the way engine status is displayed in the SQL Runner to make job and engine state easier to understand.

    SQL

    Users can now expand (|→) or collapse (←|) the catalog explorer panel when working in the SQL Editor or SQL Runner.

    SQL

    Added the ability to push down DATE_ADD and DATE_SUB functions to RDBMS sources.

    SQL

    Added the ability to push down DATE_ADD and DATE_SUB functions to RDBMS sources.

    SQL

    The queue_name column has been removed from the jobs runtime system table (SYS.PROJECT.JOBS) because queues are not supported in Dremio Cloud.

    SQL

    In SYS.PROJECT.VIEWS and SYS.PROJECT."TABLES", if an owner of a view or table has been deleted, owner_id now shows up as $deleted$ instead of the invalid user ID.

    API

    The state of a project created using the /projects API will now return as either CREATING or ACTIVE.

    API

    The requestId attribute is now optional when creating a project with the /projects API.

    API

    The following ruleSet attributes have been renamed in the Engine Rules (/rules) API: ruleName -> name, ruleCondition -> condition, ruleAction -> action, outputValue -> engineName, PLACE -> ROUTE (action attribute). Additionally, the following attributes have been removed, outputId, ruleUniqueId, and version.

    Issues Fixed
    General Updates

    Fixed an issue that was causing errors when running the CloudFormation template during the automatic cloud connect portion of the signup process.

    General Updates

    During signup, user input in the Organization and Project fields was not getting validated, resulting in a generic error when trying to submit.

    General Updates

    If password authentication was disabled after a user had started the signup process from the invite link or by entering their email address on the login page, the user could still log in with a password.

    General Updates

    Fixed an issue that was redirecting users to the login screen when trying to remove an organization from the organization selection screen.

    General Updates

    If a user was active in one or more organizations, they were unable to sign up for a new organization.

    General Updates

    Continuing the sign up process on an incomplete organization was resulting in a CloudFormation template error.

    APP

    The Reflections tab was not showing up under Project Settings for local or PUBLIC users.

    APP

    Tooltips to provide details about engine states were not always displaying when expected.

    APP

    If a user’s login session had expired and they tried to visit a page in the app, an unexpected error was showing up instead of taking the user directly to the login page.

    APP

    The tooltip to indicate the status of an engine that was starting up was not displaying unless users hovered over the engine dropdown.

    APP

    When browsing the Dremio app with Edge, scrollbars were not showing up and users had to zoom out to view certain portions of the screen.

    APP

    Not all users were showing up when searching in the User filter on the Jobs page.

    APP

    The Upload File button on the Datasets page was available all the time, regardless of whether or not it had been disabled.

    APP

    The Tableau and PowerBI buttons were not showing up or remaining hidden as expected, and they are now enabled all the time in the SQL Editor.

    APP

    If password authentication was disabled after a user had started the signup process from the invite link or by entering their email address on the login page, the user could still log in with a password.

    APP

    When adding a tag to a dataset’s catalog, a Tags are not saved error was displayed even though the tag was still applied.

    APP

    After removing a view from a space, the total number of views in the space was not getting updated in the Spaces panel.

    APP

    In some cases, if a Parquet file in a Delta Lake table had many row groups, COUNT(*) queries were failing due to a divide by 0 exception.

    SQL

    Even though ownership of a folder or table had been granted to a role, users belonging to that role were unable to access the folder or table.

    SQL

    Ownership (owner_id in SYS.ORGANIZATION.USERS) of invited users was not being set to the ID of the user who invited them.

    SQL

    IN filters were not working after applying an internal schema on a varchar file.

    SQL

    Long paths to a dataset were running into the tab names above the SQL Editor.

    SQL

    Fixed an issue that was causing some reflections to fail with a NessieReferenceConflictException error.

    API

    The Billing transactions API was not filtering results properly for a billing account.

    February 15, 2022

    What's New
    General Updates

    Audit logging is now available in Dremio Cloud. See Audit Logging and SYS.PROJECT.HISTORY.EVENTS for more information.

    General Updates

    Email verification is now required during user sign up when using username/password authentication.

    General Updates

    If a user had not completed the onboarding process during sign up, the incomplete organization was showing up on the login page and there was no way to remove the incomplete organization. The option to remove such organizations is now available.

    General Updates

    We’ve simplified and streamlined our login flow by using cookies. If a user has accepted the use of all cookies, the organization selection and login screens will be skipped if the user still has an active login session. Additionally, the main login screen will be skipped if the user has logged in previously from the current browser (even if the login session has expired).

    APP

    Dremio Cloud Admins now have access to Billing, where they can change between Dremio Standard or Enterprise editions, create and modify a billing account, and view detailed transaction data for DCU usage.

    APP

    Apache Iceberg is now a supported format for tables.

    APP

    Copyright information is now displayed directly in the help menu, and the About Dremio option has been removed. Edition details can now be found in the Billing section under Organization Settings.

    APP

    Each OAuth application listed under Organization Settings now includes the application ID.

    APP

    IP addresses are now used instead of hostnames to track replicas, meaning that DNS hostnames no longer need to be enabled in the VPC. Additionally, the Engines detail page uses Host IP instead of Hostname and lists replica IP addresses.

    APP

    Our enhanced CloudFormation Template provides updated IAM policies with narrowed permission scopes to enhance customer security.

    APP

    The is_member condition for building Engine Rules was not working for internal roles.

    APP

    The support portal link on the Help menu is only available to Admin users in organizations using the Enterprise edition of Dremio Cloud.

    APP

    We’ve added new engine states to provide more granular details about engine status. For more information, see Managing Engines.

    APP

    Users with the PUBLIC role can now view the OAuth Application page under Organization Settings.

    SQL

    Admins can grant or revoke the CONFIGURE SECURITY privilege to a user or role. Users who have been granted this privilege (directly or via role) will have full access to Authentication, External Token Providers, BI Applications, and OAuth Applications under Organization Settings.

    SQL

    Runtime system tables for tables and views are now available in Dremio Cloud. See SYS.PROJECT.“TABLES” and SYS.PROJECT.VIEWS for more information.

    SQL

    Runtime and historical jobs tables are now accessible as sys.project.jobs and sys.project.history.jobs.

    API

    An engine is no longer created when adding a project with the /projects API. Users can create engines independent of projects.

    Issues Fixed
    APP

    Clicking the help link in the Add OAuth Application dialog was resulting in a 404/page not found error.

    APP

    Fixed a Job <job-id> output doesn’t exist error message to provide more information about why the job could not be retrieved and how the user can address the issue.

    APP

    Fixed an issue that was causing an unknown error when searching on the Engines page.

    APP

    Fixed an issue that was preventing users from disabling SSO for Power BI by simply clearing the Enable checkbox and saving — the Save button was not available in this scenario.

    APP

    If a non-admin user had ownership granted for a view, the view was not showing up for the user.

    APP

    We’ve fixed an issue that would occasionally result in lost connectivity to a source.

    APP

    When adding a new project, the Cloud used for the latest project is selected by default.

    APP

    When hovering over a specific entry in the Usage graph, Dremio now shows details that include the date and DCUs used on that day.

    APP

    When reordering an engine on the Engine Routing page, both an error and a success message were being displayed.

    APP

    When viewing usage details for a specific project, changing the usage period was taking users back to the All Projects view.

    API

    Users were not able to modify any engine attributes via the Engines API unless they specified all available attributes. All attributes are optional now.

    November 2021

    Issues Fixed
    APP

    The bug that prevented you from assigning the Administrator role to users has been resolved.

    APP

    The logos for social login options have been fixed to correctly match with the text.

    APP

    Errors messages for issues with granting and revoking privileges and not having priviliges to perform an action have been enhanced to be specific and clear.

    APP

    Enhanced error message to include parsing message when the engine rule being added is syntactically incorrect (there was an UNKNOWN error being returned previously).

    October 2021

    What's New
    General Updates

    Added support for the eu-west-2 (London) region.

    General Updates

    Fixed a deadlock with processing engine scaling events, which prevented engines from scaling up and shutting down.

    General Updates

    In addition to admin users, users with the OWNERSHIP or MANAGE GRANTS privileges on an object can now transfer ownership to a new user.

    Issues Fixed
    SQL

    Improved error messages when a user attempts to run ALTER USER with a PASSWORD clause (this action is not supported through SQL).

    APP

    You can now successfully add a user as an admin on the Administrators page.

    September 2021

    What's New
    General Updates

    EU Control Plane: We now support the eu-central-1 (Frankfurt) and eu-west-1 (Ireland) AWS regions for the control plane, allowing for better latency and performance with EU-based data planes. For additional information on EU endpoints, see the Supported Regions page.

    General Updates

    The PUBLIC role no longer has privileges to modify, disable, or delete engines.

    General Updates

    Added a Default Project setting, which tools like Tableau and Power BI default to so you don’t have to specify a project ID to connect to Dremio.

    API

    EU Control Plane: The API endpoint for the control plane is https://api.eu.dremio.cloud.

    API

    Personal access tokens are supported in the authorization header.

    APP

    EU Control Plane: The App endpoint for the control plane is https://app.eu.dremio.cloud.

    APP

    The login page of https://app.dremio.cloud displayed the deleted organizations. Now, the login page only displays the active organizations.

    Issues Fixed
    API

    When an engine was created with a duplicate name through a POST operation, a 500 Internal Server Error response code was returned. The error handling has been corrected to return a 400 Bad Request response code when an engine is created with a duplicate name.

    APP

    Previously deleted organizations were displayed on the login page. This has been fixed so that only active organizations are listed on the log in page.

    APP

    The Documentation link in the Help menu now points to https://docs-beta.dremio.com.

    APP

    On the SQL Editor page, the search tab in the Datasets pop out has been removed because it the search function doesn’t currently work. It will be added back when the search issues have been resolved.

    SQL

    Querying sys.privileges to retrieve the granted privileges did not return privileges on the current organization and the current project. Organization and project-level privileges and the associated organization and project are now returned correctly when querying sys.privileges.

    Client
    A fix has been implemented to prevent a new replica from launching when client applications that use JDBC went offline after the preview engine had been disabled/enabled. The periodic offline of JDBC sources was caused by the replica name being cached, but further updates were left untracked.
    Known Issues
    Known Issues
    A new external ID is generated every time a cloud entity is edited, which breaks the IAM role trust relationship. To prevent this issue, do not edit a cloud entity when you are using IAM role credentials. If you face this issue, temporarily switch to IAM access key credentials with the same permissions as the IAM role. A fix for this issue is expected in the next update.