Dremio 14.0.0 provides read-only support for the Delta Lake data format. The feature is disabled by default, To enable support for Delta Lake, set the
dremio.deltalake.enabled support key to
true. Dremio supports unlimited splits for Delta Lake tables.
Dremio support for Delta Lake has the following limitations:
When existing Parquet datasets are converted to Delta Lake tables using Spark without rewriting the Parquet files, the converted Delta Lake table does not have statistics about the table. Dremio estimates high row counts in this case, which can lead to queries failing due to lack of sufficient memory.
Dremio does not support the Delta Lake data format for Hive and AWS Glue data sources. However, Dremio users can promote a physical data set in the Delta Lake format.