Table

Use the Catalog API to retrieve tables, format files and folders as tables, update and refresh tables, and revert tables to files and folders.

Table Object

{
  "entityType": "dataset",
  "id": "c9c11d32-0576-4200-5a5b-8c7229cb3d72",
  "type": "PHYSICAL_DATASET",
  "path": [
    "Samples",
    "samples.dremio.com",
    "Dremio University",
    "restaurant_reviews.parquet"
  ],
  "createdAt": "2024-01-13T19:52:01.894Z",
  "tag": "cb2905bb-39c0-497f-ae74-4c310d534f25",
  "accelerationRefreshPolicy": {
    "activePolicyType": "SCHEDULE",
    "refreshPeriodMs": 3600000,
    "gracePeriodMs": 10800000,
    "refreshSchedule": "0 0 8 * * ?",
    "method": "FULL",
    "neverExpire": false,
    "neverRefresh": false,
    "sourceRefreshOnDataChanges": "false"
  },
  "isMetadataExpired": false, 
  "lastMetadataRefreshAt": "2024-01-31T09:50:01.012Z", 
  "format": {
    "type": "Parquet",
    "name": "restaurant_reviews.parquet",
    "fullPath": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "restaurant_reviews.parquet"
    ],
    "ctime": 0,
    "isFolder": false,
    "location": "/samples.dremio.com/Dremio University/restaurant_reviews.parquet",
    "ignoreOtherFileFormats": "false",
    "autoCorrectCorruptDates": true
  },
  "accessControlList": {
    "users": [
      {
        "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      },
      {
        "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
        "permissions": [
          "SELECT",
          "ALTER",
          "MANAGE_GRANTS"
        ]
      }
    ],
    "roles": [
      {
        "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      }
    ]
  },
  "owner": {
    "ownerId": "30fca499-4abc-4469-7142-fc8dd29acac8",
    "ownerType": "USER"
  },
  "fields": [
    {
      "name": "_id",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "name",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "city",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "state",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "categories",
      "type": {
        "name": "LIST",
        "subSchema": [
          {
            "type": {
              "name": "VARCHAR"
            }
          }
        ]
      }
    },
    {
      "name": "review_count",
      "type": {
        "name": "BIGINT"
      }
    },
    {
      "name": "stars",
      "type": {
        "name": "DOUBLE"
      }
    },
    {
      "name": "attributes",
      "type": {
        "name": "STRUCT",
        "subSchema": [
          {
            "name": "Parking",
            "type": {
              "name": "STRUCT",
              "subSchema": [
                {
                  "name": "garage",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "street",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "lot",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "valet",
                  "type": {
                    "name": "BOOLEAN"
                  }
                }
              ]
            }
          },
          {
            "name": "Accepts Credit Cards",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Wheelchair Accessible",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Price Range",
            "type": {
              "name": "BIGINT"
            }
          }
        ]
      }
    },
    {
      "name": "date",
      "type": {
        "name": "VARCHAR"
      }
    }
  ],
  "approximateStatisticsAllowed": false
}

Table Attributes

entityType String

Type of catalog entity. For tables, the entityType is dataset.

Example: dataset

id String

Unique identifier for the table. For tables in non-Arctic sources, the id may be a UUID like c9c11d32-0576-4200-5a5b-8c7229cb3d72 or a text path. For tables in Arctic sources and the primary Arctic catalog, the id is a JSON string like {"tableKey":["myArcticSource","myFolder","myTable"],"contentId":"65b24a53-c56e-4037-8ffc-f25297614c3c","versionContext":\{"type":"BRANCH","value":"main"}}.

Example: c9c11d32-0576-4200-5a5b-8c7229cb3d72

type String

Type of dataset. For tables, the type is PHYSICAL_DATASET.

Example: PHYSICAL_DATASET

path Array of String

Path to the table within Dremio, expressed in an array. The path lists each level of hierarchy in order, from outer to inner: source first, then folder and subfolders, then the table itself as the last item in the array.

Example: ["Samples", "samples.dremio.com", "Dremio University", "restaurant_reviews.parquet"]

createdAt String

Date and time that the table was created. In UTC format.

Example: 2024-01-13T19:52:01.894Z

tag String (UUID)

Unique identifier for the table instance. Dremio changes the tag whenever the table changes and uses the tag value to ensure that PUT requests apply to the most recent version of the table.

Example: cb2905bb-39c0-497f-ae74-4c310d534f25

accelerationRefreshPolicy Object

Attributes that define the acceleration refresh policy for the table.

isMetadataExpired Boolean

If true, the metadata of the table needs to be refreshed. To refresh it, run the ALTER TABLE command, using the clause REFRESH METADATA. See ALTER TABLE.
If false, the metadata can still be used for planning queries against the table.
If NULL, metadata has never yet been collected for the table.

lastMetadataRefreshAt String

Date and time that the table metadata was last refreshed. In UTC format. If NULL, the metadata has never yet been refreshed.

Example: 2024-01-31T09:50:01.012Z

format Object

Table format attributes.

accessControlList Object

Information about users and roles with privileges on the table and the specific privileges each user or role has. May include a users array, a roles array, or both, depending on the configured access and privileges. The accessControlList object is empty if table-specific privileges are not set. Read Inheritance, Scope, and Ownership for more information. For tables in Arctic sources and the primary Arctic catalog, the table object does not include the accessControlList object.

owner Object

Information about the table's owner. For tables in Arctic sources and the primary Arctic catalog, the table object does not include the owner object.

fields Array of Object

Attributes that represent the table schema.

approximateStatisticsAllowed Boolean

If true, COUNT DISTINCT queries run on the table will return approximate results. Otherwise, false.

Example: false

Attributes of the `accelerationRefreshPolicy` Object

activePolicyType String

Option to set the policy for refreshing reflections that are defined on the source. For this option to take effect, the neverRefresh parameter must be set to false.

The possible values are:

NEVER: The reflections are never refreshed.
PERIOD: Default. The reflections are refreshed at the end of every period that is defined by refreshPeriodMs.
SCHEDULE: The reflections are refreshed according to the schedule that is set by refreshSchedule.
REFRESH_ON_DATA_CHANGES: Reflections automatically refresh for underlying tables that are in Iceberg format when new snapshots are created after an update. If the reflection refresh job finds no changes, then no data is updated. Reflections that are automatically updated based on Iceberg source table changes also update according to the source-level policy as the minimum refresh frequency.

refreshPeriodMs Integer

Refresh period for the data in all reflections for the table. In milliseconds.

Example: 3600000

refreshSchedule String

A cron expression that sets the schedule, in UTC time, according to which the reflections that are defined on the source are refreshed.

Field	Allowed Values	Allowed Special Characters
Second	0	N/A
Minute	0-59	N/A
Hour	0-23	N/A
Day of month	N/A	* ?
Month	N/A	* ?
Days of week	1-7 or SUN-SAT	, - * ?

Special Character	Description
*	Used to specify all values for a field. For `Day of month`, specifies every day of the month. For `Month`, specifies every month. For `Days of week`, specifies every day of the week.
?	Equivalent to *.
,	Used to specify two or more days in the `Days of week` field. For example, `MON,WED,FRI`.
-	Used to specify ranges in the `Days of week` field. For example, `1-3` is equivalent to `Sunday, Monday, and Tuesday`.

Examples:

0 0 0 * * ? : Refreshes every day at midnight.
0 45 15 * * 1,4,7 : Refreshes at 15:45 on Sunday, Wednesday, and Saturday.
0 15 7 ? * 2-6 : Refreshes at 7:15 on Monday and Friday.

gracePeriodMs Integer

Maximum age allowed for reflection data used to accelerate queries. In milliseconds.

Example: 10800000

method String

Method used for refreshing the data in reflections. For tables that are in the Apache Iceberg format; Parquet datasets in filesystems; or Parquet datasets, Avro datasets, or non-transactional ORC datasets in AWS Glue, the value is AUTO. In this case, the method used depends on this algorithm:

The initial refresh of a reflection is always a full refresh.
If the reflection is created from a view that uses nested group-bys, joins, unions, or window functions, then a full refresh is performed.
If the changes to the base table are only appends, then an incremental refresh based on table snapshots is performed.
If the changes to the base table include non-append operations, then a partition-based incremental refresh is attempted.
If the partitions of the base table and the partitions of the reflection are not compatible, or if either the base table or the reflection is not partitioned, then a full refresh is performed.

Read Refreshing Reflections for more information.

Enum: AUTO, FULL, INCREMENTAL

Example: FULL

refreshField String

For the INCREMENTAL refresh method, the field to refresh for the table. Used only if method is INCREMENTAL. This parameter applies only to tables that are not in the Apache Iceberg format.

Example: business_id

neverExpire Boolean

If the reflection will never expire, true. Otherwise, false.

Example: false

neverRefresh Boolean

If the reflection will never refresh, true. Otherwise, false.

Example: false

sourceRefreshOnDataChanges Boolean

If the table's source is configured so that reflections on tables in Iceberg format in the source will refresh when new snapshots are created after an update, true. Otherwise, false.

Attributes of the `format` Object

type String

Type of data in the table.

Enum: Delta, Excel, Iceberg, JSON, Parquet, Text, Unknown, XLS

Example: Parquet

name String

Table name. Dremio automatically duplicates the name of the origin file or folder to populate this value. The name of the origin file or folder cannot include the following special characters: /, :, [, or ].

Example: restaurant_reviews.parquet

fullPath Array of String

Example: ["Samples", "samples.dremio.com", "Dremio University", "restaurant_reviews.parquet"]

ctime Integer

Not used. Has the value 0.

Example: 0

isFolder Boolean

If true, the table was created from a folder. If false, the table was created from a file.

Example: false

location String

Location where the table's metadata is stored within a Dremio source, expressed in a string.

Example: /samples.dremio.com/Dremio University/restaurant_reviews.parquet

ignoreOtherFileFormats Boolean

If true, Dremio ignores all non-Parquet files in the related folder structure, and the promoted table works as if only Parquet files are in the folder structure. Otherwise, false. Included only for Parquet folders.

Example: false

metaStoreType String

Not used. Has the value HDFS.

Example: HDFS

parquetDataFormat Object

Information about data format for Parquet tables.

dataFormatTypeList Array of String

List of data format types in the table. Included only for Iceberg tables, and PARQUET is the only valid value.

Example: ["PARQUET"]

sheetName String

For tables created from files with multiple sheets, name of the sheet used to create the table.

Example: location_1

extractHeader Boolean

For tables created from files, true if Dremio extracted the table's column names from the first line of the file. Otherwise, false.

Example: false

hasMergedCells Boolean

For tables created from files, true if Dremio expanded merged cells in the file when creating the table. Otherwise, false.

Example: true

fieldDelimiter String

Character used to indicate separate fields in the table. May be , for a comma (default), \t for a tab, | for a pipe, or a custom character.

quote String

Character used for quotes in the table. May be \" for a double quote (default), ' for a single quote, or a custom character.

comment String

Character used to indicate comments in the table. May be # for a number sign (default) or a custom character.

escape String

Character used to indicate an escape in the table. May be \" for a double quote (default), ` for a back quote, \\ for a backward slash, or a custom character.

lineDelimiter String

Character used to indicate separate lines in the table. May be \r\n for a carriage return plus new line (default), \n for a new line, or a custom character.

skipFirstLine Boolean

If Dremio skipped the first line in the file or folder when creating the table, true. Otherwise, false.

Example: false

autoGenerateColumnNames Boolean

If Dremio used the existing columnn names in the file or folder for the table columns, true. Otherwise, false.

Example: true

trimHeader Boolean

If Dremio trimmed column names to a specific number of characters when creating the table, true. Otherwise, false.

Example: true

autoCorrectCorruptDates Boolean

If Dremio Cloud automatically corrects corrupted date fields in the table, true. Otherwise, false.

Example: true

Attributes of the `accessControlList` Object

users Array of Object

Information about the users that have been granted privileges on the table and the privileges each user has.

Example: [{"id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3","permissions": ["SELECT","ALTER"]},{"id": "30fca499-4abc-4469-7142-fc8dd29acac8","permissions": ["SELECT","ALTER","MANAGE_GRANTS"]}]

roles Array of Object

Information about the roles that have been granted privileges on the table and the privileges each role has.

Example: [{"id": "76a9884b-aea5-46d5-a73a-000edf23f390","permissions": ["SELECT","ALTER"]}]

Attributes of Objects in the `users` and `roles` Arrays

id String

The ID of the role or user.

Example: c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3

permissions Array of String

The privileges that the role or user has on the table. Read Table Privileges.

Example: ["SELECT", "ALTER"]

Attributes of the `owner` Object

ownerId String (UUID)

Unique identifier for the table's owner.

Example: 30fca499-4abc-4469-7142-fc8dd29acac8

ownerType String

Type of owner for the table.

Enum: USER, ROLE

Example: USER

Attributes of Objects in the `fields` Array

name String

Name of the table field.

Example: review_count

type Object

Information about the table field.

Attributes of the `type` Object

name String

Name of the table field's type.

Enum: STRUCT, LIST, UNION, INTEGER, BIGINT, FLOAT, DOUBLE, VARCHAR, VARBINARY, BOOLEAN, DECIMAL, TIME, DATE, TIMESTAMP, INTERVAL DAY TO SECOND, INTERVAL YEAR TO MONTH

Example: BIGINT

precision Integer

Total number of digits in the number. Included only for the DECIMAL type.

Example: 38

scale Integer

Number of digits to the right of the decimal point. Included only for the DECIMAL type.

Example: 0

subSchema Array of Object

List of objects that represent the field's composition. For example, a field composed of data about a restaurant might have a subSchema with an object for parking options, another for payment methods, and so on. subSchemas may be nested within other subSchemas. subSchema appears only for the STRUCT, LIST, and UNION types.

Attributes of Objects in the `subSchema` Array

name String

Name for the subSchema object.

Example: Parking

type Object

Object that contains a name attribute that provides the field's type.

Example: {"name": "BOOLEAN"}

Attributes of the `parquetDataFormat` Object

type String

Type of data in the table. Within the parquetDataFormat object, the only valid type is Parquet.

Example: Parquet

ctime Integer

Not used. Has the value 0.

Example: 0

isFolder Boolean

If true, the table was created from a folder. If false, the table was created from a file.

Example: true

autoCorrectCorruptDates Boolean

If true, Dremio Cloud automatically corrects corrupted date fields in the table. Otherwise, false.

Example: true

Formatting a File or Folder as a Table

Format a file or folder as a table so that you can query the data in Dremio.

note

To format a folder, all files in the folder must be the same format.

Method and URL

POST /v0/projects/{project-id}/catalog/{id}

Parameters

project-id Path String (UUID)

Unique identifier for the project that contains the file or folder you want to format.

Example: 1df71752-69b7-47d9-9e6c-990e6b194aa4

id Path String

Unique identifier for the file or folder you want to format. For files and folders in non-Arctic sources, the id may be a UUID or a text path. If the id is a text path, use URL-encoded format to replace special characters with their UTF-8-equivalent characters: %3A for a colon; %2F for a forward slash, and %20 for a space. For example, if the id value is dremio:/Samples/samples.dremio.com/Dremio University, the encoded id is dremio%3A%2FSamples%2Fsamples.dremio.com%2FDremio%20University. For files and folders in Arctic sources, the id is a JSON string like {"tableKey":["myArcticSource","myFolder"],"contentId":"65b24a53-c56e-4037-8ffc-f25297614c3c","versionContext":{"type":"BRANCH","value":"main"}}.

Example: c590ed7f-7142-4e1f-ba7d-94173afdc9a3

entityType Body String

Type of catalog entity. To format a file or folder as a table, the entityType is dataset.

path Body Array of String

Path to the file or folder you want to format, expressed in an array. List each level of hierarchy in order, from outer to inner: source first, then any folder and subfolders, then the file or folder itself as the last item in the array. Get the path from the file or folder's children object in the response to a Folder request.

Example: ["Samples", "samples.dremio.com", "Dremio University", "restaurant_reviews.parquet"]

type Body String

Type of dataset. For tables, the type is PHYSICAL_DATASET.

accelerationRefreshPolicy Object

Attributes that define the acceleration refresh policy for the table.

format Body String

Formatting parameters for the file or folder.

Parameters of the `accelerationRefreshPolicy` Object

activePolicyType Body String

Policy to use for refreshing reflections that are defined on the source. For this option to take effect, the neverRefresh parameter must be set to false.

The possible values are:

NEVER: The reflections are never refreshed.
PERIOD: Default. The reflections are refreshed at the end of every period that is defined by refreshPeriodMs.
SCHEDULE: The reflections are refreshed according to the schedule that is set by refreshSchedule.
REFRESH_ON_DATA_CHANGES: Reflections automatically refresh for underlying tables that are in Iceberg format when new snapshots are created after an update. If the reflection refresh job finds no changes, then no data is updated. Reflections that are automatically updated based on Iceberg source table changes also update according to the source-level policy as the minimum refresh frequency. Only available for tables in Iceberg format.

refreshPeriodMs Body Integer

Refresh period to use for the data in all reflections for the table. In milliseconds. Optional if you set activePolicyType to PERIOD. The default setting is 3600000 milliseconds or one hour, which is also the minimum amount of time that is supported.

Example: 3600000

refreshSchedule Body String

A cron expression that sets the schedule, in UTC time, according to which the reflections that are defined on the source should be refreshed. Optional if you set activePolicyType to SCHEDULE. The default refreshSchedule setting is to refresh every day at 8:00 a.m.

Field	Allowed Values	Allowed Special Characters
Second	0	N/A
Minute	0-59	N/A
Hour	0-23	N/A
Day of month	N/A	* ?
Month	N/A	* ?
Days of week	1-7 or SUN-SAT	, - * ?

Special Character	Description
*	Used to specify all values for a field. For `Day of month`, specifies every day of the month. For `Month`, specifies every month. For `Days of week`, specifies every day of the week.
?	Equivalent to *.
,	Used to specify two or more days in the `Days of week` field. For example, `MON,WED,FRI`.
-	Used to specify ranges in the `Days of week` field. For example, `1-3` is equivalent to `Sunday, Monday, and Tuesday`.

Examples:

0 0 0 * * ? : Refreshes every day at midnight.
0 45 15 * * 1,4,7 : Refreshes at 15:45 on Sunday, Wednesday, and Saturday.
0 15 7 ? * 2-6 : Refreshes at 7:15 on Monday and Friday.

gracePeriodMs Body Integer

Maximum age to allow for reflection data used to accelerate queries. In milliseconds.

Example: 10800000

method Body String

Method to use for refreshing the data in reflections. For tables that are in the Apache Iceberg format; Parquet datasets in filesystems; or Parquet datasets, Avro datasets, or non-transactional ORC datasets in AWS Glue, the value is AUTO. In this case, the method used depends on this algorithm:

The initial refresh of a reflection is always a full refresh.
If the reflection is created from a view that uses nested group-bys, joins, unions, or window functions, then a full refresh is performed.
If the changes to the base table are only appends, then an incremental refresh based on table snapshots is performed.
If the changes to the base table include non-append operations, then a partition-based incremental refresh is attempted.
If the partitions of the base table and the partitions of the reflection are not compatible, or if either the base table or the reflection is not partitioned, then a full refresh is performed.

Read Refreshing Reflections for more information.

Enum: AUTO, FULL, INCREMENTAL

Example: FULL

refreshField Body String

For the INCREMENTAL refresh method, the field to refresh for the table. Used only if the method is INCREMENTAL. This parameter applies only to tables that are not in the Apache Iceberg format.

Example: business_id

neverExpire Body Boolean

If the reflection should never expire, true. Otherwise, false.

Example: false

neverRefresh Body Boolean

If the reflection should never refresh, true. Otherwise, false.

Example: false

Parameters of the `format` Object

type Body String

Type of data in the file or folder. To format a folder, all files in the folder must be the same format.

Enum: Delta, Excel, Iceberg, JSON, Parquet, Text, Unknown, XLS

ignoreOtherFileFormats Body Boolean Optional

If Dremio should ignore all non-Parquet files in the related folder structure so that the promoted table works as if only Parquet files are in the folder structure, set to true. Otherwise, set to false (default). Optional for Parquet folders.

Example: false

skipFirstLine Body Boolean Optional

If Dremio should skip the first line in the file or folder when creating the table, true. Otherwise, false (default). Optional for Excel and Text types.

Example: true

extractHeader Body Boolean Optional

If Dremio should extract the table's column names from the first line of the file, true. Otherwise, false (default). Optional for Excel and Text types.

Example: true

hasMergedCells Body Boolean Optional

If Dremio should expand merged cells in the file when creating the table, true. Otherwise, false (default). Optional for Excel types.

Example: true

sheetName Body String Optional

For tables created from Excel files with multiple sheets, name of the sheet to use to create the table. Default is the first sheet in the file (for files with multiple sheets).

Example: location_1

fieldDelimiter Body String Optional

Character to use to indicate separate fields in the table. May be , for a comma (default), \t for a tab, | for a pipe, or a custom character. Optional for Text type.

quote Body String Optional

Character to use for quotes in the table. May be \" for a double quote (default), ' for a single quote, or a custom character. Optional for Text type.

comment Body String Optional

Character to use to indicate comments in the table. May be # for a number sign (default) or a custom character. Optional for Text type.

escape Body String Optional

Character used to indicate an escape in the table. May be " for a double quote (default), ` for a back quote, \ for a backward slash, or a custom character. Optional for Text type.

lineDelimiter Body String Optional

Character used to indicate separate lines in the table. May be \r\n for a carriage return plus new line (default), \n for a new line, or a custom character. Optional for Text type.

autoGenerateColumnNames Body Boolean Optional

If Dremio should use the existing columnn names in the file or folder for the table columns, true (default). Otherwise, false. Optional for Text type.

Example: true

trimHeader Body Boolean Optional

If Dremio should trim column names to a specific number of characters when creating the table, true. Otherwise, false (default). Optional for Text type.

Example: true

Example Request

curl -X POST 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/c9c11d32-0576-4200-5a5b-8c7229cb3d72' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "entityType": "dataset",
  "path": [
    "Samples",
    "Dremio University",
    "restaurant_reviews.parquet"
  ],
  "type": "PHYSICAL_DATASET",
  "accelerationRefreshPolicy": {
    "activePolicyType": "PERIOD",
    "refreshPeriodMs": 3600000,
    "refreshSchedule": "0 56 18 * * *",
    "gracePeriodMs": 259200000,
    "method": "AUTO",
    "neverExpire": true,
    "neverRefresh": false,
    "sourceRefreshOnDataChanges": false
  },
  "format": {
    "type": "Parquet"
  },
  "accessControlList": {
    "users": [
      {
        "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      },
      {
        "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
        "permissions": [
          "SELECT",
          "ALTER",
          "MANAGE_GRANTS"
        ]
      }
    ],
    "roles": [
      {
        "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      }
    ]
  }
}'

Example Response

{
  "entityType": "dataset",
  "id": "c9c11d32-0576-4200-5a5b-8c7229cb3d72",
  "type": "PHYSICAL_DATASET",
  "path": [
    "Samples",
    "samples.dremio.com",
    "Dremio University",
    "restaurant_reviews.parquet"
  ],
  "createdAt": "2024-01-13T19:52:01.894Z",
  "tag": "cb2905bb-39c0-497f-ae74-4c310d534f25",
  "isMetadataExpired": false,
  "lastMetadataRefreshAt": "2024-01-31T09:50:01.012Z",
  "accelerationRefreshPolicy": {
    "activePolicyType": "PERIOD",
    "refreshPeriodMs": 3600000,
    "refreshSchedule": "0 56 18 * * *",
    "gracePeriodMs": 259200000,
    "method": "AUTO",
    "neverExpire": true,
    "neverRefresh": false,
    "sourceRefreshOnDataChanges": false
  },
  "format": {
    "type": "Parquet",
    "name": "restaurant_reviews.parquet",
    "fullPath": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "restaurant_reviews.parquet"
    ],
    "ctime": 0,
    "isFolder": false,
    "location": "/samples.dremio.com/Dremio University/restaurant_reviews.parquet",
    "ignoreOtherFileFormats": false,
    "autoCorrectCorruptDates": true
  },
  "accessControlList": {
    "users": [
      {
        "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      },
      {
        "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
        "permissions": [
          "SELECT",
          "ALTER",
          "MANAGE_GRANTS"
        ]
      }
    ],
    "roles": [
      {
        "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      }
    ]
  },
  "owner": {
    "ownerId": "30fca499-4abc-4469-7142-fc8dd29acac8",
    "ownerType": "USER"
  },
  "fields": [
    {
      "name": "_id",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "name",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "city",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "state",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "categories",
      "type": {
        "name": "LIST",
        "subSchema": [
          {
            "type": {
              "name": "VARCHAR"
            }
          }
        ]
      }
    },
    {
      "name": "review_count",
      "type": {
        "name": "BIGINT"
      }
    },
    {
      "name": "stars",
      "type": {
        "name": "DOUBLE"
      }
    },
    {
      "name": "attributes",
      "type": {
        "name": "STRUCT",
        "subSchema": [
          {
            "name": "Parking",
            "type": {
              "name": "STRUCT",
              "subSchema": [
                {
                  "name": "garage",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "street",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "lot",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "valet",
                  "type": {
                    "name": "BOOLEAN"
                  }
                }
              ]
            }
          },
          {
            "name": "Accepts Credit Cards",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Wheelchair Accessible",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Price Range",
            "type": {
              "name": "BIGINT"
            }
          }
        ]
      }
    },
    {
      "name": "date",
      "type": {
        "name": "VARCHAR"
      }
    }
  ],
  "approximateStatisticsAllowed": false
}

Example Request for Excel format type

curl -X POST 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/dremio%3A%2FSamples%2Fsamples.dremio.com%2FDremio%20University%2Foracle-departments.xlsx' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "entityType": "dataset",
  "path": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "oracle-departments.xlsx"
    ],
  "type": "PHYSICAL_DATASET",
  "format": {
    "type": "Excel",
    "extractHeader": true,
    "hasMergedCells": true,
    "sheetName": "Sheet1"
    }
}'

Example Request for Text format type

curl -X POST 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/6ba3bd6e-fd27-4572-a535-77e1548283b3' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "entityType": "dataset",
  "path": [
    "Samples",
    "samples.dremio.com",
    "Dremio University",
    "airbnb_listings.csv"
  ],
  "type": "PHYSICAL_DATASET",
  "format": {
    "type": "Text",
    "fieldDelimiter": ",",
    "skipFirstLine": false,
    "extractHeader": true,
    "quote": "\"",
    "comment": "#",
    "escape": "\"",
    "lineDelimiter": "\r\n",
    "autoGenerateColumnNames": true,
    "trimHeader": false
  }
}'

Response Status Codes

200 OK

400 Bad Request

401 Unauthorized

403 Forbidden

404 Not Found

500 Internal Server Error

Retrieving a Table by ID

Retrieve a table by specifying the table's id value.

Method and URL

GET /v0/projects/{project-id}/catalog/{id}

Parameters

project-id Path String (UUID)

Unique identifier for the project that contains the table you want to retrieve.

Example: 1df71752-69b7-47d9-9e6c-990e6b194aa4

id Path String

Unique identifier for the table you want to retrieve. For tables in non-Arctic sources, the id may be a UUID or a text path. If the id is a text path, use URL-encoded format to replace special characters with their UTF-8-equivalent characters: %3A for a colon; %2F for a forward slash, and %20 for a space. For example, if the id value is dremio:/Samples/samples.dremio.com/Dremio University, the encoded id is dremio%3A%2FSamples%2Fsamples.dremio.com%2FDremio%20University. For tables in Arctic sources and the primary Arctic catalog, the id is a JSON string like {"tableKey":["myArcticSource","myFolder","myTable"],"contentId":"65b24a53-c56e-4037-8ffc-f25297614c3c","versionContext":{"type":"BRANCH","value":"main"}}.

Example: c9c11d32-0576-4200-5a5b-8c7229cb3d72

Example Request

curl -X GET 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/c9c11d32-0576-4200-5a5b-8c7229cb3d72' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json'

Example Response

{
  "entityType": "dataset",
  "id": "c9c11d32-0576-4200-5a5b-8c7229cb3d72",
  "type": "PHYSICAL_DATASET",
  "path": [
    "Samples",
    "samples.dremio.com",
    "Dremio University",
    "restaurant_reviews.parquet"
  ],
  "createdAt": "2024-01-13T19:52:01.894Z",
  "tag": "cb2905bb-39c0-497f-ae74-4c310d534f25",
  "isMetadataExpired": false,
  "lastMetadataRefreshAt": "2024-01-31T09:50:01.012Z",
  "accelerationRefreshPolicy": {
    "activePolicyType": "PERIOD",
    "refreshPeriodMs": 3600000,
    "refreshSchedule": "0 56 18 * * *",
    "gracePeriodMs": 259200000,
    "method": "AUTO",
    "neverExpire": true,
    "neverRefresh": false,
    "sourceRefreshOnDataChanges": false
  },
  "format": {
    "type": "Parquet",
    "name": "restaurant_reviews.parquet",
    "fullPath": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "restaurant_reviews.parquet"
    ],
    "ctime": 0,
    "isFolder": false,
    "location": "/samples.dremio.com/Dremio University/restaurant_reviews.parquet",
    "ignoreOtherFileFormats": false,
    "autoCorrectCorruptDates": true
  },
  "accessControlList": {
    "users": [
      {
        "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      },
      {
        "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
        "permissions": [
          "SELECT",
          "ALTER",
          "MANAGE_GRANTS"
        ]
      }
    ],
    "roles": [
      {
        "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      }
    ]
  },
  "owner": {
    "ownerId": "30fca499-4abc-4469-7142-fc8dd29acac8",
    "ownerType": "USER"
  },
  "fields": [
    {
      "name": "_id",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "name",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "city",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "state",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "categories",
      "type": {
        "name": "LIST",
        "subSchema": [
          {
            "type": {
              "name": "VARCHAR"
            }
          }
        ]
      }
    },
    {
      "name": "review_count",
      "type": {
        "name": "BIGINT"
      }
    },
    {
      "name": "stars",
      "type": {
        "name": "DOUBLE"
      }
    },
    {
      "name": "attributes",
      "type": {
        "name": "STRUCT",
        "subSchema": [
          {
            "name": "Parking",
            "type": {
              "name": "STRUCT",
              "subSchema": [
                {
                  "name": "garage",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "street",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "lot",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "valet",
                  "type": {
                    "name": "BOOLEAN"
                  }
                }
              ]
            }
          },
          {
            "name": "Accepts Credit Cards",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Wheelchair Accessible",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Price Range",
            "type": {
              "name": "BIGINT"
            }
          }
        ]
      }
    },
    {
      "name": "date",
      "type": {
        "name": "VARCHAR"
      }
    }
  ],
  "approximateStatisticsAllowed": false
}

Response Status Codes

200 OK

400 Bad Request

401 Unauthorized

403 Forbidden

404 Not Found

Retrieving a Table by Path

Retrieve a table by specifying the table's path.

Method and URL

GET /v0/projects/{project-id}/catalog/by-path/{path}

Parameters

project-id Path String (UUID)

Unique identifier for the project that contains the table you want to retrieve.

Example: 1df71752-69b7-47d9-9e6c-990e6b194aa4

path Path String

Table's location within Dremio, using forward slashes as separators. For example, for the "NYC-taxi-trips" table in the "samples.dremio.com" folder within the source "Samples," the path is Samples/samples.dremio.com/NYC-taxi-trips. If the name of any component in the path includes special characters for URLs, such as spaces, use URL encoding to replace the special characters with their UTF-8-equivalent characters. For example, "Dremio University" should be Dremio%20University in the URL path.

Example: Samples/samples.dremio.com/Dremio%20University/restaurant_reviews.parquet

Example Request

curl -X GET 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/by-path/Samples/samples.dremio.com/Dremio%20University/restaurant_reviews.parquet' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json'

Example Response

{
  "entityType": "dataset",
  "id": "c9c11d32-0576-4200-5a5b-8c7229cb3d72",
  "type": "PHYSICAL_DATASET",
  "path": [
    "Samples",
    "samples.dremio.com",
    "Dremio University",
    "restaurant_reviews.parquet"
  ],
  "createdAt": "2024-01-13T19:52:01.894Z",
  "tag": "cb2905bb-39c0-497f-ae74-4c310d534f25",
  "isMetadataExpired": false,
  "lastMetadataRefreshAt": "2024-01-31T09:50:01.012Z",
  "accelerationRefreshPolicy": {
    "activePolicyType": "PERIOD",
    "refreshPeriodMs": 3600000,
    "refreshSchedule": "0 56 18 * * *",
    "gracePeriodMs": 259200000,
    "method": "AUTO",
    "neverExpire": true,
    "neverRefresh": false,
    "sourceRefreshOnDataChanges": false
  },
  "format": {
    "type": "Parquet",
    "name": "restaurant_reviews.parquet",
    "fullPath": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "restaurant_reviews.parquet"
    ],
    "ctime": 0,
    "isFolder": false,
    "location": "/samples.dremio.com/Dremio University/restaurant_reviews.parquet",
    "ignoreOtherFileFormats": false,
    "autoCorrectCorruptDates": true
  },
  "accessControlList": {
    "users": [
      {
        "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      },
      {
        "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
        "permissions": [
          "SELECT",
          "ALTER",
          "MANAGE_GRANTS"
        ]
      }
    ],
    "roles": [
      {
        "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      }
    ]
  },
  "owner": {
    "ownerId": "30fca499-4abc-4469-7142-fc8dd29acac8",
    "ownerType": "USER"
  },
  "fields": [
    {
      "name": "_id",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "name",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "city",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "state",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "categories",
      "type": {
        "name": "LIST",
        "subSchema": [
          {
            "type": {
              "name": "VARCHAR"
            }
          }
        ]
      }
    },
    {
      "name": "review_count",
      "type": {
        "name": "BIGINT"
      }
    },
    {
      "name": "stars",
      "type": {
        "name": "DOUBLE"
      }
    },
    {
      "name": "attributes",
      "type": {
        "name": "STRUCT",
        "subSchema": [
          {
            "name": "Parking",
            "type": {
              "name": "STRUCT",
              "subSchema": [
                {
                  "name": "garage",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "street",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "lot",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "valet",
                  "type": {
                    "name": "BOOLEAN"
                  }
                }
              ]
            }
          },
          {
            "name": "Accepts Credit Cards",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Wheelchair Accessible",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Price Range",
            "type": {
              "name": "BIGINT"
            }
          }
        ]
      }
    },
    {
      "name": "date",
      "type": {
        "name": "VARCHAR"
      }
    }
  ],
  "approximateStatisticsAllowed": false
}

Response Status Codes

200 OK

400 Bad Request

401 Unauthorized

403 Forbidden

404 Not Found

Updating a Table

Update the specified table in Dremio.

Method and URL

PUT /v0/projects/{project-id}/catalog/{id}

Parameters

project-id Path String (UUID)

Unique identifier for the project that contains the table you want to update.

Example: 1df71752-69b7-47d9-9e6c-990e6b194aa4

id Path String

Unique identifier for the table you want to update. For tables that are not in an Arctic source, the id is a UUID like c9c11d32-0576-4200-5a5b-8c7229cb3d72. For tables in Arctic sources and the primary Arctic catalog, the id is a JSON string like {"tableKey":["myArcticSource","myFolder","myTable"],"contentId":"65b24a53-c56e-4037-8ffc-f25297614c3c","versionContext":{"type":"BRANCH","value":"main"}}.

Example: c9c11d32-0576-4200-5a5b-8c7229cb3d72

entityType Body String

Type of catalog entity. For tables, the entityType is dataset.

id Body String

Example: c9c11d32-0576-4200-5a5b-8c7229cb3d72

path Body Array of String

Path to the table you want to update, expressed in an array. List each level of hierarchy in order, from outer to inner: source first, then any folder and subfolders, then the table itself as the last item in the array. Get the path from the table's children object in the response to a Folder.

Example: ["Samples", "samples.dremio.com", "Dremio University", "restaurant_reviews.parquet"]

tag Body String (UUID) Optional

Unique identifier for the version of the table you want to update. If you provide a tag in the request body, Dremio uses the tag value to ensure that you are requesting to update the most recent version of the table. If you do not provide a tag, Dremio automatically updates the most recent version of the table.

Example: cb2905bb-39c0-497f-ae74-4c310d534f25

type Body String

Type of dataset. For tables, the type is PHYSICAL_DATASET.

accelerationRefreshPolicy Object

Attributes that define the acceleration refresh policy for the table.

format Body String

Formatting parameters for the table.

accessControlList Body Object Optional

Object used to specify which users and roles should have privileges on the table and the specific privileges each user and role has. May include a users array, a roles array, or both. Omit if you want the table to inherit access and privileges. Not supported for tables in Arctic sources and the primary Arctic catalog. Read Inheritance, Scope, and Ownership for more information.

Parameters of the `accelerationRefreshPolicy` Object

activePolicyType Body String

Policy to use for refreshing reflections that are defined on the source. For this option to take effect, the neverRefresh parameter must be set to false.

The possible values are:

NEVER: The reflections are never refreshed.
PERIOD: Default. The reflections are refreshed at the end of every period that is defined by refreshPeriodMs.
SCHEDULE: The reflections are refreshed according to the schedule that is set by refreshSchedule.
REFRESH_ON_DATA_CHANGES: Reflections automatically refresh for underlying tables that are in Iceberg format when new snapshots are created after an update. If the reflection refresh job finds no changes, then no data is updated. Reflections that are automatically updated based on Iceberg source table changes also update according to the source-level policy as the minimum refresh frequency. Only available for tables in Iceberg format.

refreshPeriodMs Body Integer

Example: 3600000

refreshSchedule Body String

Field	Allowed Values	Allowed Special Characters
Second	0	N/A
Minute	0-59	N/A
Hour	0-23	N/A
Day of month	N/A	* ?
Month	N/A	* ?
Days of week	1-7 or SUN-SAT	, - * ?

Special Character	Description
*	Used to specify all values for a field. For `Day of month`, specifies every day of the month. For `Month`, specifies every month. For `Days of week`, specifies every day of the week.
?	Equivalent to *.
,	Used to specify two or more days in the `Days of week` field. For example, `MON,WED,FRI`.
-	Used to specify ranges in the `Days of week` field. For example, `1-3` is equivalent to `Sunday, Monday, and Tuesday`.

Examples:

0 0 0 * * ? : Refreshes every day at midnight.
0 45 15 * * 1,4,7 : Refreshes at 15:45 on Sunday, Wednesday, and Saturday.
0 15 7 ? * 2-6 : Refreshes at 7:15 on Monday and Friday.

gracePeriodMs Body Integer

Maximum age to allow for reflection data used to accelerate queries. In milliseconds.

Example: 10800000

method Body String

The initial refresh of a reflection is always a full refresh.
If the reflection is created from a view that uses nested group-bys, joins, unions, or window functions, then a full refresh is performed.
If the changes to the base table are only appends, then an incremental refresh based on table snapshots is performed.
If the changes to the base table include non-append operations, then a partition-based incremental refresh is attempted.
If the partitions of the base table and the partitions of the reflection are not compatible, or if either the base table or the reflection is not partitioned, then a full refresh is performed.

Read Refreshing Reflections for more information.

Enum: AUTO, FULL, INCREMENTAL

Example: FULL

refreshField Body String

For the INCREMENTAL refresh method, the field to refresh for the table. Used only if the method is INCREMENTAL. This parameter applies only to tables that are not in the Apache Iceberg format.

Example: business_id

neverExpire Body Boolean

If the reflection should never expire, true. Otherwise, false.

Example: false

neverRefresh Body Boolean

If the reflection should never refresh, true. Otherwise, false.

Example: false

Parameters of the `format` Object

type Body String

Type of data in the table.

Enum: Delta, Excel, Iceberg, JSON, Parquet, Text, Unknown, XLS

ignoreOtherFileFormats Body Boolean Optional

Example: false

skipFirstLine Body Boolean Optional

If Dremio should skip the first line in the table, true. Otherwise, false (default). Optional for Excel and Text types.

Example: true

extractHeader Body Boolean Optional

If Dremio should extract the table's column names from the first line of the file, true. Otherwise, false (default). Optional for Excel and Text types.

Example: true

hasMergedCells Body Boolean Optional

If Dremio should expand merged cells in the table, true. Otherwise, false (default). Optional for Excel types.

Example: true

fieldDelimiter Body String Optional

Character to use to indicate separate fields in the table. May be , for a comma (default), \t for a tab, | for a pipe, or a custom character. Optional for Text type.

quote Body String Optional

Character to use for quotes in the table. May be \" for a double quote (default), ' for a single quote, or a custom character. Optional for Text type.

comment Body String Optional

Character to use to indicate comments for the table. May be # for a number sign (default) or a custom character. Optional for Text type.

escape Body String Optional

Character to use to indicate an escape for the table. May be \" for a double quote (default), ` for a back quote, \\ for a backward slash, or a custom character. Optional for Text type.

lineDelimiter Body String Optional

Character to use to indicate separate lines for the table. May be \r\n for a carriage return plus new line (default), \n for a new line, or a custom character. Optional for Text type.

autoGenerateColumnNames Body Boolean Optional

If Dremio should use the existing columnn names for the table columns, true (default). Otherwise, false. Optional for Text type.

Example: true

trimHeader Body Boolean Optional

If Dremio should trim column names to a specific number of characters when updating the table, true. Otherwise, false (default). Optional for Text type.

Example: true

Parameters of the `accessControlList` Object

users Body Array of Object Optional

The users that should have privileges on the table and the privileges each user should have.

Example: [{"id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3","permissions": ["SELECT","ALTER"]},{"id": "30fca499-4abc-4469-7142-fc8dd29acac8","permissions": ["SELECT","ALTER","MANAGE_GRANTS"]}]

roles Body Array of Object Optional

The roles that should have privileges on the table and the privileges each role should have.

Example: [{"id": "76a9884b-aea5-46d5-a73a-000edf23f390","permissions": ["SELECT","ALTER"]}]

Parameters of Objects in the `users` and `roles` Arrays

id Body String Optional

The ID of the role or user.

Example: c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3

permissions Body Array of String Optional

The privileges that the role or user should have on the table. Read Table Privileges for more information.

Example: ["SELECT", "ALTER"]

Example Request

curl -X PUT 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/dba1e4fe-6351-44d2-a3e0-7aa20e782bf3' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "entityType": "dataset",
  "id": "dba1e4fe-6351-44d2-a3e0-7aa20e782bf3",
  "path": [
    "Samples",
    "samples.dremio.com",
    "Dremio University",
    "airbnb_listings.csv"
  ],
  "type": "PHYSICAL_DATASET",
  "format": {
    "type": "Text",
    "fieldDelimiter": ",",
    "skipFirstLine": false,
    "extractHeader": true,
    "quote": "\"",
    "comment": "#",
    "escape": "\"",
    "lineDelimiter": "\r\n",
    "autoGenerateColumnNames": true,
    "trimHeader": true
  }
}'

Example Response

{
    "entityType": "dataset",
    "id": "dba1e4fe-6351-44d2-a3e0-7aa20e782bf3",
    "type": "PHYSICAL_DATASET",
    "path": [
        "Samples",
        "samples.dremio.com",
        "Dremio University",
        "airbnb_listings.csv"
    ],
    "createdAt": "2024-01-23T21:26:59.568Z",
    "tag": "fc1707df-35a1-45c1-87d7-5f66fb11a729",
    "isMetadataExpired": false, 
    "lastMetadataRefreshAt": "2024-01-31T09:50:01.012Z",
    "accelerationRefreshPolicy": {
      "activePolicyType": "PERIOD",
      "refreshPeriodMs": 3600000,
      "refreshSchedule": "0 56 18 * * *",
      "gracePeriodMs": 259200000,
      "method": "AUTO",
      "neverExpire": true,
      "neverRefresh": false,
      "sourceRefreshOnDataChanges": false
    },
    "format": {
        "type": "Text",
        "ctime": 0,
        "isFolder": false,
        "location": "/samples.dremio.com/Dremio University/airbnb_listings.csv",
        "fieldDelimiter": ",",
        "skipFirstLine": false,
        "extractHeader": true,
        "quote": "\"",
        "comment": "#",
        "escape": "\"",
        "lineDelimiter": "\r\n",
        "autoGenerateColumnNames": true,
        "trimHeader": true
    },
    "accessControlList": {},
    "owner": {
        "ownerId": "c590ed7f-7142-4e1f-ba7d-94173afdc9a3",
        "ownerType": "USER"
    },
    "fields": [
        {
            "name": "id",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "listing_url",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "scrape_id",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "last_scraped",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "name",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "summary",
            "type": {
                "name": "VARCHAR"
            }
        },
        {...},
        {
            "name": "reviews_per_month",
            "type": {
                "name": "VARCHAR"
            }
        }
    ],
    "approximateStatisticsAllowed": false
}

Response Status Codes

200 OK

400 Bad Request

401 Unauthorized

403 Forbidden

404 Not Found

500 Internal Server Error

Refreshing the Reflections on a Table

Refresh the reflections associated with the specified table.

Read Refreshing Reflections to learn how refreshing works.

Method and URL

POST /v0/projects/{project-id}/catalog/{id}/refresh

Parameters

project-id Path String (UUID)

Unique identifier of the project that contains the table you want to refresh.

Example: 1df71752-69b7-47d9-9e6c-990e6b194aa4

id Path String (UUID)

Unique identifier for the table you want to refresh. For tables that are not in an Arctic source, the id is a UUID like c9c11d32-0576-4200-5a5b-8c7229cb3d72. For tables in Arctic sources and the primary Arctic catalog, the id is a JSON string like {"tableKey":["myArcticSource","myFolder","myTable"],"contentId":"65b24a53-c56e-4037-8ffc-f25297614c3c","versionContext":{"type":"BRANCH","value":"main"}}.

Example: c9c11d32-0576-4200-5a5b-8c7229cb3d72

Example Request

curl -X POST 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/c9c11d32-0576-4200-5a5b-8c7229cb3d72/refresh' \
-H 'Authorization: Bearer <PersonalAccessToken>' \
-H 'Content-Type: application/json' 

Example Response

No response

Response Status Codes

204 No Content

400 Bad Request

401 Unauthorized

403 Forbidden

404 Not Found

Reverting a Table to a File or Folder

Revert a table in a source to change the data in the table back to its original format, file or folder.

caution

Tables in primary Arctic catalogs and Arctic Catalog sources are deleted rather than reverted.

Method and URL

DELETE /v0/projects/{project-id}/catalog/{id}

Parameters

project-id Path String (UUID)

Unique identifier for the project that contains the table you want to revert to a file or folder.

Example: 1df71752-69b7-47d9-9e6c-990e6b194aa4

id Path String

Unique identifier for the table you want to revert to a file or folder. For tables that are not in an Arctic source, the id is a UUID like c9c11d32-0576-4200-5a5b-8c7229cb3d72. For tables in Arctic sources, the id is a JSON string like {"tableKey":["myArcticSource","myFolder","myTable"],"contentId":"65b24a53-c56e-4037-8ffc-f25297614c3c","versionContext":{"type":"BRANCH","value":"main"}}.

Example: c9c11d32-0576-4200-5a5b-8c7229cb3d72

Example Request

curl -X DELETE 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/c9c11d32-0576-4200-5a5b-8c7229cb3d72' \
-H 'Authorization: Bearer <PersonalAccessToken>' \
-H 'Content-Type: application/json' 

Example Response

No response

Response Status Codes

204 No Content

400 Bad Request

401 Unauthorized

403 Forbidden

404 Not Found

Table Attributes​

Attributes of the accelerationRefreshPolicy Object​

Attributes of the format Object​

Attributes of the accessControlList Object​

Attributes of Objects in the users and roles Arrays​

Attributes of the owner Object​

Attributes of Objects in the fields Array​

Attributes of the type Object​

Attributes of Objects in the subSchema Array​

Attributes of the parquetDataFormat Object​

Formatting a File or Folder as a Table​

Parameters​

Parameters of the accelerationRefreshPolicy Object​

Parameters of the format Object​

Response Status Codes​

Retrieving a Table by ID​

Parameters​

Response Status Codes​

Retrieving a Table by Path​

Parameters​

Response Status Codes​

Updating a Table​

Parameters​

Parameters of the accelerationRefreshPolicy Object​

Parameters of the format Object​

Parameters of the accessControlList Object​

Parameters of Objects in the users and roles Arrays​

Response Status Codes​

Refreshing the Reflections on a Table​

Parameters​

Response Status Codes​

Reverting a Table to a File or Folder​

Parameters​

Response Status Codes​

Table Attributes

Attributes of the `accelerationRefreshPolicy` Object

Attributes of the `format` Object

Attributes of the `accessControlList` Object

Attributes of Objects in the `users` and `roles` Arrays

Attributes of the `owner` Object

Attributes of Objects in the `fields` Array

Attributes of the `type` Object

Attributes of Objects in the `subSchema` Array

Attributes of the `parquetDataFormat` Object

Formatting a File or Folder as a Table

Parameters

Parameters of the `accelerationRefreshPolicy` Object

Parameters of the `format` Object

Response Status Codes

Retrieving a Table by ID

Parameters

Response Status Codes

Retrieving a Table by Path

Parameters

Response Status Codes

Updating a Table

Parameters

Parameters of the `accelerationRefreshPolicy` Object

Parameters of the `format` Object

Parameters of the `accessControlList` Object

Parameters of Objects in the `users` and `roles` Arrays

Response Status Codes

Refreshing the Reflections on a Table

Parameters

Response Status Codes

Reverting a Table to a File or Folder

Parameters

Response Status Codes