Table

Use the Catalog API to retrieve tables, format files and folders as tables, update and refresh tables, and revert tables to files and folders.

Table Object

{
  "entityType": "dataset",
  "id": "c9c11d32-0576-4200-5a5b-8c7229cb3d72",
  "type": "PHYSICAL_DATASET",
  "path": [
    "Samples",
    "samples.dremio.com",
    "Dremio University",
    "restaurant_reviews.parquet"
  ],
  "createdAt": "2023-01-13T19:52:01.894Z",
  "tag": "cb2905bb-39c0-497f-ae74-4c310d534f25",
  "accelerationRefreshPolicy": {
    "refreshPeriodMs": 3600000,
    "gracePeriodMs": 10800000,
    "method": "FULL",
    "neverExpire": false,
    "neverRefresh": false
  },
  "format": {
    "type": "Parquet",
    "name": "restaurant_reviews.parquet",
    "fullPath": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "restaurant_reviews.parquet"
    ],
    "ctime": 0,
    "isFolder": false,
    "location": "/samples.dremio.com/Dremio University/restaurant_reviews.parquet",
    "autoCorrectCorruptDates": true
  },
  "accessControlList": {
    "users": [
      {
        "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      },
      {
        "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
        "permissions": [
          "SELECT",
          "ALTER",
          "MANAGE_GRANTS"
        ]
      }
    ],
    "roles": [
      {
        "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      }
    ]
  },
  "owner": {
    "ownerId": "30fca499-4abc-4469-7142-fc8dd29acac8",
    "ownerType": "USER"
  },
  "fields": [
    {
      "name": "_id",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "name",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "city",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "state",
      "type": {
        "name": "VARCHAR"
      }
    },
    {
      "name": "categories",
      "type": {
        "name": "LIST",
        "subSchema": [
          {
            "type": {
              "name": "VARCHAR"
            }
          }
        ]
      }
    },
    {
      "name": "review_count",
      "type": {
        "name": "BIGINT"
      }
    },
    {
      "name": "stars",
      "type": {
        "name": "DOUBLE"
      }
    },
    {
      "name": "attributes",
      "type": {
        "name": "STRUCT",
        "subSchema": [
          {
            "name": "Parking",
            "type": {
              "name": "STRUCT",
              "subSchema": [
                {
                  "name": "garage",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "street",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "lot",
                  "type": {
                    "name": "BOOLEAN"
                  }
                },
                {
                  "name": "valet",
                  "type": {
                    "name": "BOOLEAN"
                  }
                }
              ]
            }
          },
          {
            "name": "Accepts Credit Cards",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Wheelchair Accessible",
            "type": {
              "name": "BOOLEAN"
            }
          },
          {
            "name": "Price Range",
            "type": {
              "name": "BIGINT"
            }
          }
        ]
      }
    },
    {
      "name": "date",
      "type": {
        "name": "VARCHAR"
      }
    }
  ],
  "approximateStatisticsAllowed": false
}

Table Attributes

Type of catalog entity. For tables, the entityType is dataset.

Example dataset

Unique identifier for the table.

Example c9c11d32-0576-4200-5a5b-8c7229cb3d72

Type of dataset. For tables, the type is PHYSICAL_DATASET.

Example PHYSICAL_DATASET

Path to the table within Dremio, expressed in an array. The path lists each level of heirarchy in order, from outer to inner: source or space first, then folder and subfolders, then the table itself as the last item in the array.

Example [ "Samples", "samples.dremio.com", "Dremio University", "restaurant_reviews.parquet" ]

Date and time that the table was created. In UTC format.

Example 2023-01-13T19:52:01.894Z

Unique identifier for the table instance. Dremio changes the tag whenever the table changes and uses the tag value to ensure that PUT requests apply to the most recent version of the table.

Example cb2905bb-39c0-497f-ae74-4c310d534f25

Attributes that define the acceleration refresh policy for the table.

Table format attributes.

Information about users and roles with access to the table, as well as their specific permissions. May include an array of users, an array of roles, or both, depending on the configured access and permissions. Empty if table-specific access control privileges are not set and the table inherits access and permissions. For more information, read Understanding Inheritance, Scope, and Ownership.

Information about the table's owner.

Attributes that represent the table schema.

If true, COUNT DISTINCT queries run on the table will return approximate results. Otherwise, false.

Example false

`accelerationRefreshPolicy` Object Attributes

Refresh period for the data in all reflections for the table. In milliseconds.

Example 3600000

Maximum age allowed for reflection data used to accelerate queries. In milliseconds.

Example 10800000

Approach used for refreshing the data in reflections for the table. For more information, read Refreshing Reflections.

Enum FULL , INCREMENTAL

Example FULL

For INCREMENTAL refresh method, the field to refresh for the table. Used only if method is INCREMENTAL.

Example business_id

If the reflection will never expire, true. Otherwise, false.

Example false

If the reflection will never refresh, true. Otherwise, false.

Example false

`format` Object Attributes

Type of data in the table.

Enum Delta , Excel , Iceberg , JSON , Parquet , Text , Unknown , XLS

Example Parquet

Table name. Dremio automatically duplicates the name of the origin file or folder to populate this value.

Example restaurant_reviews.parquet

Example [ "Samples", "samples.dremio.com", "Dremio University", "restaurant_reviews.parquet" ]

Not used. Has the value 0.

Example 0

If true, the table was created from a folder. If false, the table was created from a file.

Example false

Location where the table's metadata is stored within a Dremio source or space, expressed in a string.

Example /samples.dremio.com/Dremio University/restaurant_reviews.parquet

Not used. Has the value HDFS.

Example HDFS

Information about data format for Parquet tables.

List of data format types in the table. Included only for Iceberg tables, and PARQUET is the only valid value.

Example [ "PARQUET" ]

For tables created from files with multiple sheets, name of the sheet used to create the table.

Example location_1

For tables created from files, true if Dremio extracted the table's column names from the first line of the file. Otherwise, false.

Example false

For tables created from files, true if Dremio expanded merged cells in the file when creating the table. Otherwise, false.

Example true

Character used to indicate separate fields in the table. May be , for a comma (default), \t for a tab, | for a pipe, or a custom character.

Character used for quotes in the table. May be \" for a double quote (default), ' for a single quote, or a custom character.

Character used to indicate comments in the table. May be # for a number sign (default) or a custom character.

Character used to indicate an escape in the table. May be " for a double quote (default), ` for a back quote, \\ for a backward slash, or a custom character.

Character used to indicate separate lines in the table. May be \r\n for a carriage return plus new line (default), \n for a new line, or a custom character.

If Dremio skipped the first line in the file or folder when creating the table, true. Otherwise, false.

Example false

If Dremio used the existing columnn names in the file or folder for the table columns, true. Otherwise, false.

Example true

If Dremio trimmed column names to a specific number of characters when creating the table, true. Otherwise, false.

Example true

If Dremio Cloud automatically corrects corrupted date fields in the table, true. Otherwise, false.

Example true

`accessControlList` Object Attributes

List of users with access to the table and their specific privileges.

Example [ { "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3", "permissions": [ "SELECT", "ALTER" ] }, { "id": "30fca499-4abc-4469-7142-fc8dd29acac8", "permissions": [ "SELECT", "ALTER", "MANAGE_GRANTS" ] } ]

List of roles whose members have access to the table and the roles' specific privileges.

Example [ { "id": "76a9884b-aea5-46d5-a73a-000edf23f390", "permissions": [ "SELECT", "ALTER" ] } ]

`user` and `role` Object Attributes

Unique identifier for the user or role with access to the table.

Example c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3

List of privileges the user or role has for the table. For more information, read Dataset Privileges.

Example [ "SELECT', "ALTER" ]

`owner` Object Attributes

Unique identifier for the table's owner.

Example 30fca499-4abc-4469-7142-fc8dd29acac8

Type of owner for the table.

Enum USER , ROLE

Example USER

`fields` Object Attributes

Name of the table field.

Example review_count

Information about the table field.

`type` Object Attributes

Name of the table field's type.

Enum STRUCT , LIST , UNION , INTEGER , BIGINT , FLOAT , DOUBLE , VARCHAR , VARBINARY , BOOLEAN , DECIMAL , TIME , DATE , TIMESTAMP , INTERVAL DAY TO SECOND , INTERVAL YEAR TO MONTH

Example BIGINT

Total number of digits in the number. Included only for the DECIMAL type.

Example 38

Number of digits to the right of the decimal point. Included only for the DECIMAL type.

Example 0

List of objects that represent the field's composition. For example, a field composed of data about a restaurant might have a subSchema with an object for parking options, another for payment methods, and so on. subSchemas may be nested within other subSchemas. subSchema appears only for the STRUCT, LIST, and UNION types.

`subSchema` Array Attributes

Name for the subSchema object.

Example Parking

Object that contains a name attribute that provides the field's type.

Example { "name": "BOOLEAN" }

`parquetDataFormat` Object Attributes

Type of data in the table. Within the parquetDataFormat object, the only valid type is Parquet.

Example Parquet

Not used. Has the value 0.

Example 0

If true, the table was created from a folder. If false, the table was created from a file.

Example true

If true, Dremio Cloud automatically corrects corrupted date fields in the table. Otherwise, false.

Example true

Format a File or Folder as a Table

Format a file or folder as a table so that you can query the data in Dremio.

note:

To format a folder, all files in the folder must be the same format.

Method and URL

POST /v0/projects/{project-id}/catalog/{id}

Parameters

Unique identifier for the project that contains the file or folder you want to format.

Example 1df71752-69b7-47d9-9e6c-990e6b194aa4

Unique identifier for the file or folder you want to format. The id may be a UUID or a text path. If the id is a text path, use encoded format to replace special characters with their UTF-8 characters: %3A for a colon; %2F for a forward slash, and %20 for a space. For example, if the id value is dremio:/Samples/samples.dremio.com/Dremio University, the encoded id is dremio%3A%2FSamples%2Fsamples.dremio.com%2FDremio%20University.

Example c590ed7f-7142-4e1f-ba7d-94173afdc9a3

Type of catalog entity. To format a file or folder as a table, the entityType is dataset.

Path to the file or folder you want to format, expressed in an array. List each level of hierarchy in order, from outer to inner: source or space first, then any folder and subfolders, then the file or folder itself as the last item in the array. Get the path from the file or folder's children object in the response to a Folder request.

Example [ "Samples', "samples.dremio.com", "Dremio University", "restaurant_reviews.parquet" ]

Type of dataset. For tables, the type is PHYSICAL_DATASET.

Formatting parameters for the file or folder.

`format` Object Parameters

Type of data in the file or folder. To format a folder, all files in the folder must be the same format.

Enum Delta , Excel , Iceberg , JSON , Parquet , Text , Unknown , XLS

If Dremio should skip the first line in the file or folder when creating the table, true. Otherwise, false (default). Optional for Excel and Text types.

Example true

If Dremio should extract the table's column names from the first line of the file, true. Otherwise, false (default). Optional for Excel and Text types.

Example true

If Dremio should expand merged cells in the file when creating the table, true. Otherwise, false (default). Optional for Excel types.

Example true

For tables created from Excel files with multiple sheets, name of the sheet to use to create the table. Default is the first sheet in the file (for files with multiple sheets).

Example location_1

Character to use to indicate separate fields in the table. May be , for a comma (default), \t for a tab, | for a pipe, or a custom character. Optional for Text type.

Character to use for quotes in the table. May be \" for a double quote (default), ' for a single quote, or a custom character. Optional for Text type.

Character to use to indicate comments in the table. May be # for a number sign (default) or a custom character. Optional for Text type.

Character used to indicate an escape in the table. May be " for a double quote (default), ` for a back quote, \\ for a backward slash, or a custom character. Optional for Text type.

Character used to indicate separate lines in the table. May be \r\n for a carriage return plus new line (default), \n for a new line, or a custom character. Optional for Text type.

If Dremio should use the existing columnn names in the file or folder for the table columns, true (default). Otherwise, false. Optional for Text type.

Example true

If Dremio should trim column names to a specific number of characters when creating the table, true. Otherwise, false (default). Optional for Text type.

Example true

Example Request

curl -X POST 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/c9c11d32-0576-4200-5a5b-8c7229cb3d72' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "entityType": "dataset",
  "path": [
    "Samples",
    "Dremio University",
    "restaurant_reviews.parquet"
  ],
  "type": "PHYSICAL_DATASET",
  "format": {
    "type": "Parquet"
  },
  "accessControlList": {
    "users": [
      {
        "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      },
      {
        "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
        "permissions": [
          "SELECT",
          "ALTER",
          "MANAGE_GRANTS"
        ]
      }
    ],
    "roles": [
      {
        "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
        "permissions": [
          "SELECT",
          "ALTER"
        ]
      }
    ]
  }
}'

Example Response

{
    "entityType": "dataset",
    "id": "c9c11d32-0576-4200-5a5b-8c7229cb3d72",
    "type": "PHYSICAL_DATASET",
    "path": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "restaurant_reviews.parquet"
    ],
    "createdAt": "2023-01-13T19:52:01.894Z",
    "tag": "cb2905bb-39c0-497f-ae74-4c310d534f25",
    "format": {
      "type": "Parquet",
      "name": "restaurant_reviews.parquet",
      "fullPath": [
        "Samples",
        "samples.dremio.com",
        "Dremio University",
        "restaurant_reviews.parquet"
      ],
      "ctime": 0,
      "isFolder": false,
      "location": "/samples.dremio.com/Dremio University/restaurant_reviews.parquet",
      "autoCorrectCorruptDates": true
    },
    "accessControlList": {
      "users": [
        {
          "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
          "permissions": [
            "SELECT",
            "ALTER"
          ]
        },
        {
          "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
          "permissions": [
            "SELECT",
            "ALTER",
            "MANAGE_GRANTS"
          ]
        }
      ],
      "roles": [
        {
          "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
          "permissions": [
            "SELECT",
            "ALTER"
          ]
        }
      ]
    },
    "owner": {
      "ownerId": "30fca499-4abc-4469-7142-fc8dd29acac8",
      "ownerType": "USER"
    },
    "fields": [
      {
        "name": "_id",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "name",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "city",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "state",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "categories",
        "type": {
          "name": "LIST",
          "subSchema": [
            {
              "type": {
                "name": "VARCHAR"
              }
            }
          ]
        }
      },
      {
        "name": "review_count",
        "type": {
          "name": "BIGINT"
        }
      },
      {
        "name": "stars",
        "type": {
          "name": "DOUBLE"
        }
      },
      {
        "name": "attributes",
        "type": {
          "name": "STRUCT",
          "subSchema": [
            {
              "name": "Parking",
              "type": {
                "name": "STRUCT",
                "subSchema": [
                  {
                    "name": "garage",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  },
                  {
                    "name": "street",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  },
                  {
                    "name": "lot",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  },
                  {
                    "name": "valet",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  }
                ]
              }
            },
            {
              "name": "Accepts Credit Cards",
              "type": {
                "name": "BOOLEAN"
              }
            },
            {
              "name": "Wheelchair Accessible",
              "type": {
                "name": "BOOLEAN"
              }
            },
            {
              "name": "Price Range",
              "type": {
                "name": "BIGINT"
              }
            }
          ]
        }
      },
      {
        "name": "date",
        "type": {
          "name": "VARCHAR"
        }
      }
    ],
    "approximateStatisticsAllowed": false
  }

Example Request for Excel format type

curl -X POST 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/dremio%3A%2FSamples%2Fsamples.dremio.com%2FDremio%20University%2Foracle-departments.xlsx' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "entityType": "dataset",
  "path": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "oracle-departments.xlsx"
    ],
  "type": "PHYSICAL_DATASET",
  "format": {
    "type": "Excel",
    "extractHeader": true,
    "hasMergedCells": true,
    "sheetName": "Sheet1"
    }
}'

Example Request for Text format type

curl -X POST 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/6ba3bd6e-fd27-4572-a535-77e1548283b3' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "entityType": "dataset",
  "path": [
    "Samples",
    "samples.dremio.com",
    "Dremio University",
    "airbnb_listings.csv"
  ],
  "type": "PHYSICAL_DATASET",
  "format": {
    "type": "Text",
    "fieldDelimiter": ",",
    "skipFirstLine": false,
    "extractHeader": true,
    "quote": "\"",
    "comment": "#",
    "escape": "\"",
    "lineDelimiter": "\r\n",
    "autoGenerateColumnNames": true,
    "trimHeader": false
  }
}'

Response Status Codes

Retrieve a Table by ID

Retrieve a table by specifying the table’s id value.

Method and URL

GET /v0/projects/{project-id}/catalog/{id}

Parameters

Unique identifier for the project that contains the table you want to retrieve.

Example 1df71752-69b7-47d9-9e6c-990e6b194aa4

Unique identifier for the table you want to retrieve.

Example c9c11d32-0576-4200-5a5b-8c7229cb3d72

Example Request

curl -X GET 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/c9c11d32-0576-4200-5a5b-8c7229cb3d72' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json'

Example Response

{
    "entityType": "dataset",
    "id": "c9c11d32-0576-4200-5a5b-8c7229cb3d72",
    "type": "PHYSICAL_DATASET",
    "path": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "restaurant_reviews.parquet"
    ],
    "createdAt": "2023-01-13T19:52:01.894Z",
    "tag": "cb2905bb-39c0-497f-ae74-4c310d534f25",
    "format": {
      "type": "Parquet",
      "name": "restaurant_reviews.parquet",
      "fullPath": [
        "Samples",
        "samples.dremio.com",
        "Dremio University",
        "restaurant_reviews.parquet"
      ],
      "ctime": 0,
      "isFolder": false,
      "location": "/samples.dremio.com/Dremio University/restaurant_reviews.parquet",
      "autoCorrectCorruptDates": true
    },
    "accessControlList": {
      "users": [
        {
          "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
          "permissions": [
            "SELECT",
            "ALTER"
          ]
        },
        {
          "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
          "permissions": [
            "SELECT",
            "ALTER",
            "MANAGE_GRANTS"
          ]
        }
      ],
      "roles": [
        {
          "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
          "permissions": [
            "SELECT",
            "ALTER"
          ]
        }
      ]
    },
    "owner": {
      "ownerId": "30fca499-4abc-4469-7142-fc8dd29acac8",
      "ownerType": "USER"
    },
    "fields": [
      {
        "name": "_id",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "name",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "city",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "state",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "categories",
        "type": {
          "name": "LIST",
          "subSchema": [
            {
              "type": {
                "name": "VARCHAR"
              }
            }
          ]
        }
      },
      {
        "name": "review_count",
        "type": {
          "name": "BIGINT"
        }
      },
      {
        "name": "stars",
        "type": {
          "name": "DOUBLE"
        }
      },
      {
        "name": "attributes",
        "type": {
          "name": "STRUCT",
          "subSchema": [
            {
              "name": "Parking",
              "type": {
                "name": "STRUCT",
                "subSchema": [
                  {
                    "name": "garage",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  },
                  {
                    "name": "street",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  },
                  {
                    "name": "lot",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  },
                  {
                    "name": "valet",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  }
                ]
              }
            },
            {
              "name": "Accepts Credit Cards",
              "type": {
                "name": "BOOLEAN"
              }
            },
            {
              "name": "Wheelchair Accessible",
              "type": {
                "name": "BOOLEAN"
              }
            },
            {
              "name": "Price Range",
              "type": {
                "name": "BIGINT"
              }
            }
          ]
        }
      },
      {
        "name": "date",
        "type": {
          "name": "VARCHAR"
        }
      }
    ],
    "approximateStatisticsAllowed": false
  }

Response Status Codes

Retrieve a Table by Path

Retrieve a table by specifying the table’s path.

Method and URL

GET /v0/projects/{project-id}/catalog/by-path/{path}

Parameters

Unique identifier for the project that contains the table you want to retrieve.

Example 1df71752-69b7-47d9-9e6c-990e6b194aa4

Table's location within Dremio, using forward slashes as separators. For example, for the "NYC-taxi-trips" table in the "samples.dremio.com" folder within the source "Samples," the path is Samples/samples.dremio.com/NYC-taxi-trips. If the name of any component in the path includes a special character for URLs, such as a space, replace the special character with its UTF-8 character. For example, "Dremio University" should be Dremio%20University in the URL path.

Example Samples/samples.dremio.com/Dremio%20University/restaurant_reviews.parquet

Example Request

curl -X GET 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/by-path/Samples/samples.dremio.com/Dremio%20University/restaurant_reviews.parquet' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json'

Example Response

{
    "entityType": "dataset",
    "id": "c9c11d32-0576-4200-5a5b-8c7229cb3d72",
    "type": "PHYSICAL_DATASET",
    "path": [
      "Samples",
      "samples.dremio.com",
      "Dremio University",
      "restaurant_reviews.parquet"
    ],
    "createdAt": "2023-01-13T19:52:01.894Z",
    "tag": "cb2905bb-39c0-497f-ae74-4c310d534f25",
    "format": {
      "type": "Parquet",
      "name": "restaurant_reviews.parquet",
      "fullPath": [
        "Samples",
        "samples.dremio.com",
        "Dremio University",
        "restaurant_reviews.parquet"
      ],
      "ctime": 0,
      "isFolder": false,
      "location": "/samples.dremio.com/Dremio University/restaurant_reviews.parquet",
      "autoCorrectCorruptDates": true
    },
    "accessControlList": {
      "users": [
        {
          "id": "c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3",
          "permissions": [
            "SELECT",
            "ALTER"
          ]
        },
        {
          "id": "30fca499-4abc-4469-7142-fc8dd29acac8",
          "permissions": [
            "SELECT",
            "ALTER",
            "MANAGE_GRANTS"
          ]
        }
      ],
      "roles": [
        {
          "id": "76a9884b-aea5-46d5-a73a-000edf23f390",
          "permissions": [
            "SELECT",
            "ALTER"
          ]
        }
      ]
    },
    "owner": {
      "ownerId": "30fca499-4abc-4469-7142-fc8dd29acac8",
      "ownerType": "USER"
    },
    "fields": [
      {
        "name": "_id",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "name",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "city",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "state",
        "type": {
          "name": "VARCHAR"
        }
      },
      {
        "name": "categories",
        "type": {
          "name": "LIST",
          "subSchema": [
            {
              "type": {
                "name": "VARCHAR"
              }
            }
          ]
        }
      },
      {
        "name": "review_count",
        "type": {
          "name": "BIGINT"
        }
      },
      {
        "name": "stars",
        "type": {
          "name": "DOUBLE"
        }
      },
      {
        "name": "attributes",
        "type": {
          "name": "STRUCT",
          "subSchema": [
            {
              "name": "Parking",
              "type": {
                "name": "STRUCT",
                "subSchema": [
                  {
                    "name": "garage",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  },
                  {
                    "name": "street",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  },
                  {
                    "name": "lot",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  },
                  {
                    "name": "valet",
                    "type": {
                      "name": "BOOLEAN"
                    }
                  }
                ]
              }
            },
            {
              "name": "Accepts Credit Cards",
              "type": {
                "name": "BOOLEAN"
              }
            },
            {
              "name": "Wheelchair Accessible",
              "type": {
                "name": "BOOLEAN"
              }
            },
            {
              "name": "Price Range",
              "type": {
                "name": "BIGINT"
              }
            }
          ]
        }
      },
      {
        "name": "date",
        "type": {
          "name": "VARCHAR"
        }
      }
    ],
    "approximateStatisticsAllowed": false
  }

Response Status Codes

Update a Table

Update the specified table in Dremio.

Method and URL

PUT /v0/projects/{project-id}/catalog/{id}

Parameters

Unique identifier for the project that contains the table you want to update.

Example 1df71752-69b7-47d9-9e6c-990e6b194aa4

Unique identifier for the table you want to update.

Example c9c11d32-0576-4200-5a5b-8c7229cb3d72

Type of catalog entity. For tables, the entityType is dataset.

Unique identifier for the table you want to update.

Example c9c11d32-0576-4200-5a5b-8c7229cb3d72

Path to the table you want to update, expressed in an array. List each level of hierarchy in order, from outer to inner: source or space first, then any folder and subfolders, then the table itself as the last item in the array. Get the path from the table's children object in the response to a Folder request.

Example [ "Samples', "samples.dremio.com", "Dremio University", "restaurant_reviews.parquet" ]

Unique identifier for the version of the table you want to update. If you provide a tag in the request body, Dremio uses the tag value to ensure that you are requesting to update the most recent version of the table. If you do not provide a tag, Dremio automatically updates the most recent version of the table.

Example cb2905bb-39c0-497f-ae74-4c310d534f25

Type of dataset. For tables, the type is PHYSICAL_DATASET.

Formatting parameters for the table.

Object used to specify which users and roles should have access to the table, as well as their specific permissions for the table. May include an array of users, an array of roles, or both. Omit if you want the table to inherit access and permissions. For more information, read Understanding Inheritance, Scope, and Ownership.

`format` Object Parameters

Type of data in the table.

Enum Delta , Excel , Iceberg , JSON , Parquet , Text , Unknown , XLS

If Dremio should skip the first line in the table, true. Otherwise, false (default). Optional for Excel and Text types.

Example true

If Dremio should extract the table's column names from the first line of the file, true. Otherwise, false (default). Optional for Excel and Text types.

Example true

If Dremio should expand merged cells in the table, true. Otherwise, false (default). Optional for Excel types.

Example true

Character to use to indicate separate fields in the table. May be , for a comma (default), \t for a tab, | for a pipe, or a custom character. Optional for Text type.

Character to use for quotes in the table. May be \" for a double quote (default), ' for a single quote, or a custom character. Optional for Text type.

Character to use to indicate comments for the table. May be # for a number sign (default) or a custom character. Optional for Text type.

Character to use to indicate an escape for the table. May be " for a double quote (default), ` for a back quote, \\ for a backward slash, or a custom character. Optional for Text type.

Character to use to indicate separate lines for the table. May be \r\n for a carriage return plus new line (default), \n for a new line, or a custom character. Optional for Text type.

If Dremio should use the existing columnn names for the table columns, true (default). Otherwise, false. Optional for Text type.

Example true

If Dremio should trim column names to a specific number of characters when updating the table, true. Otherwise, false (default). Optional for Text type.

Example true

`accessControlList` Object Parameters

List of users who should have access to the table and their specific privileges.

List of roles whose members should have access to the table and the roles' specific privileges.

Example [ { "id": "76a9884b-aea5-46d5-a73a-000edf23f390", "permissions": [ "SELECT", "ALTER" ] } ]

`user` and `role` Object Parameters

Unique identifier for the user or role with access to the table.

Example c590ed7f-b2b4-4e1f-ba7d-94173afdc9a3

List of privileges the user or role has for the table. For more information, read Dataset Privileges.

Example [ "SELECT', "ALTER" ]

Example Request

curl -X PUT 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/dba1e4fe-6351-44d2-a3e0-7aa20e782bf3' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "entityType": "dataset",
  "id": "dba1e4fe-6351-44d2-a3e0-7aa20e782bf3",
  "path": [
    "Samples",
    "samples.dremio.com",
    "Dremio University",
    "airbnb_listings.csv"
  ],
  "type": "PHYSICAL_DATASET",
  "format": {
    "type": "Text",
    "fieldDelimiter": ",",
    "skipFirstLine": false,
    "extractHeader": true,
    "quote": "\"",
    "comment": "#",
    "escape": "\"",
    "lineDelimiter": "\r\n",
    "autoGenerateColumnNames": true,
    "trimHeader": true
  }
}'

Example Response

{
    "entityType": "dataset",
    "id": "dba1e4fe-6351-44d2-a3e0-7aa20e782bf3",
    "type": "PHYSICAL_DATASET",
    "path": [
        "Samples",
        "samples.dremio.com",
        "Dremio University",
        "airbnb_listings.csv"
    ],
    "createdAt": "2023-01-23T21:26:59.568Z",
    "tag": "fc1707df-35a1-45c1-87d7-5f66fb11a729",
    "format": {
        "type": "Text",
        "ctime": 0,
        "isFolder": false,
        "location": "/samples.dremio.com/Dremio University/airbnb_listings.csv",
        "fieldDelimiter": ",",
        "skipFirstLine": false,
        "extractHeader": true,
        "quote": "\"",
        "comment": "#",
        "escape": "\"",
        "lineDelimiter": "\r\n",
        "autoGenerateColumnNames": true,
        "trimHeader": true
    },
    "accessControlList": {},
    "owner": {
        "ownerId": "c590ed7f-7142-4e1f-ba7d-94173afdc9a3",
        "ownerType": "USER"
    },
    "fields": [
        {
            "name": "id",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "listing_url",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "scrape_id",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "last_scraped",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "name",
            "type": {
                "name": "VARCHAR"
            }
        },
        {
            "name": "summary",
            "type": {
                "name": "VARCHAR"
            }
        },
        {...},
        {
            "name": "reviews_per_month",
            "type": {
                "name": "VARCHAR"
            }
        }
    ],
    "approximateStatisticsAllowed": false
}

Response Status Codes

Refresh a Table

Refresh the data stored in the specified table and the reflections associated with the table.

Read Refreshing Reflections to learn how refreshing works.

Method and URL

POST /v0/projects/{project-id}/catalog/{id}/refresh

Parameters

Unique identifier for the project that contains the table you want to refresh.

Example 1df71752-69b7-47d9-9e6c-990e6b194aa4

Unique identifier for the table you want to refresh.

Example c9c11d32-0576-4200-5a5b-8c7229cb3d72

Example Request

curl -X POST 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/c9c11d32-0576-4200-5a5b-8c7229cb3d72/refresh' \
-H 'Authorization: Bearer <PersonalAccessToken>' \
-H 'Content-Type: application/json'

Example Response

No response

Response Status Codes

Revert a Table to a File or Folder

Revert a table in a source to change the data in the table back to its original format, file or folder.

note:

If a table is saved in your home space, the revert request will delete the table entirely. The revert endpoint only changes a table back to a file or folder if the table is saved in a source.

Method and URL

DELETE /v0/projects/{project-id}/catalog/{id}

Parameters

Unique identifier for the project that contains the table you want to revert to a file or folder.

Example 1df71752-69b7-47d9-9e6c-990e6b194aa4

Unique identifier for the table you want to revert to a file or folder.

Example c9c11d32-0576-4200-5a5b-8c7229cb3d72

Example Request

curl -X DELETE 'https://api.dremio.cloud/v0/projects/1df71752-69b7-47d9-9e6c-990e6b194aa4/catalog/c9c11d32-0576-4200-5a5b-8c7229cb3d72' \
-H 'Authorization: Bearer <PersonalAccessToken>' \
-H 'Content-Type: application/json'

Example Response

No response

Response Status Codes

Was this page helpful?

Glad to hear it! Thank you for your feedback. Sorry to hear that. Thank you for your feedback.

Table

Table Attributes

accelerationRefreshPolicy Object Attributes

format Object Attributes

accessControlList Object Attributes

user and role Object Attributes

owner Object Attributes

fields Object Attributes

type Object Attributes

subSchema Array Attributes

parquetDataFormat Object Attributes

Format a File or Folder as a Table

Parameters

format Object Parameters

Response Status Codes

Retrieve a Table by ID

Parameters

Response Status Codes

Retrieve a Table by Path

Parameters

Response Status Codes

Update a Table

Parameters

format Object Parameters

accessControlList Object Parameters

user and role Object Parameters

Response Status Codes

Refresh a Table

Parameters

Response Status Codes

Revert a Table to a File or Folder

Parameters

Response Status Codes

`accelerationRefreshPolicy` Object Attributes

`format` Object Attributes

`accessControlList` Object Attributes

`user` and `role` Object Attributes

`owner` Object Attributes

`fields` Object Attributes

`type` Object Attributes

`subSchema` Array Attributes

`parquetDataFormat` Object Attributes

`format` Object Parameters

`format` Object Parameters

`accessControlList` Object Parameters

`user` and `role` Object Parameters