Catalog

Use the Catalog API to retrieve and manage sources as well as the folders, tables, and views they contain. The Catalog API also includes endpoints for retrieving lineage information for datasets and for creating and managing tags, wikis, privileges, and grants on catalog objects.

Use the Catalog API endpoint described on this page to retrieve a list of the sources in your Dremio organization. The response contains the IDs required to make requests to other Catalog API endpoints to create, retrieve, update, and delete objects in your catalog.

Catalog Object

{
  "data": [
    {
      "id": "5c2f0383-540c-462d-816a-a8be8334efe5",
      "path": [
        "default-arctic-catalog"
      ],
      "tag": "9027fe00-d27e-4d02-96a8-e7705c934888",
      "type": "CONTAINER",
      "containerType": "SOURCE",
      "stats": {
        "datasetCount": 18,
        "datasetCountBounded": false
      },
      "createdAt": "2023-08-29T15:22:12.988Z",
      "permissions": [
        "READ_METADATA",
        "READ",
        "WRITE",
        "ALTER_REFLECTION",
        "SELECT",
        "ALTER",
        "VIEW_REFLECTION",
        "MODIFY",
        "MANAGE_GRANTS",
        "CREATE_TABLE",
        "DROP",
        "EXTERNAL_QUERY",
        "INSERT",
        "TRUNCATE",
        "DELETE",
        "UPDATE",
        "EXECUTE",
        "CREATE_SOURCE",
        "ALL"
      ]
    },
    {
      "id": "b7d1494f-217e-41a3-8c40-c8d904e61419",
      "path": [
        "Samples"
      ],
      "tag": "2d55eae8-e938-4a86-b204-fed55f38f0a6",
      "type": "CONTAINER",
      "containerType": "SOURCE",
      "createdAt": "2023-08-29T15:22:21.156Z",
      "stats": {
        "datasetCount": 15,
        "datasetCountBounded": false
      },
      "permissions": [
        "READ_METADATA",
        "READ",
        "WRITE",
        "ALTER_REFLECTION",
        "SELECT",
        "ALTER",
        "VIEW_REFLECTION",
        "MODIFY",
        "MANAGE_GRANTS",
        "CREATE_TABLE",
        "DROP",
        "EXTERNAL_QUERY",
        "INSERT",
        "TRUNCATE",
        "DELETE",
        "UPDATE",
        "EXECUTE",
        "CREATE_SOURCE",
        "ALL"
      ]
    }
  ]
}

Catalog Attributes

data Array of Object

List of catalog objects in the project.

Attributes of Objects in the `data` Array

id String (UUID)

Unique identifier of the catalog object.

Example: 5c2f0383-540c-462d-816a-a8be8334efe5

path Array of String

Path of the catalog object within Dremio, expressed as an array.

Example: ["default-arctic-catalog"]

tag String

Unique identifier of the version of the catalog object. Dremio changes the tag whenever the catalog object changes and uses the tag to ensure that PUT requests apply to the most recent version of the catalog object.

Example: 9027fe00-d27e-4d02-96a8-e7705c934888

type String

Type of the catalog object. For objects that can contain other catalog objects (the only objects this endpoint retrieves), the type is CONTAINER.

Example: CONTAINER

containerType String

For catalog objects with the type CONTAINER, the type of container.

Enum: SPACE, SOURCE, FOLDER, HOME

Example: SPACE

stats Object

Information about the number of datasets in the catalog object and whether the dataset count is bounded. Appears in the response only if the request URL includes the datasetCount query parameter.

Example: {"datasetCount": 18,"datasetCountBounded": false}

createdAt String

Date and time that the catalog object was created, in UTC format.

Example: 2023-08-29T15:22:12.988Z

permissions Array of String

List of the privileges that you have on the catalog object. Only appears in the response if the request URL includes the permissions query parameter. For more information, read Privileges.

Example: ["READ_METADATA","READ","WRITE","ALTER_REFLECTION","SELECT","ALTER","VIEW_REFLECTION","MODIFY","MANAGE_GRANTS","CREATE_TABLE","DROP","EXTERNAL_QUERY","INSERT","TRUNCATE","DELETE","UPDATE","EXECUTE","CREATE_SOURCE","ALL"]

Attributes of the `stats` Object

datasetCount Integer

Number of datasets the catalog object contains.

Example: 18

datasetCountBounded Boolean

If the dataset count is bounded, the value is true. Otherwise, the value is false.

Example: false

Retrieving a Catalog

Retrieve a list of all the objects contained in the catalog for the specified project.

Method and URL

GET /v0/projects/{project-id}/catalog

Parameters

project-id Path String (UUID)

Unique identifier of the project whose catalog objects you want to retrieve.

Example: 5c2f0383-540c-462d-816a-a8be8334efe5

include Query String Optional

Include a non-default attribute in the response. The available values for the include query parameter are permissions (Enterprise-only) and datasetCount. Specify permissions to include each catalog object's permissions array in the response. Specify datasetCount to include the stats object in the response. For more information, read include and exclude Query Parameters.

Example: ?include=permissions

Example Request

curl -X GET 'https://api.dremio.cloud/v0/projects/5c2f0383-540c-462d-816a-a8be8334efe5/catalog' \
--header 'Authorization: Bearer <PersonalAccessToken>' \
--header 'Content-Type: application/json'

Example Response

{
  "data": [
    {
      "id": "5c2f0383-540c-462d-816a-a8be8334efe5",
      "path": [
        "default-arctic-catalog"
      ],
      "tag": "9027fe00-d27e-4d02-96a8-e7705c934888",
      "type": "CONTAINER",
      "containerType": "SOURCE",
      "createdAt": "2023-08-29T15:22:12.988Z"
    },
    {
      "id": "b7d1494f-217e-41a3-8c40-c8d904e61419",
      "path": [
        "Samples"
      ],
      "tag": "2d55eae8-e938-4a86-b204-fed55f38f0a6",
      "type": "CONTAINER",
      "containerType": "SOURCE",
      "createdAt": "2023-08-29T15:22:21.156Z"
    }
  ]
}

Response Status Codes

200 OK

401 Unauthorized

403 Forbidden

404 Not Found

Catalog Attributes​

Attributes of Objects in the data Array​

Attributes of the stats Object​

Retrieving a Catalog​

Parameters​

Response Status Codes​