On this page

    Edit a Catalog

    This API updates existing datasets and sources.

    Syntax

    PUT /api/v3/catalog/{id}
    

    Request Input

    The request input is one of the following:

    Response Output

    The response output is one of the following:

    Response Codes

    400 - The supplied catalog entity object is invalid.
    403 - User does not have permission to create the catalog entity.
    404 - A catalog entity with the specified ID could not be found.
    409 - A catalog entity with the specified path already exists.

    Example: Update Source

    In this example, a source entity, my_elasticsearch, is updated with "accelerationNeverRefresh": true. The ID is obtained from the POST /catalog examples.

    Note:
    Postman was used to generate samples.

    HTTP Request

    PUT localhost:9047/api/v3/catalog/a7fe6900-a9d1-4afe-aeb7-3dc5ddfca5b7

    Raw Body Input

    {
    	"entityType": "source",
    	"id": "a7fe6900-a9d1-4afe-aeb7-3dc5ddfca5b7",
    	"tag": "0",
    	"type": "ELASTIC",
    	"name": "my_elasticsearch",
    	"accelerationNeverRefresh": true,
    	"config": {
    		"hostList": [
    			{
    				"hostname": "172.25.0.94",
    				"port": 9200
    			}
    		]
    	}
    }
    

    Curl

    curl -X PUT \
      http://localhost:9047/api/v3/catalog/a7fe6900-a9d1-4afe-aeb7-3dc5ddfca5b7 \
      -H 'Authorization: _dremioo8opojj6vn4ughkvcpalpr46d6' \
      -H 'Content-Type: application/json' \
      -d '{
    	"entityType": "source",
    	"id": "a7fe6900-a9d1-4afe-aeb7-3dc5ddfca5b7",
    	"tag": "1",
    	"type": "ELASTIC",
    	"name": "my_elasticsearch",
    	"accelerationNeverRefresh": true,
    	"config": {
    		"hostList": [
    			{
    				"hostname": "172.25.0.94",
    				"port": 9200
    			}
    		]
    	}
    }'
    

    Python

    import requests
    
    url = "http://localhost:9047/api/v3/catalog/a7fe6900-a9d1-4afe-aeb7-3dc5ddfca5b7"
    
    payload = "{\n\t\"entityType\": \"source\",\n\t\"id\": \"a7fe6900-a9d1-4afe-aeb7-3dc5ddfca5b7\",\n\t\"tag\": \"1\",\n\t\"type\": \"ELASTIC\",\n\t\"name\": \"my_elasticsearch\",\n\t\"accelerationNeverRefresh\": true,\n\t\"config\": {\n\t\t\"hostList\": [\n\t\t\t{\n\t\t\t\t\"hostname\": \"172.25.0.94\",\n\t\t\t\t\"port\": 9200\n\t\t\t}\n\t\t]\n\t}\n}"
    headers = {
        'Authorization': "_dremioo8opojj6vn4ughkvcpalpr46d6",
        'Content-Type': "application/json"
        }
    
    response = requests.request("PUT", url, data=payload, headers=headers)
    
    print(response.text)
    

    Response

    {
        "entityType": "source",
        "config": {
            "scriptsEnabled": true,
            "showHiddenIndices": false,
            "showIdColumn": false,
            "readTimeoutMillis": 60000,
            "scrollTimeoutMillis": 300000,
            "usePainless": true,
            "scrollSize": 4000,
            "allowPushdownOnNormalizedOrAnalyzedFields": false,
            "warnOnRowCountMismatch": false,
            "encryptionValidationMode": "CERTIFICATE_AND_HOSTNAME_VALIDATION",
            "hostList": [
                {
                    "hostname": "172.25.0.94",
                    "port": 9200
                }
            ],
            "password": "$DREMIO_EXISTING_VALUE$",
            "authenticationType": "ANONYMOUS",
            "sslEnabled": false,
            "useWhitelist": false
        },
        "state": {
            "status": "warn",
            "messages": [
                {
                    "level": "WARN",
                    "message": "Elastic cluster health is yellow."
                }
            ]
        },
        "id": "a7fe6900-a9d1-4afe-aeb7-3dc5ddfca5b7",
        "tag": "2",
        "type": "ELASTIC",
        "name": "my_elasticsearch",
        "createdAt": "1970-01-01T00:00:00.000Z",
        "metadataPolicy": {
            "authTTLMs": 86400000,
            "namesRefreshMs": 3600000,
            "datasetRefreshAfterMs": 3600000,
            "datasetExpireAfterMs": 10800000,
            "datasetUpdateMode": "PREFETCH_QUERIED",
            "deleteUnavailableDatasets": true,
            "autoPromoteDatasets": false
        },
        "accelerationGracePeriodMs": 0,
        "accelerationRefreshPeriodMs": 0,
        "accelerationNeverExpire": false,
        "accelerationNeverRefresh": true,
        "children": [
            {
                "id": "0477f384-28ee-49e3-9dc9-9dbaf77f40d1",
                "path": [
                    "my_elasticsearch",
                    "yelp"
                ],
                "tag": "0",
                "type": "CONTAINER",
                "containerType": "FOLDER"
            }
        ],
        "accessControlList": {
            "version": "0"
        }
    }
    

    Example: Update Space

    In this example, a space entity, Pebbles, is updated by giving a user READ permission. The space entity ID is obtained from the POST /catalog examples. The user ID is obtained by executing GET localhost:9047/api/v3/user/by-name/test1 for the test1 user. See GET /user/by-name.

    Note:
    Postman was used to generate samples.

    HTTP Request

    PUT localhost:9047/api/v3/catalog/35f9ffe7-fd02-4509-aa99-ce626f123537

    Raw Body Input

    {
    	"entityType": "space",
    	"id": "35f9ffe7-fd02-4509-aa99-ce626f123537",
    	"name": "Pebbles",
    	"tag": "0",
    	"accessControlList": {
    		"version": "0",
    		"users": [
    			{
    				"id": "2002c74e-3ea0-470b-bdeb-9f680cecaebb",
    				"permissions": [
    					"READ"
    				]
    			}
    		]
    	}
    }
    

    Curl

    curl -X PUT \
      http://localhost:9047/api/v3/catalog/35f9ffe7-fd02-4509-aa99-ce626f123537 \
      -H 'Authorization: _dremioo8opojj6vn4ughkvcpalpr46d6' \
      -H 'Content-Type: application/json' \
      -d '{
    	"entityType": "space",
    	"id": "35f9ffe7-fd02-4509-aa99-ce626f123537",
    	"name": "Pebbles",
    	"tag": "0",
    	"accessControlList": {
    		"version": "0",
    		"users": [
    			{
    				"id": "2002c74e-3ea0-470b-bdeb-9f680cecaebb",
    				"permissions": [
    					"READ"
    				]
    			}
    		]
    	}
    }'
    

    Python

    import requests
    
    url = "http://localhost:9047/api/v3/catalog/35f9ffe7-fd02-4509-aa99-ce626f123537"
    
    payload = "{\n\t\"entityType\": \"space\",\n\t\"id\": \"35f9ffe7-fd02-4509-aa99-ce626f123537\",\n\t\"name\": \"Pebbles\",\n\t\"tag\": \"0\",\n\t\"accessControlList\": {\n\t\t\"version\": \"0\",\n\t\t\"users\": [\n\t\t\t{\n\t\t\t\t\"id\": \"2002c74e-3ea0-470b-bdeb-9f680cecaebb\",\n\t\t\t\t\"permissions\": [\n\t\t\t\t\t\"READ\"\n\t\t\t\t]\n\t\t\t}\n\t\t]\n\t}\n}"
    headers = {
        'Authorization': "_dremioo8opojj6vn4ughkvcpalpr46d6",
        'Content-Type': "application/json"
        }
    
    response = requests.request("PUT", url, data=payload, headers=headers)
    
    print(response.text)
    

    Response

    {
        "entityType": "space",
        "id": "35f9ffe7-fd02-4509-aa99-ce626f123537",
        "name": "Pebbles",
        "tag": "1",
        "children": [
            {
                "id": "37732cdf-c491-4a15-afbf-f0e0a7563e40",
                "path": [
                    "Pebbles",
                    "Color"
                ],
                "tag": "0",
                "type": "CONTAINER",
                "containerType": "FOLDER"
            }
        ],
        "accessControlList": {
            "users": [
                {
                    "id": "2002c74e-3ea0-470b-bdeb-9f680cecaebb",
                    "permissions": [
                        "READ"
                    ]
                }
            ],
            "version": "1"
        }
    }
    

    Example: Update Folder

    In this example, a folder entity, Color, is updated by giving a user READ permission. The folder entity ID is obtained from the POST /catalog examples. The user ID is obtained by executing GET localhost:9047/api/v3/user/by-name/test1 for the test1 user. See GET /user/by-name.

    Note:
    Postman was used to generate samples.

    HTTP Request

    PUT localhost:9047/api/v3/catalog/37732cdf-c491-4a15-afbf-f0e0a7563e40

    Raw Body Input

    {
      "entityType": "folder",
      "id": "37732cdf-c491-4a15-afbf-f0e0a7563e40",
      "path": [
        "Pebbles",
        "Color"
      ],
      "tag": "0",
      "accessControlList": {
    		"version": "0",
    		"users": [
    			{
    				"id": "2002c74e-3ea0-470b-bdeb-9f680cecaebb",
    				"permissions": [
    					"READ"
    				]
    			}
    		]
    	}
    }
    

    Curl

    curl -X PUT \
      http://localhost:9047/api/v3/catalog/37732cdf-c491-4a15-afbf-f0e0a7563e40 \
      -H 'Authorization: _dremioo8opojj6vn4ughkvcpalpr46d6' \
      -H 'Content-Type: application/json' \
      -d '{
      "entityType": "folder",
      "id": "37732cdf-c491-4a15-afbf-f0e0a7563e40",
      "path": [
        "Pebbles",
        "Color"
      ],
      "tag": "0",
      "accessControlList": {
    		"version": "0",
    		"users": [
    			{
    				"id": "2002c74e-3ea0-470b-bdeb-9f680cecaebb",
    				"permissions": [
    					"READ"
    				]
    			}
    		]
    	}
    }'
    

    Python

    import requests
    
    url = "http://localhost:9047/api/v3/catalog/37732cdf-c491-4a15-afbf-f0e0a7563e40"
    
    payload = "{\n  \"entityType\": \"folder\",\n  \"id\": \"37732cdf-c491-4a15-afbf-f0e0a7563e40\",\n  \"path\": [\n    \"Pebbles\",\n    \"Color\"\n  ],\n  \"tag\": \"0\",\n  \"accessControlList\": {\n\t\t\"version\": \"0\",\n\t\t\"users\": [\n\t\t\t{\n\t\t\t\t\"id\": \"2002c74e-3ea0-470b-bdeb-9f680cecaebb\",\n\t\t\t\t\"permissions\": [\n\t\t\t\t\t\"READ\"\n\t\t\t\t]\n\t\t\t}\n\t\t]\n\t}\n}"
    headers = {
        'Authorization': "_dremioo8opojj6vn4ughkvcpalpr46d6",
        'Content-Type': "application/json"
        }
    
    response = requests.request("PUT", url, data=payload, headers=headers)
    
    print(response.text)
    

    Response

    {
        "entityType": "folder",
        "id": "37732cdf-c491-4a15-afbf-f0e0a7563e40",
        "path": [
            "Pebbles",
            "Color"
        ],
        "tag": "1",
        "children": [
            {
                "id": "e2676566-1946-4f11-8e21-9fbb366fa77a",
                "path": [
                    "Pebbles",
                    "Color",
                    "my_elastic_dataset"
                ],
                "tag": "0",
                "type": "DATASET",
                "datasetType": "VIRTUAL"
            }
        ],
        "accessControlList": {
            "users": [
                {
                    "id": "2002c74e-3ea0-470b-bdeb-9f680cecaebb",
                    "permissions": [
                        "READ"
                    ]
                }
            ],
            "version": "1"
        }
    }
    

    Example: Update Dataset

    In this example, a virtual dataset (VDS) entity, my_elastic_dataset, is updated by selecting the content from the business PDS of the my_elasticsearch source and yelp folder. The ID is obtained from the POST /catalog examples.

    Note:
    Postman was used to generate samples.

    HTTP Request

    PUT localhost:9047/api/v3/catalog

    Raw Body Input

    {
      "entityType": "dataset",
      "id": "e2676566-1946-4f11-8e21-9fbb366fa77a",
      "path": [
        "Pebbles",
        "Color",
        "my_elastic_dataset"
      ],
      "tag": "0",
    	"type": "VIRTUAL_DATASET",
    	"sql": "select * from business",
    	"sqlContext": ["my_elasticsearch", "yelp"]
    }
    

    Curl

    curl -X PUT \
      http://localhost:9047/api/v3/catalog/e2676566-1946-4f11-8e21-9fbb366fa77a \
      -H 'Authorization: _dremioo8opojj6vn4ughkvcpalpr46d6' \
      -H 'Content-Type: application/json' \
      -d '{
      "entityType": "dataset",
      "id": "e2676566-1946-4f11-8e21-9fbb366fa77a",
      "path": [
        "Pebbles",
        "Color",
        "my_elastic_dataset"
      ],
      "tag": "0",
    	"type": "VIRTUAL_DATASET",
    	"sql": "select * from business",
    	"sqlContext": ["my_elasticsearch", "yelp"]
    }'
    

    Python

    import requests
    
    url = "http://localhost:9047/api/v3/catalog/e2676566-1946-4f11-8e21-9fbb366fa77a"
    
    payload = "{\n  \"entityType\": \"dataset\",\n  \"id\": \"e2676566-1946-4f11-8e21-9fbb366fa77a\",\n  \"path\": [\n    \"Pebbles\",\n    \"Color\",\n    \"my_elastic_dataset\"\n  ],\n  \"tag\": \"0\",\n\t\"type\": \"VIRTUAL_DATASET\",\n\t\"sql\": \"select * from business\",\n\t\"sqlContext\": [\"my_elasticsearch\", \"yelp\"]\n}"
    headers = {
        'Authorization': "_dremioo8opojj6vn4ughkvcpalpr46d6",
        'Content-Type': "application/json"
        }
    
    response = requests.request("PUT", url, data=payload, headers=headers)
    
    print(response.text)
    

    Response

    {
        "entityType": "dataset",
        "id": "e2676566-1946-4f11-8e21-9fbb366fa77a",
        "type": "VIRTUAL_DATASET",
        "path": [
            "Pebbles",
            "Color",
            "my_elastic_dataset"
        ],
        "createdAt": "2019-03-26T00:27:36.075Z",
        "tag": "1",
        "sql": "select * from business",
        "sqlContext": [
            "my_elasticsearch",
            "yelp"
        ],
        "accessControlList": {
            "version": "0"
        },
        "fields": [
            {
                "name": "hours",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "neighborhoods",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "city",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "latitude",
                "type": {
                    "name": "DOUBLE"
                }
            },
            {
                "name": "review_count",
                "type": {
                    "name": "BIGINT"
                }
            },
            {
                "name": "full_address",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "stars",
                "type": {
                    "name": "DOUBLE"
                }
            },
            {
                "name": "type",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "name",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "attributes",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "categories",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "state",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "business_id",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "open",
                "type": {
                    "name": "BOOLEAN"
                }
            },
            {
                "name": "longitude",
                "type": {
                    "name": "DOUBLE"
                }
            },
            {
                "name": "_index",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "_type",
                "type": {
                    "name": "VARCHAR"
                }
            },
            {
                "name": "_uid",
                "type": {
                    "name": "VARCHAR"
                }
            }
        ]
    }