Skip to main content

AI

Dremio's AI features are designed for business and data analysts who need a conversational data analysis agent as well as data and platform engineers who are interested using LLMs to query unstructured data and store in Iceberg tables. AI is also available for generating wikis and labels so you can rapidly develop your semantic layer in support of complete governed context for Dremio's AI Agent.

To use AI features, you need to connect to a Model Provider and then enable AI in your Project Preferences.

AI Agent​

Dremio's AI Agent is a natural language interface where you can explore datasets, analyze data, ask for an explanation of SQL structure or job performance, and visualize outputs. As you interact with the AI Agent, you will see reasoning and actions taken in response so you can review and audit the agent. All existing data permissions and functionality privileges will apply. Generative AI can make mistakes; therefore, you are strongly encouraged to verify all outputs.

AI Functions​

Write SQL with one of the following AI functions and pass a prompt along with structured or unstructured data to the LLM, then use the LLM output in structured SQL.

  • AI_GENERATE - Flexible general-purpose function for complex data extraction
  • AI_CLASSIFY - Specialized for sentiment analysis and document categorization
  • AI_COMPLETE - Optimized for creative text generation and summaries

Generate Wikis and Labels Preview​

Dremio can use AI to generate wikis and labels. In order to generate a label, Dremio bases its understanding on your schema by considering other labels that have been previously generated and those that have been created by other users. In order to generate a wiki for a dataset, Dremio takes your data into account by determining how the columns within the dataset relate to each other and to the dataset as a whole.

Data Privacy​

  • Neither Dremio nor our trained model partner (OpenAI) trains models using customer data, which reduces the risk of IP leakage through passed context information.
  • Dremio sends sample data to the model, and this data is secured with the user's credentials.
  • Dremio uses schema information and dataset tags/wikis as model context. This means that any sensitive information in your table or column names, or in the dataset tags/wikis, could be passed to a model. Neither Dremio nor our trained model partner (Open AI) persists this context.
  • Generative AI features are built on Dremio's existing, secure data platform. These features are audited by the same practices and policies, including SOC 2 Type II. For more information, see Dremio Cloud Terms of Service.