AI Enterprise
Dremio features the Dremio's AI Agent for natural language chat and AI Functions for writing SQL while using LLMs.
Configuring AI Model Providers
These features are powered by model providers that you configure for your organization, allowing you to control the specific models you want to work with or make available. To begin using AI features, first configure at least one model provider for your organization.
The Dremio's AI Agent
The Dremio's AI Agent is a natural language interface where you can explore datasets, analyze data, ask for an explanation of SQL structure or job performance, and visualize outputs. As you interact with the Dremio's AI Agent, you will see reasoning and actions taken in response so you can review and audit the agent. All existing data permissions and functionality privileges will apply. Generative AI can make mistakes; therefore, you are strongly encouraged to verify all outputs.
Use the Dremio's AI Agent
To use the Dremio's AI Agent, you can either:
- Open the chat from the SQL Runner by selecting the
in the upper right corner. - Open the chat from the Jobs Details page by selecting the in the upper right corner.
- Use the shortcut keys ⌘+Shift+G on a Mac or Ctrl+Shift+L on Windows to open the agent from either the SQL Runner or the Jobs Details page.
Explore with Dremio's AI Agent
Dremio's AI Agent will help you discover available datasets and provide a detailed breakdown of table structure, as well as offer guidance on what tables you may want to use. The more detailed the question, the better the insight that Dremio's AI Agent can provide.
| Okay Prompt | Great Prompt |
|---|---|
| What tables can I use? | Which tables or views have customer location data? |
| How can I analyze time series data? | Which tables or views can I use to do a time series analysis on customer activity? |
What is the customer_activity table? | How is the customer_activity table structured and what other tables does it relate to? |
Analyze Data with Dremio's AI Agent
Dremio's AI Agent will write and execute SQL on your behalf based on your natural language input and the information available from the semantic layer. The Dremio's AI Agent will adhere to the privileges that you have been granted in Dremio. From within the chat, you can further audit the SQL by expanding Executing SQL entries in the chat window.
| Okay Prompt | Great Prompt |
|---|---|
| I want to see analysis of customer activity | I want to see an analysis of customer purchase activity by region, by customer type for each month of the year. |
| Which customers are the most valuable? | Which customers have spent the most with us over the lifetime of the relationship? |
Visualize Insights with Dremio's AI Agent
Dremio's AI Agent will visualize insights on your behalf based on your natural language input. The details you provide, including the chart type, axis requirements, grouping, or trendlines, will be considered by the LLM. The visualization will be accompanied by insights that serve as a narrative for the chart that the Dremio's AI Agent generated. Once a visualization has been created, you can toggle between the visualization and a grid representation of the data that is used to back the visualization.
| Okay Prompt | Great Prompt |
|---|---|
| Visualize the data | Visualize the data as a bar chart with month on the x asis and sum of purchase value as the y axis |
| Create a visual trendline showing me the activity | Create a visualization with a trendline showing customer activty by month? |
The Dremio's AI Agent can return the following types of visualizations: Bar, Line, Area, Scatter, Pie, Heatmap
Explain SQL and Job
The AI Agent helps you analyze and improve SQL in your Dremio deployment. In the SQL Runner, highlight your SQL, right-click, and select Explain SQL to start a chat with an initial summary of the query’s overview, datasets, and architecture. From there, you can dive deeper into the analysis, instruct the AI Agent to optimize and ask follow-up questions about the SQL.
The AI Agent helps you analyze and optimize jobs in Dremio. On the Job Details page, click Explain Job to start a chat with an initial summary of job performance (such as skew, memory usage, and threading) and optimization suggestions. From there, you can explore the job in more detail or discuss other jobs by referencing their Job IDs.
AI Functions
Write SQL with one of the following AI functions and pass a prompt along with structured or unstructured data to the LLM, then use the LLM output in structured SQL.
| Function | How it Works | When To Use |
|---|---|---|
| AI_GENERATE | Flexible general-purpose function for processing unstructured data | Complex data extraction requiring multiple fields from source files. |
| AI_CLASSIFY | Specialized form of AI_GENERATE for sentiment analysis and document categorization, returned as VARCHAR. | Classification list provided to LLM as an Array. |
| AI_COMPLETE | Specialized form of AI_GENERATE for creative text generation and summaries, returned as VARCHAR. | Generating new data based on input. |
| LIST_FILES | Passes unstructured files to AI_GENERATE for use by LLM. | Processing unstructured data. |
Managing AI Privileges
Users will need to have the CALL MODEL privilege assigned in order to use the available model providers. LLM usage for your chosen model provider is tracked in sys.model_usage for visibility into how your organization is engaging with AI.