|
| 1 | +# Data Visualization |
| 2 | + |
| 3 | +This guide covers the code interpreter feature for generating charts and visualizations from query results. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +When enabled, the data agent can detect visualization intent in user queries (e.g., "show me a chart", "plot the data") and generate matplotlib code to create charts. The code runs in a secure, isolated environment using Azure Container Apps Dynamic Sessions. |
| 8 | + |
| 9 | +**Key features:** |
| 10 | +- Automatic detection of visualization requests |
| 11 | +- LLM-generated matplotlib code |
| 12 | +- Secure sandboxed execution with Hyper-V isolation |
| 13 | +- Native image capture (no file storage) |
| 14 | +- Support for bar charts, line charts, pie charts, scatter plots, and more |
| 15 | + |
| 16 | +## Requirements |
| 17 | + |
| 18 | +Visualization requires Azure Container Apps Dynamic Sessions. This provides: |
| 19 | + |
| 20 | +| Feature | Benefit | |
| 21 | +|---------|---------| |
| 22 | +| **Hyper-V isolation** | Each execution runs in a dedicated VM | |
| 23 | +| **Pre-installed packages** | NumPy, Pandas, Matplotlib ready to use | |
| 24 | +| **Native image capture** | `plt.show()` output captured automatically | |
| 25 | +| **Automatic cleanup** | Sessions terminate after idle timeout | |
| 26 | +| **No host access** | Code cannot access host filesystem or network | |
| 27 | + |
| 28 | +## Azure Setup |
| 29 | + |
| 30 | +### 1. Create a Container Apps Environment |
| 31 | + |
| 32 | +If you don't already have one: |
| 33 | + |
| 34 | +```bash |
| 35 | +az containerapp env create \ |
| 36 | + --name aca-env \ |
| 37 | + --resource-group rg-data-agent \ |
| 38 | + --location eastus |
| 39 | +``` |
| 40 | + |
| 41 | +### 2. Create the Session Pool |
| 42 | + |
| 43 | +```bash |
| 44 | +az containerapp sessionpool create \ |
| 45 | + --name session-pool-viz \ |
| 46 | + --resource-group rg-data-agent \ |
| 47 | + --container-type PythonLTS \ |
| 48 | + --max-sessions 100 \ |
| 49 | + --cooldown-period 300 \ |
| 50 | + --location eastus |
| 51 | +``` |
| 52 | + |
| 53 | +**Parameters:** |
| 54 | +- `--container-type PythonLTS`: Python runtime with common data science packages |
| 55 | +- `--max-sessions`: Maximum concurrent sessions |
| 56 | +- `--cooldown-period`: Seconds before idle session is terminated |
| 57 | + |
| 58 | +### 3. Get the Pool Management Endpoint |
| 59 | + |
| 60 | +```bash |
| 61 | +az containerapp sessionpool show \ |
| 62 | + --name session-pool-viz \ |
| 63 | + --resource-group rg-data-agent \ |
| 64 | + --query "properties.poolManagementEndpoint" -o tsv |
| 65 | +``` |
| 66 | + |
| 67 | +This returns a URL like: |
| 68 | +``` |
| 69 | +https://eastus.dynamicsessions.io/subscriptions/<sub>/resourceGroups/<rg>/sessionPools/<pool> |
| 70 | +``` |
| 71 | + |
| 72 | +### 4. Assign the Executor Role |
| 73 | + |
| 74 | +Grant your identity permission to execute code in the session pool: |
| 75 | + |
| 76 | +```bash |
| 77 | +# Get your user ID |
| 78 | +USER_ID=$(az ad signed-in-user show --query id -o tsv) |
| 79 | + |
| 80 | +# Get the session pool resource ID |
| 81 | +POOL_ID=$(az containerapp sessionpool show \ |
| 82 | + --name session-pool-viz \ |
| 83 | + --resource-group rg-data-agent \ |
| 84 | + --query id -o tsv) |
| 85 | + |
| 86 | +# Assign the role |
| 87 | +az role assignment create \ |
| 88 | + --role "Azure ContainerApps Session Executor" \ |
| 89 | + --assignee $USER_ID \ |
| 90 | + --scope $POOL_ID |
| 91 | +``` |
| 92 | + |
| 93 | +**Note:** For service principals or managed identities, replace `$USER_ID` with the appropriate object ID. |
| 94 | + |
| 95 | +### 5. Install the SDK |
| 96 | + |
| 97 | +```bash |
| 98 | +pip install langchain-azure-dynamic-sessions |
| 99 | +``` |
| 100 | + |
| 101 | +Or add to your `pyproject.toml`: |
| 102 | +```toml |
| 103 | +dependencies = [ |
| 104 | + "langchain-azure-dynamic-sessions>=0.1.0", |
| 105 | +] |
| 106 | +``` |
| 107 | + |
| 108 | +## Configuration |
| 109 | + |
| 110 | +### Environment Variable |
| 111 | + |
| 112 | +Set the pool endpoint: |
| 113 | + |
| 114 | +```bash |
| 115 | +export AZURE_SESSIONS_POOL_ENDPOINT="https://eastus.dynamicsessions.io/subscriptions/.../sessionPools/..." |
| 116 | +``` |
| 117 | + |
| 118 | +Or in `.env`: |
| 119 | +```bash |
| 120 | +AZURE_SESSIONS_POOL_ENDPOINT=https://eastus.dynamicsessions.io/subscriptions/.../sessionPools/... |
| 121 | +``` |
| 122 | + |
| 123 | +### YAML Configuration |
| 124 | + |
| 125 | +Enable visualization in your agent config: |
| 126 | + |
| 127 | +```yaml |
| 128 | +data_agents: |
| 129 | + - name: "sales_agent" |
| 130 | + # ... other config ... |
| 131 | + code_interpreter: |
| 132 | + enabled: true |
| 133 | + azure_sessions_endpoint: ${AZURE_SESSIONS_POOL_ENDPOINT} |
| 134 | +``` |
| 135 | +
|
| 136 | +| Setting | Description | Default | |
| 137 | +|---------|-------------|---------| |
| 138 | +| `enabled` | Enable/disable visualization | `false` | |
| 139 | +| `azure_sessions_endpoint` | Session pool management endpoint URL | - | |
| 140 | + |
| 141 | +### System Prompt |
| 142 | + |
| 143 | +To enable visualization detection, include `visualization_requested` in your response format: |
| 144 | + |
| 145 | +```yaml |
| 146 | +system_prompt: | |
| 147 | + You are a SQL expert for the sales database. |
| 148 | +
|
| 149 | + {schema_context} |
| 150 | +
|
| 151 | + ## Response Format |
| 152 | +
|
| 153 | + Provide your response as JSON with these fields: |
| 154 | + - "thinking": Step-by-step reasoning about the query |
| 155 | + - "sql_query": The generated SQL query |
| 156 | + - "explanation": Brief explanation of what the query does |
| 157 | + - "visualization_requested": Set to true if the user asks for a chart, graph, plot, or visualization |
| 158 | +``` |
| 159 | + |
| 160 | +## How It Works |
| 161 | + |
| 162 | +``` |
| 163 | +┌─────────────────────────────────────────────────────────────────┐ |
| 164 | +│ User Query │ |
| 165 | +│ "Show me a bar chart of sales by region" │ |
| 166 | +└─────────────────────────────────────────┬───────────────────────┘ |
| 167 | + │ |
| 168 | + ▼ |
| 169 | +┌─────────────────────────────────────────────────────────────────┐ |
| 170 | +│ SQL Generation LLM │ |
| 171 | +│ Generates SQL + sets visualization_requested: true │ |
| 172 | +└─────────────────────────────────────────┬───────────────────────┘ |
| 173 | + │ |
| 174 | + ▼ |
| 175 | +┌─────────────────────────────────────────────────────────────────┐ |
| 176 | +│ Database Query │ |
| 177 | +│ Execute SQL, return result rows │ |
| 178 | +└─────────────────────────────────────────┬───────────────────────┘ |
| 179 | + │ |
| 180 | + ▼ |
| 181 | +┌─────────────────────────────────────────────────────────────────┐ |
| 182 | +│ Visualization LLM │ |
| 183 | +│ Generates matplotlib code based on data + user question │ |
| 184 | +└─────────────────────────────────────────┬───────────────────────┘ |
| 185 | + │ |
| 186 | + ▼ |
| 187 | +┌─────────────────────────────────────────────────────────────────┐ |
| 188 | +│ Azure Container Apps Dynamic Sessions │ |
| 189 | +│ • Code executed in Hyper-V isolated container │ |
| 190 | +│ • plt.show() output captured automatically │ |
| 191 | +│ • Image returned as base64 PNG │ |
| 192 | +└─────────────────────────────────────────┬───────────────────────┘ |
| 193 | + │ |
| 194 | + ▼ |
| 195 | +┌─────────────────────────────────────────────────────────────────┐ |
| 196 | +│ Response │ |
| 197 | +│ Text explanation + embedded chart image │ |
| 198 | +└─────────────────────────────────────────────────────────────────┘ |
| 199 | +``` |
| 200 | +
|
| 201 | +### Execution Flow |
| 202 | +
|
| 203 | +1. **Intent Detection**: The SQL LLM sets `visualization_requested: true` when it detects chart/graph/plot intent |
| 204 | +2. **SQL Execution**: Query runs against the database, returning structured data |
| 205 | +3. **Code Generation**: A second LLM call generates matplotlib code tailored to the data and question |
| 206 | +4. **Sandboxed Execution**: Code runs in Azure Sessions with automatic image capture |
| 207 | +5. **Response Assembly**: Text response and chart image are combined for display |
| 208 | +
|
| 209 | +## Example Queries |
| 210 | +
|
| 211 | +These prompts trigger visualization: |
| 212 | +
|
| 213 | +| Query | Chart Type | |
| 214 | +|-------|------------| |
| 215 | +| "Show me a bar chart of sales by region" | Bar chart | |
| 216 | +| "Visualize the top 10 customers by revenue" | Horizontal bar | |
| 217 | +| "Plot monthly revenue trends for 2024" | Line chart | |
| 218 | +| "Create a pie chart of transaction types" | Pie chart | |
| 219 | +| "Graph the distribution of order values" | Histogram | |
| 220 | +| "Compare Q1 vs Q2 performance" | Grouped bar | |
| 221 | +
|
| 222 | +## Further Reading |
| 223 | +
|
| 224 | +- [Azure Container Apps Dynamic Sessions](https://learn.microsoft.com/azure/container-apps/sessions) |
| 225 | +- [Session Pool Management](https://learn.microsoft.com/azure/container-apps/sessions-code-interpreter) |
| 226 | +- [LangChain Azure Dynamic Sessions](https://python.langchain.com/docs/integrations/tools/azure_dynamic_sessions) |
0 commit comments