Skip to content

Commit f7ecda0

Browse files
jpmc-githubjpmc-github
authored
Merge pull request #92 from sandeepkhot/DATA_QUALITY_AGENT
Data quality agent & DBMS Scheduler agent
2 parents 890ca38 + 1cfe2ce commit f7ecda0

6 files changed

Lines changed: 3534 additions & 0 deletions

File tree

‎autonomous-ai-agents/data_quality_agent/README.md‎

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
# Select AI - Data Quality Check Agent for Oracle Autonomous Database
2+
3+
## Release Metadata
4+
5+
- Release Version: `1.1`
6+
- Release Date: `19-May-2026`
7+
8+
## Overview
9+
10+
The **Data Quality Check Agent** provides schema-aware data quality assessment for Oracle Autonomous Database tables using Select AI Agent tools.
11+
12+
It supports:
13+
14+
- Table profiling
15+
- Null/duplicate/outlier detection
16+
- Quality score computation with history tracking
17+
- Drift detection based on recent vs baseline score windows
18+
- Issue listing with severity and remediation guidance
19+
- Safe remediation preview and controlled apply mode
20+
- OML Services monitoring setup and run trigger hooks
21+
22+
For definitions of **Tool**, **Task**, **Agent**, and **Agent Team**, see the top-level guide: [README](../README.md#simple-agent-execution-flow).
23+
24+
---
25+
26+
## Repository Contents
27+
28+
```text
29+
.
30+
├── database_quality_check_tools.sql
31+
│ ├── Installer bootstrap and grants
32+
│ ├── DATABASE_QUALITY package (core DQ logic)
33+
│ ├── SELECT_AI_DATA_QUALITY_AGENT package (tool wrappers)
34+
│ └── Tool registration
35+
│
36+
├── database_quality_check_agent.sql
37+
│ ├── Task definition (DATA_QUALITY_TASKS)
38+
│ ├── Agent creation (DATA_QUALITY_ADVISOR)
39+
│ ├── Team creation (DATA_QUALITY_TEAM)
40+
│ └── Default target schema behavior (DQ_TARGET_SCHEMA)
41+
│
42+
└── README.md
43+
```
44+
45+
---
46+
47+
## Architecture Overview
48+
49+
```text
50+
User Request
51+
↓
52+
DATA_QUALITY_TASKS
53+
↓
54+
DATA_QUALITY_ADVISOR Reasoning
55+
├── PROFILE_TABLE_TOOL
56+
├── DETECT_NULLS_TOOL
57+
├── DETECT_DUPLICATES_TOOL
58+
├── DETECT_OUTLIERS_TOOL
59+
├── DETECT_DRIFT_TOOL
60+
├── GENERATE_QUALITY_RULES_TOOL
61+
├── EVALUATE_QUALITY_SCORE_TOOL
62+
├── LIST_QUALITY_ISSUES_TOOL
63+
├── SUGGEST_REMEDIATION_TOOL
64+
├── APPLY_REMEDIATION_TOOL
65+
├── SETUP_OML_DATA_MONITORING_TOOL
66+
└── RUN_OML_DATA_MONITORING_TOOL
67+
↓
68+
Issue Summary + Severity + Quality Score + Next Action
69+
```
70+
71+
---
72+
73+
## Prerequisites
74+
75+
- Oracle Autonomous AI Database (26ai recommended)
76+
- Select AI and `DBMS_CLOUD_AI_AGENT` enabled
77+
- `ADMIN` (or equivalent privileged user) for installation
78+
- A valid AI profile (`DBMS_CLOUD_AI.CREATE_PROFILE`)
79+
- Object privileges from install schema to target data schema tables (for cross-schema checks)
80+
81+
For OML monitoring tools:
82+
83+
- `SELECTAI_AGENT_CONFIG` entries for `AGENT='DATA_QUALITY'`:
84+
- `OML_MONITORING_ENDPOINT`
85+
- `OML_MONITORING_CREDENTIAL`
86+
87+
For controlled remediation apply:
88+
89+
- `SELECTAI_AGENT_CONFIG` entry for `AGENT='DATA_QUALITY'`:
90+
- `REMEDIATION_APPROVAL_CODE`
91+
92+
---
93+
94+
## Installation
95+
96+
Run as `ADMIN` (or privileged user) from this folder:
97+
98+
```sql
99+
sqlplus admin@<adb_connect_string> @database_quality_check_tools.sql
100+
sqlplus admin@<adb_connect_string> @database_quality_check_agent.sql
101+
```
102+
103+
Prompts in tools script:
104+
105+
- `SCHEMA_NAME` (schema where package/tools are installed)
106+
107+
Prompts in agent script:
108+
109+
- `SCHEMA_NAME` (same install schema)
110+
- `AI_PROFILE_NAME`
111+
- `DQ_TARGET_SCHEMA` (default schema for DQ checks; if blank uses `SCHEMA_NAME`)
112+
113+
Important:
114+
115+
- Re-run `database_quality_check_agent.sql` whenever task instructions are changed.
116+
117+
---
118+
119+
## Internal Tables
120+
121+
Created in install schema (if missing):
122+
123+
- `DQ_RUN_HISTORY$`:
124+
- score history per table/run
125+
- stores score component metrics and worst severity
126+
- `DQ_FINDINGS$`:
127+
- issue registry with severity, recommendation, and optional fix SQL
128+
- `DQ_OML_MONITORS$`:
129+
- registered OML monitor metadata and last run response
130+
131+
---
132+
133+
## Tool-to-Function Mapping
134+
135+
| Tool | Function | Purpose |
136+
|---|---|---|
137+
| `PROFILE_TABLE_TOOL` | `select_ai_data_quality_agent.profile_table` | Baseline profile |
138+
| `DETECT_NULLS_TOOL` | `select_ai_data_quality_agent.detect_nulls` | Null issue detection |
139+
| `DETECT_DUPLICATES_TOOL` | `select_ai_data_quality_agent.detect_duplicates` | Duplicate detection |
140+
| `DETECT_OUTLIERS_TOOL` | `select_ai_data_quality_agent.detect_outliers` | Outlier detection |
141+
| `DETECT_DRIFT_TOOL` | `select_ai_data_quality_agent.detect_drift` | Drift analysis |
142+
| `GENERATE_QUALITY_RULES_TOOL` | `select_ai_data_quality_agent.generate_quality_rules` | Rule suggestions |
143+
| `EVALUATE_QUALITY_SCORE_TOOL` | `select_ai_data_quality_agent.evaluate_quality_score` | Score + persistence |
144+
| `LIST_QUALITY_ISSUES_TOOL` | `select_ai_data_quality_agent.list_quality_issues` | Issue review |
145+
| `SUGGEST_REMEDIATION_TOOL` | `select_ai_data_quality_agent.suggest_remediation` | SQL guidance |
146+
| `APPLY_REMEDIATION_TOOL` | `select_ai_data_quality_agent.apply_remediation` | Preview/apply fix SQL |
147+
| `SETUP_OML_DATA_MONITORING_TOOL` | `select_ai_data_quality_agent.setup_oml_data_monitoring` | Register OML monitor |
148+
| `RUN_OML_DATA_MONITORING_TOOL` | `select_ai_data_quality_agent.run_oml_data_monitoring` | Trigger OML monitor run |
149+
150+
---
151+
152+
## Operational Behavior
153+
154+
- If `owner_name` is omitted, agent defaults to `DQ_TARGET_SCHEMA`.
155+
- For schema-wide requests (for example, “all tables”), task instruction is configured to auto-discover tables and not ask user to list table names.
156+
- `APPLY_REMEDIATION_TOOL`:
157+
- default mode is `PREVIEW`
158+
- `APPLY` requires matching `approval_code`
159+
- SQL safety checks block unsafe statements
160+
161+
---
162+
163+
## Example Prompts
164+
165+
- `Check null issues in SALES and show columns with null_count, null_rate_pct, and severity.`
166+
- `Detect duplicates in SALES using all columns and show duplicate_row_count, duplicate_rate_pct, and severity.`
167+
- `Find numeric outliers in SALES using z-score threshold 3 and rank by severity.`
168+
- `Evaluate quality score for SALES and explain null, duplicate, outlier, and drift components.`
169+
- `Evaluate quality score for every table in the default target schema and return table-wise summary.`
170+
- `List open HIGH severity quality issues for SALES with recommendation and generated_fix_sql.`
171+
- `Preview remediation for issue_id 1 on SALES.`
172+
- `Apply remediation for issue_id 1 on SALES with execute_mode APPLY and approval_code <code>.`
173+
174+
OML examples:
175+
176+
- `Set up OML data monitoring for SH.SALES with monitor name SH_SALES_DQ_MON, baseline query "<baseline_sql>", and new-data query "<new_sql>".`
177+
- `Run OML data monitoring for monitor SH_SALES_DQ_MON and return the job response.`
178+
179+
---
180+
181+
## Troubleshooting
182+
183+
- `ORA-00942` during package compilation:
184+
- Re-run `database_quality_check_tools.sql`; it pre-creates `DQ_*` tables.
185+
- Agent asks for table list during “all tables” request:
186+
- Re-run `database_quality_check_agent.sql` to recreate task with latest instructions.
187+
- OML monitoring tool errors with missing config:
188+
- Insert required keys in `SELECTAI_AGENT_CONFIG` for `AGENT='DATA_QUALITY'`.
189+
- Apply remediation blocked:
190+
- Ensure `REMEDIATION_APPROVAL_CODE` is configured and supplied as `approval_code`.

‎autonomous-ai-agents/data_quality_agent/database_quality_check_agent.sql‎

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
rem ============================================================================
2+
rem LICENSE
3+
rem Copyright (c) 2026 Oracle and/or its affiliates.
4+
rem Licensed under the Universal Permissive License (UPL), Version 1.0
5+
rem https://oss.oracle.com/licenses/upl/
6+
rem
7+
rem NAME
8+
rem database_quality_check_agent.sql
9+
rem
10+
rem DESCRIPTION
11+
rem Installer and configuration script for Data Quality Check AI Agent Team.
12+
rem
13+
rem RELEASE VERSION
14+
rem 1.0
15+
rem
16+
rem RELEASE DATE
17+
rem 18-May-2026
18+
rem ============================================================================
19+
20+
SET SERVEROUTPUT ON
21+
SET VERIFY OFF
22+
23+
PROMPT ======================================================
24+
PROMPT Data Quality Check AI Agent Installer
25+
PROMPT ======================================================
26+
27+
VAR v_schema VARCHAR2(128)
28+
EXEC :v_schema := '&SCHEMA_NAME';
29+
30+
VAR v_ai_profile_name VARCHAR2(128)
31+
EXEC :v_ai_profile_name := '&AI_PROFILE_NAME';
32+
33+
PROMPT
34+
PROMPT DQ_TARGET_SCHEMA:
35+
PROMPT Schema to inspect by default for data quality checks.
36+
PROMPT If blank, SCHEMA_NAME is used as default.
37+
PROMPT
38+
39+
VAR v_dq_target_schema VARCHAR2(128)
40+
EXEC :v_dq_target_schema := '&DQ_TARGET_SCHEMA';
41+
42+
DECLARE
43+
l_sql VARCHAR2(500);
44+
l_schema VARCHAR2(128);
45+
l_session_user VARCHAR2(128);
46+
BEGIN
47+
l_schema := DBMS_ASSERT.SIMPLE_SQL_NAME(:v_schema);
48+
l_session_user := SYS_CONTEXT('USERENV', 'SESSION_USER');
49+
50+
IF UPPER(l_schema) <> UPPER(l_session_user) THEN
51+
l_sql := 'GRANT EXECUTE ON DBMS_CLOUD_AI_AGENT TO ' || l_schema;
52+
EXECUTE IMMEDIATE l_sql;
53+
54+
l_sql := 'GRANT EXECUTE ON DBMS_CLOUD_AI TO ' || l_schema;
55+
EXECUTE IMMEDIATE l_sql;
56+
57+
l_sql := 'GRANT EXECUTE ON DBMS_CLOUD TO ' || l_schema;
58+
EXECUTE IMMEDIATE l_sql;
59+
ELSE
60+
DBMS_OUTPUT.PUT_LINE('Skipping grants for schema ' || l_schema ||
61+
' (same as session user).');
62+
END IF;
63+
64+
DBMS_OUTPUT.PUT_LINE('Grants completed.');
65+
END;
66+
/
67+
68+
BEGIN
69+
EXECUTE IMMEDIATE 'ALTER SESSION SET CURRENT_SCHEMA = ' || :v_schema;
70+
END;
71+
/
72+
73+
CREATE OR REPLACE PROCEDURE install_data_quality_check_agent(
74+
p_install_schema IN VARCHAR2,
75+
p_profile_name IN VARCHAR2,
76+
p_dq_target_schema IN VARCHAR2
77+
)
78+
AUTHID DEFINER
79+
AS
80+
l_target_schema VARCHAR2(128);
81+
BEGIN
82+
l_target_schema := UPPER(TRIM(NVL(p_dq_target_schema, p_install_schema)));
83+
84+
DBMS_OUTPUT.PUT_LINE('--------------------------------------------');
85+
DBMS_OUTPUT.PUT_LINE('Starting Data Quality Check AI installation');
86+
DBMS_OUTPUT.PUT_LINE('--------------------------------------------');
87+
88+
BEGIN
89+
DBMS_CLOUD_AI_AGENT.DROP_TASK('DATA_QUALITY_TASKS');
90+
EXCEPTION
91+
WHEN OTHERS THEN
92+
NULL;
93+
END;
94+
95+
DBMS_CLOUD_AI_AGENT.CREATE_TASK(
96+
task_name => 'DATA_QUALITY_TASKS',
97+
description => 'Task for data quality profiling, scoring, and remediation planning',
98+
attributes => '{
99+
"instruction": "You are a Data Quality specialist for Oracle Autonomous Database. '
100+
|| 'Default target schema for data quality checks is ' || l_target_schema || '. '
101+
|| 'If the user does not provide owner_name, use owner_name=' || l_target_schema || '. '
102+
|| 'If the user provides a different schema explicitly, use that schema. '
103+
|| 'Cross-schema analysis is allowed when object privileges are granted to the install schema. '
104+
|| 'When the user asks for all tables or schema-wide analysis, automatically discover table names from the target schema and run checks without asking the user to provide table lists. '
105+
|| 'If no owner_name is provided in schema-wide requests, use owner_name=' || l_target_schema || '. '
106+
|| 'Do not ask the user for table names when this can be derived from ALL_TABLES/USER_TABLES metadata. '
107+
|| 'Use PROFILE_TABLE_TOOL first to establish table baseline when user provides owner/table. '
108+
|| 'Use DETECT_NULLS_TOOL, DETECT_DUPLICATES_TOOL, and DETECT_OUTLIERS_TOOL to identify quality issues with severity. '
109+
|| 'Use GENERATE_QUALITY_RULES_TOOL to propose enforceable quality rules. '
110+
|| 'Use EVALUATE_QUALITY_SCORE_TOOL to compute/store overall quality score and history point. '
111+
|| 'Use DETECT_DRIFT_TOOL to identify recent score drift against baseline history. '
112+
|| 'Use SETUP_OML_DATA_MONITORING_TOOL for automated OML Services data monitoring setup when requested. '
113+
|| 'Use RUN_OML_DATA_MONITORING_TOOL to trigger OML monitoring jobs and report run response. '
114+
|| 'Use LIST_QUALITY_ISSUES_TOOL for issue review. '
115+
|| 'Use SUGGEST_REMEDIATION_TOOL to produce practical SQL-based fixes. '
116+
|| 'Only use APPLY_REMEDIATION_TOOL in PREVIEW mode unless the user explicitly asks to apply changes and provides approval_code. '
117+
|| 'Always return: issue summary, severity, quality score, and next remediation step. '
118+
|| 'User request: {query}",
119+
"tools": [
120+
"PROFILE_TABLE_TOOL",
121+
"DETECT_NULLS_TOOL",
122+
"DETECT_DUPLICATES_TOOL",
123+
"DETECT_OUTLIERS_TOOL",
124+
"DETECT_DRIFT_TOOL",
125+
"SETUP_OML_DATA_MONITORING_TOOL",
126+
"RUN_OML_DATA_MONITORING_TOOL",
127+
"GENERATE_QUALITY_RULES_TOOL",
128+
"EVALUATE_QUALITY_SCORE_TOOL",
129+
"LIST_QUALITY_ISSUES_TOOL",
130+
"SUGGEST_REMEDIATION_TOOL",
131+
"APPLY_REMEDIATION_TOOL"
132+
],
133+
"enable_human_tool": "true"
134+
}'
135+
);
136+
DBMS_OUTPUT.PUT_LINE('Created task DATA_QUALITY_TASKS');
137+
138+
BEGIN
139+
DBMS_CLOUD_AI_AGENT.DROP_AGENT('DATA_QUALITY_ADVISOR');
140+
EXCEPTION
141+
WHEN OTHERS THEN
142+
NULL;
143+
END;
144+
145+
DBMS_CLOUD_AI_AGENT.CREATE_AGENT(
146+
agent_name => 'DATA_QUALITY_ADVISOR',
147+
attributes =>
148+
'{' ||
149+
'"profile_name":"' || p_profile_name || '",' ||
150+
'"role":"You are a Data Quality Advisor. You profile data, detect anomalies and drift, compute quality scores, and recommend safe remediation steps for Oracle Autonomous Database tables."' ||
151+
'}',
152+
description => 'AI agent for Oracle Autonomous Database data quality monitoring and remediation guidance'
153+
);
154+
DBMS_OUTPUT.PUT_LINE('Created agent DATA_QUALITY_ADVISOR');
155+
156+
BEGIN
157+
DBMS_CLOUD_AI_AGENT.DROP_TEAM('DATA_QUALITY_TEAM');
158+
EXCEPTION
159+
WHEN OTHERS THEN
160+
NULL;
161+
END;
162+
163+
DBMS_CLOUD_AI_AGENT.CREATE_TEAM(
164+
team_name => 'DATA_QUALITY_TEAM',
165+
attributes => '{
166+
"agents":[{"name":"DATA_QUALITY_ADVISOR","task":"DATA_QUALITY_TASKS"}],
167+
"process":"sequential"
168+
}'
169+
);
170+
171+
DBMS_OUTPUT.PUT_LINE('Created team DATA_QUALITY_TEAM');
172+
DBMS_OUTPUT.PUT_LINE('--------------------------------------------');
173+
DBMS_OUTPUT.PUT_LINE('Data Quality Check AI installation COMPLETE');
174+
DBMS_OUTPUT.PUT_LINE('--------------------------------------------');
175+
END install_data_quality_check_agent;
176+
/
177+
178+
PROMPT Executing installer procedure ...
179+
BEGIN
180+
install_data_quality_check_agent(
181+
p_install_schema => :v_schema,
182+
p_profile_name => :v_ai_profile_name,
183+
p_dq_target_schema => :v_dq_target_schema
184+
);
185+
END;
186+
/
187+
188+
ALTER SESSION SET CURRENT_SCHEMA = ADMIN;
189+
190+
PROMPT ======================================================
191+
PROMPT Installation finished successfully
192+
PROMPT ======================================================

0 commit comments

Comments
 (0)