Skip to content

Commit 40191e2

Browse files
committed
Refactor and enhance agent tasks and tools
- Updated task definitions in notebooks to use more descriptive field names (e.g., 'query' to 'question'). - Changed the default task in domain tuning notebook from "sentiment" to "plan_execute". - Improved agent behavior optimization by refining prompts and adding explanations for model choices. - Enhanced search functionality in tools to provide better ticket search results and counts. - Updated calculations for plan quality and self-correct accuracy to align with new output structures. - Added MIPROv2 optimization step to improve agent responses based on vague prompts. - Adjusted dataset for search agent to include more complex queries and answers. - Updated kernel specifications across notebooks to use Python 3.13.12.
1 parent 4a12acc commit 40191e2

10 files changed

Lines changed: 247 additions & 168 deletions

File tree

notebooks/03_domain_tuning.ipynb

Lines changed: 15 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@
8383
"outputs": [],
8484
"source": [
8585
"# Ändere den Task hier (z.B. \"sentiment\", \"math_word\", \"fact_verification\")\n",
86-
"TASK = \"sentiment\"\n",
86+
"TASK = \"plan_execute\"\n",
8787
"\n",
8888
"task = get_task(TASK)\n",
8989
"print(f\"📋 {task.name}\")\n",
@@ -102,73 +102,6 @@
102102
"cell_type": "markdown",
103103
"id": "6",
104104
"metadata": {},
105-
"source": [
106-
"## 3. Jetzt mit deinen echten Daten\n",
107-
"\n",
108-
"Generische Aufgaben wie Sentiment oder Mathe sind nett — aber der **echte Wert** zeigt sich bei **deinen eigenen Daten**.\n",
109-
"\n",
110-
"Wir laden echte Support-Tickets aus dem Projekt. Jedes Ticket hat:\n",
111-
"- **Zusammenfassung** — was ist das Problem?\n",
112-
"- **Kategorie** — Network, Software, Hardware, ...\n",
113-
"- **Priorität** — Critical, High, Medium, Low\n",
114-
"- **Team** — welches Team kümmert sich?\n",
115-
"\n",
116-
"Das Modell soll neue Tickets automatisch klassifizieren.\n"
117-
]
118-
},
119-
{
120-
"cell_type": "code",
121-
"execution_count": null,
122-
"id": "7",
123-
"metadata": {},
124-
"outputs": [],
125-
"source": [
126-
"task = get_task(\"ticket_routing\")\n",
127-
"examples = task.load_examples()\n",
128-
"\n",
129-
"print(f\"📊 {len(examples)} echte Tickets geladen\\n\")\n",
130-
"for ex in examples[:5]:\n",
131-
" print(f\" 📋 {str(ex.summary)[:80]}...\")\n",
132-
" print(f\" → Kategorie: {ex.category} | Priorität: {ex.priority} | Team: {ex.assigned_group}\")\n",
133-
" print()\n"
134-
]
135-
},
136-
{
137-
"cell_type": "markdown",
138-
"id": "8",
139-
"metadata": {},
140-
"source": [
141-
"## 4. Generischer Prompt → wie gut ist er?\n",
142-
"\n",
143-
"Erst testen wir ohne Tuning — nur ein generischer Prompt: \"Klassifiziere dieses Ticket.\"\n",
144-
"\n",
145-
"Die Metrik bewertet drei Felder gleichzeitig:\n",
146-
"- Priorität korrekt? → **40%** Gewicht (am wichtigsten!)\n",
147-
"- Kategorie korrekt? → **35%** Gewicht\n",
148-
"- Richtiges Team? → **25%** Gewicht\n"
149-
]
150-
},
151-
{
152-
"cell_type": "code",
153-
"execution_count": null,
154-
"id": "9",
155-
"metadata": {},
156-
"outputs": [],
157-
"source": [
158-
"print(f\"⏳ Teste generischen Prompt auf {MODEL}...\\n\")\n",
159-
"\n",
160-
"baseline = run_baseline(\"ticket_routing\", max_eval=10)\n",
161-
"\n",
162-
"display_score(\"Generischer Prompt (Baseline)\", baseline.score)\n",
163-
"display_results_table(baseline.individual_scores[:5])\n",
164-
"\n",
165-
"print(f\"\\n👆 {baseline.score:.0%} — das Modell kennt eure Teams und Kategorien nicht!\")\n"
166-
]
167-
},
168-
{
169-
"cell_type": "markdown",
170-
"id": "10",
171-
"metadata": {},
172105
"source": [
173106
"## 5. Domain-Tuning → der Unterschied\n",
174107
"\n",
@@ -178,7 +111,7 @@
178111
{
179112
"cell_type": "code",
180113
"execution_count": null,
181-
"id": "11",
114+
"id": "7",
182115
"metadata": {},
183116
"outputs": [],
184117
"source": [
@@ -192,7 +125,7 @@
192125
},
193126
{
194127
"cell_type": "markdown",
195-
"id": "12",
128+
"id": "8",
196129
"metadata": {},
197130
"source": [
198131
"## 6. Was du gerade gesehen hast\n",
@@ -214,7 +147,7 @@
214147
{
215148
"cell_type": "code",
216149
"execution_count": null,
217-
"id": "13",
150+
"id": "9",
218151
"metadata": {},
219152
"outputs": [],
220153
"source": [
@@ -226,7 +159,7 @@
226159
},
227160
{
228161
"cell_type": "markdown",
229-
"id": "14",
162+
"id": "10",
230163
"metadata": {},
231164
"source": [
232165
"## ⏭️ Weiter geht's!\n",
@@ -239,13 +172,21 @@
239172
],
240173
"metadata": {
241174
"kernelspec": {
242-
"display_name": "Python 3",
175+
"display_name": ".venv (3.13.12)",
243176
"language": "python",
244177
"name": "python3"
245178
},
246179
"language_info": {
180+
"codemirror_mode": {
181+
"name": "ipython",
182+
"version": 3
183+
},
184+
"file_extension": ".py",
185+
"mimetype": "text/x-python",
247186
"name": "python",
248-
"version": "3.11.0"
187+
"nbconvert_exporter": "python",
188+
"pygments_lexer": "ipython3",
189+
"version": "3.13.12"
249190
}
250191
},
251192
"nbformat": 4,

0 commit comments

Comments
 (0)