Add agent discovery symlinks, HelpOnErrorParser, PEP 723 deps, DefaultAzureCredential

* Add agent discovery symlinks for fine-tuning skill

Add symlinks so coding agents (Copilot, Claude, Codex) can auto-discover
the fine-tuning skill from their conventional paths:

- .github/skills/azure-ai-fine-tuning -> ../../Skills
- .claude/skills/azure-ai-fine-tuning -> ../../Skills
- .agents/skills/azure-ai-fine-tuning -> ../../Skills

This reuses the existing comprehensive skill (SFT, DPO, RFT with graders
and tools, 8 scripts, 11 reference docs, 5 workflows) rather than creating
a separate thinner skill.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add HelpOnErrorParser, PEP 723 deps, DefaultAzureCredential, and AI Agent Skills docs

Adopts three patterns from PR #27:

1. HelpOnErrorParser in common.py — prints full --help on invalid args
2. PEP 723 inline script metadata (# /// script) on all scripts —
   enables 'uv run scripts/submit_training.py' with auto-installed deps
3. DefaultAzureCredential fallback in get_clients() — works without
   API key when az CLI or Managed Identity is available

Also adds AI Agent Skills section to README with usage instructions
for Copilot (VS Code + CLI), Claude Code, and Codex agents.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix: scripts now use HelpOnErrorParser, fix credential fallback bug

- All 7 scripts now import and use HelpOnErrorParser from common.py
  (previously added the class but scripts still used argparse.ArgumentParser)
- Fix bug in DefaultAzureCredential fallback: when base_url is set but
  api_key is None and credential fails, no longer creates client with
  api_key=None (which would fail silently on first API call)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix: DPO system message bug, cross-dir import path, dedup imports

- convert_dataset.py: Fix DPO generation to include system messages when
  generating non-preferred responses from base model (was generating without
  system prompt but recording it in the DPO input — distributional mismatch)
- All scripts: Add sys.path.insert for common.py import to work when
  scripts are run from outside the scripts/ directory
- Clean up duplicate os/sys imports

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: AliciaFrame

.agents/skills/azure-ai-fine-tuning

@@ -0,0 +1 @@
+../../Skills

.claude/skills/azure-ai-fine-tuning

@@ -0,0 +1 @@
+../../Skills

.github/skills/azure-ai-fine-tuning

@@ -0,0 +1 @@
+../../Skills

README.md

@@ -7,6 +7,7 @@ This repository contains **12 end-to-end demos** and **sample datasets** for fin
 - [Quick Start](#-quick-start)
 - [Demos](#-demos)
 - [Sample Datasets](#-sample-datasets)
+- [AI Agent Skills](#-ai-agent-skills)
 - [Prerequisites](#-prerequisites)
 - [Contributing](#contributing)
 
@@ -66,6 +67,52 @@ Ready-to-use datasets for testing fine-tuning techniques in the **[Sample_Datase
 
 ---
 
+## 🤖 AI Agent Skills
+
+This repo includes a fine-tuning skill that coding agents can auto-discover and use to help you submit, monitor, and evaluate fine-tuning jobs.
+
+| Agent | Skill Path | Auto-discovery |
+|-------|-----------|----------------|
+| **GitHub Copilot** (VS Code / CLI) | [.github/skills/azure-ai-fine-tuning](.github/skills/azure-ai-fine-tuning) | ✅ Automatic |
+| **Claude Code** | [.claude/skills/azure-ai-fine-tuning](.claude/skills/azure-ai-fine-tuning) | ✅ Automatic |
+| **Codex / other agents** | [.agents/skills/azure-ai-fine-tuning](.agents/skills/azure-ai-fine-tuning) | ✅ Automatic |
+
+All three paths are symlinks to the canonical skill at **[Skills/](Skills/)**, which includes:
+- **SKILL.md** — Agent instructions covering SFT, DPO, and RFT workflows
+- **8 scripts** — submit, monitor, deploy, evaluate, validate, score, convert, distill
+- **11 reference docs** — hyperparameters, dataset formats, agentic RFT, cost management, and more
+- **5 guided workflows** — full pipeline, dataset creation, iterative training, diagnosis
+- **Sample data** — SFT, DPO, and RFT example JSONL files
+
+### Using with GitHub Copilot (VS Code)
+
+1. Open this repo in VS Code with Copilot Chat enabled.
+2. Ask a fine-tuning task (e.g., *"help me submit an SFT job with my dataset"*).
+3. Copilot auto-discovers the skill from `.github/skills/` and follows the workflow.
+
+### Using with Copilot CLI
+
+```bash
+cd /path/to/this/repo
+copilot
+# Then ask: "Submit an SFT fine-tuning job with my training data"
+```
+
+### Using with Claude Code
+
+```bash
+cd /path/to/this/repo
+claude
+# Then ask: "Fine-tune gpt-4.1-mini on my dataset"
+```
+
+Scripts support `uv` for zero-setup execution (PEP 723 inline dependencies):
+```bash
+uv run Skills/scripts/submit_training.py --help
+```
+
+---
+
 ## ✅ Prerequisites
 
 Before running any demo, ensure you have:

Skills/scripts/check_training.py

@@ -1,3 +1,10 @@
+# /// script
+# dependencies = [
+#   "openai>=1.0",
+#   "azure-identity",
+#   "azure-ai-projects",
+# ]
+# ///
 """
 check_training.py — Analyze training curves, detect overfitting, list checkpoints.
 
@@ -13,6 +20,8 @@
 import json
 import os
 import sys
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from common import HelpOnErrorParser
 
 import openai
 
@@ -160,7 +169,7 @@ def analyze_job(client, job_id, download_csv=None):
 
 
 def main():
-    parser = argparse.ArgumentParser(description="Analyze fine-tuning training curves")
+    parser = HelpOnErrorParser(description="Analyze fine-tuning training curves")
     parser.add_argument("--base-url", default=os.environ.get("OPENAI_BASE_URL"),
                         help="Project /v1/ URL (preferred). Uses openai.OpenAI().")
     parser.add_argument("--endpoint", default=os.environ.get("AZURE_OPENAI_ENDPOINT"),

Skills/scripts/common.py

@@ -1,10 +1,10 @@
 """
 common.py — Shared Azure AI Foundry authentication and client setup.
 
-Adapted from foundry-ft agent's common.py, with our enhancements:
-- Supports /v1/ project endpoint (preferred) and Azure endpoint (fallback)
-- REST API fallback for OSS models
-- No .env dependency — works with az CLI tokens directly
+Supports three connection methods in order of preference:
+1. /v1/ project endpoint (simplest, preferred)
+2. Foundry SDK with DefaultAzureCredential (no API key needed, cloud-native)
+3. Azure OpenAI endpoint (classic)
 
 Usage:
     from common import get_clients, upload_file
@@ -13,17 +13,30 @@
     clients = get_clients(base_url="https://<resource>.services.ai.azure.com/api/projects/<project>/openai/v1/",
                           api_key="KEY")
 
-    # Method 2: Foundry SDK
+    # Method 2: Foundry SDK (DefaultAzureCredential — no API key needed)
     clients = get_clients(project_endpoint="https://<resource>.services.ai.azure.com/api/projects/<project>")
 
     # Method 3: Azure OpenAI endpoint
     clients = get_clients(azure_endpoint="https://<resource>.openai.azure.com",
                           api_key="KEY")
 """
+import argparse
 import os
 import sys
 
 
+class HelpOnErrorParser(argparse.ArgumentParser):
+    """ArgumentParser that prints full help when arguments are invalid.
+    
+    Standard ArgumentParser only prints a one-line usage summary on error,
+    which isn't helpful for first-time users. This prints the full --help.
+    """
+
+    def error(self, message):
+        self.print_help(sys.stderr)
+        self.exit(2, f"\nerror: {message}\n")
+
+
 def get_clients(base_url=None, azure_endpoint=None, project_endpoint=None, api_key=None):
     """Initialize and return OpenAI-compatible client.
 
@@ -40,9 +53,22 @@ def get_clients(base_url=None, azure_endpoint=None, project_endpoint=None, api_k
 
     if base_url:
         import openai
-        client = openai.OpenAI(base_url=base_url, api_key=api_key)
-        print(f"✅ Connected via /v1/ project endpoint")
-        return client, "project-v1"
+        # If no API key, try DefaultAzureCredential for token-based auth
+        if not api_key:
+            try:
+                from azure.identity import DefaultAzureCredential
+                credential = DefaultAzureCredential()
+                token = credential.get_token("https://cognitiveservices.azure.com/.default")
+                client = openai.OpenAI(base_url=base_url, api_key=token.token)
+                print(f"✅ Connected via /v1/ project endpoint (DefaultAzureCredential)")
+                return client, "project-v1-aad"
+            except Exception as e:
+                print(f"⚠️ No API key and DefaultAzureCredential failed: {e}")
+                # Fall through to Method 2/3
+        else:
+            client = openai.OpenAI(base_url=base_url, api_key=api_key)
+            print(f"✅ Connected via /v1/ project endpoint")
+            return client, "project-v1"
 
     # Method 2: Foundry SDK
     project_endpoint = project_endpoint or os.environ.get("AZURE_AI_PROJECT_ENDPOINT")

Skills/scripts/convert_dataset.py

@@ -1,3 +1,8 @@
+# /// script
+# dependencies = [
+#   "openai>=1.0",
+# ]
+# ///
 """
 convert_dataset.py — Convert between SFT, DPO, and RFT dataset formats.
 
@@ -22,6 +27,8 @@
 import os
 import sys
 import time
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from common import HelpOnErrorParser
 
 
 def parquet_to_sft(input_path, output_path, user_col, assistant_col, system_prompt=None):
@@ -92,7 +99,7 @@ def sft_to_dpo(input_path, output_path, endpoint, api_key, base_model):
 
             # Generate a non-preferred response from the base model
             try:
-                gen_msgs = [user_msg]
+                gen_msgs = system_msgs + [user_msg]
                 resp = client.chat.completions.create(
                     model=base_model,
                     messages=gen_msgs,
@@ -154,7 +161,7 @@ def dpo_to_sft(input_path, output_path, system_prompt=None):
 
 
 def main():
-    parser = argparse.ArgumentParser(description="Convert between fine-tuning dataset formats")
+    parser = HelpOnErrorParser(description="Convert between fine-tuning dataset formats")
     parser.add_argument("--input", required=True, help="Input file path")
     parser.add_argument("--output", required=True, help="Output file path")
     parser.add_argument("--format", required=True,

Skills/scripts/deploy_model.py

@@ -1,3 +1,9 @@
+# /// script
+# dependencies = [
+#   "openai>=1.0",
+#   "azure-identity",
+# ]
+# ///
 """
 deploy_model.py — Deploy fine-tuned models on Azure AI Foundry via ARM REST API.
 
@@ -16,6 +22,8 @@
 import subprocess
 import sys
 import time
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from common import HelpOnErrorParser
 
 import requests
 
@@ -167,7 +175,7 @@ def list_deployments(sub, rg, account):
 
 
 def main():
-    parser = argparse.ArgumentParser(description="Deploy fine-tuned models on Azure AI Foundry")
+    parser = HelpOnErrorParser(description="Deploy fine-tuned models on Azure AI Foundry")
     parser.add_argument("--sub", default=DEFAULT_SUB, help="Azure subscription ID")
     parser.add_argument("--rg", default=DEFAULT_RG, help="Resource group")
     parser.add_argument("--account", default=DEFAULT_ACCOUNT, help="Cognitive Services account")

Skills/scripts/evaluate_model.py

@@ -1,3 +1,9 @@
+# /// script
+# dependencies = [
+#   "openai>=1.0",
+#   "azure-identity",
+# ]
+# ///
 """
 evaluate_model.py — Custom 2-dimension LLM judge evaluator for fine-tuned models.
 
@@ -30,6 +36,8 @@
 import sys
 import time
 from concurrent.futures import ThreadPoolExecutor, as_completed
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from common import HelpOnErrorParser
 
 import openai
 
@@ -149,7 +157,7 @@ def grade_response(judge_client, judge_model, prompt, reference, output, max_ret
 
 
 def main():
-    parser = argparse.ArgumentParser(description="Evaluate a fine-tuned model with LLM judge")
+    parser = HelpOnErrorParser(description="Evaluate a fine-tuned model with LLM judge")
     parser.add_argument("--base-url", default=os.environ.get("OPENAI_BASE_URL"),
                         help="Project /v1/ URL (preferred). Uses openai.OpenAI().")
     parser.add_argument("--endpoint", default=os.environ.get("AZURE_OPENAI_ENDPOINT"),

Skills/scripts/generate_distillation_data.py

@@ -1,3 +1,9 @@
+# /// script
+# dependencies = [
+#   "openai>=1.0",
+#   "azure-identity",
+# ]
+# ///
 """
 generate_distillation_data.py — Generate training data from a teacher model for distillation.
 
@@ -31,6 +37,8 @@
 import re
 import sys
 import time
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from common import HelpOnErrorParser
 
 import openai
 
@@ -123,7 +131,7 @@ def grade_output(client, judge_model, output, retries=3):
 
 
 def main():
-    parser = argparse.ArgumentParser(description="Generate distillation training data from a teacher model")
+    parser = HelpOnErrorParser(description="Generate distillation training data from a teacher model")
     parser.add_argument("--endpoint", default=os.environ.get("AZURE_OPENAI_ENDPOINT"))
     parser.add_argument("--api-key", default=os.environ.get("AZURE_OPENAI_API_KEY"))
     parser.add_argument("--teacher", required=True, help="Teacher model deployment name")