feat(sagemaker-ai): update skills and bump version to 1.1.0 (#144)

SageMakerAIAgentSkills (mainline ee2105d):
- finetuning: add continuous customization reference, EULA fixes, RLVR notebook UX improvements
- finetuning-setup: add benchmark-based model selection + 8 benchmark references + model-licenses
- model-evaluation: add custom scorer workflow, reward function creation, MLflow support;
  rename notebook_cells.py -> llmaaj_evaluator.py; add nova/oss reward function templates
- model-deployment: EULA and deploy script fixes, model-licenses reference
- dataset-transformation: notebook structure/writing guide updates
- planning: scope activation to SageMaker model customization domain

X-AI-Prompt: Sync SageMakerAIAgentSkills mainline into awslabs/agent-plugins sagemaker-ai plugin and bump version to 1.1.0
X-AI-Tool: Kiro CLI

Co-authored-by: Alain Krok <alkrok@amazon.com>

Author: pilgd-aws

.claude-plugin/marketplace.json

@@ -161,7 +161,7 @@
       "name": "sagemaker-ai",
       "source": "./plugins/sagemaker-ai",
       "tags": ["aws", "sagemaker", "machine-learning", "generative-ai"],
-      "version": "1.0.0"
+      "version": "1.1.0"
     }
   ]
 }

plugins/sagemaker-ai/.claude-plugin/plugin.json

@@ -18,5 +18,5 @@
   "license": "Apache-2.0",
   "name": "sagemaker-ai",
   "repository": "https://github.com/awslabs/agent-plugins",
-  "version": "1.0.0"
+  "version": "1.1.0"
 }

plugins/sagemaker-ai/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "sagemaker-ai",
-  "version": "1.0.0",
+  "version": "1.1.0",
   "description": "Build, train, and deploy AI models with deep AWS AI/ML expertise brought directly into your coding assistants, covering the surface area of Amazon SageMaker AI.",
   "author": {
     "name": "Amazon Web Services",

plugins/sagemaker-ai/skills/dataset-transformation/SKILL.md

@@ -22,11 +22,9 @@ Transforms a data set provided by the user into their desired format. All transf
 3. **Don't read files until you need them.** Only read reference files when you've reached the workflow step that requires them and the user has confirmed the direction. Never read ahead.
 4. **No narration.** Don't explain what you're about to do or what you just did. Share outcomes and ask questions. Keep responses short and focused.
 5. **No repetition.** If you said something before a tool call, don't repeat it after. Only share new information.
-6. **Do not deviate from the Workflow.** The steps listed in the workflow should be followed exactly as described. Progress from Step 1 to Step 10 to complete the task. Do not deviate from the workflow!
+6. **Do not deviate from the Workflow.** The steps listed in the workflow should be followed exactly as described. Progress from Step 1 to Step 11 to complete the task. Do not deviate from the workflow!
 7. **Always end with a question.** Whenever you pause for user input, acknowledgment, or feedback, your response must end with a question. Never leave the user with a statement and expect them to know they need to respond.
-8. **Never overwrite existing files — append instead.** If a target notebook already exists, do NOT overwrite it. Append new cells to the existing file. Notify the user that the file already exists and that you will be appending to it.
-9. **Avoid filename collisions.** When creating a new file, check if a file with the same name already exists. If it does, rename the new file by appending a numeric suffix (e.g., `transform_dataset_2.ipynb`) before writing.
-10. **Default output format is JSONL.** Unless the user explicitly requests a different file format, the transformed dataset should be written as `.jsonl` (JSON Lines — one JSON object per line).
+8. **Default output format is JSONL.** Unless the user explicitly requests a different file format, the transformed dataset should be written as `.jsonl` (JSON Lines — one JSON object per line).
 
 ## Known Dataset Formats Reference
 
@@ -142,7 +140,18 @@ Continue iterating with the user's feedback — update the notebook cell in plac
 
 ⏸ Wait for user.
 
-### Step 8: Generate the execution cells in the notebook
+### Step 8: Determine notebook target
+
+Check if the project notebook already exists at `<project-dir>/notebooks/<project-name>.ipynb`.
+
+- If it exists → ask: _"Would you like me to append the transformation cells to the existing notebook, or create a new one?"_
+- If it doesn't exist → create it
+
+When appending, add a markdown header cell `## Dataset Transformation` as a section divider before the new cells.
+
+⏸ Wait for user.
+
+### Step 9: Generate the execution cells in the notebook
 
 **Before writing the notebook, read:**
 
@@ -168,7 +177,7 @@ Read the reference guide at `references/dataset_transformation_code.md` and foll
 
 ⏸ Wait for user.
 
-### Step 9: Determine and confirm execution mode
+### Step 10: Determine and confirm execution mode
 
 Check the size of the input dataset:
 
@@ -198,20 +207,20 @@ Do not execute until the user approves. If the user rejects the recommendation,
 
 If local execution:
 
-- Add a cell that runs the transformation by importing from the `.py` files already on disk (written by the agent during Steps 7–8): import `transform_dataset` from `transform_fn`, load the dataset, transform, and save output. Scripts are located in `<project-dir>/scripts/`.
+- Add a cell that runs the transformation by importing from the `.py` files already on disk (written by the agent during Steps 7 and 9): import `transform_dataset` from `transform_fn`, load the dataset, transform, and save output. Scripts are located in `<project-dir>/scripts/`.
 
 If SageMaker Processing Job:
 
 - Add a cell that submits and monitors the Processing Job inline using the V3 SageMaker SDK directly (FrameworkProcessor, ProcessingInput, ProcessingOutput, etc.). Create a FrameworkProcessor with the SKLearn 1.2-1 image, configure inputs/outputs, and call `processor.run(wait=True, logs=True)` to block the cell and stream logs until the job completes. See `scripts/transformation_tools.py` for reference implementation details.
 - Inform the user they can run this cell to kick off and monitor the job.
 
-**Important:** The agent must NOT execute the full dataset transformation itself. The notebook cells are generated for the user to review and run. Only sample data (from Steps 7–8) should be transformed by the agent for validation purposes.
+**Important:** The agent must NOT execute the full dataset transformation itself. The notebook cells are generated for the user to review and run. Only sample data (from Steps 7 and 9) should be transformed by the agent for validation purposes.
 
 > "I've added the execution cell to the notebook. You can run it to transform the full dataset. Would you like to review the notebook before running it?"
 
 ⏸ Wait for user.
 
-### Step 10: Verify and confirm with the user
+### Step 11: Verify and confirm with the user
 
 For this step, you need: **to verify the output looks correct and confirm with the user.**
 

plugins/sagemaker-ai/skills/dataset-transformation/references/notebook_structure.md

@@ -41,6 +41,7 @@ Each line of code is a separate string in `source`, ending with `\n` (except the
 - Escape quotes inside strings: `\"`
 - No trailing commas in arrays or objects
 - 2-space indentation
-- Use `fs_write` with `command: create` to write the complete notebook JSON
+- Write notebooks using your standard file write tool to create the `.ipynb` file with the complete notebook JSON, OR use notebook MCP tools (e.g., `create_notebook`, `add_cell`) if available in the current environment
+- **DON'T** use bash commands, shell scripts, or `echo`/`cat` piping to generate notebooks
 - Markdown cell 0: `"cell_type": "markdown"`, no `execution_count` or `outputs`
 - Wrap all cells in `{"cells": [...], "metadata": {...}, "nbformat": 4, "nbformat_minor": 4}`

plugins/sagemaker-ai/skills/dataset-transformation/references/notebook_writing_guide.md

@@ -26,9 +26,7 @@ But in a Jupyter notebook, each line must be a separate string in a JSON array:
 }
 ```
 
-## The Solution: Use fs_write with JSON Structure
-
-**ALWAYS use the `fs_write` tool with `command: create` to write notebooks.**
+## The Solution: Write as JSON Structure
 
 ### Correct Notebook Structure (Pretty-Print Format)
 
@@ -82,7 +80,7 @@ Use **2-space indentation** (pretty-print format) for consistent, readable forma
 ❌ **DON'T** forget to escape quotes in strings
 ❌ **DON'T** add trailing commas to last array elements
 
-✅ **DO** use fs_write with the complete JSON structure
+✅ **DO** use your standard file write tool to write the complete notebook JSON, or notebook MCP tools (e.g., `create_notebook`, `add_cell`) if available
 ✅ **DO** add `\n` to end of each line in source arrays
 ✅ **DO** validate JSON structure before writing
 ✅ **DO** use proper escaping for special characters

plugins/sagemaker-ai/skills/directory-management/SKILL.md

@@ -30,5 +30,5 @@ When working with the agent, all generated files are organized under an project
 ├── scripts/                # Generated Python scripts
 │   ├── <project-name>_transform_fn.py
 └── notebooks/              # Generated Jupyter notebooks
-    ├── <project-name>_training.ipynb
+    ├── <project-name>.ipynb
 ```

plugins/sagemaker-ai/skills/finetuning-setup/SKILL.md

@@ -40,16 +40,12 @@ Guides the user through selecting a base model and fine-tuning technique based o
 
 ### Step 2: Select Base Model
 
-1. Read `use_case_spec.md` to understand the use case and success criteria.
-2. Restate the use case in one sentence.
-3. Always retrieve the full list of available SageMaker Hub model names by running: `python finetuning-setup/scripts/get_model_names.py <hub-name>` — even if the user has already mentioned a model name or family. Do not skip this step or filter the results.
-4. Present all available models to the user, grouped by model family (e.g., Llama, Mistral, Gemma) for readability.
-5. Ask the user to pick the exact model ID from the list.
-6. Validate the selected model exists in the retrieved list before proceeding.
+First, retrieve all available SageMaker Hub model names by running: `python finetuning-setup/scripts/get_model_names.py <hub-name>`.
 
-EXTREMELY IMPORTANT: NEVER recommend or suggest any particular model based on the context you have. YOU ARE ALLOWED ONLY to display the list of models
-as given by the script. DO NOT add your own recommendation or suggestion after displaying the list of models to tell which model is correct. Present this
-statement to the user: "Which model would you like to use? Please type the exact model name from the above list." and allow the user to select the model.
+Present all available models to the user with their licenses before making any recommendations. Cross-reference the model list with `references/model-licenses.md` and display each as `<model name> - [<license>](<url>)`. For example: "Qwen3-4B - [Apache 2.0](https://huggingface.co/Qwen/Qwen3-4B/blob/main/LICENSE)"
+
+If you already know the model the user wants to use (from conversation context or planning files), confirm that it's in the list, display its license, and move on. Otherwise, help the user pick a model following the instructions in `references/model-selection.md`.
+**Important:** Make sure to remember this list of available models when helping with model selection. Don't recommend a model that's not available to the user.
 
 ### Step 3: Determine Finetuning Technique
 
@@ -72,4 +68,6 @@ Here's what we've selected:
 
 ## References
 
+- `references/model-selection.md` — Model selection instructions and benchmark descriptions
 - `references/finetune_technique_selection_guide.md` — Technique guidance
+- `references/model-licenses.md` — Model license information for display during model selection

plugins/sagemaker-ai/skills/finetuning-setup/references/benchmarks/agenticIndex.md

@@ -0,0 +1,49 @@
+# Agentic Index
+
+AA's composite agentic score: GDPval-AA (real-world tasks, 44 occupations) + τ²-bench Telecom.
+
+**Use this for:** Autonomous agents, workflow automation, tool-using assistants.
+
+**Source:** Artificial Analysis (artificialanalysis.ai), April 2026.
+"—" = no data — infer from similar models in the same family, but tell the user you're inferring.
+
+| #  | Model                               | Family      | Score |
+| -- | ----------------------------------- | ----------- | ----- |
+| 1  | GPT-OSS 120B (mode: high)           | OpenAI      | 37.9  |
+| 2  | Nova 2.0 Lite (mode: high)          | Amazon Nova | 37.3  |
+| 3  | Nova 2.0 Lite (mode: medium)        | Amazon Nova | 32.9  |
+| 4  | GPT-OSS 120B (mode: low)            | OpenAI      | 28.0  |
+| 5  | Nova 2.0 Lite (mode: low)           | Amazon Nova | 27.8  |
+| 6  | GPT-OSS 20B (mode: high)            | OpenAI      | 27.6  |
+| 7  | GPT-OSS 20B (mode: low)             | OpenAI      | 21.9  |
+| 8  | Nova 2.0 Lite (mode: non-reasoning) | Amazon Nova | 21.1  |
+| 9  | Qwen3 14B (mode: reasoning)         | Qwen        | 14.4  |
+| 10 | Qwen3 14B (mode: non-reasoning)     | Qwen        | 13.6  |
+| 11 | Qwen3 32B (mode: reasoning)         | Qwen        | 13.4  |
+| 12 | Qwen3 8B (mode: reasoning)          | Qwen        | 12.6  |
+| 13 | Qwen3 8B (mode: non-reasoning)      | Qwen        | 11.6  |
+| 14 | Llama 3.3 70B Instruct              | Meta Llama  | 9.1   |
+| 15 | Qwen3 1.7B (mode: reasoning)        | Qwen        | 8.7   |
+| 16 | Qwen3 1.7B (mode: non-reasoning)    | Qwen        | 7.2   |
+| 17 | Qwen3 0.6B (mode: reasoning)        | Qwen        | 7.0   |
+| 18 | Nova Lite                           | Amazon Nova | 5.8   |
+| 19 | Llama 3.1 8B Instruct               | Meta Llama  | 5.5   |
+| 20 | Llama 4 Scout 17B                   | Meta Llama  | 5.2   |
+| 21 | Qwen3 0.6B (mode: non-reasoning)    | Qwen        | 4.9   |
+| 22 | Nova Pro                            | Amazon Nova | 4.7   |
+| 23 | Nova Micro                          | Amazon Nova | 4.7   |
+| 24 | Llama 3.2 1B Instruct               | Meta Llama  | 0.0   |
+| —  | Llama 3.2 3B Instruct               | Meta Llama  | —     |
+| —  | Qwen3 32B (mode: non-reasoning)     | Qwen        | —     |
+| —  | Qwen3 4B (mode: reasoning)          | Qwen        | —     |
+| —  | Qwen3 4B (mode: non-reasoning)      | Qwen        | —     |
+| —  | Qwen2.5 72B Instruct                | Qwen        | —     |
+| —  | Qwen2.5 32B Instruct                | Qwen        | —     |
+| —  | DeepSeek R1 Distill Llama 70B       | DeepSeek    | —     |
+| —  | DeepSeek R1 Distill Llama 8B        | DeepSeek    | —     |
+| —  | DeepSeek R1 Distill Qwen 32B        | DeepSeek    | —     |
+| —  | DeepSeek R1 Distill Qwen 14B        | DeepSeek    | —     |
+| —  | DeepSeek R1 Distill Qwen 1.5B       | DeepSeek    | —     |
+| —  | Qwen2.5 14B Instruct                | Qwen        | —     |
+| —  | Qwen2.5 7B Instruct                 | Qwen        | —     |
+| —  | DeepSeek R1 Distill Qwen 7B         | DeepSeek    | —     |

plugins/sagemaker-ai/skills/finetuning-setup/references/benchmarks/codingIndex.md

@@ -0,0 +1,49 @@
+# Coding Index
+
+AA's composite coding score: Terminal-Bench Hard (agentic SWE/sysadmin) + SciCode (scientific programming).
+
+**Use this for:** Software engineering, scientific computing, system administration, code-heavy tasks.
+
+**Source:** Artificial Analysis (artificialanalysis.ai), April 2026.
+"—" = no data — infer from similar models in the same family, but tell the user you're inferring.
+
+| #  | Model                               | Family      | Score |
+| -- | ----------------------------------- | ----------- | ----- |
+| 1  | GPT-OSS 120B (mode: high)           | OpenAI      | 28.6  |
+| 2  | Nova 2.0 Lite (mode: medium)        | Amazon Nova | 23.9  |
+| 3  | Nova 2.0 Lite (mode: high)          | Amazon Nova | 23.4  |
+| 4  | GPT-OSS 20B (mode: high)            | OpenAI      | 18.5  |
+| 5  | GPT-OSS 120B (mode: low)            | OpenAI      | 15.5  |
+| 6  | GPT-OSS 20B (mode: low)             | OpenAI      | 14.4  |
+| 7  | Qwen3 32B (mode: reasoning)         | Qwen        | 13.8  |
+| 8  | Nova 2.0 Lite (mode: low)           | Amazon Nova | 13.6  |
+| 9  | Qwen3 14B (mode: reasoning)         | Qwen        | 13.1  |
+| 10 | Nova 2.0 Lite (mode: non-reasoning) | Amazon Nova | 12.5  |
+| 11 | Qwen3 14B (mode: non-reasoning)     | Qwen        | 12.4  |
+| 12 | Qwen2.5 72B Instruct                | Qwen        | 11.9  |
+| 13 | DeepSeek R1 Distill Llama 70B       | DeepSeek    | 11.4  |
+| 14 | Nova Pro                            | Amazon Nova | 11.0  |
+| 15 | Llama 3.3 70B Instruct              | Meta Llama  | 10.7  |
+| 16 | Qwen3 8B (mode: reasoning)          | Qwen        | 9.0   |
+| 17 | Qwen3 8B (mode: non-reasoning)      | Qwen        | 7.1   |
+| 18 | Llama 4 Scout 17B                   | Meta Llama  | 6.7   |
+| 19 | Nova Lite                           | Amazon Nova | 5.1   |
+| 20 | Llama 3.1 8B Instruct               | Meta Llama  | 4.9   |
+| 21 | Nova Micro                          | Amazon Nova | 4.1   |
+| 22 | Qwen3 1.7B (mode: non-reasoning)    | Qwen        | 2.3   |
+| 23 | Qwen3 1.7B (mode: reasoning)        | Qwen        | 1.4   |
+| 24 | Qwen3 0.6B (mode: non-reasoning)    | Qwen        | 1.4   |
+| 25 | Qwen3 0.6B (mode: reasoning)        | Qwen        | 0.9   |
+| 26 | Llama 3.2 1B Instruct               | Meta Llama  | 0.6   |
+| —  | Llama 3.2 3B Instruct               | Meta Llama  | —     |
+| —  | Qwen3 32B (mode: non-reasoning)     | Qwen        | —     |
+| —  | Qwen3 4B (mode: reasoning)          | Qwen        | —     |
+| —  | Qwen3 4B (mode: non-reasoning)      | Qwen        | —     |
+| —  | Qwen2.5 32B Instruct                | Qwen        | —     |
+| —  | DeepSeek R1 Distill Llama 8B        | DeepSeek    | —     |
+| —  | DeepSeek R1 Distill Qwen 32B        | DeepSeek    | —     |
+| —  | DeepSeek R1 Distill Qwen 14B        | DeepSeek    | —     |
+| —  | DeepSeek R1 Distill Qwen 1.5B       | DeepSeek    | —     |
+| —  | Qwen2.5 14B Instruct                | Qwen        | —     |
+| —  | Qwen2.5 7B Instruct                 | Qwen        | —     |
+| —  | DeepSeek R1 Distill Qwen 7B         | DeepSeek    | —     |