Skip to content

Commit f16c9f7

Browse files
vaibhavjindalclaude
vaibhavjindal
and
claude
authored
Improve liger-autopatch skill to enable modifying existing monkey-patches (#1177)
## Summary Extends the `liger-autopatch` Claude Code skill to support **modifying existing monkey-patches**, not just adding new models. Previously, the skill only triggered for new model support, so modification tasks (e.g., adding `LigerReLUSquared` to nemotron in #1172) bypassed the skill entirely and missed its guidelines. Changes to 2 skill files: **SKILL.md:** - Broadened skill description to trigger on modification-related keywords (update, fix, change, extend, etc.) - Added **Mode Detection** section that routes to create vs modify pipelines - Added **Modify Pipeline** with 3 stages: Change Impact Analysis → Apply Changes → Validate - Validate stage lists all convergence test commands explicitly (including multimodal) **code-generator.md:** - Added **Mode** section distinguishing create vs modify mode - Added **Modification Checklist** with 7 rules (R1-R7): - R1: Both patching levels (class-level + instance-level) - R2: New parameter with default - R3: Update docstring - R4: Update tests - R5: Check revert function in test/utils.py - R6: Run convergence tests (all 6 files including multimodal) - R7: Update README.md - Added **Common Modification Patterns** for: adding activation kernels, adding norm variants, fixing missing instance patching, updating for upstream HF changes ## Testing Done Tested by asking Claude to add `relu_squared` to nemotron's existing monkey-patch. Verified the skill triggers in modify mode and follows the modification checklist. - Hardware Type: N/A (skill files only, no code changes) - [x] run `make test` to ensure correctness - [x] run `make checkstyle` to ensure code style - [x] run `make test-convergence` to ensure convergence --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 35cf6a0 commit f16c9f7

2 files changed

Lines changed: 134 additions & 4 deletions

File tree

‎.claude/skills/liger-autopatch/SKILL.md‎

Lines changed: 46 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,20 @@
11
---
22
name: liger-autopatch
3-
description: "Adds Liger Kernel support for a new HuggingFace Transformers model. Generates lce_forward, monkey-patch function, tests, and README entry. Use when adding a new model to Liger Kernel, when a user asks to patch an unsupported model, or when extending MODEL_TYPE_TO_APPLY_LIGER_FN."
3+
description: "Adds Liger Kernel support for a new HuggingFace Transformers model, or modifies existing monkey-patching. Generates lce_forward, monkey-patch function, tests, and README entry. Use when adding a new model to Liger Kernel, when a user asks to patch an unsupported model, when extending MODEL_TYPE_TO_APPLY_LIGER_FN, or when modifying/updating/fixing an existing monkey-patch (e.g., adding a new kernel to an already-supported model, fixing instance patching, updating a patch for upstream HF changes)."
44
---
55

66
# Liger Auto-Patch
77

8-
Adds Liger Kernel optimization support for a new HuggingFace model through a 3-stage pipeline with human review between stages.
8+
Adds Liger Kernel optimization support for a new HuggingFace model, or modifies existing monkey-patching, through a staged pipeline with human review between stages. Supports creating new model patches and modifying existing ones.
99

10-
## Pipeline
10+
## Mode Detection
11+
12+
- **Create mode**: User asks to add/patch/support a new model → full pipeline (Analyze → Generate → Validate)
13+
- **Modify mode**: User asks to update/fix/change/extend an existing monkey-patch → lighter pipeline (Change Impact Analysis → Apply Changes → Validate)
14+
15+
Keywords that suggest modify mode: update, fix, change, add [kernel] to [existing model], extend, modify, new activation, new norm, bug in patch, upstream changed
16+
17+
## Pipeline (Create Mode)
1118

1219
### Stage 1: Analyze
1320

@@ -47,6 +54,42 @@ Runs instance patching test, convergence test, and lint check. Retries up to 3 t
4754

4855
**Human checkpoint:** Report final test results.
4956

57+
## Pipeline (Modify Mode)
58+
59+
### Stage 1: Change Impact Analysis
60+
61+
Read the existing `apply_liger_kernel_to_{model_type}` function in `monkey_patch.py` and the relevant section of the upstream HF `modeling_{model_type}.py`. Produce a short change plan:
62+
63+
- What is being added/changed/fixed
64+
- Which Liger kernel(s) are involved
65+
- Which files need modification (subset of the 13 files from create mode)
66+
- What the expected behavior should be after the change
67+
68+
**Human checkpoint:** Present the change plan. Confirm before proceeding.
69+
70+
### Stage 2: Apply Changes
71+
72+
Spawn the **Code Generator** agent (read [code-generator.md](code-generator.md)) in **modify mode**.
73+
74+
**Human checkpoint:** Present changes for review.
75+
76+
### Stage 3: Validate
77+
78+
Spawn the **Validator** agent (read [validator.md](validator.md)). This stage is **mandatory** — do not skip it. At minimum, run:
79+
80+
1. Instance patching test: `pytest test/transformers/test_monkey_patch.py -k "{model_type}" -xvs`
81+
2. All convergence tests for the model:
82+
- `pytest test/convergence/bf16/test_mini_models.py -k "{model_type}" -xvs` (FLCE, bf16)
83+
- `pytest test/convergence/bf16/test_mini_models_with_logits.py -k "{model_type}" -xvs` (non-FLCE, bf16)
84+
- `pytest test/convergence/fp32/test_mini_models.py -k "{model_type}" -xvs` (FLCE, fp32)
85+
- `pytest test/convergence/fp32/test_mini_models_with_logits.py -k "{model_type}" -xvs` (non-FLCE, fp32)
86+
- If VL (multimodal) model, also run:
87+
- `pytest test/convergence/bf16/test_mini_models_multimodal.py -k "{model_type}" -xvs`
88+
- `pytest test/convergence/fp32/test_mini_models_multimodal.py -k "{model_type}" -xvs`
89+
3. Checkstyle: `make checkstyle`
90+
91+
**Human checkpoint:** Report final test results.
92+
5093
## Reference Files
5194

5295
- [decision-matrix.md](decision-matrix.md) — 12 architectural decisions to resolve per model

‎.claude/skills/liger-autopatch/code-generator.md‎

Lines changed: 88 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,11 @@
11
# Code Generator Agent
22

3-
Takes a confirmed model profile and generates all files to add Liger Kernel support.
3+
Takes a confirmed model profile (create mode) or a change plan (modify mode) and generates or modifies files for Liger Kernel support.
4+
5+
## Mode
6+
7+
- **Create mode** (default): Generating all files for a new model. Follow the full "Files to Generate" list below.
8+
- **Modify mode**: Making targeted changes to an existing monkey-patch. Follow the "Modification Checklist" section instead.
49

510
## Pre-Requisites
611

@@ -84,3 +89,85 @@ Add row to the Patching table under "### Patching":
8489
- Follow exact patterns from existing code — do not innovate on style
8590
- When modifying existing files, insert new entries in **alphabetical order** alongside similar existing entries. Never append to the end of a section — find the correct alphabetical position.
8691
- After generating all files, run `make checkstyle` to verify formatting. If it fails, run `ruff check . --fix && ruff format .` to auto-fix, then verify with `make checkstyle` again.
92+
93+
## Modification Checklist (Modify Mode)
94+
95+
Before making changes, read the existing implementation:
96+
1. Read `apply_liger_kernel_to_{model_type}` in `monkey_patch.py`
97+
2. Read the existing test in `test_monkey_patch.py` for this model
98+
3. Read the relevant HF modeling source for context
99+
100+
### Rules for All Modifications
101+
102+
**R1. Both patching levels.** If adding a new kernel, it must appear in BOTH:
103+
- Class-level patching (the main body of `apply_liger_kernel_to_{model_type}`)
104+
- Instance-level patching (the `if model is not None` block)
105+
106+
Omitting one is the most common mistake.
107+
108+
**R2. New parameter with default.** Every new kernel gets a bool parameter on the
109+
apply function signature (e.g., `relu_squared: bool = True`). Default should be `True`
110+
for kernels that are safe to enable by default, `False` otherwise.
111+
112+
**R3. Update docstring.** Update the function's docstring to:
113+
- Add an `Args` entry for the new parameter
114+
- Remove any stale notes that the new kernel invalidates
115+
(e.g., "squared ReLU is not supported" → remove if you're adding it)
116+
117+
**R4. Update tests.** In the existing `test_apply_liger_kernel_to_instance_for_{model_type}`:
118+
- Add import for the new Liger kernel class
119+
- Add "not yet patched" assertion before `_apply_liger_kernel_to_instance`
120+
- Add "correctly patched" assertion after
121+
- Follow the exact pattern of existing assertions in the same test
122+
123+
**R5. Check revert function.** Read `revert_liger_kernel_to_{model_type}` in `test/utils.py`.
124+
The revert function uses `importlib.reload(modeling_{model_type})` to undo all patches.
125+
This handles most cases automatically, but check if the new kernel requires additional
126+
revert logic (e.g., if the kernel patches something outside the modeling module, or
127+
replaces a global like `ACT2FN` that `importlib.reload` won't fully restore). Update
128+
the revert function if needed.
129+
130+
**R6. Run convergence tests.** Don't modify convergence test files unless the change
131+
requires it (e.g., new mini model config fields). But DO run existing convergence
132+
tests in the Validate stage to verify no regression. This is critical — the Validator
133+
agent (Stage 3) handles this, but if you are generating code without a separate
134+
Validate stage, run these yourself:
135+
```bash
136+
pytest test/convergence/bf16/test_mini_models.py -k "{model_type}" -xvs
137+
pytest test/convergence/bf16/test_mini_models_with_logits.py -k "{model_type}" -xvs
138+
pytest test/convergence/fp32/test_mini_models.py -k "{model_type}" -xvs
139+
pytest test/convergence/fp32/test_mini_models_with_logits.py -k "{model_type}" -xvs
140+
```
141+
For VL (multimodal) models, also run:
142+
```bash
143+
pytest test/convergence/bf16/test_mini_models_multimodal.py -k "{model_type}" -xvs
144+
pytest test/convergence/fp32/test_mini_models_multimodal.py -k "{model_type}" -xvs
145+
```
146+
147+
**R7. Update README.md.** If the change adds a visibly new capability to the model's
148+
row in the patching table (e.g., a new operation), update the supported operations list.
149+
150+
### Common Modification Patterns
151+
152+
**Adding an activation kernel (e.g., relu_squared for nemotron):**
153+
- Import the Liger kernel class at the top of `monkey_patch.py`
154+
- Add bool parameter to apply function signature
155+
- Class-level: replace in `ACT2FN` dict or replace the MLP class
156+
- Instance-level: patch each `decoder_layer`'s activation/MLP
157+
- Test: `assert isinstance` checks on the activation/MLP
158+
159+
**Adding a norm variant:**
160+
- Add bool parameter to apply function
161+
- Class-level: replace the norm class
162+
- Instance-level: use `_patch_rms_norm_module` or `_patch_layer_norm_module` on all norm attrs
163+
- Test: `assert isinstance` checks on norm modules
164+
165+
**Fixing missing instance patching:**
166+
- Read the class-level patching to see what's patched
167+
- Add corresponding instance-level patches in the `if model is not None` block
168+
- Test: add assertions that were missing
169+
170+
**Updating for upstream HF changes:**
171+
- Compare the current HF modeling file against what the patch assumes
172+
- Update class names, attribute names, forward signatures as needed
173+
- May require updating `lce_forward` if the base model's forward changed

0 commit comments

Comments
 (0)