Support multimethod in runner#17228
Conversation
Specify which method to call in the runner. Update CLAUDE.md for fbcode/xplat. Differential Revision: [D92225533](https://our.internmc.facebook.com/intern/diff/D92225533/) [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17228
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 1 Unrelated FailureAs of commit a0cea55 with merge base aa2f683 ( NEW FAILURE - The following job has failed:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
Specify which method to call in the runner. Update CLAUDE.md for fbcode/xplat. Differential Revision: [D92225533](https://our.internmc.facebook.com/intern/diff/D92225533/) [ghstack-poisoned]
There was a problem hiding this comment.
Pull request overview
This PR adds multimethod support to the LLM runner infrastructure, allowing users to specify which method to execute in the model instead of hardcoding "forward". This is part of a larger stack implementing multimethod and LoRA support.
Changes:
- Added
method_nameparameter toTextDecoderRunnerand related factory functions, with "forward" as the default - Updated all runner code to use the configurable method name instead of hardcoded "forward"
- Added command-line flag
--method_nameto the llama example - Included minor documentation updates to CLAUDE.md about fbcode/xplat conventions
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| extension/llm/runner/text_decoder_runner.h | Added method_name constructor parameter, member variable, and getter method |
| extension/llm/runner/text_decoder_runner.cpp | Updated constructor and step() to use configurable method_name; fixed typo in error message |
| extension/llm/runner/test/test_text_decoder_runner.cpp | Added tests for default and custom method names |
| extension/llm/runner/llm_runner_helper.h | Added method_name parameter to create_text_llm_runner function signatures |
| extension/llm/runner/llm_runner_helper.cpp | Propagated method_name through factory functions; added logging for method name |
| examples/models/llama/runner/runner.h | Added method_name parameter to create_llama_runner functions |
| examples/models/llama/runner/runner.cpp | Propagated method_name through create_llama_runner implementations |
| examples/models/llama/main.cpp | Added --method_name command-line flag |
| CLAUDE.md | Added documentation about fbcode/xplat directory structure; minor formatting fixes |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Specify which method to call in the runner. Update CLAUDE.md for fbcode/xplat. Differential Revision: [D92225533](https://our.internmc.facebook.com/intern/diff/D92225533/) [ghstack-poisoned]
Specify which method to call in the runner. Update CLAUDE.md for fbcode/xplat. Differential Revision: [D92225533](https://our.internmc.facebook.com/intern/diff/D92225533/) [ghstack-poisoned]
Pull Request resolved: #17228 Specify which method to call in the runner. ghstack-source-id: 340390608 @exported-using-ghexport Differential Revision: [D92225533](https://our.internmc.facebook.com/intern/diff/D92225533/)
fe9b74f
into
gh/lucylq/131/base
This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #17228 by @lucylq ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/lucylq/131/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/lucylq/131/head Merge bot PR base: https://github.com/pytorch/executorch/tree/main Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/lucylq/131/orig Differential Revision: [D92225533](https://our.internmc.facebook.com/intern/diff/D92225533/) @diff-train-skip-merge Co-authored-by: Github Executorch <github_executorch@arm.com>
Stack from ghstack (oldest at bottom):
Specify which method to call in the runner.
Update CLAUDE.md for fbcode/xplat.
Differential Revision: D92225533