Add --chat-template-file to mount custom chat template (fixes #1783)

[rhatdan]

Users can pass a host path to a chat template file; it is mounted into the container at /mnt/models/chat_template.file and passed to the inference engine. Enables using fixed or custom templates without redownloading the model.

Made-with: Cursor

Fixes: #1783

Summary by Sourcery

Add support for providing a host chat template file that is mounted into the container and passed through to the inference engine.

New Features:

• Accept a --chat-template-file CLI option for run, serve, and perplexity commands to supply a custom chat template from the host.
• Propagate the chat template file path through the runtime context so it is available to the inference engine configuration.
• Allow llama.cpp engine configs to prioritize a user-specified chat template file over the model’s default template when present.

Enhancements:

• Extend container mount setup to bind-mount a user-provided chat template file into a fixed in-container path for use by runtimes.

[sourcery-ai]

Reviewer's Guide

Adds support for a new --chat-template-file CLI option that mounts a user-provided chat template file into the container and wires it through to the llama.cpp engine, preferring the user’s file over the model’s default template when provided.

File-Level Changes

Change	Details	Files
Wire a user-provided chat template file into container mounts for all relevant execution paths.	In setup_mounts, read chat_template_file from args using getattr with a default of None. When a valid file path is provided, compute its container mount path with get_container_mount_path and add a bind mount to MNT_CHAT_TEMPLATE_FILE with read-only and relabel options. Apply this mounting logic in both the early-return (engine-specific mount method) branch and the standard reference-file mount path so the feature works for all engines.	ramalama/transports/base.py
Expose a --chat-template-file CLI flag and propagate it through the runtime context.	Add a --chat-template-file argument for perplexity, run, and serve commands, storing the value as chat_template_file on the args namespace. Extend RamalamaArgsContext with a chat_template_file field. Populate RamalamaArgsContext.chat_template_file with MNT_CHAT_TEMPLATE_FILE when a chat_template_file was provided on the CLI, so downstream components see the container path rather than the host path.	ramalama/cli.py ramalama/command/context.py
Update llama.cpp engine spec to prioritize the user-specified chat template file over the model’s default and only pass the flag when a template is available.	Change the --chat-template-file value to use args.chat_template_file when present, falling back to model.chat_template_path otherwise. Tighten the condition for emitting --chat-template-file so it is only passed when either args.chat_template_file or model.chat_template_path exists and the model is not multimodal (no mmproj_path).	inference-spec/engines/llama.cpp.yaml

Possibly linked issues

• Allow passing additional config files (e.g. chat templates) into the model container #1783: PR adds --chat-template-file and container mount, directly enabling external chat templates as requested in the issue.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the inference engine's flexibility by allowing users to supply custom chat templates directly from their local filesystem. This change enables the use of fixed or personalized templates without requiring a model re-download, streamlining the process of experimenting with different conversational formats and improving user control over the model's output structure.

Highlights

• New CLI Argument: Introduced a new command-line argument, --chat-template-file, allowing users to specify a custom chat template file from their host system.
• Dynamic Chat Template Usage: Updated the llama.cpp engine configuration to prioritize the user-provided --chat-template-file over the model's default chat template path, enabling flexible template usage.
• Container Mounting: Implemented logic to automatically bind mount the specified host chat template file into the container at /mnt/models/chat_template.file, making it accessible to the inference engine.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• inference-spec/engines/llama.cpp.yaml
  • Updated the value field for --chat-template-file to use the new args.chat_template_file if provided, falling back to model.chat_template_path.
  • Modified the if condition for --chat-template-file to ensure it's only applied when a template file is available and no mmproj_path is present.
• ramalama/cli.py
  • Added the --chat-template-file argument to the CLI parser for perplexity, run, and serve commands, with a help description for its purpose.
• ramalama/command/context.py
  • Imported MNT_CHAT_TEMPLATE_FILE for consistent path definition.
  • Added chat_template_file as an optional string attribute to the RamalamaArgsContext class.
  • Populated the ctx.chat_template_file with the container mount path if the --chat-template-file argument was provided.
• ramalama/transports/base.py
  • Imported MNT_CHAT_TEMPLATE_FILE for consistent path definition.
  • Added logic within setup_mounts to check for a user-provided chat_template_file and, if found, create a read-only bind mount into the container at the predefined MNT_CHAT_TEMPLATE_FILE path.

Activity

• No human activity (comments, reviews) has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[sourcery-ai]

Hey - I've left some high level feedback:

• In setup_mounts, the --mount entries for chat_template_file use src={container_chat_template}; for a bind mount the src should be the host path, so consider using the original chat_template_file as src and MNT_CHAT_TEMPLATE_FILE as destination rather than the container-mapped path.
• The chat_template_file mounting logic is duplicated in two branches of setup_mounts; consider extracting this into a small helper or shared block to reduce repetition and keep the behavior consistent if it needs to change later.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- In `setup_mounts`, the `--mount` entries for `chat_template_file` use `src={container_chat_template}`; for a bind mount the `src` should be the host path, so consider using the original `chat_template_file` as `src` and `MNT_CHAT_TEMPLATE_FILE` as `destination` rather than the container-mapped path.
- The `chat_template_file` mounting logic is duplicated in two branches of `setup_mounts`; consider extracting this into a small helper or shared block to reduce repetition and keep the behavior consistent if it needs to change later.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

This pull request introduces a new --chat-template-file command-line option, allowing users to specify a custom chat template. The changes are well-implemented across the CLI argument parsing, command context, and the inference engine specification. My review includes one suggestion to refactor a piece of duplicated code in ramalama/transports/base.py to improve long-term maintainability.

[olliewalsh]

IIRC it's more complex than just bind mounting - have to consider oci images, kube/quadlet etc...

[olliewalsh]

Another option might be to support overriding the chat template in the ramalama convert cli - use it to generate an oci image/artifact with an overridden template? That would work in all cases I expect

[github-actions]

A friendly reminder that this PR had no activity for 30 days.