feat(<agentsight>): 引入 Rust 版本的 Chat Template 和 Tokenizer 以替代 Python 实现

[chengshuyi]

Description

将 Chat Template 和 Tokenizer 的实现从 Python 迁移到 Rust，通过 llm-tokenizer crate 提供轻量级、高性能的替代方案：

核心功能：

Chat Template 解析：支持主流 LLM 模型（如 Llama、GPT、Claude 等）的 chat template 格式解析和渲染
Token 计数：提供准确的 token 编码和解码功能，支持流式处理
轻量级部署：Rust 编译后的二进制文件无需额外运行时依赖，可独立部署

Related Issue

• Related issue linked (recommended for new features and non-trivial changes)

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional change)
• Performance improvement
• CI/CD or build changes

Scope

• cosh (copilot-shell)
• agent-sec-core
• os-skills
• agentsight
• Multiple / Project-wide

Checklist

• I have read the Contributing Guide
• My code follows the project's code style
• I have added tests that prove my fix is effective or that my feature works
• I have updated the documentation accordingly
• For cosh: Lint passes, type check passes, and tests pass
• For agent-sec-core (Rust): cargo clippy -- -D warnings and cargo fmt --check pass
• For agent-sec-core (Python): Ruff format and pytest pass
• For os-skills: Skill directory structure is valid and shell scripts pass syntax check
• Lock files are up to date (package-lock.json / Cargo.lock)

Testing

Additional Notes

[chengshuyi]

lgtm