Skip to content

feat: add TSP (Tree-like Self-Play) for Secure Code LLMs#1

Merged
TheBinKing merged 2 commits into
Narwhal-Lab:mainfrom
Easonnoway:add-tsp-project
May 23, 2026
Merged

feat: add TSP (Tree-like Self-Play) for Secure Code LLMs#1
TheBinKing merged 2 commits into
Narwhal-Lab:mainfrom
Easonnoway:add-tsp-project

Conversation

@Easonnoway
Copy link
Copy Markdown
Contributor

Summary

Add the Learn from Your Mistakes: Tree-like Self-Play for Secure Code LLMs project to the hub.

TSP is a training framework that improves code generation security by identifying CWE risk nodes in the generation process and using targeted self-play with DPO training to teach models to distinguish secure code paths from vulnerable alternatives.

This PR includes the core TSP pipeline:

  • Step 0: CWE risk node annotation via GPT-4o
  • Step 1: vLLM code generation at vulnerability decision points
  • Step 2: DPO preference pair creation and format conversion
  • Step 3: DPO training via LLaMA-Factory (with configs for CodeLlama-7B and Qwen2.5-Coder-7B)

Test plan

  • Verify directory structure matches hub conventions (similar to IsYourPromptPoisoningCode)
  • Confirm README format is consistent with existing project
  • Review that no hardcoded secrets or absolute paths are present
  • Check Python syntax for all scripts

Add the "Learn from Your Mistakes" project implementing TSP, a training
framework that improves code generation security by identifying CWE risk
nodes and using DPO training with targeted self-play at vulnerability
decision points.

Includes:
- Step 0-3 TSP pipeline (annotation, inference, data processing, training)
- RQ1-3 evaluation scripts (SecurityEval, CWE_Eval, generalization, ablation)
- Training configs for CodeLlama-7B and Qwen2.5-Coder-7B
Remove RQ1-3 evaluation code and references from README as per project
scope decision. Keep only the core TSP training pipeline (step0-3).
@TheBinKing TheBinKing merged commit 0530bad into Narwhal-Lab:main May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants