Skip to content

Add GEPA prompt optimization tutorial and clean up Qwen3 configs#3642

Merged
copybara-service[bot] merged 1 commit intomainfrom
save-gepa-notebook
Apr 22, 2026
Merged

Add GEPA prompt optimization tutorial and clean up Qwen3 configs#3642
copybara-service[bot] merged 1 commit intomainfrom
save-gepa-notebook

Conversation

@darisoy
Copy link
Copy Markdown
Collaborator

@darisoy darisoy commented Apr 10, 2026

Description

This PR introduces a new tutorial notebook demonstrating how to use GEPA (Generic Evaluation and Prompt Adaptation) to optimize system prompts for MaxText models. It also includes minor configuration cleanups for Qwen3 models in the checkpoint conversion utility.

The specific changes include:

  • New Tutorial Notebook (src/maxtext/examples/maxtext_with_gepa.ipynb):
    • Provides environment setup instructions for a single TPU GCE VM using uv.
    • Covers downloading and converting the qwen3-4b checkpoint.
    • Demonstrates how to start and serve the model using MaxText/vLLM on a TPU.
    • Walks through using GEPA to optimize the model's system prompt against the AIME dataset.
    • Includes a before-and-after comparison of the model's accuracy to showcase the optimization results.
  • Config Cleanup (src/maxtext/checkpoint_conversion/utils/hf_model_configs.py):
    • Removes the use_sliding_window parameter from the Qwen3 configurations, as it is no longer supported.

Tests

The changes primarily consist of an example notebook. The notebook contains reproducible steps to run end-to-end tests on a TPU VM, which covers:

  • Checkpoint conversion of Qwen3-4B.
  • vLLM serving initialization.
  • Executing the GEPA optimization loop and evaluating both the seed prompt and optimized prompt outputs.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@shralex shralex self-requested a review April 10, 2026 23:54
@darisoy darisoy force-pushed the save-gepa-notebook branch 5 times, most recently from fcddfc5 to e27c88b Compare April 16, 2026 20:01
@darisoy darisoy force-pushed the save-gepa-notebook branch from e27c88b to 0dba38c Compare April 16, 2026 20:06
Comment thread src/maxtext/examples/maxtext_with_gepa.ipynb
Comment thread src/maxtext/examples/maxtext_with_gepa.ipynb Outdated
Comment thread src/maxtext/examples/maxtext_with_gepa.ipynb
Comment thread src/maxtext/examples/maxtext_with_gepa.ipynb
@darisoy darisoy force-pushed the save-gepa-notebook branch 2 times, most recently from d18ab79 to 5092209 Compare April 18, 2026 15:14
Comment thread README.md
@darisoy darisoy force-pushed the save-gepa-notebook branch 2 times, most recently from 360cbc2 to 36d7beb Compare April 18, 2026 23:48
@darisoy darisoy requested a review from jacoguzo as a code owner April 19, 2026 18:18
Comment thread .github/workflows/run_jupyter_notebooks.yml Outdated
Comment thread src/maxtext/examples/maxtext_with_gepa.ipynb Outdated
@darisoy darisoy force-pushed the save-gepa-notebook branch from 3e88788 to 0be7b1b Compare April 20, 2026 18:02
@darisoy darisoy force-pushed the save-gepa-notebook branch 4 times, most recently from 63901ff to 9b6f8c2 Compare April 21, 2026 20:54
@darisoy darisoy force-pushed the save-gepa-notebook branch 2 times, most recently from c9677b8 to 633eca2 Compare April 21, 2026 23:03
@copybara-service copybara-service Bot merged commit 8a17c3d into main Apr 22, 2026
94 of 101 checks passed
@copybara-service copybara-service Bot deleted the save-gepa-notebook branch April 22, 2026 04:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants