Skip to content

Commit 43458fd

Browse files
authored
fix: Update the Maxtext installing cmds (GoogleCloudPlatform#1019)
This PR updates the setup process for the `jetstream_benchmark_serving` DAG. - Upgrades the Python virtual environment to 3.12. - Introduces uv as the package manager. - Installs MaxText from PyPI using uv pip install. This aligns with the official recommended installation guide from the MaxText repository: https://maxtext.readthedocs.io/en/latest/guides/install_maxtext.html
1 parent 7d71e4d commit 43458fd

1 file changed

Lines changed: 18 additions & 6 deletions

File tree

dags/inference/configs/jetstream_benchmark_serving_gce_config.py

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -55,18 +55,30 @@ def get_config(
5555
# Download jetstream and maxtext
5656
f"if [ ! -d maxtext ]; then git clone {maxtext_branch} https://github.com/google/maxtext.git; fi",
5757
f"if [ ! -d JetStream ]; then git clone {jetstream_branch} https://github.com/google/JetStream.git; fi",
58-
# Create a python virtual environment
5958
"sudo apt-get -y update",
60-
"sudo apt-get -y install python3.10-venv",
6159
"sudo apt-get -y install jq",
62-
"python -m venv .env",
63-
"source .env/bin/activate",
64-
# Setup MaxText & JetStream
65-
f"cd maxtext && bash setup.sh MODE={test_mode.value} && cd ..",
6660
"cd JetStream && pip install -e . && cd benchmarks && pip install -r requirements.in",
6761
"pip install torch --index-url https://download.pytorch.org/whl/cpu",
62+
"cd ..",
6863
)
6964

65+
# Setup uv and and instll maxtext from PypI (Recommended by the maxtext repo: https://maxtext.readthedocs.io/en/latest/guides/install_maxtext.html)
66+
setup_maxtext_cmds = (
67+
# Install uv using the standalone installer and add to PATH permanently in the current session
68+
"curl -LsSf https://astral.sh/uv/install.sh | sh",
69+
'export PATH="$HOME/.local/bin:$PATH"',
70+
# Make the PATH change permanent for subsequent sessions (optional, but good practice)
71+
"echo 'export PATH=\"$HOME/.local/bin:$PATH\"' >> ~/.bashrc",
72+
"source ~/.bashrc",
73+
"uv venv --python 3.12 venv-312 --seed",
74+
"source venv-312/bin/activate",
75+
"pip install uv",
76+
"uv pip install maxtext --resolution=lowest",
77+
"install_maxtext_github_deps",
78+
)
79+
80+
set_up_cmds += setup_maxtext_cmds
81+
7082
additional_metadata_dict = {
7183
"model_name": f"{model_configs['model_name']}",
7284
"model_mode": f"{model_configs['model_mode']}",

0 commit comments

Comments
 (0)