|
| 1 | +# Copyright 2023–2026 Google LLC |
| 2 | +# |
| 3 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
| 4 | +# you may not use this file except in compliance with the License. |
| 5 | +# You may obtain a copy of the License at |
| 6 | +# |
| 7 | +# https://www.apache.org/licenses/LICENSE-2.0 |
| 8 | +# |
| 9 | +# Unless required by applicable law or agreed to in writing, software |
| 10 | +# distributed under the License is distributed on an "AS IS" BASIS, |
| 11 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 12 | +# See the License for the specific language governing permissions and |
| 13 | +# limitations under the License. |
| 14 | + |
1 | 15 |
|
2 | 16 | """ |
3 | 17 | A unified tool to inspect checkpoint structures for: |
4 | | -1. HuggingFace/PyTorch source files (.safetensors, .pth) |
5 | | -2. MaxText Model Architecture (on-the-fly, no weights loaded) |
6 | | -3. Saved Orbax Checkpoints (metadata only) |
| 18 | +1. HuggingFace/PyTorch (need load weight) |
| 19 | +2. MaxText Model Architecture (lightweight, no weights loaded) |
| 20 | +3. Orbax Checkpoints (lightweight, no weights loaded) |
7 | 21 |
|
8 | 22 | Usage Examples: |
9 | 23 | [Mode 1: HF/PyTorch] |
10 | | - python inspect_checkpoint.py hf --path <local_hf_path> --format <safetensors | pth> |
| 24 | + python src/MaxText/utils/ckpt_conversion/inspect_checkpoint.py hf --path <local_hf_path> --format <safetensors | pth> |
11 | 25 | [Mode 2: MaxText Arch] |
12 | | - python inspect_checkpoint.py maxtext --model_name <maxtext_model_name> --scan_layers <True | False> |
| 26 | + python src/MaxText/utils/ckpt_conversion/inspect_checkpoint.py maxtext --model_name <maxtext_model_name> --scan_layers <True | False> |
13 | 27 | [Mode 3: Orbax] |
14 | | - python inspect_checkpoint.py orbax --path <local_orbax_path | gcs_orbax_path> |
15 | | -
|
16 | | -
|
17 | | -cd ~/maxtext |
18 | | -SCRIPT=~/maxtext/src/MaxText/utils/ckpt_conversion/inspect_checkpoint.py |
19 | | -python inspect_checkpoint.py hf --path <local_hf_path> --format safetensors |
20 | | -python $SCRIPT maxtext --model_name deepseek3.2-671b --scan_layers False |
21 | | -python $SCRIPT maxtext --model_name deepseek3.2-671b --scan_layers True |
| 28 | + python src/MaxText/utils/ckpt_conversion/inspect_checkpoint.py orbax --path <local_orbax_path | gcs_orbax_path> |
22 | 29 | """ |
23 | 30 |
|
24 | 31 | import argparse |
|
0 commit comments