[Bug]: Gemma4 INT4 AWQ quantization fails during export: Gemma4Config has no attribute vocab_size

### System Info

## Bug Description

INT4 AWQ quantization of Gemma-4-E2B-it fails during TensorRT-LLM checkpoint export.

The quantization/calibration stage completes successfully, but export fails because modelopt expects `model.config.vocab_size`, while Gemma4Config stores it under `model.config.text_config.vocab_size`.

## Environment

- TensorRT-LLM: 1.3.0rc20
- Docker image:
  nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20
- GPU: NVIDIA RTX 4060
- Model:
  google/gemma-4-E2B-it
- Transformers: 5.5.4
- Python: 3.12

## Reproduce

Command:

```bash
root@cfda36728cbe:/app/tensorrt_llm/examples/quantization# python quantize.py --model_dir /app/tensorrt_llm/hf_local/gemma-4-E2B-it --qformat int4_awq --awq_block_size 64 --tp_size 1 --output_dir /app/tensorrt_llm/models/gemma-4-e2b-it-int4-awq
Skipping import of cpp extensions due to incompatible torch version 2.11.0a0+eb65b36914.nv26.02 for torchao version 0.15.0             Please see https://github.com/pytorch/ao/issues/2919 for more info
/usr/local/lib/python3.12/dist-packages/modelopt/torch/__init__.py:36: UserWarning: transformers version 5.5.4 is incompatible with nvidia-modelopt and may cause issues. Please install recommended version with `pip install nvidia-modelopt[hf]` if working with HF models.
  _warnings.warn(
[TensorRT-LLM] TensorRT LLM version: 1.3.0rc20
`torch_dtype` is deprecated! Use `dtype` instead!
Loading weights: 100%|████████████████████████████████████████████████████████████████████| 1951/1951 [00:00<00:00, 3132.51it/s]
Some parameters are on the meta device because they were offloaded to the cpu.
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
README.md: 100%|███████████████████████████████████████████████████████████████████████████| 15.6k/15.6k [00:00<00:00, 48.0MB/s]
3.0.0/train-00000-of-00003.parquet: 100%|████████████████████████████████████████████████████| 257M/257M [00:09<00:00, 27.2MB/s]
3.0.0/train-00001-of-00003.parquet: 100%|████████████████████████████████████████████████████| 257M/257M [00:06<00:00, 39.3MB/s]
3.0.0/train-00002-of-00003.parquet: 100%|████████████████████████████████████████████████████| 259M/259M [00:06<00:00, 37.2MB/s]
3.0.0/validation-00000-of-00001.parquet: 100%|█████████████████████████████████████████████| 34.7M/34.7M [00:01<00:00, 29.4MB/s]
3.0.0/test-00000-of-00001.parquet: 100%|███████████████████████████████████████████████████| 30.0M/30.0M [00:01<00:00, 23.4MB/s]
Generating train split: 100%|████████████████████████████████████████████████| 287113/287113 [00:02<00:00, 133294.21 examples/s]
Generating validation split: 100%|█████████████████████████████████████████████| 13368/13368 [00:00<00:00, 136809.16 examples/s]
Generating test split: 100%|███████████████████████████████████████████████████| 11490/11490 [00:00<00:00, 131229.40 examples/s]
Registered <class 'transformers.models.gemma4.modeling_gemma4.Gemma4TextAttention'> to _QuantAttention for KV Cache quantization
Registered <class 'transformers.models.gemma4.modeling_gemma4.Gemma4VisionAttention'> to _QuantAttention for KV Cache quantization
Registered <class 'transformers.models.gemma4.modeling_gemma4.Gemma4AudioAttention'> to _QuantAttention for KV Cache quantization
Inserted 1773 quantizers
Caching activation statistics for awq_lite...
Searching awq_lite parameters...
Loading extension modelopt_cuda_ext...
Loaded extension modelopt_cuda_ext in 81.6 seconds
/usr/local/lib/python3.12/dist-packages/modelopt/torch/export/model_config_export.py:549: UserWarning: Cannot export model to the model_config. The modelopt-optimized model state_dict can be saved with torch.save for further inspection.
  warn(
Traceback (most recent call last):
  File "/app/tensorrt_llm/examples/quantization/quantize.py", line 160, in <module>
    quantize_and_export(
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/quantization/quantize_by_modelopt.py", line 959, in quantize_and_export
    export_tensorrt_llm_checkpoint(
  File "/usr/local/lib/python3.12/dist-packages/modelopt/torch/export/model_config_export.py", line 553, in export_tensorrt_llm_checkpoint
    raise e
  File "/usr/local/lib/python3.12/dist-packages/modelopt/torch/export/model_config_export.py", line 487, in export_tensorrt_llm_checkpoint
    for (
  File "/usr/local/lib/python3.12/dist-packages/modelopt/torch/export/model_config_export.py", line 152, in torch_to_tensorrt_llm_checkpoint
    vocab_size = model.config.vocab_size
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/transformers/configuration_utils.py", line 422, in __getattribute__
    return super().__getattribute__(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Gemma4Config' object has no attribute 'vocab_size'
root@cfda36728cbe:/app/tensorrt_llm/examples/quantization# 
```

### Who can help?

@Tracin 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

## Reproduction

Using the official TensorRT-LLM quantization example, INT4 AWQ quantization of Gemma-4-E2B-it fails during the export stage.

Environment:

- TensorRT-LLM: 1.3.0rc20
- Docker image: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20
- Model: google/gemma-4-E2B-it

Run:

```bash
export TLLM_DEBUG_MODE=1

python quantize.py \
    --model_dir /app/tensorrt_llm/hf_local/gemma-4-E2B-it \
    --qformat int4_awq \
    --awq_block_size 64 \
    --tp_size 1 \
    --output_dir /app/tensorrt_llm/models/gemma-4-e2b-it-int4-awq
```

The calibration stage completes successfully:

```
Inserted 1773 quantizers
Caching activation statistics for awq_lite...
Searching awq_lite parameters...
Loading extension modelopt_cuda_ext...
```

However, export fails with:

```
AttributeError: 'Gemma4Config' object has no attribute 'vocab_size'
```

The error occurs at:

```
/usr/local/lib/python3.12/dist-packages/modelopt/torch/export/model_config_export.py

line 152:

vocab_size = model.config.vocab_size
```

Gemma4Config stores the vocabulary size in:

```python
model.config.text_config.vocab_size
```

instead of:

```python
model.config.vocab_size
```

### Expected behavior

The quantization process should successfully complete and export a TensorRT-LLM checkpoint for the Gemma-4-E2B-it model.

Expected behavior:

- INT4 AWQ quantization finishes without errors.
- The model is exported to the specified `--output_dir`.
- The output directory contains a valid TensorRT-LLM checkpoint that can be used with `trtllm-build`.

Example expected output:

```
/app/tensorrt_llm/models/gemma-4-e2b-it-int4-awq/
├── config.json
├── model_config.json
├── model.safetensors
└── tokenizer files
```

The quantization/export stage should not fail when using a valid Hugging Face `Gemma4Config` where `vocab_size` is stored under `text_config.vocab_size`.

### actual behavior

The quantization process does not complete successfully.

The AWQ calibration stage finishes successfully, but the export step fails with an AttributeError.

Actual output:

```
Inserted 1773 quantizers
Caching activation statistics for awq_lite...
Searching awq_lite parameters...
Loading extension modelopt_cuda_ext...
Loaded extension modelopt_cuda_ext in 81.6 seconds

Traceback (most recent call last):
  File "/app/tensorrt_llm/examples/quantization/quantize.py", line 160, in <module>
    quantize_and_export(
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/quantization/quantize_by_modelopt.py", line 959, in quantize_and_export
    export_tensorrt_llm_checkpoint(
  File "/usr/local/lib/python3.12/dist-packages/modelopt/torch/export/model_config_export.py", line 152, in torch_to_tensorrt_llm_checkpoint
    vocab_size = model.config.vocab_size
AttributeError: 'Gemma4Config' object has no attribute 'vocab_size'
```

No TensorRT-LLM checkpoint is generated in the specified `--output_dir`.

The failure occurs during the model export stage after quantization parameters have been computed.

### additional notes

Additional notes:

The issue appears to be related to the compatibility between TensorRT-LLM/modelopt export logic and the Hugging Face Gemma4Config structure.

I verified that the Gemma4 configuration does contain the vocabulary size, but it is stored as:

```python
model.config.text_config.vocab_size
```

and not directly as:

```python
model.config.vocab_size
```

Verification:

```python
from transformers import AutoConfig

config = AutoConfig.from_pretrained(
    "/app/tensorrt_llm/hf_local/gemma-4-E2B-it"
)

print(type(config))
# <class 'transformers.models.gemma4.configuration_gemma4.Gemma4Config'>

print(hasattr(config, "vocab_size"))
# False

print(config.text_config.vocab_size)
# 262144
```

A possible compatibility fix may be to handle multimodal configs by falling back to `text_config.vocab_size` when `config.vocab_size` is unavailable.

The quantization process itself appears to work correctly until the TensorRT-LLM checkpoint export step.

### Before submitting a new issue...

- [x] Make sure you already searched for relevant issues, and checked the [documentation](https://nvidia.github.io/TensorRT-LLM/) and [examples](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples) for answers to frequently asked questions.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Bug]: Gemma4 INT4 AWQ quantization fails during export: Gemma4Config has no attribute vocab_size #16171

System Info

Bug Description

Environment

Reproduce

Who can help?

Information

Tasks

Reproduction

Reproduction

Expected behavior

actual behavior

additional notes

Before submitting a new issue...

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Uh oh!

[Bug]: Gemma4 INT4 AWQ quantization fails during export: Gemma4Config has no attribute vocab_size #16171

Description

System Info

Bug Description

Environment

Reproduce

Who can help?

Information

Tasks

Reproduction

Reproduction

Expected behavior

actual behavior

additional notes

Before submitting a new issue...

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions