Dynamic Dimensions Guide

A practical guide for choosing dimension override values when converting ONNX models to WebNN format.

Understanding Dynamic Dimensions
Inspection Methods
Common Values by Model Type
Decision-Making Process
Troubleshooting

Understanding Dynamic Dimensions

What Are Dynamic Dimensions?

ONNX models often use symbolic dimensions (like batch_size, sequence_length) instead of fixed numbers. This allows the same model to handle different input sizes.

Example ONNX input shape:

input_ids: [batch_size, sequence_length]  # Dynamic

In many models, shape-driving expressions must become static for conversion:

input_ids: [1, 128]  # Static

Why Provide Overrides?

WebNN executes in browsers and edge devices where:

Memory must be allocated upfront
Shape-driving expressions (for example, reshape targets) must be resolvable
Performance is optimized for specific sizes

webnn-graph can preserve unresolved symbolic input metadata in v2 graphs with --experimental-dynamic-inputs, but conversion still needs concrete values when dynamic shape math cannot be folded.

Inspection Methods

Method 1: Using Python + ONNX

pip install onnxslim
onnxslim --inspect model.onnx

Method 2: Using Netron

Install Netron: pip install netron
Open model: netron model.onnx
Click on input nodes to see shape information
Look for dimension parameters like batch_size, seq_len, etc.

Method 3: Check Model Documentation

Most models on Hugging Face include dimension information:

# Visit the model page
# https://huggingface.co/<org>/<model-name>

# Look for:
# - "Model Details" section
# - "max_seq_length" in config
# - Example usage code

Method 4: Check the Sidecar File

webnn-graph supports automatic dimension discovery via .dims.json files:

# If model.onnx has a model.dims.json file, check it:
cat model.dims.json

Example content:

{
  "freeDimensionOverrides": {
    "batch_size": 1,
    "sequence_length": 128
  }
}

Common Values by Model Type

Text / NLP Models (Transformers, BERT, GPT)

Typical dimensions:

--override-dim batch_size=1
--override-dim sequence_length=<128|256|512>

Choosing sequence_length:

Model Type	Recommended	Max	Use Case
Sentence embeddings (MiniLM, MPNet)	128	256	Sentences, titles, short text
Document classification (BERT)	256	512	Paragraphs, articles
Question answering (BERT, RoBERTa)	384	512	Q&A pairs with context
Text generation (GPT-2)	512	1024	Long-form generation
Long-document (Longformer)	1024	4096	Full documents

How to determine:

# Check tokenizer max length
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("model-name")
print(f"Max length: {tokenizer.model_max_length}")

Common dimension parameter names:

batch_size, batch, N, B
sequence_length, seq_len, max_len, T, L
hidden_size, hidden_dim (usually fixed, not dynamic)

Vision Models (CNNs, Vision Transformers)

Typical dimensions:

--override-dim batch_size=1
--override-dim height=224
--override-dim width=224

Standard image sizes by architecture:

Architecture	Size	Notes
ResNet-50/101	224×224	ImageNet standard
EfficientNet-B0	224×224	Scales up with variants
EfficientNet-B7	600×600	Higher accuracy, slower
Vision Transformer (ViT-B)	224×224 or 384×384	Two common variants
MobileNet V2/V3	224×224	Mobile-optimized
YOLO (object detection)	416×416 or 640×640	Detection-specific
Semantic segmentation	512×512 or 1024×1024	Full-resolution

How to determine:

# Check preprocessing configuration
from transformers import AutoImageProcessor
processor = AutoImageProcessor.from_pretrained("model-name")
print(f"Size: {processor.size}")

Common dimension parameter names:

batch_size, batch, N, B
height, H, image_height
width, W, image_width
channels, C (usually 3 for RGB, 1 for grayscale)

Audio Models (Speech Recognition, Audio Classification)

Typical dimensions:

--override-dim batch_size=1
--override-dim sequence_length=<varies>  # Based on audio duration

Determining sequence_length for audio:

# Formula: sequence_length = sample_rate * duration_seconds / hop_length

# Example for Wav2Vec2 (16kHz, 10 seconds)
sample_rate = 16000
duration = 10  # seconds
hop_length = 320  # model-specific
sequence_length = (sample_rate * duration) // hop_length
# Result: ~500 for 10-second audio

Common values:

Whisper: Processes 30-second chunks → sequence_length based on mel spectrogram frames
Wav2Vec2: Variable based on audio duration
Audio classification: Often 16000 samples (1 second at 16kHz)

Decision-Making Process

Step-by-Step Guide

Step 1: Identify Dynamic Dimensions That Need Values

Try converting without overrides first:

./webnn-graph convert-onnx --input model.onnx

If conversion cannot resolve required dims, error output will indicate what to set:

Error: unresolved dynamic dimension(s) require explicit overrides:
 - input 'input_ids' dim 'batch_size': --override-dim batch_size=<value>
 - input 'input_ids' dim 'sequence_length': --override-dim sequence_length=<value>

Step 2: Determine Model Type

Look at the model filename or documentation:

*bert*, *roberta*, *gpt* → Text model
*resnet*, *efficientnet*, *vit* → Vision model
*wav2vec*, *whisper* → Audio model

Step 3: Start with Conservative Values

Always start with:

batch_size=1 (single inference)
Smallest reasonable size for other dimensions

Why start small?

Faster conversion and testing
Less memory usage
Easier to debug
Can always increase later

Step 4: Look Up Standard Values

Use the tables in Common Values by Model Type above.

Step 5: Test and Iterate

# Test with initial values
./webnn-graph convert-onnx \
  --input model.onnx \
  --override-dim batch_size=1 \
  --override-dim sequence_length=128

# If successful, test inference
# If shapes are wrong, adjust and retry

Example Workflows

Example 1: BERT Sentence Embeddings

# Model: sentence-transformers/all-MiniLM-L12-v2
# Task: Generate sentence embeddings

# 1. Check documentation
# Hugging Face says: max_seq_length = 256

# 2. Choose conservative value
# Most sentences < 128 tokens, so start there

# 3. Convert
./webnn-graph convert-onnx \
  --input all-MiniLM-L12-v2.onnx \
  --override-dim batch_size=1 \
  --override-dim sequence_length=128 \
  --optimize

# 4. If you need longer sequences, increase
./webnn-graph convert-onnx \
  --input all-MiniLM-L12-v2.onnx \
  --override-dim batch_size=1 \
  --override-dim sequence_length=256 \
  --optimize

Example 2: ResNet Image Classification

# Model: ResNet-50
# Task: Image classification

# 1. Standard size for ResNet is 224×224
# 2. No need to check - this is well-known

# 3. Convert
./webnn-graph convert-onnx \
  --input resnet50.onnx \
  --override-dim batch_size=1 \
  --override-dim height=224 \
  --override-dim width=224

Example 3: Unknown Custom Model

# 1. Inspect the model
python -c "
import onnx
model = onnx.load('custom_model.onnx')
for inp in model.graph.input:
    print(inp.name, inp.type.tensor_type.shape)
"

# Output shows: data [N, 3, H, W]
# This is an image model (3 channels)

# 2. Try standard image sizes
./webnn-graph convert-onnx \
  --input custom_model.onnx \
  --override-dim N=1 \
  --override-dim H=224 \
  --override-dim W=224

# 3. If conversion fails, check error messages
# 4. Try other common sizes: 256, 299, 384, 512

Troubleshooting

Problem: "Dynamic dimensions require explicit overrides"

Solution: Some model paths still need concrete values for symbolic dimensions.

# Error shows which dimensions need values
Error: unresolved dynamic dimension(s) require explicit overrides:
 - input 'input' dim 'height': --override-dim height=<value>

# Provide the missing dimensions
./webnn-graph convert-onnx \
  --input model.onnx \
  --override-dim height=224 \
  --override-dim width=224

Problem: "Shape inference failed"

Cause: The dimension values you provided are incompatible with the model's operations.

Solution:

Check if the model has specific size requirements:

# Some models only work with specific sizes
# e.g., YOLO expects multiples of 32

Try multiples of common factors:

# For CNNs: try 224, 256, 288, 320, 384, 416, 512
# For transformers: try 128, 256, 384, 512

Enable optimization to help with shape inference:

./webnn-graph convert-onnx \
  --input model.onnx \
  --optimize \
  --override-dim ...

Problem: "Constant folding failed"

Cause: The model uses dynamic operations that can't be resolved with the given dimensions.

Solution:

Make sure you've provided ALL dynamic dimensions:

# Check for missing dimensions
python -c "
import onnx
model = onnx.load('model.onnx')
for inp in model.graph.input:
    for dim in inp.type.tensor_type.shape.dim:
        if dim.dim_param:
            print(f'Dynamic: {dim.dim_param}')
"

Use the --optimize flag:

./webnn-graph convert-onnx \
  --input model.onnx \
  --optimize \
  --override-dim batch_size=1 \
  --override-dim sequence_length=128

Problem: Conversion succeeds but inference fails

Cause: The dimension values are too small for your actual input data.

Solution:

Check your actual input size:

// In JavaScript
console.log("Input shape:", inputData.shape);

Increase dimensions to match:

# If your inputs are 256 tokens but you used 128
./webnn-graph convert-onnx \
  --input model.onnx \
  --override-dim sequence_length=256  # Increased from 128

Problem: Out of memory during conversion

Cause: Dimension values are too large.

Solution:

Reduce to smaller values:

# Instead of 512, try 256
# Instead of 256, try 128

Use batch_size=1 (not higher):

--override-dim batch_size=1  # Always use 1 for inference

Best Practices

1. Always Start with batch_size=1

For inference in WebNN (browsers/edge devices):

--override-dim batch_size=1

Only increase batch size if you're doing batch processing server-side.

2. Match Your Use Case

Choose dimensions based on your actual inputs:

Short texts (tweets, titles): sequence_length=128
Medium texts (articles): sequence_length=256
Long texts (documents): sequence_length=512+

3. Consider Memory Constraints

Larger dimensions = more memory:

Memory ∝ batch_size × sequence_length × hidden_size

For browser inference, prefer smaller dimensions.

4. Use Standard Values When Possible

Standard sizes are well-tested:

Images: 224, 256, 384, 512
Text: 128, 256, 512, 1024
These are powers of 2 or common multiples

5. Document Your Choices

Create a .dims.json file alongside your model:

{
  "freeDimensionOverrides": {
    "batch_size": 1,
    "sequence_length": 128
  },
  "notes": "128 tokens handles 95% of our sentences. Max length is 256 if needed."
}

6. Test with Real Data

After conversion, test with actual inputs:

// Verify converted model works with your data
const result = await context.compute(graph, {
  input_ids: actualInputIds,  // Your real input
  attention_mask: actualMask
});

Quick Reference

Text Models

--override-dim batch_size=1 --override-dim sequence_length=128

Vision Models

--override-dim batch_size=1 --override-dim height=224 --override-dim width=224

Unknown Model

# 1. Try without overrides (see error message)
# 2. Check model documentation
# 3. Start with smallest reasonable values
# 4. Iterate based on errors/results

Additional Resources

ONNX Model Zoo - Standard models with known shapes
Hugging Face Model Hub - Model documentation
Netron - Visual model inspector
WebNN Specification - WebNN requirements

Summary

The decision process:

Try converting without overrides and capture unresolved dimensions from the error
Determine model type (text/vision/audio)
Look up standard values for that model type
Start with conservative (small) values
Test and iterate as needed

Most common pattern:

./webnn-graph convert-onnx \
  --input model.onnx \
  --optimize \
  --override-dim batch_size=1 \
  --override-dim <other_dims>=<standard_value>

Remember: batch_size=1 is almost always correct for inference!

Uh oh!

FilesExpand file tree

dynamic-dimensions-guide.md

Latest commit

History

dynamic-dimensions-guide.md

File metadata and controls

Dynamic Dimensions Guide

Table of Contents

Understanding Dynamic Dimensions

What Are Dynamic Dimensions?

Why Provide Overrides?

Inspection Methods

Method 1: Using Python + ONNX

Method 2: Using Netron

Method 3: Check Model Documentation

Method 4: Check the Sidecar File

Common Values by Model Type

Text / NLP Models (Transformers, BERT, GPT)

Vision Models (CNNs, Vision Transformers)

Audio Models (Speech Recognition, Audio Classification)

Decision-Making Process

Step-by-Step Guide

Step 1: Identify Dynamic Dimensions That Need Values

Step 2: Determine Model Type

Step 3: Start with Conservative Values

Step 4: Look Up Standard Values

Step 5: Test and Iterate

Example Workflows

Example 1: BERT Sentence Embeddings

Example 2: ResNet Image Classification

Example 3: Unknown Custom Model

Troubleshooting

Problem: "Dynamic dimensions require explicit overrides"

Problem: "Shape inference failed"

Problem: "Constant folding failed"

Problem: Conversion succeeds but inference fails

Problem: Out of memory during conversion

Best Practices

1. Always Start with batch_size=1

2. Match Your Use Case

3. Consider Memory Constraints

4. Use Standard Values When Possible

5. Document Your Choices

6. Test with Real Data

Quick Reference

Text Models

Vision Models

Unknown Model

Additional Resources

Summary