Eval bug: regression introduced in #20503

[EAddario]

Name and Version

Any versions post #20503

Operating systems

Mac

GGML backends

Metal

Hardware

All hardware

Models

Possibly all models

Problem description & steps to reproduce

LLM_TN_IMPL::str() now includes a check to verify if a given tensor is defined in the current model architecture's model_tensors list.

If a model includes tensors that might not be strictly listed in the architecture definition, like position_embd or token_types for example, the console is spammed with many

str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1

To reproduce, simply quantize a model like: llama-quantize Qwen3.5-9B-F16.gguf Model-Q4_K.gguf Q4_K 12

First Bad Commit

#20503

Relevant log output

llama-quantize Qwen3.5-9B-F16.gguf Model-Q4_K.gguf Q4_K 12
main: build = 8563 (1f5d15e66)
main: built with AppleClang 17.0.0.17000604 for Darwin arm64
main: quantizing 'Qwen3.5-9B-F16.gguf' to 'Model-Q4_K.gguf' as Q4_K using 12 threads
llama_model_loader: loaded meta data with 41 key-value pairs and 427 tensors from Qwen3.5-9B-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen35
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Qwen3.5 9B
llama_model_loader: - kv   3:                           general.basename str              = Qwen3.5
llama_model_loader: - kv   4:                         general.size_label str              = 9B
llama_model_loader: - kv   5:                            general.license str              = apache-2.0
llama_model_loader: - kv   6:                       general.license.link str              = https://huggingface.co/Qwen/Qwen3.5-9...
llama_model_loader: - kv   7:                   general.base_model.count u32              = 1
llama_model_loader: - kv   8:                  general.base_model.0.name str              = Qwen3.5 9B Base
llama_model_loader: - kv   9:          general.base_model.0.organization str              = Qwen
llama_model_loader: - kv  10:              general.base_model.0.repo_url str              = https://huggingface.co/Qwen/Qwen3.5-9...
llama_model_loader: - kv  11:                               general.tags arr[str,1]       = ["image-text-to-text"]
llama_model_loader: - kv  12:                         qwen35.block_count u32              = 32
llama_model_loader: - kv  13:                      qwen35.context_length u32              = 262144
llama_model_loader: - kv  14:                    qwen35.embedding_length u32              = 4096
llama_model_loader: - kv  15:                 qwen35.feed_forward_length u32              = 12288
llama_model_loader: - kv  16:                qwen35.attention.head_count u32              = 16
llama_model_loader: - kv  17:             qwen35.attention.head_count_kv u32              = 4
llama_model_loader: - kv  18:             qwen35.rope.dimension_sections arr[i32,4]       = [11, 11, 10, 0]
llama_model_loader: - kv  19:                      qwen35.rope.freq_base f32              = 10000000.000000
llama_model_loader: - kv  20:    qwen35.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  21:                qwen35.attention.key_length u32              = 256
llama_model_loader: - kv  22:              qwen35.attention.value_length u32              = 256
llama_model_loader: - kv  23:                          general.file_type u32              = 1
llama_model_loader: - kv  24:                     qwen35.ssm.conv_kernel u32              = 4
llama_model_loader: - kv  25:                      qwen35.ssm.state_size u32              = 128
llama_model_loader: - kv  26:                     qwen35.ssm.group_count u32              = 16
llama_model_loader: - kv  27:                  qwen35.ssm.time_step_rank u32              = 32
llama_model_loader: - kv  28:                      qwen35.ssm.inner_size u32              = 4096
llama_model_loader: - kv  29:             qwen35.full_attention_interval u32              = 4
llama_model_loader: - kv  30:                qwen35.rope.dimension_count u32              = 64
llama_model_loader: - kv  31:               general.quantization_version u32              = 2
llama_model_loader: - kv  32:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  33:                         tokenizer.ggml.pre str              = qwen35
llama_model_loader: - kv  34:                      tokenizer.ggml.tokens arr[str,248320]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  35:                  tokenizer.ggml.token_type arr[i32,248320]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  36:                      tokenizer.ggml.merges arr[str,247587]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  37:                tokenizer.ggml.eos_token_id u32              = 248046
llama_model_loader: - kv  38:            tokenizer.ggml.padding_token_id u32              = 248044
llama_model_loader: - kv  39:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  40:                    tokenizer.chat_template str              = {%- set image_count = namespace(value...
llama_model_loader: - type  f32:  177 tensors
llama_model_loader: - type  f16:  250 tensors
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
...

[donington]

I quickly tested against b7825 and b8565 and can confirm that the spam happens in the latest build but not the previous (these were just builds I had available to test).

Same result as EAddario, lots of position_embd and token_types spewed to the console.

Here's the specific change causing the noise: src/llama-arch.cpp:2758

[0xA50C1A1]

Same thing for Ministral 3 14B Instruct 2512:

llama-quantize Ministral-3-14B-Instruct-2512-RP-FFT.BF16.gguf Ministral-3-14B-Instruct-2512-RP-FFT.Q6_K.gguf Q6_K
main: build = 8664 (9c699074c)
main: built with GNU 13.3.0 for Linux x86_64
main: quantizing 'Ministral-3-14B-Instruct-2512-RP-FFT.BF16.gguf' to 'Ministral-3-14B-Instruct-2512-RP-FFT.Q6_K.gguf' as Q6_K
llama_model_loader: loaded meta data with 42 key-value pairs and 363 tensors from Ministral-3-14B-Instruct-2512-RP-FFT.BF16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = mistral3
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Ministral 3 14B Instruct 2512 RP FFT
llama_model_loader: - kv   3:                            general.version str              = 2512
llama_model_loader: - kv   4:                           general.finetune str              = Instruct-RP-FFT
llama_model_loader: - kv   5:                           general.basename str              = Ministral-3
llama_model_loader: - kv   6:                         general.size_label str              = 14B
llama_model_loader: - kv   7:                       mistral3.block_count u32              = 40
llama_model_loader: - kv   8:                    mistral3.context_length u32              = 262144
llama_model_loader: - kv   9:                  mistral3.embedding_length u32              = 5120
llama_model_loader: - kv  10:               mistral3.feed_forward_length u32              = 16384
llama_model_loader: - kv  11:              mistral3.attention.head_count u32              = 32
llama_model_loader: - kv  12:           mistral3.attention.head_count_kv u32              = 8
llama_model_loader: - kv  13:                 mistral3.rope.scaling.type str              = yarn
llama_model_loader: - kv  14:               mistral3.rope.scaling.factor f32              = 16.000000
llama_model_loader: - kv  15: mistral3.rope.scaling.original_context_length u32              = 16384
llama_model_loader: - kv  16:       mistral3.rope.scaling.yarn_beta_fast f32              = 32.000000
llama_model_loader: - kv  17:       mistral3.rope.scaling.yarn_beta_slow f32              = 1.000000
llama_model_loader: - kv  18:                    mistral3.rope.freq_base f32              = 1000000000.000000
llama_model_loader: - kv  19:  mistral3.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  20:              mistral3.attention.key_length u32              = 128
llama_model_loader: - kv  21:            mistral3.attention.value_length u32              = 128
llama_model_loader: - kv  22:                          general.file_type u32              = 32
llama_model_loader: - kv  23:                        mistral3.vocab_size u32              = 131072
llama_model_loader: - kv  24:              mistral3.rope.dimension_count u32              = 128
llama_model_loader: - kv  25:  mistral3.rope.scaling.yarn_log_multiplier f32              = 1.000000
llama_model_loader: - kv  26:       mistral3.attention.temperature_scale f32              = 0.100000
llama_model_loader: - kv  27:               general.quantization_version u32              = 2
llama_model_loader: - kv  28:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  29:                         tokenizer.ggml.pre str              = tekken
llama_model_loader: - kv  30:                      tokenizer.ggml.tokens arr[str,131072]  = ["<unk>", "<s>", "</s>", "[INST]", "[...
llama_model_loader: - kv  31:                  tokenizer.ggml.token_type arr[i32,131072]  = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ...
llama_model_loader: - kv  32:                      tokenizer.ggml.merges arr[str,269443]  = ["Ġ Ġ", "Ġ t", "e r", "i n", "Ġ �...
llama_model_loader: - kv  33:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  34:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  35:            tokenizer.ggml.unknown_token_id u32              = 0
llama_model_loader: - kv  36:            tokenizer.ggml.padding_token_id u32              = 11
llama_model_loader: - kv  37:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  38:               tokenizer.ggml.add_sep_token bool             = false
llama_model_loader: - kv  39:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  40:                    tokenizer.chat_template str              = {#- Default system message if no syst...
llama_model_loader: - kv  41:            tokenizer.ggml.add_space_prefix bool             = false
llama_model_loader: - type  f32:   81 tensors
llama_model_loader: - type bf16:  282 tensors
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 15827 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15827 MiB
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1

[MoonRide303]

Observing the same str: cannot properly format tensor name messages when quantizing gemma-4-31B-it models using b8680:

Details

llama-quantize.exe --imatrix .\gemma-4-31B-it-imatrix.gguf .\gemma-4-31B-it-heretic-F16.gguf .\gemma-4-31B-it-heretic-IQ3_XS.gguf IQ3_XS
load_imatrix: imatrix datasets=['/training_dir/calibration_datav5.txt']
load_imatrix: loaded 410 importance matrix entries from .\gemma-4-31B-it-imatrix.gguf computed on 886 chunks
prepare_imatrix: have 410 importance matrix entries
main: build = 8680 (15f786e65)
main: built with MSVC 19.44.35222.0 for x64
main: quantizing '.\gemma-4-31B-it-heretic-F16.gguf' to '.\gemma-4-31B-it-heretic-IQ3_XS.gguf' as IQ3_XS
llama_model_loader: loaded meta data with 44 key-value pairs and 833 tensors from .\gemma-4-31B-it-heretic-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = gemma4
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 64
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
llama_model_loader: - kv   5:                               general.name str              = Gemma 4 31B It Heretic
llama_model_loader: - kv   6:                           general.finetune str              = it-heretic
llama_model_loader: - kv   7:                           general.basename str              = gemma-4
llama_model_loader: - kv   8:                         general.size_label str              = 31B
llama_model_loader: - kv   9:                         gemma4.block_count u32              = 60
llama_model_loader: - kv  10:                      gemma4.context_length u32              = 262144
llama_model_loader: - kv  11:                    gemma4.embedding_length u32              = 5376
llama_model_loader: - kv  12:                 gemma4.feed_forward_length u32              = 21504
llama_model_loader: - kv  13:                gemma4.attention.head_count u32              = 32
llama_model_loader: - kv  14:             gemma4.attention.head_count_kv arr[i32,60]      = [16, 16, 16, 16, 16, 4, 16, 16, 16, 1...
llama_model_loader: - kv  15:                      gemma4.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  16:                  gemma4.rope.freq_base_swa f32              = 10000.000000
llama_model_loader: - kv  17:    gemma4.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  18:                gemma4.attention.key_length u32              = 512
llama_model_loader: - kv  19:              gemma4.attention.value_length u32              = 512
llama_model_loader: - kv  20:                          general.file_type u32              = 1
llama_model_loader: - kv  21:             gemma4.final_logit_softcapping f32              = 30.000000
llama_model_loader: - kv  22:            gemma4.attention.sliding_window u32              = 1024
llama_model_loader: - kv  23:          gemma4.attention.shared_kv_layers u32              = 0
llama_model_loader: - kv  24:    gemma4.embedding_length_per_layer_input u32              = 0
llama_model_loader: - kv  25:    gemma4.attention.sliding_window_pattern arr[bool,60]     = [true, true, true, true, true, false,...
llama_model_loader: - kv  26:            gemma4.attention.key_length_swa u32              = 256
llama_model_loader: - kv  27:          gemma4.attention.value_length_swa u32              = 256
llama_model_loader: - kv  28:                gemma4.rope.dimension_count u32              = 512
llama_model_loader: - kv  29:            gemma4.rope.dimension_count_swa u32              = 256
llama_model_loader: - kv  30:               general.quantization_version u32              = 2
llama_model_loader: - kv  31:                       tokenizer.ggml.model str              = gemma4
llama_model_loader: - kv  32:                      tokenizer.ggml.tokens arr[str,262144]  = ["<pad>", "<eos>", "<bos>", "<unk>", ...
llama_model_loader: - kv  33:                      tokenizer.ggml.scores arr[f32,262144]  = [-1000.000000, -1000.000000, -1000.00...
llama_model_loader: - kv  34:                  tokenizer.ggml.token_type arr[i32,262144]  = [3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  35:                      tokenizer.ggml.merges arr[str,514906]  = ["\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n", ...
llama_model_loader: - kv  36:                tokenizer.ggml.bos_token_id u32              = 2
llama_model_loader: - kv  37:                tokenizer.ggml.eos_token_id u32              = 1
llama_model_loader: - kv  38:            tokenizer.ggml.unknown_token_id u32              = 3
llama_model_loader: - kv  39:            tokenizer.ggml.padding_token_id u32              = 0
llama_model_loader: - kv  40:               tokenizer.ggml.mask_token_id u32              = 4
llama_model_loader: - kv  41:                    tokenizer.chat_template str              = {%- macro format_parameters(propertie...
llama_model_loader: - kv  42:            tokenizer.ggml.add_space_prefix bool             = false
llama_model_loader: - kv  43:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - type  f32:  422 tensors
llama_model_loader: - type  f16:  411 tensors
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 16375 MiB):
  Device 0: NVIDIA GeForce RTX 4080, compute capability 8.9, VMM: yes, VRAM: 16375 MiB

llama_model_quantize_impl: have importance matrix data with 410 entries
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
(...)

A bit annoying, as it's being repeated like few hundreds times. @JohannesGaessler could you take look at it, by any chance?

[EAddario]

Issue fixed by #21531