Trying to run benchmark.py script results "too many values to upack error"

[prasadwrites]

I am trying to run the benchmark.py script using as shown below :

mpirun -n 1 python3 benchmarks/python/benchmark.py -m starcoder_15.5b --mode plugin --batch_size "1;4;8;" --input_output_len "200;200"

This results in following error:

/python/gpt_benchmark.py", line 206, in get_config
for inlen, outlen in self.in_out_lens:
ValueError: too many values to unpack (expected 2)

Seems one should not unpack the values from self.in_out_lens to inlen and outlen as seen in the gpt_benchmark.py

Any one know how to fix that error ?

[Advait251206]

The error suggests that self.in_out_lens is not being parsed into a list of (input_len, output_len) tuples as the benchmark expects.

In gpt_benchmark.py, the code:

for inlen, outlen in self.in_out_lens:

assumes that each element of self.in_out_lens is a pair, e.g.:

[(200, 200), (512, 128)]

A ValueError: too many values to unpack (expected 2) typically means that one of the elements is being interpreted as something else (for example, a string or a list with more than two values), rather than a 2-tuple.

A couple of things to check:

• Verify the expected format for the --input_output_len argument for the TensorRT-LLM version you're using. The CLI parsing has changed across releases, and older benchmark scripts may expect a different separator or syntax.
• Try removing the trailing semicolon from your arguments. For example:

  --batch_size "1;4;8"
  --input_output_len "200;200"

  An extra delimiter can sometimes result in an empty or malformed entry being parsed.
• Print or inspect self.in_out_lens just before the failing loop to confirm its actual contents. That will quickly reveal whether the argument parser is producing the expected data structure or if it's a parsing bug.

If self.in_out_lens turns out to be something like:

"200;200"

instead of:

[(200, 200)]

then the issue is likely in the argument parsing rather than in the unpacking itself.

Could you also share:

• The exact TensorRT-LLM commit or release you're using.
• The implementation of the CLI parser for --input_output_len (or the relevant section of benchmark.py).
• The value of self.in_out_lens immediately before the loop.

That would help determine whether this is a usage issue or a bug in that particular release.

---

If this answer helped or pointed you in the right direction, I'd appreciate it if you could mark it as the accepted answer so it's easier for others with the same issue to find.

Also, if you found my contribution useful, I'd appreciate it if you could check out my GitHub profile, follow me, and star any repositories you find interesting.

GitHub: https://github.com/Advait251206