I believe it's important to record the actual guidellm version with the test results.
In the past I have seen significant metric variances between guidellm versions running the same workload on the same platform (identical hardware and vllm version).
Currently the ansible controller console emits the GuideLLM container image-name as
"latest", which is not specific enough.
Also the GuideLLM generated 'benchmarks.json' does contain a field labeled 'guidellm_version', I think it would be wise to promote this info to also be in the test-metadata.json file.
I believe it's important to record the actual guidellm version with the test results.
In the past I have seen significant metric variances between guidellm versions running the same workload on the same platform (identical hardware and vllm version).
Currently the ansible controller console emits the GuideLLM container image-name as
"latest", which is not specific enough.
Also the GuideLLM generated 'benchmarks.json' does contain a field labeled 'guidellm_version', I think it would be wise to promote this info to also be in the test-metadata.json file.