Skip to content

Coerce float timestamps to int for HF output.#1291

Merged
superdosh merged 3 commits into
mainfrom
hf-timestamp-coercion
Sep 15, 2025
Merged

Coerce float timestamps to int for HF output.#1291
superdosh merged 3 commits into
mainfrom
hf-timestamp-coercion

Conversation

@superdosh

@superdosh superdosh commented Sep 15, 2025

Copy link
Copy Markdown
Contributor

This command failed with the error described in #1287

poetry run modelgauge run-sut --sut meta-llama/Llama-4-Maverick-17B-128E-Instruct:sambanova:hfrelay --prompt "hello"

Command works after this fix.

Adding: I chose to truncate to int because the official HF docs state that the output should be an int, not float: https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#response

@github-actions

github-actions Bot commented Sep 15, 2025

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@superdosh superdosh marked this pull request as ready for review September 15, 2025 18:59
@superdosh superdosh requested a review from a team as a code owner September 15, 2025 18:59

@wpietri wpietri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And the first one in with a fix is Vishal! 🚀

fake_sut.evaluate(request)


def test_huggingface_chat_completion_output_created_coercion():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Very clear documentation in test form.

@bkorycki bkorycki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@superdosh superdosh merged commit 10c47ae into main Sep 15, 2025
2 checks passed
@superdosh superdosh deleted the hf-timestamp-coercion branch September 15, 2025 19:46
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 15, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants