Make test sut options consistent and easier to access.#1425
Conversation
superdosh
had a problem deploying
to
Scheduled Testing
GitHub Actions
Failure
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
superdosh
had a problem deploying
to
Scheduled Testing
GitHub Actions
Failure
superdosh
temporarily deployed
to
Scheduled Testing
GitHub Actions
Inactive
superdosh
requested review from
bkorycki,
bollacker,
dhosterman,
rogthefrog and
wpietri
|
@bollacker adding you to confirm the numbers. |
| """ | ||
|
|
||
| hazards = Hazards() | ||
| _sut_options = SUTOptions( |
There was a problem hiding this comment.
Is this something that could be in a superclass of the safe test and the security test, or is that too much of a twee "optimization"?
There was a problem hiding this comment.
It is, actually: https://github.com/mlcommons/modelbench/blob/main/src/modelgauge/base_test.py#L30
I wasn't sure we'd want the same default in the superclass though? I'm not sure; could be convinced.
wpietri
left a comment
wpietri
left a comment
There was a problem hiding this comment.
Nice! A good step forward.
|
Holding on merging this until after break. |
bkorycki
left a comment
bkorycki
left a comment
There was a problem hiding this comment.
I don't really understand what this is achieving to be honest. But go ahead and merge it if you think its necessary!
|
Discussed live with @bkorycki and we're on the same page! |
4 participants
No description provided.