Add evals with Inspect guide to Inference Providers

[dvsrepo]

No description provided.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[davanstrien]

Very nice! Love the example! Made a few small comments and nit suggestion for language (feel free to ignore!)

[pcuenca]

Very cool! 🔥

[NathanHB]

great blog post ! Don't know if it's in scope of the blog but lighteval is moving towards using inspect-ai as well. I'm adding a few utilities for comparing and evaluating hf-inference-providers. For example I added :all to be able to compare all the providers in one line. If you think that's interesting might be worth mentioning at the end!

[pcuenca]

it's in scope of the blog but lighteval

I had the same thought, not sure if in scope either. Maybe worth adding just a sentence with a link, but your call (of course) @dvsrepo

[dvsrepo]

great blog post ! Don't know if it's in scope of the blog but lighteval is moving towards using inspect-ai as well. I'm adding a few utilities for comparing and evaluating hf-inference-providers. For example I added :all to be able to compare all the providers in one line. If you think that's interesting might be worth mentioning at the end!

it's in scope of the blog but lighteval

I had the same thought, not sure if in scope either. Maybe worth adding just a sentence with a link, but your call (of course) @dvsrepo

@NathanHB, it would be awesome to add a reference/references to lighteval, even at each section (example) level. Would you be open to add some changes, or you prefer I add them?

[NathanHB]

Added a small line at the end, feel free to rephrase if needed :)

[dvsrepo]

Added a small line at the end, feel free to rephrase if needed :)

Great @NathanHB ! I rephrased it a bit and moved it to the second bullet point, let me know if the working seems right to you

[dvsrepo]

we're all set from my side, thanks a lot for the reviews @pcuenca @davanstrien @NathanHB !

[Vaibhavs10]

Nice! Looks great! 🔥

[Vaibhavs10]

Merged this - open PRs are detrimental to progress ^^ (we can iterate afterwards if needed)

[dvsrepo]

Merged this - open PRs are detrimental to progress ^^ (we can iterate afterwards if needed)

Thanks for merging @Vaibhavs10 !