Update provider spec & upgrade to llama-stack 0.2.22

[dmaniloff]

• Split into inline & remote modules.
• Update run-inline.yaml and run-remote.yaml to include new APIs and adjust paths.
• Remove deprecated provider YAML files and implement new inline and remote provider structures.
• Modify inline and remote demo notebooks to reflect updated model identifiers and re-run w/ new LLS version.
• Update tests & docs.
• Version bump.

[saichandrapandraju]

LGTM! Thanks @dmaniloff

[nathan-weinberg]

Generally this LGTM, but please wait for @cdoern for approve before merging

[cdoern]

comment about module usage, looks like its moving in the right direction!

[dmaniloff]

@cdoern here's what @saichandrapandraju and i discussed, please let us know what you think:

option 1: leave as-is, it will work.

• why? we currently specify the package name and version in build.yaml, that will pull in the provider package and install OK. i just re-tested w/ 0.2.22
• we then specify the module in run.yaml and use the .inline / .remote specifications
• the only issue w/ this approach is that the auto-generated run.yaml that results from running llama stack build will have module==package name which is incorrect. not ideal, but we are providing a custom run.yaml anyways.

option 2: make the default provider the remote one.

• this means just adding a top-level provider.py and from there pointing to the default one. ie., from .remote.provider import get_provider_spec
• add documentation in the package to explain to users that if they want the inline version they need to modify run.yaml to use .inline in the module arg.

suggestion: given that we do provide a customized run.yaml i think for now option 1 is good, also taking in to account that you have your refactor in the queue to simplify this process.

ps: as i write this i realize we also have a third option which combines both 1 & 2, by specifying the remote version of the provider as default (to be on the safe side so that top-level imports just work and also the auto-generated run.yaml works), and we fully specify the .inline and .remote in the yamls of the red hat lls distro to show how they can be selected.

[cdoern]

If this works, I am good with it.

I just want to confirm:

the build yaml module is llama_stack_provider_ragas. and then the run.yaml module will be llama_stack_provider_ragas.remote and this works, right?

[dmaniloff]

Correct! And we will provide a default too, so that llama_stack_provider_ragas as a run module will work as well.

[cdoern]

a note. When you bump to 0.2.23+ this will break: ogx-ai/ogx#3378

[ruivieira]

#21

[dmaniloff]

@nathan-weinberg i will merge this, then release a new version, then update my comments to the correct version of llama_stack_provider_ragas in opendatahub-io/ogx-distribution#52