Feature Request: ggml model cache and kv cache in remote rpc server

[vanmilleru]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

provide model cache and kv cache in remote rpc server

Motivation

Sometimes people need to load large models over low bandwidth Ethernet, which takes forever.

Possible Implementation

It's nice to have a check sum of the sent parameter or use the path to gguf file as the model cache key on the remote server. This will make startup and switching model work faster.

[vanmilleru]

the other way to speed up is to allow setting a local path gguf file(s) on the rpc server and perform "speculative loading" when file names maches

[am17an]

rpc-server has --cache, are you using that?