Add support of using local NVME for model storage

[surajssd]

Component

Helm Chart

Desired use case or feature

Right now there is no way to choose how to deal with the model cache. If you choose the hugging face then the model is by default stored in memory:

llm-d-deployer/charts/llm-d/templates/modelservice/presets/basic-gpu-with-nixl-preset.yaml

Lines 234 to 237 in c9e16e9

	{{ `{{ if .HFModelName }}` }}
	- name: model-cache
	emptyDir: {}
	{{ `{{ end }}` }}

And the other way is to use pvc.

---

I believe that we can offer a user the place to store model on the host nvme disks, when the model is being downloaded from the huggingface.

Proposed solution

Allow user to specify the hostpath or volume type when the model is being downloaded from HF.

Alternatives

No response

Additional context or screenshots

No response

[l4b4r4b4b4]

hmmm not sure I understand, but I am using longhorn as storage backend and this creates the volume, downloads and stores model on disk of the node that has the decoder deployed on. in my case a machine with nvme.

[surajssd]

@l4b4r4b4b4 I was saying if we allow users to also provide a way to use other volume types and not be constrained to either PVC or download from HF and store in memory. There are host of options that one could use: https://kubernetes.io/docs/concepts/storage/volumes/