Hidden States Feature for serverless endpoints

[PatrickO94]

Technical Feedback

I think it'd be great if serverless endpoints with chat-completion models would have an option to return the hidden states.
(exactly like HF transformers let you do model = AutoModel.from_pretrained("some-model") --> model(**inputs, output_hidden_states=True)). As far as I understand this is currently only possible for Managed Compute, but making serverless endpoints less of a total black-box can be very useful too, while giving users the option for token-based billing.
Is such a feature planned?

Desired Outcome

The ability to obtain hidden states after the forward pass of the initial input, before (or separate from) text generation.

Current Workaround

None