Performance optimizations for stateless inference)

[ravi9]

Optimize performance for stateless inference focusing on latency and throughput.

[ravi9]

Individual issues are created for optimizing performance for stateless inference