Fix and Speedup megatron_mmlu by >10x via prefill scoring and global batching#1280

kevalmorabia97 merged 4 commits intomainNVIDIA/Model-Optimizer:mainfrom

kmorabia/speedup-megatron-mmluNVIDIA/Model-Optimizer:kmorabia/speedup-megatron-mmluCopy head branch name to clipboard