Merge pull request #291 from NYU-RTS/minor-cleanup

ml/ai cleanup

Author: s-sajid-ali

docs/genai/01_getting_started/01_intro.mdx

@@ -6,7 +6,7 @@ If you're looking to harness Generative AI for administrative or classroom use,
 
 Welcome to Pythia, the generative AI platform for research workflows. As part of the Pythia platform, the following capabilities are offered:
 - [Access to externally hosted LLMs](../01_getting_started/02_llm_gateway.mdx)
-- [HPC resources for fine tuning LLMs](../../hpc/08_ml_ai_hpc/06_llm_fine_tuning.md)
+- [HPC resources for fine tuning LLMs](../../hpc/08_ml_ai_hpc/05_llm_fine_tuning.md)
 
 :::info[Personal use]
 If you want to access NYU provided LLMs for personal use, proceed to https://gemini.google.com/app with your NYU credentials. 

docs/genai/04_how_to_guides/04_llm_fine_tuning.md

@@ -1,3 +1,3 @@
 # Fine tuning
 
-Please look into harnessing RAG before attempting to fine-tune a model. For open-weight models, you can use the HPC cluster to perform LoRA fine-tuning as [described here](../../hpc/08_ml_ai_hpc/06_llm_fine_tuning.md).
+Please look into harnessing RAG before attempting to fine-tune a model. For open-weight models, you can use the HPC cluster to perform LoRA fine-tuning as [described here](../../hpc/08_ml_ai_hpc/05_llm_fine_tuning.md).

…s/hpc/08_ml_ai_hpc/06_llm_fine_tuning.md …s/hpc/08_ml_ai_hpc/05_llm_fine_tuning.mddocs/hpc/08_ml_ai_hpc/06_llm_fine_tuning.md renamed to docs/hpc/08_ml_ai_hpc/05_llm_fine_tuning.md docs/hpc/08_ml_ai_hpc/06_llm_fine_tuning.md renamed to docs/hpc/08_ml_ai_hpc/05_llm_fine_tuning.md

@@ -0,0 +1,23 @@
+# Overview
+This directory provides two primary pathways for deploying and running Large Language Models (LLMs) on the NYU Torch cluster: Hugging Face Transformers (for research/experimentation) and vLLM (for high-performance serving).
+
+## 1. Basic Inference (Hugging Face)
+This method is ideal for feature extraction, embeddings, or small-scale batch processing. Persistence is achieved by setting up the environment within an `ext3` overlay that can be mounted to an Apptainer container.
+
+**Workflow:**
+Environment: Launch an Apptainer container with a read/write overlay.
+Persistence: Install `conda` and libraries directly into `/ext3`.
+Execution: Use AutoModel to load weights and perform a forward pass.
+Key File: `huggingface.py`
+Ideal for: Extracting `last_hidden_state` (embeddings) or sentiment classification.
+
+## 2. High-Performance Serving (vLLM)
+vLLM is the recommended tool for production-level throughput and low-latency inference. It utilizes `PagedAttention` to manage memory efficiently. Please find our guide on [deploying LLMs with vLLM on Torch here](./03_vLLM.md).
+
+**Why vLLM?**
+Speed: higher throughput than standard backends on Torch.
+Compatibility: Drop-in replacement for OpenAI API.
+
+**Deployment Options:**
+Online: Use `vllm serve` to start an HTTP server accessible via curl or OpenAI clients.
+Offline: Use the LLM class within Python for processing large datasets without a server.