Skip to content

Commit eb7d267

Browse files
committed
ml/ai cleanup
modified: docs/genai/01_getting_started/01_intro.mdx modified: docs/genai/04_how_to_guides/04_llm_fine_tuning.md renamed: docs/hpc/08_ml_ai_hpc/06_llm_fine_tuning.md -> docs/hpc/08_ml_ai_hpc/05_llm_fine_tuning.md renamed: docs/hpc/08_ml_ai_hpc/LLM Inference/llm_inferenceoverview.md -> docs/hpc/08_ml_ai_hpc/LLM Inference/01_llm_inferenceoverview.md renamed: docs/hpc/08_ml_ai_hpc/LLM Inference/05_run_hf_model.md -> docs/hpc/08_ml_ai_hpc/LLM Inference/02_run_hf_model.md renamed: docs/hpc/08_ml_ai_hpc/LLM Inference/07_vLLM.md -> docs/hpc/08_ml_ai_hpc/LLM Inference/03_vLLM.md
1 parent e2360a9 commit eb7d267

6 files changed

Lines changed: 5 additions & 5 deletions

File tree

docs/genai/01_getting_started/01_intro.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ If you're looking to harness Generative AI for administrative or classroom use,
66

77
Welcome to Pythia, the generative AI platform for research workflows. As part of the Pythia platform, the following capabilities are offered:
88
- [Access to externally hosted LLMs](../01_getting_started/02_llm_gateway.mdx)
9-
- [HPC resources for fine tuning LLMs](../../hpc/08_ml_ai_hpc/06_llm_fine_tuning.md)
9+
- [HPC resources for fine tuning LLMs](../../hpc/08_ml_ai_hpc/05_llm_fine_tuning.md)
1010

1111
:::info[Personal use]
1212
If you want to access NYU provided LLMs for personal use, proceed to https://gemini.google.com/app with your NYU credentials.
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
# Fine tuning
22

3-
Please look into harnessing RAG before attempting to fine-tune a model. For open-weight models, you can use the HPC cluster to perform LoRA fine-tuning as [described here](../../hpc/08_ml_ai_hpc/06_llm_fine_tuning.md).
3+
Please look into harnessing RAG before attempting to fine-tune a model. For open-weight models, you can use the HPC cluster to perform LoRA fine-tuning as [described here](../../hpc/08_ml_ai_hpc/05_llm_fine_tuning.md).
File renamed without changes.

docs/hpc/08_ml_ai_hpc/LLM Inference/llm_inferenceoverview.md renamed to docs/hpc/08_ml_ai_hpc/LLM Inference/01_llm_inferenceoverview.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,17 @@
22
This directory provides two primary pathways for deploying and running Large Language Models (LLMs) on the NYU Torch/Greene cluster: Hugging Face Transformers (for research/experimentation) and vLLM (for high-performance serving).
33

44
## 1. Basic Inference (Hugging Face)
5-
This method is ideal for feature extraction, embeddings, or small-scale batch processing. It relies on a persistent Singularity environment using an .ext3 overlay.
5+
This method is ideal for feature extraction, embeddings, or small-scale batch processing. It relies on a persistent Singularity environment using an `ext3` overlay.
66

77
**Workflow:**
88
Environment: Launch an Apptainer container with a read/write overlay.
9-
Persistence: Install Conda/Miniforge and libraries directly into /ext3.
9+
Persistence: Install Conda/Miniforge and libraries directly into `/ext3`.
1010
Execution: Use AutoModel to load weights and perform a forward pass.
1111
Key File: huggingface.py
1212
Ideal for: Extracting last_hidden_state (embeddings) or sentiment classification.
1313

1414
## 2. High-Performance Serving (vLLM)
15-
vLLM is the recommended tool for production-level throughput and low-latency inference. It utilizes PagedAttention to manage memory efficiently.
15+
vLLM is the recommended tool for production-level throughput and low-latency inference. It utilizes `PagedAttention` to manage memory efficiently. Please find our guide on [deploying LLMs with vLLM on Torch here](./03_vLLM.md).
1616

1717
**Why vLLM?**
1818
Speed: higher throughput than standard backends on Torch.

docs/hpc/08_ml_ai_hpc/LLM Inference/05_run_hf_model.md renamed to docs/hpc/08_ml_ai_hpc/LLM Inference/02_run_hf_model.md

File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)