Skip to content

Commit 2870386

Browse files
JeresheaPriya Savithiri
Jereshea
and
Priya Savithiri
authored
Update README files for EPYC support (#2157)
Signed-off-by: Jereshea J M <jejohnma@amd.com> Co-authored-by: Priya Savithiri <pbaskara@amd.com>
1 parent 4e53c71 commit 2870386

12 files changed

Lines changed: 61 additions & 27 deletions

File tree

‎AudioQnA/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ The table below lists currently available deployment options. They outline in de
7474
| ---------------------- | ----------------- | ---------------------------------------------------------------- |
7575
| On-premise Deployments | Docker compose | [AudioQnA deployment on Xeon](./docker_compose/intel/cpu/xeon) |
7676
| | | [AudioQnA deployment on Gaudi](./docker_compose/intel/hpu/gaudi) |
77+
| | | [AudioQnA deployment on AMD EPYC](./docker_compose/amd/cpu/epyc) |
7778
| | | [AudioQnA deployment on AMD ROCm](./docker_compose/amd/gpu/rocm) |
7879
| | Kubernetes | [Helm Charts](./kubernetes/helm) |
7980

@@ -83,6 +84,7 @@ The table below lists currently available deployment options. They outline in de
8384
| ----------------- | --------------------- | ----------------------------------- | ------------ |
8485
| Docker Compose | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | Intel Gaudi |
8586
| Docker Compose | vLLM, TGI, GPT-SoVITS | meta-llama/Meta-Llama-3-8B-Instruct | Intel Xeon |
87+
| Docker Compose | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | AMD EPYC |
8688
| Docker Compose | vLLM, TGI | Intel/neural-chat-7b-v3-3 | AMD ROCm |
8789
| Helm Charts | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | Intel Gaudi |
8890
| Helm Charts | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | Intel Xeon |

‎ChatQnA/README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,8 @@ The table below lists currently available deployment options. They outline in de
102102
| | | [ChatQnA deployment on AI PC](./docker_compose/intel/cpu/aipc/README.md) |
103103
| | | [ChatQnA deployment on Gaudi](./docker_compose/intel/hpu/gaudi/README.md) |
104104
| | | [ChatQnA deployment on Nvidia GPU](./docker_compose/nvidia/gpu/README.md) |
105-
| | | [ChatQnA deployment on AMD ROCm](./docker_compose/amd/gpu/rocm/README.md) |
105+
| | | [ChatQnA deployment on AMD EPYC](./docker_compose/amd/cpu/epyc/README.md) |
106+
| | | [ChatQnA deployment on AMD ROCm](./docker_compose/amd/cpu/epyc/README.md) |
106107
| Cloud Platforms Deployment on AWS, GCP, Azure, IBM Cloud,Oracle Cloud, [Intel® Tiber™ AI Cloud](https://ai.cloud.intel.com/) | Docker Compose | [Getting Started Guide: Deploy the ChatQnA application across multiple cloud platforms](https://github.com/opea-project/docs/tree/main/getting-started/README.md) |
107108
| | Kubernetes | [Helm Charts](./kubernetes/helm/README.md) |
108109
| Automated Terraform Deployment on Cloud Service Providers | AWS | [Terraform deployment on 4th Gen Intel Xeon with Intel AMX using meta-llama/Meta-Llama-3-8B-Instruct ](https://github.com/intel/terraform-intel-aws-vm/tree/main/examples/gen-ai-xeon-opea-chatqna) |

‎CodeGen/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@ This CodeGen example can be deployed manually on various hardware platforms usin
110110
| :-------------- | :------------------- | :----------------------------------------------------------------------- |
111111
| Intel Xeon CPU | Single Node (Docker) | [Xeon Docker Compose Guide](./docker_compose/intel/cpu/xeon/README.md) |
112112
| Intel Gaudi HPU | Single Node (Docker) | [Gaudi Docker Compose Guide](./docker_compose/intel/hpu/gaudi/README.md) |
113+
| AMD EPYC CPU | Single Node (Docker) | [EPYC Docker Compose Guide](./docker_compose/amd/cpu/epyc/README.md) |
113114
| AMD ROCm GPU | Single Node (Docker) | [ROCm Docker Compose Guide](./docker_compose/amd/gpu/rocm/README.md) |
114115
| Intel Xeon CPU | Kubernetes (Helm) | [Kubernetes Helm Guide](./kubernetes/helm/README.md) |
115116
| Intel Gaudi HPU | Kubernetes (Helm) | [Kubernetes Helm Guide](./kubernetes/helm/README.md) |
@@ -144,6 +145,7 @@ Intel® Optimized Cloud Modules for Terraform provide an automated way to deploy
144145
| ----------------- | -------------- | ------------------------------ | ------------ |
145146
| Docker Compose | vLLM, TGI | Qwen/Qwen2.5-Coder-7B-Instruct | Intel Gaudi |
146147
| Docker Compose | vLLM, TGI | Qwen/Qwen2.5-Coder-7B-Instruct | Intel Xeon |
148+
| Docker Compose | vLLM, TGI | Qwen/Qwen2.5-Coder-7B-Instruct | AMD EPYC |
147149
| Docker Compose | vLLM, TGI | Qwen/Qwen2.5-Coder-7B-Instruct | AMD ROCm |
148150
| Helm Charts | vLLM, TGI | Qwen/Qwen2.5-Coder-7B-Instruct | Intel Gaudi |
149151
| Helm Charts | vLLM, TGI | Qwen/Qwen2.5-Coder-7B-Instruct | Intel Xeon |

‎CodeTrans/README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ The CodeTrans application is an end-to-end workflow that leverages the capabilit
1616

1717
The CodeTrans example is implemented using the component-level microservices defined in [GenAIComps](https://github.com/opea-project/GenAIComps).
1818

19-
This Code Translation use case demonstrates Text Generation Inference across multiple platforms. Currently, we provide examples for [Intel Gaudi2](https://www.intel.com/content/www/us/en/products/details/processors/ai-accelerators/gaudi-overview.html) and [Intel Xeon Scalable Processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon.html), and we invite contributions from other hardware vendors to expand OPEA ecosystem.
19+
This Code Translation use case demonstrates Text Generation Inference across multiple platforms. Currently, we provide examples for [Intel Gaudi2](https://www.intel.com/content/www/us/en/products/details/processors/ai-accelerators/gaudi.html), [Intel Xeon Scalable Processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon.html) and [AMD EPYC™ Processors](https://www.amd.com/en/products/processors/server/epyc.html), and we invite contributions from other hardware vendors to expand OPEA ecosystem.
2020

2121
## Deployment Options
2222

@@ -26,6 +26,7 @@ The table below lists currently available deployment options. They outline in de
2626
| ---------------------- | -------------------- | --------------------------------------------------------------------------- |
2727
| On-premise Deployments | Docker compose | [CodeTrans deployment on Xeon](./docker_compose/intel/cpu/xeon/README.md) |
2828
| | | [CodeTrans deployment on Gaudi](./docker_compose/intel/hpu/gaudi/README.md) |
29+
| | | [CodeTrans deployment on AMD EPYC](./docker_compose/amd/cpu/epyc/README.md) |
2930
| | | [CodeTrans deployment on AMD ROCm](./docker_compose/amd/gpu/rocm/README.md) |
3031
| | Kubernetes | [Helm Charts](./kubernetes/helm/README.md) |
3132
| | Azure | Work-in-progress |
@@ -37,6 +38,7 @@ The table below lists currently available deployment options. They outline in de
3738
| ----------------- | -------------- | ---------------------------------- | ------------ |
3839
| Docker Compose | vLLM, TGI | mistralai/Mistral-7B-Instruct-v0.3 | Intel Gaudi |
3940
| Docker Compose | vLLM, TGI | mistralai/Mistral-7B-Instruct-v0.3 | Intel Xeon |
41+
| Docker Compose | vLLM, TGI | Qwen/Qwen2.5-Coder-7B-Instruct | AMD EPYC |
4042
| Docker Compose | vLLM, TGI | Qwen/Qwen2.5-Coder-7B-Instruct | AMD ROCm |
4143
| Helm Charts | vLLM, TGI | mistralai/Mistral-7B-Instruct-v0.3 | Intel Gaudi |
4244
| Helm Charts | vLLM, TGI | mistralai/Mistral-7B-Instruct-v0.3 | Intel Xeon |

‎DocSum/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ The table below lists currently available deployment options. They outline in de
7777
| ---------------------- | ---------------------- | -------------------------------------------------------------- |
7878
| On-premise Deployments | Docker Compose (Xeon) | [DocSum deployment on Xeon](./docker_compose/intel/cpu/xeon) |
7979
| | Docker Compose (Gaudi) | [DocSum deployment on Gaudi](./docker_compose/intel/hpu/gaudi) |
80+
| | Docker Compose (EPYC) | [DocSum deployment on AMD EPYC](./docker_compose/amd/cpu/epyc) |
8081
| | Docker Compose (ROCm) | [DocSum deployment on AMD ROCm](./docker_compose/amd/gpu/rocm) |
8182

8283
## Validated Configurations
@@ -85,6 +86,7 @@ The table below lists currently available deployment options. They outline in de
8586
| ----------------- | -------------- | ----------------------------------- | ------------ |
8687
| Docker Compose | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | Intel Gaudi |
8788
| Docker Compose | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | Intel Xeon |
89+
| Docker Compose | vLLM, TGI | Intel/neural-chat-7b-v3-3 | AMD EPYC |
8890
| Docker Compose | vLLM, TGI | Intel/neural-chat-7b-v3-3 | AMD ROCm |
8991
| Helm Charts | vLLM, TGI | Intel/neural-chat-7b-v3-3 | Intel Gaudi |
9092
| Helm Charts | vLLM, TGI | Intel/neural-chat-7b-v3-3 | Intel Xeon |

‎MultimodalQnA/README.md‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ flowchart LR
9292
9393
```
9494

95-
This MultimodalQnA use case performs Multimodal-RAG using LangChain, Redis VectorDB and Text Generation Inference on [Intel Gaudi2](https://www.intel.com/content/www/us/en/products/details/processors/ai-accelerators/gaudi-overview.html) and [Intel Xeon Scalable Processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon.html), and we invite contributions from other hardware vendors to expand the example.
95+
This MultimodalQnA use case performs Multimodal-RAG using LangChain, Redis VectorDB and Text Generation Inference on [Intel Gaudi2](https://www.intel.com/content/www/us/en/products/details/processors/ai-accelerators/gaudi.html), [Intel Xeon Scalable Processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon.html) and [AMD EPYC™ Processors](https://www.amd.com/en/products/processors/server/epyc.html), and we invite contributions from other hardware vendors to expand the example.
9696

9797
## Deployment Options
9898

@@ -106,4 +106,14 @@ The table below lists currently available deployment options. They outline in de
106106
| ----------------- | -------------- | --------------------------------- | ------------- | ------------ |
107107
| Docker Compose | LLAVA | llava-hf/llava-1.5-7b-hf | Milvus, Redis | Intel Xeon |
108108
| Docker Compose | LLAVA | llava-hf/llava-v1.6-vicuna-13b-hf | Redis | Intel Gaudi |
109+
| Docker Compose | LLAVA | llava-hf/llava-1.5-7b-hf | Milvus, Redis | AMD EPYC |
110+
| Docker Compose | TGI, vLLM | Xkev/Llama-3.2V-11B-cot | Redis | AMD ROCm |
111+
112+
## Validated Configurations
113+
114+
| **Deploy Method** | **LLM Engine** | **LLM Model** | **Database** | **Hardware** |
115+
| ----------------- | -------------- | --------------------------------- | ------------- | ------------ |
116+
| Docker Compose | LLAVA | llava-hf/llava-1.5-7b-hf | Milvus, Redis | Intel Xeon |
117+
| Docker Compose | LLAVA | llava-hf/llava-v1.6-vicuna-13b-hf | Redis | Intel Gaudi |
118+
| Docker Compose | LLAVA | llava-hf/llava-1.5-7b-hf | Milvus, Redis | AMD EPYC |
109119
| Docker Compose | TGI, vLLM | Xkev/Llama-3.2V-11B-cot | Redis | AMD ROCm |

‎ProductivitySuite/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,9 +154,11 @@ The table below lists the available deployment options and their implementation
154154
| Platform | Deployment Method | Link |
155155
| ---------- | ----------------- | --------------------------------------------------------------- |
156156
| Intel Xeon | Docker compose | [Deployment on Xeon](./docker_compose/intel/cpu/xeon/README.md) |
157+
| AMD EPYC | Docker compose | [Deployment on EPYC](./docker_compose/amd/cpu/epyc/README.md) |
157158

158159
## Validated Configurations
159160

160161
| **Deploy Method** | **LLM Engine** | **LLM Model** | **Hardware** |
161162
| ----------------- | -------------- | ------------------------- | ------------ |
162163
| Docker Compose | vLLM | Intel/neural-chat-7b-v3-3 | Intel Xeon |
164+
| Docker Compose | vLLM | Intel/neural-chat-7b-v3-3 | AMD EPYC |

0 commit comments

Comments
 (0)