Skip to content

Commit bc03ba2

Browse files
chensuyuepre-commit-ci[bot]
chensuyue
and
pre-commit-ci[bot]
authored
Add example validated configurations for ChatQnA (#2057)
Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent a6e9984 commit bc03ba2

5 files changed

Lines changed: 38 additions & 240 deletions

File tree

‎ChatQnA/README.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,3 +120,14 @@ For ChatQnA specific tracing and metrics monitoring, follow [OpenTelemetry on Ch
120120
## FAQ Generation Application
121121

122122
FAQ Generation Application leverages the power of large language models (LLMs) to revolutionize the way you interact with and comprehend complex textual data. By harnessing cutting-edge natural language processing techniques, our application can automatically generate comprehensive and natural-sounding frequently asked questions (FAQs) from your documents, legal texts, customer queries, and other sources. We merged the FaqGen into the ChatQnA example, which utilize LangChain to implement FAQ Generation and facilitate LLM inference using Text Generation Inference on Intel Xeon and Gaudi2 processors.
123+
124+
## Validated Configurations
125+
126+
| **Deploy Method** | **LLM Engine** | **LLM Model** | **Embedding** | **Vector Database** | **Reranking** | **Guardrails** | **Hardware** |
127+
| ----------------- | -------------- | ----------------------------------- | ------------- | ---------------------------------------- | ------------- | -------------- | ------------ |
128+
| Docker Compose | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | TEI | Redis | w/, w/o | w/, w/o | Intel Gaudi |
129+
| Docker Compose | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | TEI | Redis, Mariadb, Milvus, Pinecone, Qdrant | w/, w/o | w/o | Intel Xeon |
130+
| Docker Compose | Ollama | llama3.2 | TEI | Redis | w/ | w/o | Intel AIPC |
131+
| Docker Compose | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | TEI | Redis | w/ | w/o | AMD ROCm |
132+
| Helm Charts | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | TEI | Redis | w/, w/o | w/, w/o | Intel Gaudi |
133+
| Helm Charts | vLLM, TGI | meta-llama/Meta-Llama-3-8B-Instruct | TEI | Redis, Milvus, Qdrant | w/, w/o | w/o | Intel Xeon |

‎LEGAL_INFORMATION.md‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,6 @@ Generative AI Examples is licensed under [Apache License Version 2.0](http://www
99
This software includes components that have separate copyright notices and licensing terms.
1010
Your use of the source code for these components is subject to the terms and conditions of the following licenses.
1111

12-
- [Third Party Programs](third-party-programs.txt)
13-
14-
See the accompanying [license](LICENSE) file for full license text and copyright notices.
15-
1612
## Citation
1713

1814
If you use Generative AI Examples in your research, use the following BibTeX entry.

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,10 @@ Deployment is based on released docker images by default - check [docker image l
8282

8383
Check [here](./supported_examples.md) for detailed information of supported examples, models, hardwares, etc.
8484

85+
## Validated Configurations
86+
87+
Check [here](./validated_configurations.md) for the validated configurations of GenAIExamples, including hardware and software versions that have been tested for each release.
88+
8589
## Contributing to OPEA
8690

8791
Welcome to the OPEA open-source community! We are thrilled to have you here and excited about the potential contributions you can bring to the OPEA platform. Whether you are fixing bugs, adding new GenAI components, improving documentation, or sharing your unique use cases, your contributions are invaluable.

‎third-party-programs.txt‎

Lines changed: 0 additions & 236 deletions
This file was deleted.

‎validated_configurations.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Validated Configurations
2+
3+
This document provides an overview of the validated configurations for the GenAIExamples release test, including hardware and software versions that have been tested and confirmed to work together.
4+
Example specific test matrix can be found in examples' README.md files, for example [ChatQnA Config](./ChatQnA/README.md#validated-configurations).
5+
6+
## v1.3 Release Test Config Overview
7+
8+
| **HW/SW Stake** | **Description** |
9+
| ---------------------- | ------------------------------------------------------------------------------------------------------------ |
10+
| **Validated Hardware** | Intel Gaudi AI Accelerators (2nd, 3rd) |
11+
| | Intel Xeon Scalable processor (4th, 6th) |
12+
| | Intel Arc Graphics GPU (A770) |
13+
| | AMD Instinct MI300X Accelerators (CDNA3) |
14+
| **Validated Software** | Ubuntu 22.04 |
15+
| | Habana v1.20 ([link](https://docs.habana.ai/en/v1.20.1/Installation_Guide/index.html)) |
16+
| | ROCm v6.4.0 ([link](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/quick-start.html)) |
17+
| | Docker version 28.0.4 |
18+
| | Docker Compose version v2.34.0 |
19+
| | Kubernetes v1.29.15 |
20+
| | HabanaAI vLLM v0.6.6.post1+Gaudi-1.20.0 |
21+
| | vLLM v0.8.3 (Xeon, ROCm) |
22+
| | TGI v2.4.0 (Xeon), v2.3.1 (Gaudi), v2.4.1 (ROCm) |
23+
| | TEI v1.6 |

0 commit comments

Comments
 (0)