Skip to content

Commit 19a7942

Browse files
dkashkindkashkin
authored
Add index to root README and cleanup the titles in child READMEs. (#175)
* Add index to root README and cleanup names * Remove the child readme link
1 parent fb3e88a commit 19a7942

11 files changed

Lines changed: 21 additions & 35 deletions

File tree

‎docs/mkdocs.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -200,7 +200,6 @@ nav:
200200
- Interactive Spatial Reasoning: genai-on-vertex-ai/gemini/prompting_recipes/spatial_reasoning/spatial_reasoning_app_for_gemini2.ipynb
201201
- Model Upgrades:
202202
- Overview: genai-on-vertex-ai/gemini/model_upgrades/README.md
203-
- Text Classification: genai-on-vertex-ai/gemini/model_upgrades/text_classification/README.md
204203

205204
- Evaluation:
206205
- Overview: genai-on-vertex-ai/gemini/README.md

‎genai-on-vertex-ai/gemini/model_upgrades/README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,11 @@ The goal is to accelerate the process of upgrading to the latest version of Gemi
1010
- Command line script based on Vertex GenAI Evaluation Service
1111
- Command line script based on Promptfoo
1212
- Eval Recipes can be customized by replacing the prompt template and the evaluation dataset.
13+
14+
15+
| Eval Recipe | Vertex AI Colab | Vertex AI Script | Promptfoo |
16+
| -------- | ------- | -------- | ------- |
17+
| Document QnA | [view](./document_qna/vertex_colab/document_qna_eval.ipynb) | [view](./document_qna/vertex_script/README.md) | [view](./document_qna/promptfoo/README.md) |
18+
| Summarization | [view](./summarization/vertex_colab/summarization_eval.ipynb) | [view](./summarization/vertex_script/README.md) | [view](./summarization/promptfoo/README.md) |
19+
| Text Classification | [view](./text_classification/vertex_colab/text_classification_eval.ipynb) | [view](./text_classification/vertex_script/README.md) | [view](./text_classification/promptfoo/README.md) |
20+

‎genai-on-vertex-ai/gemini/model_upgrades/document_qna/promptfoo/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Document QnA
1+
# Document Question Answering
22
### _Eval Recipe for model migration_
33

44
This Eval Recipe demonstrates how to compare performance of a Document Question Answering prompt with Gemini 1.0 and Gemini 2.0 using a labeled dataset and open source evaluation tool [Promptfoo](https://www.promptfoo.dev/).

‎genai-on-vertex-ai/gemini/model_upgrades/document_qna/vertex_colab/document_qna_eval.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
"id": "LbBTb55fx_qN"
2727
},
2828
"source": [
29-
"## **Text Classification Eval Recipe**\n",
29+
"## **Document Question Answering Eval Recipe**\n",
3030
"\n",
3131
"This Eval Recipe demonstrates how to compare performance of two models on a document question answering prompt using [Vertex AI Evaluation Service](https://cloud.google.com/vertex-ai/generative-ai/docs/models/evaluation-overview)."
3232
]

‎genai-on-vertex-ai/gemini/model_upgrades/document_qna/vertex_script/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Text Classification
1+
# Document Question Answering
22
### _Eval Recipe for model migration_
33

44
This Eval Recipe demonstrates how to compare performance of a document question answering prompt with Gemini 1.0 and Gemini 2.0 using [Vertex AI Evaluation Service](https://cloud.google.com/vertex-ai/generative-ai/docs/models/evaluation-overview).

‎genai-on-vertex-ai/gemini/model_upgrades/document_qna/vertex_script/eval.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ def run_eval(experiment_name: str, baseline_model: str, candidate_model: str, pr
3434
print(f"Baseline model score: {baseline_results.summary_metrics['question_answering_quality/mean']*20:.1f}%")
3535
print(f"Candidate model score: {candidate_results.summary_metrics['question_answering_quality/mean']*20:.1f}%")
3636

37+
def export_eval_run():
38+
detail_df = eval_result.metrics_table.to_dict(orient="records")
39+
3740
if __name__ == '__main__':
3841
if os.getenv("PROJECT_ID", "your-project-id") == "your-project-id":
3942
raise ValueError("Please configure your Google Cloud Project ID.")

‎genai-on-vertex-ai/gemini/model_upgrades/summarization/promptfoo/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Document QnA
1+
# Summarization
22
### _Eval Recipe for model migration_
33

44
This Eval Recipe demonstrates how to compare performance of a Summarization prompt with Gemini 1.0 and Gemini 2.0 using a labeled dataset and open source evaluation tool [Promptfoo](https://www.promptfoo.dev/).

‎genai-on-vertex-ai/gemini/model_upgrades/summarization/vertex_script/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Text Classification
1+
# Summarization
22
### _Eval Recipe for model migration_
33

44
This Eval Recipe demonstrates how to compare performance of a summarization prompt with Gemini 1.0 and Gemini 2.0 using [Vertex AI Evaluation Service](https://cloud.google.com/vertex-ai/generative-ai/docs/models/evaluation-overview).

‎genai-on-vertex-ai/gemini/model_upgrades/text_classification/promptfoo/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,16 +37,16 @@ git sparse-checkout set genai-on-vertex-ai/gemini/model_upgrades && \
3737
git pull origin main
3838
```
3939

40-
2. Install Promptfoo using [these instructions](https://www.promptfoo.dev/docs/installation/).
40+
1. Install Promptfoo using [these instructions](https://www.promptfoo.dev/docs/installation/).
4141

42-
3. Navigate to the Eval Recipe directory in terminal and run the command `promptfoo eval`.
42+
1. Navigate to the Eval Recipe directory in terminal and run the command `promptfoo eval`.
4343

4444
``` bash
4545
cd genai-on-vertex-ai/gemini/model_upgrades/text_classification/promptfoo
4646
promptfoo eval
4747
```
4848

49-
4. Run `promptfoo view` to analyze the eval results. You can switch the Display option to `Show failures only` in order to investigate any underperforming prompts.
49+
1. Run `promptfoo view` to analyze the eval results. You can switch the Display option to `Show failures only` in order to investigate any underperforming prompts.
5050

5151
## How to customize this Eval Recipe:
5252

‎genai-on-vertex-ai/gemini/model_upgrades/text_classification/vertex_colab/text_classification_eval.ipynb‎

Lines changed: 1 addition & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -55,31 +55,7 @@
5555
" <img width=\"32px\" src=\"https://upload.wikimedia.org/wikipedia/commons/9/91/Octicons-mark-github.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
5656
" </a>\n",
5757
" </td>\n",
58-
"</table>\n",
59-
"\n",
60-
"<div style=\"clear: both;\"></div>\n",
61-
"\n",
62-
"<b>Share to:</b>\n",
63-
"\n",
64-
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/applied-ai-engineering-samples/blob/main/genai-on-vertex-ai/gemini/model_upgrades/text_classification/vertex_colab/text_classification_eval.ipynb\" target=\"_blank\">\n",
65-
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
66-
"</a>\n",
67-
"\n",
68-
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/applied-ai-engineering-samples/blob/main/genai-on-vertex-ai/gemini/model_upgrades/text_classification/vertex_colab/text_classification_eval.ipynb\" target=\"_blank\">\n",
69-
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
70-
"</a>\n",
71-
"\n",
72-
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/applied-ai-engineering-samples/blob/main/genai-on-vertex-ai/gemini/model_upgrades/text_classification/vertex_colab/text_classification_eval.ipynb\" target=\"_blank\">\n",
73-
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/53/X_logo_2023_original.svg\" alt=\"X logo\">\n",
74-
"</a>\n",
75-
"\n",
76-
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/applied-ai-engineering-samples/blob/main/genai-on-vertex-ai/gemini/model_upgrades/text_classification/vertex_colab/text_classification_eval.ipynb\" target=\"_blank\">\n",
77-
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
78-
"</a>\n",
79-
"\n",
80-
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/applied-ai-engineering-samples/blob/main/genai-on-vertex-ai/gemini/model_upgrades/text_classification/vertex_colab/text_classification_eval.ipynb\" target=\"_blank\">\n",
81-
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
82-
"</a>"
58+
"</table>"
8359
]
8460
},
8561
{

0 commit comments

Comments
 (0)