Skip to content

Commit b58365e

Browse files
committed
Added a notebook evaluating recommendations produced by different embedding models.
Signed-off-by: ArionDas <ariondasad@gmail.com>
1 parent efbb422 commit b58365e

2 files changed

Lines changed: 3074 additions & 0 deletions

File tree

cookbook/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,5 @@ The following notebooks exemplify workflow steps, features, and possible uses of
1414
## Evaluation
1515

1616
1. [Test Recommendations with a Prompt Dataset](./test_recommendations.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/IBM/responsible-prompting-api/blob/develop/cookbook/test_recommendations.ipynb)
17+
2. [Evaluate Embedding Model](./evaluate_embedding_model.ipynb) - Intrinsic embedding quality metrics (inter-cluster distance, misclassification rate, intra-cluster K-means distance).
18+
3. [Embedding Model Comparison: Red Team Evaluation](./embeddings_comparison_red_team.ipynb) - Extrinsic task-level evaluation comparing how different embedding models affect recommendation quality using the red team dataset. Computes accuracy, precision, recall, and F1-score for add and remove recommendations.

0 commit comments

Comments
 (0)