You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: search.json
+7Lines changed: 7 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -83,6 +83,13 @@
83
83
"section": "Discussion on evaluating a topic model",
84
84
"text": "Discussion on evaluating a topic model\nTopic model evaluation is an active domain of research that goes beyond the scope of this tutorial. We propose an overview of the methods that exist and how to quickly tell if your topic model can be used or need to be refined.\nIn short: quantitative methods are impractical and one should focus more on the qualitative evaluation.\n\nQualitative evaluation\nThere is no one way around qualitatively evaluate your BERTopic, however the point is \n\n\nQuantitative evaluation\nIn this section we introduce different metrics that can be used to evaluate your topic model. However, we mainly included it to warn you of the complexity behind evaluating a topic model and that there is no one-fit-all solution.\n\nFirst, choosing the coherence score by itself can have a large influence on the difference in performance you will find between models. For example, NPMI and UCI may each lead to quite different values. Second, the coherence score only tells a part of the story. Perhaps your purpose is more classification than having the most coherent words or perhaps you want as diverse topics as possible. These use cases require vastly different evaluation metrics to be used.\nResponse to “How to evaluate the performance of the model?” by Maarten Grootendorst source\n\nThere are two types of metrics that you could use:\n\nCluster metrics — ie focus on the group making. There exist a lot of metrics, but few are fit to our situation: unsupervised learning with density based algorithms. In our experience, optimising these metrics result in a sub-optimal solutions as illustrated bellow. Read more\nTopic representation metrics — ie focus on how relevant the keywords are. Although some metrics exist their utility is limited: good score does not necessarily align with what expert consider good topic models, and they are not good scores to optimise (Stammbach et al., 2023). Read more"
"text": "Precompute your embeddings\nPre-computing the embeddings is a good practice as it will prevent from computing them at each run, but also because it allows you to use a broader spectrum of embedding models that could ne necessarily be used with BERTopic15. We retrieve the embeddings and the documents\nds = load_from_disk(\"path/to/file\")\ndocs = np.array(ds[f\"resumes.en\"]) # Number of documents : 6500\nembeddings = np.array(ds[\"embedding\"]) # shape : (6500, 768)\nThe columns in the dataset are the same as before in addition to an embedding column containing the embeddings of the resumes."
<h2class="anchored" data-anchor-id="precompute-your-embeddings">Precompute your embeddings</h2>
1312
1315
<p>Pre-computing the embeddings is a good practice as it will prevent from computing them at each run, but also because it allows you to use a broader spectrum of embedding models that could ne necessarily be used with BERTopic<ahref="#fn15" class="footnote-ref" id="fnref15" role="doc-noteref"><sup>15</sup></a>. We retrieve the embeddings and the documents</p>
<h2class="anchored" data-anchor-id="save-your-instance-locally">Save your instance locally</h2>
1319
1323
<p>For reproducibility purposes, and more generally, to save your work, BERTopic lets you do that with the <code>save</code> method. Two parameters of importance:</p>
0 commit comments