fix(p1b-oom): fold Sol OOM docs + release-hygiene findings

P1b-OOM disposition (Sol/ultra out-of-model review). Gate targets B1/B2/eval
confirmed clean; folding the doc/config/metadata findings only.

- README: clustered heatmap is unconditional (clustered= flag is not gated);
  TF-IDF experimental variant requires distance_metric="raw_weighted" (F10, F11)
- notebook: align method labels to README — method 2 "Jaccard" -> "Co-occurrence";
  PMI described as PPMI, cosine-only; core comment typo co-occurrnece->co-occurrence
- notebook kernelspec cmap_vis_replication -> python3 (portable; was local env) (F4)
- .gitignore: re-exclude function/__pycache__/ and the foreign CDPH
  org_cooccurrence_network.py so neither can reach the public repo (F8, F9)

Deferred (tracked follow-up): F2 empty-seed None-guard, F3 singleton memory,
F5 pytest dep, F6 parser char-bound, F1 chunking (pre-existing), F7 nltk-at-collection.

Author: Corey-Abramson

.gitignore

@@ -165,3 +165,7 @@ __marimo__/
 !/tests/**
 !pytest.ini
 tests/__pycache__/
+# Re-exclude bytecode under the un-ignored source dirs
+function/__pycache__/
+# Foreign module from another lane (CDPH org-cooccurrence network) — keep it out of this public repo
+function/org_cooccurrence_network.py

README.md

@@ -341,13 +341,13 @@ Word-to-word similarity can be computed with one of four methods, chosen with `c
 - **1 — RoBERTa embeddings:** contextual embeddings compared with cosine similarity.
 - **2 — Co-occurrence:** `cosine` uses frequency-sensitive context vectors; the default uses the Jaccard index (set-based, binary overlap).
 - **3 — PMI (Positive Pointwise Mutual Information):** supported with `distance_metric = "cosine"` **only**. PMI builds context vectors whose similarity is meaningful under cosine; other metrics are not defined for this method, so pairing PMI with a non-cosine metric is an unsupported configuration rather than a defect. Use `"cosine"` with method 3.
-- **4 — TF-IDF weighted co-occurrence:** `cosine` (or `default`) uses cosine similarity; other values select an experimental unnormalized-sum variant.
+- **4 — TF-IDF weighted co-occurrence:** `cosine` (or `default`) uses cosine similarity; `distance_metric = "raw_weighted"` selects an experimental unnormalized-sum variant. No other `distance_metric` value is supported for this method.
 
 Combining a method and metric outside these supported pairings — most notably PMI with a non-cosine metric — will not produce a usable similarity matrix. This is expected behavior: select a supported method/metric pair.
 
 ### Hierarchical Clustering (Ward Linkage) in the Clustered Heatmap
 
-When `clustered = True`, the heatmap also shows a dendrogram-ordered version that groups words using hierarchical clustering (Ward linkage) applied to the rows and columns of the word-by-word similarity matrix. This places similar words next to one another and reveals nested groupings.
+The heatmap output always includes a dendrogram-ordered version, shown side by side with the standard (unclustered) heatmap, that groups words using hierarchical clustering (Ward linkage) applied to the rows and columns of the word-by-word similarity matrix. This places similar words next to one another and reveals nested groupings.
 
 Ward linkage here operates on each word's full similarity *profile* (its row/column in the matrix). It is a deliberate choice for producing interpretable dendrograms and word groupings, not a transformation of the individual similarity values. The values shown in the standard (unclustered) heatmap are unchanged; the clustered view only reorders and annotates them.
 

function/vis_tool_core.py

@@ -379,7 +379,7 @@ def run_pmi(sentences, selected_words, context_window, cache_file, epsilon=1e-9)
 
     return filtered_embeddings, None, pmi_matrix
 
-def run_tfidf_overlap(sentences, selected_words, context_window, all_candidates, cache_file): # Get TF-IDF co-occurrnece matrix 
+def run_tfidf_overlap(sentences, selected_words, context_window, all_candidates, cache_file): # Get TF-IDF co-occurrence matrix
     """
     Calculates similarity based on TF-IDF vectors of context windows.
     The primary and recommended method is 'cosine' similarity.

visualization_toolkit_final.ipynb

@@ -16,7 +16,7 @@
     "2. **Run Setup + Helper Functions top-to-bottom** — these load packages, define directories, stopwords / word-families, validators, and the plotting functions.\n",
     "3. **Run a tool section** — Basic (word cloud), Intermediate (heatmaps, t-SNE, networks), or Advanced (full / interactive). Edit the user-configuration block at the top of each section.\n",
     "\n",
-    "**Key parameters (set per tool):** `clustering_method` 1 = RoBERTa, 2 = Jaccard, 3 = PMI, 4 = TF-IDF; `distance_metric` `\"cosine\"` or `\"default\"`. Design rationale for the environment-configuration and word-family setup steps is in [README → Design Notes](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit#design-notes).\n",
+    "**Key parameters (set per tool):** `clustering_method` 1 = RoBERTa, 2 = Co-occurrence, 3 = PMI, 4 = TF-IDF; `distance_metric` `\"cosine\"` or `\"default\"`. Design rationale for the environment-configuration and word-family setup steps is in [README → Design Notes](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit#design-notes).\n",
     "\n",
     "## Core principles \n",
     "- Abramson, C. M., Prendergast, T., Li, Z., & Dohan, D. (2026). \"Qualitative Research in an Era of Artificial Intelligence: A Pragmatic Approach to Data Analysis, Workflow, and Computation.\" *Annual Review of Sociology, 52* (in press). https://doi.org/10.1146/annurev-soc-011824-104836\n",
@@ -899,14 +899,14 @@
     "        print(\"Method: RoBERTa – Shows semantic relationships based on contextual embeddings\")\n",
     "    elif input_data.clustering_method == 2:\n",
     "        if input_data.distance_metric == \"cosine\":\n",
-    "            print(\"Method: Jaccard (cosine) – Uses context window vectors to compute cosine-based similarity between word usage patterns\")\n",
+    "            print(\"Method: Co-occurrence (cosine) – Uses context window vectors to compute cosine-based similarity between word usage patterns\")\n",
     "        elif input_data.distance_metric == \"default\":\n",
-    "            print(\"Method: Jaccard (default) – Uses binary co-occurrence counts within a context window to capture word overlap\")\n",
+    "            print(\"Method: Co-occurrence (Jaccard index, default) – Uses binary co-occurrence counts within a context window to capture word overlap\")\n",
     "    elif input_data.clustering_method == 3:\n",
     "        if input_data.distance_metric == \"cosine\":\n",
     "            print(\"Method: PMI (cosine) – Highlights statistically significant word associations using cosine similarity of PMI-weighted context vectors\")\n",
     "        elif input_data.distance_metric == \"default\":\n",
-    "            print(\"Method: PMI (default) – Highlights statistically significant word associations using raw PMI scores\")\n",
+    "            print(\"Method: PMI (default) – Highlights statistically significant word associations using positive PMI (PPMI) scores\")\n",
     "    elif input_data.clustering_method == 4:\n",
     "        if input_data.distance_metric == \"raw_weighted\":\n",
     "            print(\"Method: TF-IDF (raw weighted) – Uses raw TF-IDF-weighted co-occurrence scores for word associations\")\n",
@@ -1176,14 +1176,14 @@
     "        print(\"Method: RoBERTa – Shows semantic relationships based on contextual embeddings\")\n",
     "    elif input_data.clustering_method == 2:\n",
     "        if input_data.distance_metric == \"cosine\":\n",
-    "            print(\"Method: Jaccard (cosine) – Uses context window vectors to compute cosine-based similarity between word usage patterns\")\n",
+    "            print(\"Method: Co-occurrence (cosine) – Uses context window vectors to compute cosine-based similarity between word usage patterns\")\n",
     "        elif input_data.distance_metric == \"default\":\n",
-    "            print(\"Method: Jaccard (default) – Uses binary co-occurrence counts within a context window to capture word overlap\")\n",
+    "            print(\"Method: Co-occurrence (Jaccard index, default) – Uses binary co-occurrence counts within a context window to capture word overlap\")\n",
     "    elif input_data.clustering_method == 3:\n",
     "        if input_data.distance_metric == \"cosine\":\n",
     "            print(\"Method: PMI (cosine) – Highlights statistically significant word associations using cosine similarity of PMI-weighted context vectors\")\n",
     "        elif input_data.distance_metric == \"default\":\n",
-    "            print(\"Method: PMI (default) – Highlights statistically significant word associations using raw PMI scores\")\n",
+    "            print(\"Method: PMI (default) – Highlights statistically significant word associations using positive PMI (PPMI) scores\")\n",
     "    elif input_data.clustering_method == 4:\n",
     "        if input_data.distance_metric == \"raw_weighted\":\n",
     "            print(\"Method: TF-IDF (raw weighted) – Uses raw TF-IDF-weighted co-occurrence scores for word associations\")\n",
@@ -1583,14 +1583,14 @@
     "        print(\"Method: RoBERTa – Shows semantic relationships based on contextual embeddings\")\n",
     "    elif input_data.clustering_method == 2:\n",
     "        if input_data.distance_metric == \"cosine\":\n",
-    "            print(\"Method: Jaccard (cosine) – Uses context window vectors to compute cosine-based similarity between word usage patterns\")\n",
+    "            print(\"Method: Co-occurrence (cosine) – Uses context window vectors to compute cosine-based similarity between word usage patterns\")\n",
     "        elif input_data.distance_metric == \"default\":\n",
-    "            print(\"Method: Jaccard (default) – Uses binary co-occurrence counts within a context window to capture word overlap\")\n",
+    "            print(\"Method: Co-occurrence (Jaccard index, default) – Uses binary co-occurrence counts within a context window to capture word overlap\")\n",
     "    elif input_data.clustering_method == 3:\n",
     "        if input_data.distance_metric == \"cosine\":\n",
     "            print(\"Method: PMI (cosine) – Highlights statistically significant word associations using cosine similarity of PMI-weighted context vectors\")\n",
     "        elif input_data.distance_metric == \"default\":\n",
-    "            print(\"Method: PMI (default) – Highlights statistically significant word associations using raw PMI scores\")\n",
+    "            print(\"Method: PMI (default) – Highlights statistically significant word associations using positive PMI (PPMI) scores\")\n",
     "    elif input_data.clustering_method == 4:\n",
     "        if input_data.distance_metric == \"raw_weighted\":\n",
     "            print(\"Method: TF-IDF (raw weighted) – Uses raw TF-IDF-weighted co-occurrence scores for word associations\")\n",
@@ -2033,14 +2033,14 @@
     "        print(\"Method: RoBERTa – Shows semantic relationships based on contextual embeddings\")\n",
     "    elif input_data.clustering_method == 2:\n",
     "        if input_data.distance_metric == \"cosine\":\n",
-    "            print(\"Method: Jaccard (cosine) – Uses context window vectors to compute cosine-based similarity between word usage patterns\")\n",
+    "            print(\"Method: Co-occurrence (cosine) – Uses context window vectors to compute cosine-based similarity between word usage patterns\")\n",
     "        elif input_data.distance_metric == \"default\":\n",
-    "            print(\"Method: Jaccard (default) – Uses binary co-occurrence counts within a context window to capture word overlap\")\n",
+    "            print(\"Method: Co-occurrence (Jaccard index, default) – Uses binary co-occurrence counts within a context window to capture word overlap\")\n",
     "    elif input_data.clustering_method == 3:\n",
     "        if input_data.distance_metric == \"cosine\":\n",
     "            print(\"Method: PMI (cosine) – Highlights statistically significant word associations using cosine similarity of PMI-weighted context vectors\")\n",
     "        elif input_data.distance_metric == \"default\":\n",
-    "            print(\"Method: PMI (default) – Highlights statistically significant word associations using raw PMI scores\")\n",
+    "            print(\"Method: PMI (default) – Highlights statistically significant word associations using positive PMI (PPMI) scores\")\n",
     "    elif input_data.clustering_method == 4:\n",
     "        if input_data.distance_metric == \"raw_weighted\":\n",
     "            print(\"Method: TF-IDF (raw weighted) – Uses raw TF-IDF-weighted co-occurrence scores for word associations\")\n",
@@ -2605,7 +2605,7 @@
     "    use_custom_stoplist    = True\n",
     "\n",
     "    # Core Analysis\n",
-    "    clustering_method = 3           # 1 = RoBERTa, 2 = Jaccard, 3 = PMI, 4 = TF-IDF\n",
+    "    clustering_method = 3           # 1 = RoBERTa, 2 = Co-occurrence, 3 = PMI, 4 = TF-IDF\n",
     "    distance_metric   = \"cosine\"   \n",
     "    \n",
     "# Note on clustering method and distance metric:\n",
@@ -2614,14 +2614,14 @@
     "#    - distance_metric is always \"default\" (ignored internally)\n",
     "#    - Uses contextual embeddings directly\n",
     "#\n",
-    "# 2. Jaccard (clustering_method = 2)\n",
+    "# 2. Co-occurrence (clustering_method = 2)\n",
     "#    - \"default\": context-window overlap measured by Jaccard index\n",
     "#                 (set-based, binary overlap score)\n",
     "#    - \"cosine\" : context vectors built from co-occurrence counts,\n",
     "#                 compared using cosine similarity (frequency-sensitive)\n",
     "#\n",
     "# 3. PMI (clustering_method = 3)\n",
-    "#    - \"default\": classic PMI co-occurrence scores\n",
+    "#    - \"default\": positive PMI (PPMI) scores (PMI supports \"cosine\" only)\n",
     "#    - \"cosine\" : PMI-weighted context vectors compared with cosine similarity\n",
     "#                 (captures similarity of PMI distributions rather than raw scores)\n",
     "#\n",
@@ -3045,7 +3045,7 @@
     "    use_custom_stoplist    = True\n",
     "\n",
     "    # Core Analysis\n",
-    "    clustering_method = 2           # 1 = RoBERTa, 2 = Jaccard, 3 = PMI, 4 = TF-IDF\n",
+    "    clustering_method = 2           # 1 = RoBERTa, 2 = Co-occurrence, 3 = PMI, 4 = TF-IDF\n",
     "    distance_metric   = \"cosine\"   \n",
     "    \n",
     "    # Note on clustering method and distance metric:\n",
@@ -3241,7 +3241,7 @@
     "   - If not provided, the default list will be used.\n",
     "\n",
     "4. Clustering Method & Parameters:\n",
-    "   - Choose from: 1 = RoBERTa, 2 = Jaccard, 3 = PMI, 4 = TF-IDF.\n",
+    "   - Choose from: 1 = RoBERTa, 2 = Co-occurrence, 3 = PMI, 4 = TF-IDF.\n",
     "   - For methods 2–4, specify distance metric (default or cosine) and context window size.\n",
     "\n",
     "5. Analysis Filters:\n",
@@ -3402,7 +3402,7 @@
     "            while True:\n",
     "                try:\n",
     "                    clustering_method = int(\n",
-    "                        input(\"Choose clustering method 1=RoBERTa, 2=Jaccard, 3=PMI, 4=TF-IDF [Default 1]: \").strip() or \"1\"\n",
+    "                        input(\"Choose clustering method 1=RoBERTa, 2=Co-occurrence, 3=PMI, 4=TF-IDF [Default 1]: \").strip() or \"1\"\n",
     "                        )\n",
     "                    if clustering_method in (1, 2, 3, 4):\n",
     "                        break\n",
@@ -3542,9 +3542,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "cmap_vis_replication",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "cmap_vis_replication"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {