Release v0.9.5: reproducible Colab-from-GitHub + pinned online version (#3)

* fix(pipeline): B1-B5 ordering/guards, N1-N3 stats, t-SNE metric and determinism

excluded_codes and group replace before embedding; RoBERTa chunk tokenize;
cross-POS multiset preserve; edge labels use raw similarity; t-SNE precomputed
distance for similarity matrices with deterministic keep order.

* chore: stop tracking pytest bytecode under tests/

* docs(notebook): restore Core principles block in usage guide

Keep author-facing intro markdown with ARS and Ethnography citations.

* chore(notebook): pin kernelspec to cmap_vis_replication

* fix(notebook,pipeline): B1/B2 wrapper ordering + safe list-column parser

Reorder all four teaching-notebook wrappers (run_heatmap_pipeline,
run_tsne_pipeline, run_heatmap_network_plain_pipeline,
run_heatmap_network_pipeline) to match the core pipeline contract:
exclude codes before sentence extraction (B1), and rebuild
filtered_sentences after group-label replacement so the co-occurrence /
PMI / TF-IDF methods embed the replaced text (B2).

Replace eval() on stringified codes/data_group cells with a bounded,
list[str]-only parse_list_column_cell in vis_tool_core.py and all four
notebook wrappers; it fails loud on tuples, malformed literals,
non-string items, and oversized cells instead of executing arbitrary code.

Add discriminating wrapper-ordering tests and list-column parser tests.
Document the supported clustering method/metric pairs (PMI = cosine only)
and the Ward-linkage clustered-heatmap design note in the README.

* fix(p1b-oom): fold Sol OOM docs + release-hygiene findings

P1b-OOM disposition (Sol/ultra out-of-model review). Gate targets B1/B2/eval
confirmed clean; folding the doc/config/metadata findings only.

- README: clustered heatmap is unconditional (clustered= flag is not gated);
  TF-IDF experimental variant requires distance_metric="raw_weighted" (F10, F11)
- notebook: align method labels to README — method 2 "Jaccard" -> "Co-occurrence";
  PMI described as PPMI, cosine-only; core comment typo co-occurrnece->co-occurrence
- notebook kernelspec cmap_vis_replication -> python3 (portable; was local env) (F4)
- .gitignore: re-exclude function/__pycache__/ and the foreign CDPH
  org_cooccurrence_network.py so neither can reach the public repo (F8, F9)

Deferred (tracked follow-up): F2 empty-seed None-guard, F3 singleton memory,
F5 pytest dep, F6 parser char-bound, F1 chunking (pre-existing), F7 nltk-at-collection.

* Release v0.9.5: reproducible Colab-from-GitHub notebook + pinned online version

- On Colab the notebook now self-installs: it clones the pinned release tag
  (v0.9.5) into a tag-specific folder and verifies the checkout is exactly that
  tag before importing, so the online notebook and the function/ package always
  come from the same version. This is a no-op outside Colab (local runs
  unaffected).
- Point the Open-in-Colab badge and links (README, paper, notebook) at the
  pinned tag so the online version is reproducible rather than tracking a moving
  branch.
- Add a note for users importing from ATLAS.ti / NVivo / MAXQDA (CMAP QDPX
  Converter).
- Widen supported torch to <2.9 so it runs on the current Colab torch (2.8).
- Add CITATION.cff.

* Add Beyond Text (2015) + open-access links; widen supported torch/scikit-learn

- Core Principles: add Abramson & Dohan 2015, "Beyond Text: Using Arrays to
  Represent and Analyze Ethnographic Data" (Sociological Methodology) with an
  open-access link, and add an open-access link for the 2018 computational
  ethnography paper.
- Widen the supported/required ranges for the packages Colab provides so the
  environment check passes on current Colab: scikit-learn to <1.8 and torch to
  <3 (kept the notebook check and requirements.txt in sync).

* Reword Colab setup: "packages Colab does not ship" -> "packages that aren't already on Colab"

Plainer language in the Colab setup cell's guidance and comment; no behavior change.

* Collapse Setup and Helper Functions cells by default in Colab (cellView=form)

Author: Corey-Abramson

.gitignore

@@ -148,6 +148,7 @@ __marimo__/
 !README.md
 !visualization_toolkit_final.ipynb
 !LICENSE.md
+!CITATION.cff
 # PNG files are now ignored
 *.png
 !environment.yml
@@ -159,3 +160,13 @@ __marimo__/
 !/input
 !/documentation
 !/documentation/**
+!/function
+!/function/**
+!/tests
+!/tests/**
+!pytest.ini
+tests/__pycache__/
+# Re-exclude bytecode under the un-ignored source dirs
+function/__pycache__/
+# Foreign module from another lane (CDPH org-cooccurrence network) — keep it out of this public repo
+function/org_cooccurrence_network.py

CITATION.cff

@@ -0,0 +1,17 @@
+cff-version: 1.2.0
+message: "If you use this software, please cite it as below."
+title: "CMAP Visualization Toolkit"
+type: software
+authors:
+  - family-names: "Abramson"
+    given-names: "Corey M."
+    affiliation: "Rice University"
+  - family-names: "Nian"
+    given-names: "Yuhan (Victoria)"
+    affiliation: "Rice University"
+version: "0.9.5"
+date-released: "2026-07-18"
+doi: "10.5281/zenodo.17162829"
+license: "BSD-3-Clause"
+repository-code: "https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit"
+url: "https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit"

README.md

@@ -2,7 +2,7 @@
 
 An easy-to-use toolkit for visualizing patterns in qualitative data, helping researchers see and share connections between words, concepts and themes alongside in-depth accounts.
 
-[![GitHub release](https://img.shields.io/github/v/release/Computational-Ethnography-Lab/cmap_visualization_toolkit?include_prereleases)](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit/releases)
+[![GitHub release](https://img.shields.io/github/v/release/Computational-Ethnography-Lab/cmap_visualization_toolkit?include_prereleases)](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit/releases) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Computational-Ethnography-Lab/cmap_visualization_toolkit/blob/v0.9.5/visualization_toolkit_final.ipynb)
 
 ## Table of Contents
 - [Overview](#overview)
@@ -44,7 +44,7 @@ Coming from ATLAS.ti, NVivo, or MAXQDA? The [CMAP QDPX Converter](https://github
 
 ### Notebook Version & Local Installation
 
-For an introduction, in an easy to use online version (not for sensitive data) check out the collab version [here](https://colab.research.google.com/drive/1n90EDMSiXhIaOULUMPJ4W4hqdZCh1NQw?usp=sharing)  
+For an introduction, in an easy to use online version (not for sensitive data) check out the Colab version [here](https://colab.research.google.com/github/Computational-Ethnography-Lab/cmap_visualization_toolkit/blob/v0.9.5/visualization_toolkit_final.ipynb) — it opens the current notebook straight from this repo and self-installs on first run.  
 You can find a short tutorial on using the toolkit in collab [here](https://vimeo.com/1122226315)). You can read a general description in the working paper [here](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit/blob/main/documentation/paper.md).
 ---
 
@@ -334,6 +334,23 @@ This section also includes validation checks to:
 
 These definitions are critical for interpretability in downstream visualization and clustering.
 
+### Clustering Methods and Supported Distance Metrics
+
+Word-to-word similarity can be computed with one of four methods, chosen with `clustering_method` and paired with a `distance_metric`:
+
+- **1 — RoBERTa embeddings:** contextual embeddings compared with cosine similarity.
+- **2 — Co-occurrence:** `cosine` uses frequency-sensitive context vectors; the default uses the Jaccard index (set-based, binary overlap).
+- **3 — PMI (Positive Pointwise Mutual Information):** supported with `distance_metric = "cosine"` **only**. PMI builds context vectors whose similarity is meaningful under cosine; other metrics are not defined for this method, so pairing PMI with a non-cosine metric is an unsupported configuration rather than a defect. Use `"cosine"` with method 3.
+- **4 — TF-IDF weighted co-occurrence:** `cosine` (or `default`) uses cosine similarity; `distance_metric = "raw_weighted"` selects an experimental unnormalized-sum variant. No other `distance_metric` value is supported for this method.
+
+Combining a method and metric outside these supported pairings — most notably PMI with a non-cosine metric — will not produce a usable similarity matrix. This is expected behavior: select a supported method/metric pair.
+
+### Hierarchical Clustering (Ward Linkage) in the Clustered Heatmap
+
+The heatmap output always includes a dendrogram-ordered version, shown side by side with the standard (unclustered) heatmap, that groups words using hierarchical clustering (Ward linkage) applied to the rows and columns of the word-by-word similarity matrix. This places similar words next to one another and reveals nested groupings.
+
+Ward linkage here operates on each word's full similarity *profile* (its row/column in the matrix). It is a deliberate choice for producing interpretable dendrograms and word groupings, not a transformation of the individual similarity values. The values shown in the standard (unclustered) heatmap are unchanged; the clustered view only reorders and annotates them.
+
 ## Troubleshooting
 
 Here are solutions to common issues you might encounter:

documentation/paper.md

@@ -61,7 +61,7 @@ The main paper charts organization and functions. Full mathematical details, rel
 
 
 # CMAP Organization 
-CMAP can be run in either a Jupyter environment [via Github](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit.git) or Google Colab [Colab Link](https://colab.research.google.com/drive/1n90EDMSiXhIaOULUMPJ4W4hqdZCh1NQw?authuser=1#scrollTo=1jgH13I3xLbA&uniqifier=1). 
+CMAP can be run in either a Jupyter environment [via Github](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit.git) or Google Colab [Colab Link](https://colab.research.google.com/github/Computational-Ethnography-Lab/cmap_visualization_toolkit/blob/v0.9.5/visualization_toolkit_final.ipynb). 
 Colab is recommended for learning the methods and experimenting with public datasets. For sensitive data or extended development, users can clone the GitHub repository and run the included installation script locally. 
 
 ```bash