|
7 | 7 | "source": [ |
8 | 8 | "# Visualization Toolkit Usage Guide\n", |
9 | 9 | "\n", |
10 | | - "A suite of semantic-analysis and visualization tools for word-level patterns in qualitative text data — word clouds, t-SNE semantic maps, word/code heatmaps, and semantic networks. An annotated online version is on [Colab](https://colab.research.google.com/drive/1n90EDMSiXhIaOULUMPJ4W4hqdZCh1NQw?usp=sharing) (not for sensitive data); see also the [video tutorial](https://vimeo.com/1122226315) and the [working-paper description](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit/blob/main/documentation/paper.md).\n", |
| 10 | + "A suite of semantic-analysis and visualization tools for word-level patterns in qualitative text data — word clouds, t-SNE semantic maps, word/code heatmaps, and semantic networks. An annotated online version is on [Colab](https://colab.research.google.com/github/Computational-Ethnography-Lab/cmap_visualization_toolkit/blob/v0.9.5/visualization_toolkit_final.ipynb) (not for sensitive data); see also the [video tutorial](https://vimeo.com/1122226315) and the [working-paper description](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit/blob/main/documentation/paper.md).\n", |
11 | 11 | "\n", |
12 | 12 | "**Full documentation — install, usage, data schema, design notes, license — is in the [README](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit#readme).** This notebook is meant to be run top-to-bottom: the **Setup** and **Helper Function** cells are background setup (run them once, in order); then use the **Analytics Tool** sections to generate and customize visuals.\n", |
13 | 13 | "\n", |
14 | 14 | "### How to run\n", |
15 | | - "1. **Prepare your data** — a `.csv`/DataFrame with at least a `text` column (optional `project`, `codes`, `data_group` for subsetting). Full schema: [Teaching → Data Schema Example (CMAP)](https://github.com/Computational-Ethnography-Lab/teaching#data-schema-example-cmap).\n", |
| 15 | + "1. **Prepare your data** — a `.csv`/DataFrame with at least a `text` column (optional `project`, `codes`, `data_group` for subsetting). Full schema: [Teaching → Data Schema Example (CMAP)](https://github.com/Computational-Ethnography-Lab/teaching#data-schema-example-cmap). Coming from ATLAS.ti, NVivo, or MAXQDA? The [CMAP QDPX Converter](https://github.com/Computational-Ethnography-Lab/cmap_qdpx_converter) turns a QDPX (REFI-QDA) export into the CMAP-schema CSV this toolkit reads — no manual reshaping.\n", |
16 | 16 | "2. **Run Setup + Helper Functions top-to-bottom** — these load packages, define directories, stopwords / word-families, validators, and the plotting functions.\n", |
17 | 17 | "3. **Run a tool section** — Basic (word cloud), Intermediate (heatmaps, t-SNE, networks), or Advanced (full / interactive). Edit the user-configuration block at the top of each section.\n", |
18 | 18 | "\n", |
|
30 | 30 | "Full citation with the Zenodo DOI and contributor list is in the [README](https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit#policies). Questions: <corey.abramson@rice.edu>.\n" |
31 | 31 | ] |
32 | 32 | }, |
| 33 | + { |
| 34 | + "cell_type": "markdown", |
| 35 | + "id": "colabsetupmd01", |
| 36 | + "metadata": {}, |
| 37 | + "source": [ |
| 38 | + "## Colab setup (run first)\n", |
| 39 | + "\n", |
| 40 | + "Running on **Google Colab**? Run the next cell first — it clones the toolkit repo (for the `function/` modules and the sample `data/data.csv`) and installs the few packages Colab does not ship. It does nothing outside Colab, so a local install is unaffected. Then continue with **Setup** below." |
| 41 | + ] |
| 42 | + }, |
| 43 | + { |
| 44 | + "cell_type": "code", |
| 45 | + "execution_count": null, |
| 46 | + "id": "colabsetupcode01", |
| 47 | + "metadata": {}, |
| 48 | + "outputs": [], |
| 49 | + "source": [ |
| 50 | + "# =============================================================================\n", |
| 51 | + "# COLAB SETUP (RUN FIRST ON COLAB)\n", |
| 52 | + "# On Google Colab this clones the toolkit repo so the `function/` package and\n", |
| 53 | + "# the sample `data/data.csv` are on disk, then installs the packages Colab does\n", |
| 54 | + "# not ship. It is a NO-OP outside Colab, so local (conda / Jupyter) runs are\n", |
| 55 | + "# unaffected. Colab's preinstalled torch / transformers / numpy / pandas / scipy\n", |
| 56 | + "# / scikit-learn stack is left untouched (avoids a slow, CUDA-breaking downgrade).\n", |
| 57 | + "# =============================================================================\n", |
| 58 | + "import sys\n", |
| 59 | + "import os\n", |
| 60 | + "import subprocess\n", |
| 61 | + "\n", |
| 62 | + "if \"google.colab\" in sys.modules:\n", |
| 63 | + " REPO = \"cmap_visualization_toolkit\"\n", |
| 64 | + " REPO_URL = \"https://github.com/Computational-Ethnography-Lab/cmap_visualization_toolkit.git\"\n", |
| 65 | + " RELEASE_TAG = \"v0.9.5\" # pinned release tag: notebook + function/ come from the same ref\n", |
| 66 | + " # Tag-specific checkout dir: a stale main / other-release checkout left in a\n", |
| 67 | + " # reused Colab runtime cannot be silently used in place of the pinned tag.\n", |
| 68 | + " CHECKOUT = REPO + \"_\" + RELEASE_TAG\n", |
| 69 | + " if os.path.basename(os.getcwd()) != CHECKOUT:\n", |
| 70 | + " if not os.path.isdir(CHECKOUT):\n", |
| 71 | + " subprocess.run([\"git\", \"clone\", \"--branch\", RELEASE_TAG, \"--depth\", \"1\", REPO_URL, CHECKOUT], check=True)\n", |
| 72 | + " os.chdir(CHECKOUT)\n", |
| 73 | + " # Fail loud if the checkout is not exactly the pinned tag (guards a reused,\n", |
| 74 | + " # partial, or wrong-ref directory that would bypass the version pin).\n", |
| 75 | + " _on_tag = subprocess.run([\"git\", \"describe\", \"--tags\", \"--exact-match\"],\n", |
| 76 | + " capture_output=True, text=True).stdout.strip()\n", |
| 77 | + " if _on_tag != RELEASE_TAG:\n", |
| 78 | + " raise RuntimeError(\n", |
| 79 | + " \"Colab setup: checkout is not at pinned tag \" + RELEASE_TAG +\n", |
| 80 | + " \" (got \" + repr(_on_tag) + \"). Delete /content/\" + CHECKOUT + \" and re-run this cell.\"\n", |
| 81 | + " )\n", |
| 82 | + " sys.path.insert(0, os.getcwd())\n", |
| 83 | + " subprocess.run(\n", |
| 84 | + " [sys.executable, \"-m\", \"pip\", \"install\", \"-q\",\n", |
| 85 | + " \"wordcloud>=1.9,<2\",\n", |
| 86 | + " \"sentence-transformers>=2.2,<3.6\",\n", |
| 87 | + " \"gensim>=4.3,<5\",\n", |
| 88 | + " \"dash>=2.10,<3\",\n", |
| 89 | + " \"dill>=0.3.6,<0.4\",\n", |
| 90 | + " \"python-dotenv>=0.15,<2\",\n", |
| 91 | + " \"pydantic>=1.10,<3\",\n", |
| 92 | + " \"nbimporter\"],\n", |
| 93 | + " check=True,\n", |
| 94 | + " )\n", |
| 95 | + " print(\"[OK] Colab setup complete — repo at \" + RELEASE_TAG + \", dependencies installed.\")" |
| 96 | + ] |
| 97 | + }, |
33 | 98 | { |
34 | 99 | "cell_type": "markdown", |
35 | 100 | "id": "1316a766", |
|
167 | 232 | " \"sklearn\": \">=1.2,<1.5\",\n", |
168 | 233 | " \"sentence_transformers\": \">=2.2,<3.6\",\n", |
169 | 234 | " \"transformers\": \">=4.36,<5.1\",\n", |
170 | | - " \"torch\": \">=2.1,<2.8\",\n", |
| 235 | + " \"torch\": \">=2.1,<2.9\",\n", |
171 | 236 | " \"matplotlib\": \">=3.7,<4\",\n", |
172 | 237 | " \"seaborn\": \">=0.12,<1\",\n", |
173 | 238 | " \"networkx\": \">=3,<4\",\n", |
|
0 commit comments