Some basic documentation

.gitignore

@@ -57,7 +57,9 @@ local_settings.py
 instance/
 
 # Sphinx documentation
-docs/_build/
+docs/build/
+docs/source/examples
+docs/**/sg_execution_times.rst
 
 # MkDocs documentation
 /site/

docs/examples/GALLERY_HEADER.rst

@@ -0,0 +1,4 @@
+Examples
+--------
+
+Some introductory tutorials on interacting with the connectivity API.

docs/examples/example.py

@@ -0,0 +1,14 @@
+"""
+A basic example.
+================
+
+Introductory text.
+
+"""
+
+# %%
+# Look, some cell-based text!
+
+import brainglobe_data_api_connectivity as bdac
+
+print(bdac.__version__)

docs/source/conf.py

@@ -9,6 +9,7 @@
 import os
 import sys
 from importlib.metadata import version as get_version
+from pathlib import Path
 
 # Used when building API docs, put the dependencies
 # of any class you are documenting here
@@ -42,6 +43,7 @@
     "sphinx.ext.autosummary",
     "sphinx.ext.viewcode",
     "sphinx.ext.intersphinx",
+    "sphinx_gallery.gen_gallery",
     "sphinx_sitemap",
     "myst_parser",
 ]
@@ -69,6 +71,16 @@
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]
 
+SPHINX_GALLERY_OUTPUT_DIR = "examples/"
+
+# Sphinx gallery configuration settings
+sphinx_gallery_conf = {
+    "examples_dirs": Path(__file__).parent / "../examples/",
+    "filename_pattern": "/*.py",
+    "gallery_dirs": Path(__file__).parent / SPHINX_GALLERY_OUTPUT_DIR,
+    "run_stale_examples": True,
+}
+
 # Automatically generate stub pages for API
 autosummary_generate = True
 autodoc_default_flags = ["members", "inherited-members"]
@@ -81,6 +93,8 @@
     # to ensure that include files (partial pages) aren't built, exclude them
     # https://github.com/sphinx-doc/sphinx/issues/1965#issuecomment-124732907
     "**/includes/**",
+    f"{SPHINX_GALLERY_OUTPUT_DIR}/*ipynb",
+    f"{SPHINX_GALLERY_OUTPUT_DIR}/*.py",
 ]
 
 # -- Options for HTML output -------------------------------------------------

docs/source/connectivity-data.md

@@ -0,0 +1,117 @@
+# Connectivity Data
+We can divide connectivity data into:
+- a network consisting of nodes, edges, and weights
+- the metadata (everything we know about the regions, connections, and dataset)
+
+While graph analysis can be done using only the network, the metadata is essential for guiding analyses, understanding results, and making the work reproducible.
+
+## Networks
+
+Mathematical networks are an abstract representation of connections between a collection of objects.
+A network consists of a collection of **nodes** that are connected to each other by (directed) **edges**, with each edge possessing a **weight** that characterises the connection.
+As such, if we have some connectivity data;
+
+- The brain regions are represented by the nodes.
+  If the brain regions are provided with unique identifiers, these same identifiers can be used for the nodes.
+- The connections between the brain regions are represented by edges.
+- The strength of the connections are represented by the edge weights.
+
+The network that represents a particular dataset can be fully described simply by listing the connections, in what is often called an **edge list** or **edge table**.
+
+This is what our API expects to receive as an input.
+
+An edge table is just a three-column CSV file, where each row specifies one connection in the form:
+
+```text
+source node, target node, connection strength
+```
+
+### Example: Edge Table and Corresponding Network
+
+Our connectivity information can be used to construct our edge table CSV file:
+
+```text
+l-a, l-b, 1.0
+l-a, r-a, 0.5
+l-b, r-a, 0.5
+l-b, r-b, 0.5
+r-a, r-b, 1.0
+r-b, l-a, 0.1
+r-b, r-a, 0.1
+```
+
+This edge table fully describes the network that will represent our connectivity data; there will be four nodes (`l-a`, `l-b`, `r-a`, `r-b`) and a total of seven edges.
+
+We can illustrate this network:
+
+![An illustration of the example connectivity data, as a network.](./_static/example-connectivity-network.svg)
+
+and we can also make a few observations:
+
+- The connection between `l-a` and `l-b` is bidirectional.
+  However, the connection from `l-a` to `l-b` is stronger (strength 1) than the connection from `l-b` to `l-a` (strength 0.5).
+- The connection between `r-b` and `l-a` is unidirectional, directed from `r-b` into `l-a`.
+  It is also one of the weakest connections in the network, with a strength of 0.1.
+- Each edge in the edge list corresponds to a **direct** connection between the regions.
+- In addition to the direct connections, all of the regions are indirectly connected to each other (since there is a path that can be taken to each any particular region from any other region).
+
+:::{note}
+Often the description of nodes, edges, and weights given here is all you need. But sometimes you may need a more precise, mathematical description of how nodes, edges, and weights are defined, for this see [Mathematical Description of a Network](./connectivity-math.md).
+:::
+
+## Metadata
+Although a great deal can be learned from the network structure alone, to make sense of what the nodes and edges actually represent you also need the metadata.
+
+Connectivity metadata generally consists of the following components:
+- **Dataset Metadata**: Information about the dataset, or study that generated the dataset.
+  These may be fields such as the species, sex, or age of the animal from which the connectivity information was obtained, the institute at which the experiment was conducted, etc.
+- **Node Metadata**: Information about the brain-regions, whose connections have been explored.
+  Typically they will be given some form of unique identifier, as well as names, abbreviations, any groupings of regions, and other data like their spatial coordinates.
+- **Edge Metadata**: Information about the connections.
+  This may consist of information about the studies in which they were examined, any special reference names they might have, or any user- or neurological-groupings of the connections themselves.
+
+### Example: Metadata
+
+**Dataset Metadata**
+
+|         |                  |
+| ------------ | --------------------- |
+| project      | Alpha–Bravo Connectivity |
+| contributors | Jane Doe              |
+| year         | 2024                  |
+| species      | Rattus bravalpha   |
+| age          | 12 weeks              |
+| sex          | F                     |
+
+
+**Node Metadata**
+
+| name  | abbr | side of brain |
+| ----- | -----| -------------- |
+| alpha | l‑a  | left           |
+| bravo | l‑b  | left           |
+| alpha | r‑a  | right          |
+| bravo | r‑b  | right          |
+
+**Note**: The `abbr` column provides the unique identifiers used in the edge table.
+
+**Edge Metadata**
+
+| source | target | method     | reference        | notes |
+| ------ | ------ | ---------- | ---------------- | ----- |
+| l‑a    | l‑b    | anterograde | Doe & Roe, 1992  |       |
+| l‑a    | r‑a    | retrograde | Doe & Roe, 1992  |       |
+| l‑b    | r‑a    | anterograde | Doe et al., 1999 |       |
+| l‑b    | r‑b    | retrograde | Doe et al., 1999 |       |
+| r‑a    | r‑b    | anterograde | Doe et al., 1999 |       |
+| r‑b    | l‑a    | retrograde | Li et al., 2012  |       |
+| r‑b    | l‑b    | anterograde | Doe et al., 1996 | injection too large to interpret positive labeling; evidence used for “absent” classification |
+| r‑b    | r‑a    | retrograde | Li et al., 2012  |       |
+
+**Note**: The `source` and `target` columns provide the unique identifiers used in the edge table.
+
+## Identifiers
+
+It is important to be able to match the metadata to the network structure so that each node and connection in the edge table can be related back to the biological regions and studies they represent.
+
+In this example, the common identifiers used the abbreviations `l‑a`, `l‑b`, `r‑a`, and `r‑b`. These appear in the edge table as the node labels, and the same labels appear in the node metadata so that each entry in the network can be matched to the corresponding brain region.

docs/source/connectivity-math.md

@@ -0,0 +1,58 @@
+# Mathematical Description of a Network
+
+Whilst we can get by with a colloquial description of networks, in some places it will be necessary for us to use a more mathematical description of these objects.
+
+We aim to set a standard for our notation and conventions in this section.
+
+## Networks
+A network—also called a *graph* in mathematical contexts—is written as $G = (V, E)$.
+
+It consists of two components:
+1. A set of **nodes** (or *vertices*), denoted by $V$.
+2. A set of **edges**, denoted by $E$.
+
+When modelling brain connectivity, the nodes ($V$) represent the **brain regions**, and the edges ($E$) represent the **connections** between them.
+
+### Nodes
+The nodes are indexed by some counter, so we typically write $V := \{ v_0, v_1, ..., v_N\}$ where each $v_i$ is one node, uniquely identified by the value of $i$.
+
+As such, when referring to nodes it is common to drop the $v_i$ notation and simply say "node $i$" in place of "node $v_i$".
+
+### Edges
+Each edge $e\in E$ is written as $e = (i, j, w)$ (or $e = (v_i, v_j, w)$).
+
+$(i, j, w)$ represents a directed, weighted edge that:
+
+- starts **from** node $i$
+- goes **to** node $j$
+- with **weight** $w$
+
+The **edge table** that describes a network can then simply be seen as a listing of the edges $e\in E$.
+
+For an example of how an edge table corresponds to a network, see [Example: Edge Table and Corresponding Network](./connectivity-data.md#example-edge-table-and-corresponding-network).
+
+### Internal Indexes
+If the brain regions are provided with unique identifiers, these same unique identifiers can be used as the indexes $i$ for the mathematical description of the nodes.
+
+However, the tools used in this package prefer the use of integer indexes to identify the nodes, rather than names or abbreviations.
+
+The API separates the metadata from the underlying network object itself. The nodes in the `.network` use integer indexing, which is referred to as their "internal indexes".
+
+<!-- FIXME: add API link for Connections.network -->
+
+Where this is important, the package keeps a record of the correspondence between "internal indexes" for the nodes and the user-facing identifiers for the brain regions they represent.
+
+These "internal indexes" are appended as a column to the node metadata, so there is a means of translating between the neuroscientific information about a brain region and its abstract representation.
+
+Several functions also exist to aid in obtaining a selection of nodes by matching metadata criteria, and vice-versa.
+
+#### Example: Internal Indexes
+
+When the [example edge table](./connectivity-data.md#example-edge-table-and-corresponding-network) is loaded, the package assigns an internal integer index to each unique region label. This mapping allows the network to be stored and processed using integers, while still preserving the original region labels for interpretation.
+
+| region label | internal index |
+|--------|----------------|
+| l‑a    | 0              |
+| l‑b    | 1              |
+| r‑a    | 2              |
+| r‑b    | 3              |

docs/source/index.md

@@ -5,5 +5,20 @@
    :maxdepth: 2
    :caption: Contents:
 
-   getting_started
+   connectivity-math
+   connectivity-data
+   examples/index
 ```
+
+This package acts as an interface between brain connectivity data and graph‑based analyses.
+
+The purpose of the package is to provide neuroscientists with an interface for asking questions about a connectivity network without having to translate those questions into mathematical graph operations.
+
+For example, the question "what is the shortest path between brain areas A and B that passes through region C?" translates to "determine the shortest path between region A and region C, and then the shortest path between region C and region B, and concatenate them", which can be handled using well-established network algorithms.
+
+In practice, the API relies on the [`rustworkx`](https://www.rustworkx.org/) library for all network-related queries. Connectivity data is represented by a [`PyDiGraph` class](https://www.rustworkx.org/apiref/rustworkx.PyDiGraph.html), which is exposed through the `Connections.network` attribute to allow for flexible querying of the network structure if specialised analyses are required.
+
+## Connectivity Data
+See [Connectivity Data](./connectivity-data.md) for details on the structure of the input data, including networks and metadata.
+
+## Interacting with the Data

pyproject.toml

@@ -48,9 +48,11 @@ dev = [
   "setuptools-scm",
 ]
 docs = [
+  "matplotlib",
   "pydata-sphinx-theme",
   "sphinx",
   "sphinx_autodoc_typehints",
+  "sphinx-gallery",
   "sphinx_sitemap",
   "myst_parser[linkify]",
 ]