Skip to content

Commit 017e81c

Browse files
daxpryceCopilot
andauthored
feat!(crates): thirdparty interop support (#69)
* refactor: extract NetworkView trait, genericize local moving phase - Define NetworkView trait with GAT-based neighbors iterator, replacing the NetworkDetails trait entirely - Implement NetworkView for CompactNetwork and LabeledNetwork - Make full_network_clustering generic over N: NetworkView - Add to_compact_network() default method for materialization during recursive aggregation - Include num_edges(), total_edge_weight_per_node(), node_weights() as default methods on the trait - Update resolution::adjust_resolution to accept impl NetworkView - All existing tests pass unchanged Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add CsrNetworkView for zero-copy CSR graph support - CsrNetworkView holds borrowed indptr/indices/data/node_weights slices - Validates structural invariants at construction time (monotonic indptr, bounds checking, finite non-negative weights) - Precomputes total_edge_weight, total_node_weight, total_self_links - Implements NetworkView with efficient CsrNeighborIterator - Integration test verifies CSR → CompactNetwork → Leiden produces correct community detection results Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add leiden_csr Python function with zero-copy scipy CSR support - Add ScipyCsrView in pyo3 crate implementing NetworkView directly on scipy's i64 indptr, i32 indices, f64 data arrays (zero-copy) - Add leiden_view() entry point in network_partitions that runs the initial local-moving phase on a generic NetworkView, only materializing to CompactNetwork when recursion is needed - Release the GIL during computation (unsafe Send+Sync on the view, safe because numpy arrays are held alive by the Python caller frame) - Only allocation during construction: node_weights (row sums, O(nnz)) - Python API: leiden_csr(indptr, indices, data, n_nodes, ...) returns (quality, Dict[int, int]) - Add numpy 0.28 dependency to pyo3 crate Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add petgraph and sprs interop with integration tests Add feature-gated NetworkView implementations for petgraph::UnGraph<f64,f64> and sprs::CsMatI<f64,usize>, enabling zero-copy Leiden community detection on third-party graph representations. - PetgraphNetworkView: wraps &UnGraph with precomputed totals - SprsNetworkView: wraps &CsMatI (CSR) with optional node weights - 25 integration tests covering basic properties, community detection, karate club, disconnected components, self-loops, and validation - Fix pre-existing bug in CompactNetwork::total_edge_weight_per_node (was using connection_start as node_id) - Add leiden_view vs leiden equivalence test - Bump crate versions: network_partitions 0.3.0, cli 0.3.0 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: modernize README with current versions and remove CLA section - Rust stable (edition 2024), not nightly - Python 3.9+, wheels for 3.9-3.13 - x86_64 + aarch64 platforms - Updated repo URL to graspologic-org - Removed Microsoft CLA and Code of Conduct sections - Added uv build instructions - Simplified privacy section Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address PR #69 review feedback - Self-loop handling: exclude diagonal entries from neighbors_for(), node_weights, and total_edge_weight across CsrNetworkView, ScipyCsrView, and SprsNetworkView. Self-loops are tracked separately via total_self_links_edge_weight() per codebase convention. - Remove unnecessary unsafe impl Send/Sync on ScipyCsrView (the type is automatically Send+Sync via its field types). - Validate trials > 0 in leiden_csr mediator (was panicking on unwrap). - Hoist to_compact_network() out of the trial loop in leiden_csr (was O(nnz) allocation per trial, now materialized once). - Fix asymmetric test graph in interop/mod.rs (node 0 had a spurious bridge edge to node 3 that wasn't reciprocated). - Update NetworkView trait docs to clarify self-loop convention. - Add weight validation (finite, non-negative) to ScipyCsrView::new(). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address second round of PR #69 review feedback - petgraph: neighbors_for now skips self-loop edges; num_edges excludes self-loops - sprs: neighbors_for now skips diagonal entries; num_edges excludes self-loops ((nnz - diag) / 2) - scipy_csr: accept use_modularity param to correctly set node weights (degree sums for modularity, 1.0 for CPM); num_edges precomputed excluding diagonals - csr_view: num_edges now precomputed excluding diagonal entries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address third round of PR #69 review feedback - sprs num_edges: count all diagonal entries per row (not just presence) to handle duplicate (i,i) non-zeros correctly - csr_view: add indptr[0] == 0 validation - csr_view: fix doc to clarify symmetry is assumed, not enforced Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address remaining PR #69 review feedback - petgraph: validate edge weights (finite, non-negative) at construction; document node weight requirements for modularity vs CPM mode - sprs: validate edge weights with InvalidWeight error variant; document node weight requirements for modularity vs CPM mode - lib.rs: fix leiden_csr docstring to not claim fully zero-copy run Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: convert PetgraphNetworkView::new to return Result; add leiden_csr Python tests - PetgraphNetworkView::new now returns Result<Self, PetgraphValidationError> instead of panicking on invalid input - Added PetgraphValidationError enum with NonContiguousNodeIndices and InvalidEdgeWeight variants - Added Python tests for leiden_csr: basic two-triangle graph, deterministic seeding, and trials-improves-or-equals quality Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: add numpy and scipy to Python test dependencies Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * build: migrate CI and docs to uv, add numpy/scipy dependencies - Replace setup-python + pip with astral-sh/setup-uv across all CI jobs - Add numpy>=1.24 and scipy>=1.10 as project dependencies in pyproject.toml - Update README build instructions to use uv exclusively - Remove Windows -i python.exe workaround (uv handles Python discovery) - Use 'uv run --with' for one-off tool invocations (toml, twine) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: clarify GIL release safety contract for leiden_csr The input arrays must not be mutated from another thread during computation. This is the caller's responsibility — under CPython's normal single-threaded usage this is naturally satisfied. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 7155f6a commit 017e81c

26 files changed

Lines changed: 2958 additions & 137 deletions

.github/workflows/build.yml

Lines changed: 9 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -15,15 +15,10 @@ jobs:
1515
runs-on: "ubuntu-latest"
1616
steps:
1717
- uses: actions/checkout@v6
18-
- name: Set up Python
19-
uses: actions/setup-python@v6
20-
with:
21-
python-version: "3.14"
18+
- uses: astral-sh/setup-uv@v6
2219
- name: Materialize build number
2320
run: |
24-
pip install -U pip
25-
pip install toml
26-
python .github/build/manifest_version.py packages/pyo3/Cargo.toml version.txt
21+
uv run --with toml python .github/build/manifest_version.py packages/pyo3/Cargo.toml version.txt
2722
- uses: actions/upload-artifact@v4
2823
with:
2924
name: cargo-toml
@@ -73,10 +68,7 @@ jobs:
7368
artifact_name: dist-macos-universal2
7469
steps:
7570
- uses: actions/checkout@v6
76-
- name: Set up Python
77-
uses: actions/setup-python@v6
78-
with:
79-
python-version: "3.14"
71+
- uses: astral-sh/setup-uv@v6
8072
- uses: actions/download-artifact@v4
8173
with:
8274
name: cargo-toml
@@ -104,14 +96,15 @@ jobs:
10496
with:
10597
command: build
10698
target: ${{ matrix.target }}
107-
args: ${{ matrix.args }} -i ${{env.pythonLocation}}\python.exe
99+
args: ${{ matrix.args }}
108100

109101
- name: Python Unittests
110102
if: ${{ !matrix.manylinux || matrix.manylinux != 'musllinux_1_2' }}
111103
run: |
112104
cd packages/pyo3
113-
pip install ../../target/wheels/*.whl
114-
python -m unittest
105+
uv venv
106+
uv pip install ../../target/wheels/*.whl
107+
uv run python -m unittest
115108
shell: bash
116109

117110
- uses: actions/upload-artifact@v4
@@ -128,10 +121,7 @@ jobs:
128121
contents: write
129122
steps:
130123
- uses: actions/checkout@v6
131-
- name: Set up Python
132-
uses: actions/setup-python@v6
133-
with:
134-
python-version: "3.14"
124+
- uses: astral-sh/setup-uv@v6
135125
- uses: actions/download-artifact@v4
136126
with:
137127
name: dist-linux-x64
@@ -174,15 +164,12 @@ jobs:
174164
GRASPOLOGIC_VERSION=$(cat version/version.txt)
175165
echo "GRASPOLOGIC_VERSION=$GRASPOLOGIC_VERSION" >> $GITHUB_ENV
176166
echo $GRASPOLOGIC_VERSION
177-
- name: Install dependencies
178-
run: |
179-
python -m pip install -U pip setuptools wheel twine
180167
- name: Publish with twine
181168
env:
182169
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
183170
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
184171
run: |
185-
twine upload dist/*
172+
uv run --with twine twine upload dist/*
186173
- name: Create Github Release
187174
env:
188175
GH_TOKEN: ${{ github.token }}

README.md

Lines changed: 30 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,58 +1,51 @@
11
# graspologic-native
22

3-
[graspologic](https://github.com/microsoft/graspologic) is a python package for graph statistics.
3+
[graspologic](https://github.com/graspologic-org/graspologic) is a Python package for graph statistics.
44

5-
Some functionality can be best served if compiled into a python native module, both for performance purposes and to
6-
share that functionality with web assembly.
7-
8-
`graspologic-native` is a repository that holds Rust packages. The core packages will be published as crate libraries,
9-
and a package using [pyo3](https://github.com/pyo3/pyo3) will expose the functionality of that library to Python.
5+
Some functionality is best served compiled into a native Python extension module for performance. `graspologic-native` holds the Rust implementation of the [Leiden community detection algorithm](https://arxiv.org/abs/1810.08473), exposed to Python via [PyO3](https://github.com/pyo3/pyo3) and built with [maturin](https://github.com/PyO3/maturin).
106

117
## Requirements
12-
- Rust nightly 1.37+ (we are currently using 1.40)
13-
- Python 3.5+ (we are currently using 3.8)
14-
- 64 bit operating system
8+
9+
- Rust stable (edition 2024)
10+
- Python 3.9+
11+
- [uv](https://github.com/astral-sh/uv)
12+
- 64-bit operating system
1513

1614
## Published Versions
17-
We currently build for x86_64 platforms only, Windows, macOS, and Ubuntu, for python versions 3.6 - 3.12.
1815

19-
## Building
20-
If for any reason, the published wheels do not match your architecture or if you have a particularly old version of glibc that isn't sufficiently accounted for in our current build matrix, or you just want to build it yourself, the following build instructions should help.
16+
We build wheels for x86_64 and aarch64 on Linux, macOS (universal2), and Windows, for Python 3.9–3.13.
2117

22-
Note that these instructions are for Linux specifically, though they also should work for MacOS. Unfortunately, the instructions for Windows are a bit more convoluted and I will comment the sections that deviate between the three, as I'm aware of issues.
18+
## Building
2319

24-
Before running these instructions, ensure you have installed Rust on your system and you have the Python development headers (e.g. `python3.8-dev`) for your system.
20+
If the published wheels don't match your platform, or you want to build from source:
2521

2622
```bash
27-
rustup default nightly
28-
git clone git@github.com:microsoft/graspologic-native.git
29-
cd graspologic-native
30-
python3.8 -m venv venv
31-
pip install -U pip setuptools wheel
32-
pip install maturin
33-
cd packages/pyo3
34-
maturin build --release -i python3.8 # this is where things break on windows. instead of `python3.8` here, you will need the full path to the correct python.exe on your windows machine, something like `-i "C:\python38\bin\python.exe"`
23+
git clone git@github.com:graspologic-org/graspologic-native.git
24+
cd graspologic-native/packages/pyo3
25+
uv build
3526
```
3627

37-
Presuming a successful build, your output wheel should be in: `graspologic-native/target/wheels/`
28+
The output wheel will be in `dist/`.
3829

39-
## Contributing
30+
To install in a local virtual environment for development:
4031

41-
This project welcomes contributions and suggestions. Most contributions require you to
42-
agree to a Contributor License Agreement (CLA) declaring that you have the right to,
43-
and actually do, grant us the rights to use your contribution. For details, visit
44-
https://cla.microsoft.com.
32+
```bash
33+
cd graspologic-native/packages/pyo3
34+
uv sync
35+
uv pip install -e .
36+
```
37+
38+
## Testing
4539

46-
When you submit a pull request, a CLA-bot will automatically determine whether you need
47-
to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the
48-
instructions provided by the bot. You will only need to do this once across all repositories using our CLA.
40+
```bash
41+
cd packages/pyo3
42+
uv run python -m unittest
43+
```
4944

50-
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
51-
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
52-
or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
45+
## Contributing
5346

54-
# Privacy
47+
This project welcomes contributions and suggestions. Please open an issue or pull request on GitHub.
5548

56-
`graspologic-native` does not collect, store, or transmit any information of any kind back to Microsoft.
49+
## Privacy
5750

58-
For your convenience, here is the link to the general [Microsoft Privacy Statement](https://privacy.microsoft.com/en-us/privacystatement/).
51+
`graspologic-native` does not collect, store, or transmit any information of any kind.

packages/cli/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "cli"
3-
version = "0.2.0"
3+
version = "0.3.0"
44
authors = ["Dax Pryce <daxpryce@microsoft.com>"]
55
edition = "2024"
66
license = "MIT"
Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,17 @@
11
[package]
22
name = "network_partitions"
3-
version = "0.2.0"
3+
version = "0.3.0"
44
authors = ["Dax Pryce <daxpryce@microsoft.com>"]
55
edition = "2024"
66
license = "MIT"
77
description = "Leiden community detection as per https://arxiv.org/abs/1810.08473"
88

9+
[features]
10+
default = []
11+
petgraph = ["dep:petgraph"]
12+
sprs = ["dep:sprs"]
13+
914
[dependencies]
1015
rand = "0.10"
16+
petgraph = { version = "0.8", optional = true }
17+
sprs = { version = "0.11", optional = true, default-features = false }
Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
// Copyright (c) Microsoft Corporation.
2+
// Licensed under the MIT license.
3+
4+
//! Interop implementations for third-party graph libraries.
5+
//!
6+
//! Each submodule is gated behind a feature flag.
7+
8+
#[cfg(feature = "petgraph")]
9+
pub mod petgraph_interop;
10+
11+
#[cfg(feature = "sprs")]
12+
pub mod sprs_interop;
13+
14+
#[cfg(test)]
15+
mod tests {
16+
use crate::leiden::{leiden, leiden_view};
17+
use crate::network::CompactNetwork;
18+
use rand::SeedableRng;
19+
use rand::rngs::SmallRng;
20+
21+
/// Verify leiden_view produces equivalent results to leiden on the same CompactNetwork.
22+
#[test]
23+
fn test_leiden_view_vs_leiden_on_compact_network() {
24+
// Two triangles (weight 10) connected by a weak bridge (weight 0.01) between nodes 2-3.
25+
// Node weights = sum of incident non-self-loop edge weights (modularity convention).
26+
// Node 0: edges to 1(10), 2(10) → weight 20
27+
// Node 1: edges to 0(10), 2(10) → weight 20
28+
// Node 2: edges to 0(10), 1(10), 3(0.01) → weight 20.01
29+
// Node 3: edges to 2(0.01), 4(10), 5(10) → weight 20.01
30+
// Node 4: edges to 3(10), 5(10) → weight 20
31+
// Node 5: edges to 3(10), 4(10) → weight 20
32+
let nodes = vec![
33+
(20.0_f64, 0_usize),
34+
(20.0, 2),
35+
(20.01, 4),
36+
(20.01, 7),
37+
(20.0, 10),
38+
(20.0, 12),
39+
];
40+
let neighbors = vec![
41+
(1_usize, 10.0_f64),
42+
(2, 10.0), // node 0: neighbors 1, 2
43+
(0, 10.0),
44+
(2, 10.0), // node 1: neighbors 0, 2
45+
(0, 10.0),
46+
(1, 10.0),
47+
(3, 0.01), // node 2: neighbors 0, 1, 3
48+
(2, 0.01),
49+
(4, 10.0),
50+
(5, 10.0), // node 3: neighbors 2, 4, 5
51+
(3, 10.0),
52+
(5, 10.0), // node 4: neighbors 3, 5
53+
(3, 10.0),
54+
(4, 10.0), // node 5: neighbors 3, 4
55+
];
56+
let compact = CompactNetwork::from(nodes, neighbors, 0.0);
57+
58+
let mut rng1 = SmallRng::seed_from_u64(42);
59+
let (_improved1, c1) = leiden(
60+
&compact,
61+
None,
62+
Some(1),
63+
None,
64+
None,
65+
&mut rng1,
66+
true,
67+
None,
68+
None,
69+
)
70+
.unwrap();
71+
72+
let mut rng2 = SmallRng::seed_from_u64(42);
73+
let (_improved2, c2) = leiden_view(
74+
&compact,
75+
None,
76+
Some(1),
77+
None,
78+
None,
79+
&mut rng2,
80+
true,
81+
Some(10),
82+
None,
83+
)
84+
.unwrap();
85+
86+
// Both should find 2 communities
87+
assert_eq!(c1.next_cluster_id(), 2);
88+
assert_eq!(c2.next_cluster_id(), 2);
89+
// Same assignments
90+
for i in 0..6 {
91+
assert_eq!(
92+
c1.cluster_at(i).unwrap(),
93+
c2.cluster_at(i).unwrap(),
94+
"Node {i} cluster mismatch between leiden and leiden_view"
95+
);
96+
}
97+
}
98+
}

0 commit comments

Comments
 (0)