Skip to content

Commit 51c3854

Browse files
lmeyerovlmeyerov
authored
Merge pull request #875 from graphistry/docs/collections-support
docs: add collections usage
2 parents 8fe2df2 + a9980d6 commit 51c3854

10 files changed

Lines changed: 324 additions & 0 deletions

File tree

‎demos/more_examples/graphistry_features/collections.ipynb‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Collections in PyGraphistry\n",
8+
"\n",
9+
"Collections define labeled subsets of a graph (nodes, edges, or subgraphs) using full GFQL. They enable advanced, layered styling that overrides base encodings when you need precise highlights.\n",
10+
"\n",
11+
"Use collections when you want:\n",
12+
"- baseline encodings (for example, by entity type) plus overlays for alerts or critical paths\n",
13+
"- multiple overlapping highlights with a priority order\n",
14+
"- a UI panel for toggling focused subsets on and off\n",
15+
"\n",
16+
"Collections are evaluated in priority order, with higher priority collections overriding lower ones for styling.\n"
17+
]
18+
},
19+
{
20+
"cell_type": "markdown",
21+
"metadata": {},
22+
"source": [
23+
"In this notebook, we build sets using GFQL AST helpers, combine them with intersections, and apply node and edge colors. Collections can be based on nodes, edges, or multi-step graph traversals (Chain).\n"
24+
]
25+
},
26+
{
27+
"cell_type": "code",
28+
"metadata": {},
29+
"execution_count": null,
30+
"outputs": [],
31+
"source": [
32+
"from pathlib import Path\n",
33+
"import pandas as pd\n",
34+
"import graphistry\n",
35+
"from graphistry import collection_set, collection_intersection, n, e_forward, Chain\n",
36+
"\n",
37+
"edges = pd.read_csv(Path('demos/data/honeypot.csv'))\n",
38+
"g = graphistry.edges(edges, \"attackerIP\", \"victimIP\")\n"
39+
]
40+
},
41+
{
42+
"cell_type": "code",
43+
"metadata": {},
44+
"execution_count": null,
45+
"outputs": [],
46+
"source": [
47+
"# Use Chain to select subgraphs (nodes + edges) by edge attributes\n",
48+
"collections = [\n",
49+
" collection_set(\n",
50+
" expr=Chain([n(), e_forward({\"vulnName\": \"MS08067 (NetAPI)\"}), n()]),\n",
51+
" id='netapi',\n",
52+
" name='MS08067 (NetAPI)',\n",
53+
" node_color='#00BFFF',\n",
54+
" edge_color='#00BFFF',\n",
55+
" ),\n",
56+
" collection_set(\n",
57+
" expr=Chain([n(), e_forward({\"victimPort\": 445.0}), n()]),\n",
58+
" id='port445',\n",
59+
" name='Port 445',\n",
60+
" node_color='#32CD32',\n",
61+
" edge_color='#32CD32',\n",
62+
" ),\n",
63+
" collection_intersection(\n",
64+
" sets=['netapi', 'port445'],\n",
65+
" name='NetAPI + 445',\n",
66+
" node_color='#AABBCC',\n",
67+
" edge_color='#AABBCC',\n",
68+
" ),\n",
69+
"]\n",
70+
"\n",
71+
"g2 = g.collections(\n",
72+
" collections=collections,\n",
73+
" show_collections=True,\n",
74+
" collections_global_node_color='CCCCCC',\n",
75+
" collections_global_edge_color='CCCCCC',\n",
76+
")\n",
77+
"\n",
78+
"g2._url_params\n"
79+
]
80+
},
81+
{
82+
"cell_type": "code",
83+
"metadata": {},
84+
"execution_count": null,
85+
"outputs": [],
86+
"source": [
87+
"# Render (requires graphistry.register(...))\n",
88+
"g2.plot()\n"
89+
]
90+
},
91+
{
92+
"cell_type": "markdown",
93+
"metadata": {},
94+
"source": [
95+
"## Notes and validation\n",
96+
"\n",
97+
"- Order matters: earlier collections override later ones.\n",
98+
"- Use collections for priority-based subsets and overlaps; use `encode_*` for simple column-driven colors.\n",
99+
"- Helper constructors: `graphistry.collection_set(...)` and `graphistry.collection_intersection(...)` return JSON-friendly dicts (AST inputs wrap to `gfql_chain`).\n",
100+
"- Provide `id` for sets used by intersections.\n",
101+
"- Global colors apply to nodes/edges not in any collection; `#` is optional.\n",
102+
"- Use `validate='strict'` to raise, or `warn=False` to silence warnings.\n",
103+
"\n",
104+
"Wire protocol and pre-encoded strings:\n",
105+
"\n",
106+
"```python\n",
107+
"collections_wire = [\n",
108+
" {\n",
109+
" \"type\": \"set\",\n",
110+
" \"name\": \"Wire Protocol Example\",\n",
111+
" \"node_color\": \"#AA00AA\",\n",
112+
" \"expr\": {\n",
113+
" \"type\": \"gfql_chain\",\n",
114+
" \"gfql\": [\n",
115+
" {\"type\": \"Node\", \"filter_dict\": {\"status\": \"purchased\"}}\n",
116+
" ]\n",
117+
" }\n",
118+
" }\n",
119+
"]\n",
120+
"g.collections(collections=collections_wire)\n",
121+
"\n",
122+
"g.collections(collections=encoded_collections, encode=False)\n",
123+
"```\n"
124+
]
125+
},
126+
{
127+
"cell_type": "markdown",
128+
"metadata": {},
129+
"source": [
130+
"Run `g2.plot()` in a notebook session with valid credentials to render inline.\n"
131+
]
132+
},
133+
{
134+
"cell_type": "markdown",
135+
"metadata": {},
136+
"source": [
137+
"## Overlap priority example\n",
138+
"\n",
139+
"Earlier collections override later ones when they overlap.\n",
140+
"\n",
141+
"```python\n",
142+
"collections_priority = [\n",
143+
" collection_set(\n",
144+
" expr=Chain([n(), e_forward({\"vulnName\": \"MS08067 (NetAPI)\"}), n()]),\n",
145+
" id=\"netapi\",\n",
146+
" name=\"MS08067 (NetAPI)\",\n",
147+
" node_color=\"#FFAA00\",\n",
148+
" edge_color=\"#FFAA00\",\n",
149+
" ),\n",
150+
" collection_set(\n",
151+
" expr=Chain([n(), e_forward({\"victimPort\": 445.0}), n()]),\n",
152+
" id=\"port445\",\n",
153+
" name=\"Port 445\",\n",
154+
" node_color=\"#00BFFF\",\n",
155+
" edge_color=\"#00BFFF\",\n",
156+
" ),\n",
157+
"]\n",
158+
"g.collections(collections=collections_priority)\n",
159+
"```\n"
160+
]
161+
},
162+
{
163+
"cell_type": "markdown",
164+
"metadata": {},
165+
"source": [
166+
"For more on color encodings, see the [Color encodings notebook](encodings-colors.ipynb).\n"
167+
]
168+
}
169+
],
170+
"metadata": {
171+
"kernelspec": {
172+
"display_name": "Python 3",
173+
"language": "python",
174+
"name": "python3"
175+
},
176+
"language_info": {
177+
"name": "python",
178+
"version": "3.x"
179+
}
180+
},
181+
"nbformat": 4,
182+
"nbformat_minor": 5
183+
}

‎docs/source/10min.rst‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,28 @@ Example visualization:
161161

162162
Now, edges are colored based on the type of vulnerability, helping you distinguish different attack types.
163163

164+
Advanced: Collections for layered highlights
165+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
166+
167+
Use collections when you want GFQL-driven subsets (nodes, edges, or subgraphs) to override base encodings.
168+
This is useful for overlays like alerts or critical paths that take precedence over your normal color rules.
169+
170+
.. code-block:: python
171+
172+
from graphistry import collection_set, n
173+
174+
collections = [
175+
collection_set(
176+
expr=n({"vip": True}),
177+
name="VIP",
178+
node_color="#FF8800",
179+
)
180+
]
181+
g.collections(collections=collections, show_collections=True).plot()
182+
183+
See the :doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>` and
184+
:doc:`GFQL docs </gfql/index>` for full details.
185+
164186
Adjusting Sizes, Labels, Icons, Badges, and More
165187
------------------------------------------------
166188

‎docs/source/cheatsheet.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -560,6 +560,11 @@ g.encode_point_color('type', as_categorical=True,
560560
categorical_mapping={"cat": "red", "sheep": "blue"}, default_mapping='#CCC')
561561
```
562562

563+
For subset-based coloring and conditional styling across multiple encodings, use Collections
564+
via `g.collections(...)` with GFQL AST helpers. See
565+
{doc}`Layout settings </visualization/layout/settings>`
566+
and the [Collections tutorial notebook](demos/more_examples/graphistry_features/collections.ipynb).
567+
563568
For more in-depth examples, check out the tutorials on [colors](https://github.com/graphistry/pygraphistry/tree/master/demos/more_examples/graphistry_features/encodings-colors.ipynb).
564569

565570
### Custom icons and badges

‎docs/source/gfql/quick.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -637,6 +637,9 @@ Run graph algorithms like PageRank, community detection, and layouts directly wi
637637
# Results have x, y coordinates for visualization
638638
result.plot()
639639
640+
Tip: For subset-based coloring after GFQL, use ``result.collections(...)`` and see
641+
:doc:`/visualization/layout/settings`.
642+
640643
Remote Graph References
641644
-----------------------
642645

‎docs/source/gfql/remote.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,13 @@ Run chain remotely and fetch results
1919
2020
``gfql_remote()`` accepts the same input types as local ``gfql()``:
2121

22+
.. note::
23+
Collections are visualization URL settings; apply them after GFQL results
24+
(for example, ``g2.collections(...)``). The GFQL remote/upload APIs do not
25+
accept collections payloads yet.
26+
27+
Method :meth:`chain_remote <graphistry.compute.ComputeMixin.ComputeMixin.chain_remote>` runs chain remotely and fetches the computed graph
28+
2229
- **Chain / List[ASTObject]**: Native GFQL chain syntax (as above).
2330
- **Cypher string**: Compiled locally, sent as wire-protocol JSON.
2431
- **ASTLet / Let dict**: DAG patterns with named bindings.

‎docs/source/gfql/spec/wire_protocol.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -595,6 +595,50 @@ Example:
595595

596596
**Note**: The `timezone` field is optional for DateTime values and defaults to "UTC" if omitted. This ensures consistent behavior across systems while allowing explicit timezone specification when needed.
597597

598+
## Collections Payloads
599+
600+
Collections are Graphistry visualization overlays that use GFQL wire protocol operations to define subsets
601+
of nodes, edges, or subgraphs. They are applied in priority order, with earlier collections overriding later
602+
ones for styling.
603+
604+
### Collection Set
605+
606+
Collection sets wrap GFQL operations in a `gfql_chain` object:
607+
608+
```json
609+
{
610+
"type": "set",
611+
"id": "purchasers",
612+
"name": "Purchasers",
613+
"node_color": "#00BFFF",
614+
"expr": {
615+
"type": "gfql_chain",
616+
"gfql": [
617+
{"type": "Node", "filter_dict": {"status": "purchased"}}
618+
]
619+
}
620+
}
621+
```
622+
623+
### Collection Intersection
624+
625+
Intersections reference previously defined set IDs:
626+
627+
```json
628+
{
629+
"type": "intersection",
630+
"name": "High Value Purchasers",
631+
"node_color": "#AA00AA",
632+
"expr": {
633+
"type": "intersection",
634+
"sets": ["purchasers", "vip"]
635+
}
636+
}
637+
```
638+
639+
For Python examples and helper constructors, see the
640+
:doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>`.
641+
598642
## Examples
599643

600644
### `MATCH ... RETURN` Row Pipeline

‎docs/source/notebooks/visualization.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ Encodings
1313
Sizes <../demos/more_examples/graphistry_features/encodings-sizes.ipynb>
1414
Icons <../demos/more_examples/graphistry_features/encodings-icons.ipynb>
1515
Badges <../demos/more_examples/graphistry_features/encodings-badges.ipynb>
16+
Collections <../demos/more_examples/graphistry_features/collections.ipynb>
1617

1718
Geographic (Kepler.gl)
1819
----------------------

‎docs/source/visualization/10min.rst‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -224,6 +224,25 @@ You can encode your graph attributes visually using colors, sizes, icons, and mo
224224
- :meth:`graphistry.PlotterBase.PlotterBase.encode_edge_color`
225225
- :meth:`graphistry.PlotterBase.PlotterBase.encode_edge_icon`
226226

227+
* **Collections (advanced coloring)**: Define subsets using GFQL AST helpers and color them consistently:
228+
229+
.. code-block:: python
230+
231+
from graphistry import collection_set, n
232+
233+
collections = [
234+
collection_set(
235+
expr=n({"subscribed_to_newsletter": True}),
236+
name="Subscribers",
237+
node_color="#32CD32",
238+
)
239+
]
240+
g.collections(collections=collections, show_collections=True).plot()
241+
242+
See :doc:`Layout settings <layout/settings>` and the
243+
:doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>`.
244+
Tip: order matters (earlier collections override later ones) and intersections require set IDs.
245+
227246
* **Bind**: Simpler data-driven settings are done through :meth:`graphistry.PlotterBase.PlotterBase.bind`:
228247

229248
.. code-block:: python

‎docs/source/visualization/index.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ Visualize
22
=============
33

44
We recommend getting started with :ref:`10 Minutes to PyGraphistry <10min-pygraphistry>`, :ref:`10 Minutes to Graphistry Visualization<10min-viz>`, and the :ref:`layout guide <layout-guide>`.
5+
For advanced, subset-based coloring, see :ref:`Layout settings <layout-settings>` and the
6+
:doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>`.
57

68
For static image export (documentation, reports), see the `static rendering tutorial <../demos/demos_databases_apis/graphviz/static_rendering.ipynb>`_.
79

‎docs/source/visualization/layout/settings.rst‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,44 @@ Use :meth:`graphistry.PlotterBase.PlotterBase.scene_settings` to modify the appe
3636
- ``edge_opacity``: Range 0.0 to 1.0 (0.0 fully transparent, 1.0 fully opaque, displayed as 0-100 in UI)
3737
- ``point_opacity``: Range 0.0 to 1.0 (0.0 fully transparent, 1.0 fully opaque, displayed as 0-100 in UI)
3838

39+
Encodings (Color, Size, Icons)
40+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
41+
42+
Use the ``encode_*`` methods to style nodes and edges based on columns (for example, color by entity type).
43+
See the :doc:`Color encodings notebook </demos/more_examples/graphistry_features/encodings-colors>` for full examples.
44+
45+
Collections
46+
~~~~~~~~~~~
47+
48+
Collections define labeled subsets (nodes, edges, or subgraphs) using full GFQL and apply layered styling
49+
that overrides base encodings. Use them to call out alerts or critical paths on top of your standard color
50+
encodings, with priority-based overrides when subsets overlap.
51+
52+
For a full walkthrough, see the :doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>`.
53+
For GFQL syntax, see :doc:`GFQL documentation </gfql/index>`.
54+
For schema details, see `Collections URL options <https://hub.graphistry.com/docs/api/1/rest/url/#url-collections>`_.
55+
56+
.. code-block:: python
57+
58+
from graphistry import collection_set, n
59+
60+
collections = [
61+
collection_set(
62+
expr=n({"subscribed_to_newsletter": True}),
63+
id="newsletter_subscribers",
64+
name="Newsletter Subscribers",
65+
node_color="#32CD32",
66+
)
67+
]
68+
69+
g2 = g.collections(
70+
collections=collections,
71+
show_collections=True,
72+
collections_global_node_color="CCCCCC",
73+
collections_global_edge_color="CCCCCC",
74+
)
75+
g2.plot()
76+
3977
4078
Styling the Background and Foreground
4179
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

0 commit comments

Comments
 (0)