diff --git a/demos/more_examples/graphistry_features/collections.ipynb b/demos/more_examples/graphistry_features/collections.ipynb new file mode 100644 index 0000000000..71864b3e06 --- /dev/null +++ b/demos/more_examples/graphistry_features/collections.ipynb @@ -0,0 +1,183 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Collections in PyGraphistry\n", + "\n", + "Collections define labeled subsets of a graph (nodes, edges, or subgraphs) using full GFQL. They enable advanced, layered styling that overrides base encodings when you need precise highlights.\n", + "\n", + "Use collections when you want:\n", + "- baseline encodings (for example, by entity type) plus overlays for alerts or critical paths\n", + "- multiple overlapping highlights with a priority order\n", + "- a UI panel for toggling focused subsets on and off\n", + "\n", + "Collections are evaluated in priority order, with higher priority collections overriding lower ones for styling.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this notebook, we build sets using GFQL AST helpers, combine them with intersections, and apply node and edge colors. Collections can be based on nodes, edges, or multi-step graph traversals (Chain).\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "execution_count": null, + "outputs": [], + "source": [ + "from pathlib import Path\n", + "import pandas as pd\n", + "import graphistry\n", + "from graphistry import collection_set, collection_intersection, n, e_forward, Chain\n", + "\n", + "edges = pd.read_csv(Path('demos/data/honeypot.csv'))\n", + "g = graphistry.edges(edges, \"attackerIP\", \"victimIP\")\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "execution_count": null, + "outputs": [], + "source": [ + "# Use Chain to select subgraphs (nodes + edges) by edge attributes\n", + "collections = [\n", + " collection_set(\n", + " expr=Chain([n(), e_forward({\"vulnName\": \"MS08067 (NetAPI)\"}), n()]),\n", + " id='netapi',\n", + " name='MS08067 (NetAPI)',\n", + " node_color='#00BFFF',\n", + " edge_color='#00BFFF',\n", + " ),\n", + " collection_set(\n", + " expr=Chain([n(), e_forward({\"victimPort\": 445.0}), n()]),\n", + " id='port445',\n", + " name='Port 445',\n", + " node_color='#32CD32',\n", + " edge_color='#32CD32',\n", + " ),\n", + " collection_intersection(\n", + " sets=['netapi', 'port445'],\n", + " name='NetAPI + 445',\n", + " node_color='#AABBCC',\n", + " edge_color='#AABBCC',\n", + " ),\n", + "]\n", + "\n", + "g2 = g.collections(\n", + " collections=collections,\n", + " show_collections=True,\n", + " collections_global_node_color='CCCCCC',\n", + " collections_global_edge_color='CCCCCC',\n", + ")\n", + "\n", + "g2._url_params\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "execution_count": null, + "outputs": [], + "source": [ + "# Render (requires graphistry.register(...))\n", + "g2.plot()\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Notes and validation\n", + "\n", + "- Order matters: earlier collections override later ones.\n", + "- Use collections for priority-based subsets and overlaps; use `encode_*` for simple column-driven colors.\n", + "- Helper constructors: `graphistry.collection_set(...)` and `graphistry.collection_intersection(...)` return JSON-friendly dicts (AST inputs wrap to `gfql_chain`).\n", + "- Provide `id` for sets used by intersections.\n", + "- Global colors apply to nodes/edges not in any collection; `#` is optional.\n", + "- Use `validate='strict'` to raise, or `warn=False` to silence warnings.\n", + "\n", + "Wire protocol and pre-encoded strings:\n", + "\n", + "```python\n", + "collections_wire = [\n", + " {\n", + " \"type\": \"set\",\n", + " \"name\": \"Wire Protocol Example\",\n", + " \"node_color\": \"#AA00AA\",\n", + " \"expr\": {\n", + " \"type\": \"gfql_chain\",\n", + " \"gfql\": [\n", + " {\"type\": \"Node\", \"filter_dict\": {\"status\": \"purchased\"}}\n", + " ]\n", + " }\n", + " }\n", + "]\n", + "g.collections(collections=collections_wire)\n", + "\n", + "g.collections(collections=encoded_collections, encode=False)\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Run `g2.plot()` in a notebook session with valid credentials to render inline.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Overlap priority example\n", + "\n", + "Earlier collections override later ones when they overlap.\n", + "\n", + "```python\n", + "collections_priority = [\n", + " collection_set(\n", + " expr=Chain([n(), e_forward({\"vulnName\": \"MS08067 (NetAPI)\"}), n()]),\n", + " id=\"netapi\",\n", + " name=\"MS08067 (NetAPI)\",\n", + " node_color=\"#FFAA00\",\n", + " edge_color=\"#FFAA00\",\n", + " ),\n", + " collection_set(\n", + " expr=Chain([n(), e_forward({\"victimPort\": 445.0}), n()]),\n", + " id=\"port445\",\n", + " name=\"Port 445\",\n", + " node_color=\"#00BFFF\",\n", + " edge_color=\"#00BFFF\",\n", + " ),\n", + "]\n", + "g.collections(collections=collections_priority)\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For more on color encodings, see the [Color encodings notebook](encodings-colors.ipynb).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.x" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/10min.rst b/docs/source/10min.rst index 8910be19f8..d8cfd79e14 100644 --- a/docs/source/10min.rst +++ b/docs/source/10min.rst @@ -161,6 +161,28 @@ Example visualization: Now, edges are colored based on the type of vulnerability, helping you distinguish different attack types. +Advanced: Collections for layered highlights +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use collections when you want GFQL-driven subsets (nodes, edges, or subgraphs) to override base encodings. +This is useful for overlays like alerts or critical paths that take precedence over your normal color rules. + +.. code-block:: python + + from graphistry import collection_set, n + + collections = [ + collection_set( + expr=n({"vip": True}), + name="VIP", + node_color="#FF8800", + ) + ] + g.collections(collections=collections, show_collections=True).plot() + +See the :doc:`Collections tutorial notebook ` and +:doc:`GFQL docs ` for full details. + Adjusting Sizes, Labels, Icons, Badges, and More ------------------------------------------------ diff --git a/docs/source/cheatsheet.md b/docs/source/cheatsheet.md index bbe887b382..67df59371c 100644 --- a/docs/source/cheatsheet.md +++ b/docs/source/cheatsheet.md @@ -560,6 +560,11 @@ g.encode_point_color('type', as_categorical=True, categorical_mapping={"cat": "red", "sheep": "blue"}, default_mapping='#CCC') ``` +For subset-based coloring and conditional styling across multiple encodings, use Collections +via `g.collections(...)` with GFQL AST helpers. See +{doc}`Layout settings ` +and the [Collections tutorial notebook](demos/more_examples/graphistry_features/collections.ipynb). + For more in-depth examples, check out the tutorials on [colors](https://github.com/graphistry/pygraphistry/tree/master/demos/more_examples/graphistry_features/encodings-colors.ipynb). ### Custom icons and badges diff --git a/docs/source/gfql/quick.rst b/docs/source/gfql/quick.rst index dcaa3fcdc5..e98fd2d326 100644 --- a/docs/source/gfql/quick.rst +++ b/docs/source/gfql/quick.rst @@ -637,6 +637,9 @@ Run graph algorithms like PageRank, community detection, and layouts directly wi # Results have x, y coordinates for visualization result.plot() +Tip: For subset-based coloring after GFQL, use ``result.collections(...)`` and see +:doc:`/visualization/layout/settings`. + Remote Graph References ----------------------- diff --git a/docs/source/gfql/remote.rst b/docs/source/gfql/remote.rst index cf5fd36f80..5b24d08e8c 100644 --- a/docs/source/gfql/remote.rst +++ b/docs/source/gfql/remote.rst @@ -19,6 +19,13 @@ Run chain remotely and fetch results ``gfql_remote()`` accepts the same input types as local ``gfql()``: +.. note:: + Collections are visualization URL settings; apply them after GFQL results + (for example, ``g2.collections(...)``). The GFQL remote/upload APIs do not + accept collections payloads yet. + +Method :meth:`chain_remote ` runs chain remotely and fetches the computed graph + - **Chain / List[ASTObject]**: Native GFQL chain syntax (as above). - **Cypher string**: Compiled locally, sent as wire-protocol JSON. - **ASTLet / Let dict**: DAG patterns with named bindings. diff --git a/docs/source/gfql/spec/wire_protocol.md b/docs/source/gfql/spec/wire_protocol.md index 9e1ca858d7..1612a90394 100644 --- a/docs/source/gfql/spec/wire_protocol.md +++ b/docs/source/gfql/spec/wire_protocol.md @@ -595,6 +595,50 @@ Example: **Note**: The `timezone` field is optional for DateTime values and defaults to "UTC" if omitted. This ensures consistent behavior across systems while allowing explicit timezone specification when needed. +## Collections Payloads + +Collections are Graphistry visualization overlays that use GFQL wire protocol operations to define subsets +of nodes, edges, or subgraphs. They are applied in priority order, with earlier collections overriding later +ones for styling. + +### Collection Set + +Collection sets wrap GFQL operations in a `gfql_chain` object: + +```json +{ + "type": "set", + "id": "purchasers", + "name": "Purchasers", + "node_color": "#00BFFF", + "expr": { + "type": "gfql_chain", + "gfql": [ + {"type": "Node", "filter_dict": {"status": "purchased"}} + ] + } +} +``` + +### Collection Intersection + +Intersections reference previously defined set IDs: + +```json +{ + "type": "intersection", + "name": "High Value Purchasers", + "node_color": "#AA00AA", + "expr": { + "type": "intersection", + "sets": ["purchasers", "vip"] + } +} +``` + +For Python examples and helper constructors, see the +:doc:`Collections tutorial notebook `. + ## Examples ### `MATCH ... RETURN` Row Pipeline diff --git a/docs/source/notebooks/visualization.rst b/docs/source/notebooks/visualization.rst index 2774803be3..4456c8026a 100644 --- a/docs/source/notebooks/visualization.rst +++ b/docs/source/notebooks/visualization.rst @@ -13,6 +13,7 @@ Encodings Sizes <../demos/more_examples/graphistry_features/encodings-sizes.ipynb> Icons <../demos/more_examples/graphistry_features/encodings-icons.ipynb> Badges <../demos/more_examples/graphistry_features/encodings-badges.ipynb> + Collections <../demos/more_examples/graphistry_features/collections.ipynb> Geographic (Kepler.gl) ---------------------- diff --git a/docs/source/visualization/10min.rst b/docs/source/visualization/10min.rst index 60e7f10c33..9e35b5502e 100644 --- a/docs/source/visualization/10min.rst +++ b/docs/source/visualization/10min.rst @@ -224,6 +224,25 @@ You can encode your graph attributes visually using colors, sizes, icons, and mo - :meth:`graphistry.PlotterBase.PlotterBase.encode_edge_color` - :meth:`graphistry.PlotterBase.PlotterBase.encode_edge_icon` +* **Collections (advanced coloring)**: Define subsets using GFQL AST helpers and color them consistently: + + .. code-block:: python + + from graphistry import collection_set, n + + collections = [ + collection_set( + expr=n({"subscribed_to_newsletter": True}), + name="Subscribers", + node_color="#32CD32", + ) + ] + g.collections(collections=collections, show_collections=True).plot() + + See :doc:`Layout settings ` and the + :doc:`Collections tutorial notebook `. + Tip: order matters (earlier collections override later ones) and intersections require set IDs. + * **Bind**: Simpler data-driven settings are done through :meth:`graphistry.PlotterBase.PlotterBase.bind`: .. code-block:: python diff --git a/docs/source/visualization/index.rst b/docs/source/visualization/index.rst index 5aaba1745e..cf85e2b7a2 100644 --- a/docs/source/visualization/index.rst +++ b/docs/source/visualization/index.rst @@ -2,6 +2,8 @@ Visualize ============= We recommend getting started with :ref:`10 Minutes to PyGraphistry <10min-pygraphistry>`, :ref:`10 Minutes to Graphistry Visualization<10min-viz>`, and the :ref:`layout guide `. +For advanced, subset-based coloring, see :ref:`Layout settings ` and the +:doc:`Collections tutorial notebook `. For static image export (documentation, reports), see the `static rendering tutorial <../demos/demos_databases_apis/graphviz/static_rendering.ipynb>`_. diff --git a/docs/source/visualization/layout/settings.rst b/docs/source/visualization/layout/settings.rst index 8014e73fa8..4abfb71eb7 100644 --- a/docs/source/visualization/layout/settings.rst +++ b/docs/source/visualization/layout/settings.rst @@ -36,6 +36,44 @@ Use :meth:`graphistry.PlotterBase.PlotterBase.scene_settings` to modify the appe - ``edge_opacity``: Range 0.0 to 1.0 (0.0 fully transparent, 1.0 fully opaque, displayed as 0-100 in UI) - ``point_opacity``: Range 0.0 to 1.0 (0.0 fully transparent, 1.0 fully opaque, displayed as 0-100 in UI) +Encodings (Color, Size, Icons) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use the ``encode_*`` methods to style nodes and edges based on columns (for example, color by entity type). +See the :doc:`Color encodings notebook ` for full examples. + +Collections +~~~~~~~~~~~ + +Collections define labeled subsets (nodes, edges, or subgraphs) using full GFQL and apply layered styling +that overrides base encodings. Use them to call out alerts or critical paths on top of your standard color +encodings, with priority-based overrides when subsets overlap. + +For a full walkthrough, see the :doc:`Collections tutorial notebook `. +For GFQL syntax, see :doc:`GFQL documentation `. +For schema details, see `Collections URL options `_. + +.. code-block:: python + + from graphistry import collection_set, n + + collections = [ + collection_set( + expr=n({"subscribed_to_newsletter": True}), + id="newsletter_subscribers", + name="Newsletter Subscribers", + node_color="#32CD32", + ) + ] + + g2 = g.collections( + collections=collections, + show_collections=True, + collections_global_node_color="CCCCCC", + collections_global_edge_color="CCCCCC", + ) + g2.plot() + Styling the Background and Foreground ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~