diff --git a/.gitignore b/.gitignore index 081226101..c6739ae5a 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,7 @@ pnpm-lock.yaml # misc .DS_Store .vercel + +.claude/ +CLAUDE.md +.release-tracking/ diff --git a/pages/advanced-algorithms/available-algorithms.mdx b/pages/advanced-algorithms/available-algorithms.mdx index c1ccf8cd2..bf5f9648b 100644 --- a/pages/advanced-algorithms/available-algorithms.mdx +++ b/pages/advanced-algorithms/available-algorithms.mdx @@ -74,6 +74,7 @@ If you want to know more and learn how this affects you, read our [announcement] | Algorithms | Lang | Description | |---------------------------------------------------------------------------------------------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------| +| [gnn](/advanced-algorithms/available-algorithms/gnn) | Python | Export and import graph data in PyTorch Geometric (PyG) and TensorFlow GNN (TF-GNN) formats for GNN training pipelines. | | [link_prediction with GNN](/advanced-algorithms/available-algorithms/gnn_link_prediction) | Python | Module for predicting links in the graph by using graph neural networks. | | [node_classification with GNN](/advanced-algorithms/available-algorithms/gnn_node_classification) | Python | Graph neural network-based node classification module | | [node2vec](/advanced-algorithms/available-algorithms/node2vec) | Python | An algorithm for calculating node embeddings on static graph. | diff --git a/pages/advanced-algorithms/available-algorithms/_meta.ts b/pages/advanced-algorithms/available-algorithms/_meta.ts index 6d885d0b6..6795c2595 100644 --- a/pages/advanced-algorithms/available-algorithms/_meta.ts +++ b/pages/advanced-algorithms/available-algorithms/_meta.ts @@ -21,6 +21,7 @@ export default { "elasticsearch_synchronization": "elasticsearch_synchronization", "embeddings": "embeddings", "export_util": "export_util", + "gnn": "gnn", "gnn_link_prediction": "gnn_link_prediction", "gnn_node_classification": "gnn_node_classification", "graph_analyzer": "graph_analyzer", diff --git a/pages/advanced-algorithms/available-algorithms/embeddings.mdx b/pages/advanced-algorithms/available-algorithms/embeddings.mdx index feb1be677..8cf2ba301 100644 --- a/pages/advanced-algorithms/available-algorithms/embeddings.mdx +++ b/pages/advanced-algorithms/available-algorithms/embeddings.mdx @@ -1,6 +1,6 @@ --- title: embeddings -description: Calculate sentence embeddings on node strings using pytorch. +description: Calculate sentence embeddings on node strings using a local SentenceTransformer model or any remote embedding provider (OpenAI, Ollama, Cohere, Voyage, Mistral, Jina, Bedrock, ...). --- # embeddings @@ -9,7 +9,19 @@ import { Cards } from 'nextra/components' import GitHub from '/components/icons/GitHub' import { Callout } from 'nextra/components' -The embeddings module provides tools for calculating sentence embeddings on node strings using pytorch. +The embeddings module computes sentence embeddings for text — either for the string +properties of nodes in the graph, or for an ad‑hoc list of strings. Two backends +are supported: + +- **Local** (default): a `SentenceTransformer` model from Hugging Face, running on CPU or CUDA inside the Memgraph process. +- **Remote**: any embedding provider supported by [LiteLLM](https://docs.litellm.ai/docs/providers) — e.g. OpenAI, Ollama, Voyage, Mistral, Bedrock, etc. (and any OpenAI‑compatible endpoint). + +Routing is decided by the `model_name` configuration key: a LiteLLM‑style +provider prefix (for example `openai/text-embedding-3-small`, +`ollama/nomic-embed-text`) is executed remotely; anything else (a bare name +like `all-MiniLM-L6-v2` or an HF path like `BAAI/bge-small-en-v1.5`) is loaded +locally. See [Remote providers](#remote-providers) for the full list and +credentials. The `device` parameter can be one of the following: @@ -69,10 +94,10 @@ documentation](/advanced-algorithms/install-mage). {

Output:

} -- `success: bool` ➡ Whether the embeddings computation was successful. +- `success: bool` ➡ Whether the embeddings computation was successful. `false` on any unrecoverable error (network, auth, invalid input); the procedure never throws. - `embeddings: List[List[float]]|NULL` ➡ The list of embeddings. Only returned if the `return_embeddings` parameter is set to `true` in the configuration, otherwise `NULL`. -- `dimension: int` ➡ The dimension of the embeddings. +- `dimension: int|NULL` ➡ The dimension of the embeddings. `NULL` when `success` is `false`, or for empty input on the remote path. {

Usage:

} @@ -126,17 +151,30 @@ This procedure can be used to return a list of embeddings when given a list of s | Name | Type | Default | Description | |----------------------------|--------------|-------------------|----------------------------------------------------------------------------------------------------------| -| `model_name` | string | `"all-MiniLM-L6-v2"` | The name of the model to use for the embeddings computation, provided by the `sentence-transformers` library. | -| `batch_size` | int | `2000` | The batch size to use for the embeddings computation. | -| `chunk_size` | int | `48` | The number of batches per "chunk". This is used when computing embeddings across multiple GPUs, as this has to be done by spawning multiple processes. Each spawned process computes the embeddings for a single chunk. | -| `device` | NULL\|string\| int\|List[string\|int] | `NULL` | The device to use for the embeddings computation. | +| `model_name` | string | `"all-MiniLM-L6-v2"` | Model to use. A bare name or HF path (e.g. `"BAAI/bge-small-en-v1.5"`) is loaded locally via `sentence-transformers`. A LiteLLM provider prefix (e.g. `"openai/text-embedding-3-small"`, `"ollama/nomic-embed-text"`) routes the call to that remote provider — see [Remote providers](#remote-providers). | +| `batch_size` | int | `2000` | The batch size to use for the embeddings computation (local path only). | +| `chunk_size` | int | `48` | Number of batches per "chunk" when spawning per-GPU worker processes (local multi-GPU path only). | +| `device` | NULL\|string\| int\|List[string\|int] | `NULL` | The device to use for the local embeddings computation. Ignored on the remote path. | + +The following keys only apply when `model_name` routes to a remote provider: + +| Name | Type | Default | Description | +|----------------------------|--------------|----------------|-------------------------------------------------------------------------------------------------------------------------------------| +| `api_base` | string\|NULL | `NULL` | Override the provider's HTTP endpoint. Needed when pointing at a self‑hosted Ollama on a non‑default host or an enterprise proxy. For most providers, LiteLLM already knows the default. | +| `input_type` | string | `"document"` | Forwarded to providers that distinguish document vs query embeddings (Voyage, Cohere). Use `"query"` when embedding retrieval queries. | +| `dimensions` | int\|NULL | `NULL` | Target output dimension for providers that support server‑side truncation (e.g. OpenAI `text-embedding-3-*`). | +| `timeout` | int | `60` | Per‑HTTP‑call timeout in seconds. | +| `num_retries` | int | `3` | Automatic retries on transient failures (429, 5xx, connection errors), handled by LiteLLM with backoff. | +| `normalize` | bool | `True` | L2‑normalize returned vectors client‑side so behavior matches the local path (which normalizes by default). | +| `remote_batch_size` | int\|NULL | `NULL` | Items per HTTP request. If `NULL`, the default matches each provider's documented hard cap — **2048** for OpenAI/Azure, **1000** for Voyage, **96** for Cohere. Other providers get **256**. Override with `remote_batch_size` for throughput tuning. | +| `concurrency` | int | `4` | Number of in‑flight HTTP requests per call. Higher values improve single‑query latency for large inputs but multiply against concurrent Cypher queries — be mindful of provider rate limits. | {

Output:

} -- `success: bool` ➡ Whether the embeddings computation was successful. -- `embeddings: List[List[float]]` ➡ The list of embeddings. -- `dimension: int` ➡ The dimension of the embeddings. +- `success: bool` ➡ Whether the embeddings computation was successful. `false` on any unrecoverable error; the procedure never throws. +- `embeddings: List[List[float]]|NULL` ➡ The list of embeddings, or `NULL` on failure. +- `dimension: int|NULL` ➡ The dimension of the embeddings. `NULL` when `success` is `false`. {

Usage:

} @@ -158,13 +196,189 @@ The key `model_name` is used to specify the name of the model to use for the emb {

Output:

} -- `model_info: mgp.Map` ➡ The information about the model used for the embeddings computation. +- `info: mgp.Map` ➡ The information about the model used for the embeddings computation. Contents: + +| Name | Type | Description | +|----------------------------|--------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `model_name` | string | The model name that was supplied (e.g. `"all-MiniLM-L6-v2"`, `"openai/text-embedding-3-small"`). | +| `dimension` | int | The dimension of the embeddings. On the remote path this is discovered by making a single probe call to the provider (cached per `(model_name, api_base)` for the process). | +| `max_sequence_length` | int\|NULL | The maximum input sequence length for the local SentenceTransformer model. `NULL` on the remote path — providers don't expose this uniformly. | + +## Remote providers + +To use a remote embedding provider, set `model_name` to a LiteLLM +provider‑prefixed name. The procedure routes the call through +[LiteLLM](https://litellm.ai/); any bare name or HF path +continues to load locally via `sentence-transformers` as before. A few +working examples: + +- `"openai/text-embedding-3-small"` — OpenAI, 1536 dims (or set `dimensions` for server‑side truncation) +- `"openai/text-embedding-3-large"` — OpenAI, 3072 dims +- `"azure/"` — Azure OpenAI +- `"ollama/nomic-embed-text"` — self‑hosted Ollama +- `"cohere/embed-english-v3.0"` — Cohere (pass `input_type: "query"` for queries) +- `"voyage/voyage-3"` — Voyage AI (pass `input_type: "query"` for queries) +- `"mistral/mistral-embed"` — Mistral +- `"jina_ai/jina-embeddings-v3"` — Jina +- `"bedrock/amazon.titan-embed-text-v2:0"` — AWS Bedrock + +For the authoritative list of supported providers and model identifiers see +the [LiteLLM providers index](https://docs.litellm.ai/docs/providers). + +### Credentials + +API keys are **not** accepted through Cypher — they are read from the Memgraph +process environment by LiteLLM, using the canonical variable name for each +provider. Set the relevant variable in the environment where Memgraph is +running (container `-e` flag, systemd unit, shell export, etc.): + +| Provider prefix | Env vars LiteLLM reads | +|-----------------|-------------------------------------------------------------------------------| +| `openai/` | `OPENAI_API_KEY` (and `OPENAI_API_BASE` if set) | +| `azure/` | `AZURE_API_KEY`, `AZURE_API_BASE`, `AZURE_API_VERSION` | +| `cohere/` | `COHERE_API_KEY` | +| `voyage/` | `VOYAGE_API_KEY` | +| `jina_ai/` | `JINA_AI_API_KEY` | +| `mistral/` | `MISTRAL_API_KEY` | +| `ollama/` | No key; `api_base` defaults to `http://localhost:11434` | +| `bedrock/` | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION_NAME` | +| `vertex_ai/` | `GOOGLE_APPLICATION_CREDENTIALS` (path to a service‑account JSON) | +| `huggingface/` | `HUGGINGFACE_API_KEY` | +| `together_ai/` | `TOGETHERAI_API_KEY` | +| `fireworks_ai/` | `FIREWORKS_AI_API_KEY` | +| `replicate/` | `REPLICATE_API_KEY` | +| `deepinfra/` | `DEEPINFRA_API_KEY` | +| `groq/` | `GROQ_API_KEY` | + +If the required variable is missing, the call returns `success: false` and +the reason is logged by Memgraph (look for lines starting with +`Remote path failed:`). + +### Running the container with API access + +When Memgraph runs in Docker, provider API keys must be passed through to +**the container's environment**, not just exported in your host shell. For +example, with OpenAI: + +```bash +docker run -p 7687:7687 \ + -e OPENAI_API_KEY="$OPENAI_API_KEY" \ + memgraph/memgraph-mage +``` + +You can verify the variable made it across with +`docker inspect --format '{{.Config.Env}}' `. + +The same pattern applies to every other provider — replace `OPENAI_API_KEY` +with the variable(s) from the table above (e.g. `-e COHERE_API_KEY=...`, +`-e AWS_ACCESS_KEY_ID=... -e AWS_SECRET_ACCESS_KEY=...`). + +#### Reaching a local Ollama from Memgraph + +Ollama is a special case because the daemon runs outside Memgraph. Three common setups: + +1. **Memgraph runs natively (not in Docker)** — the default `api_base` + (`http://localhost:11434`) works out of the box. +2. **Memgraph in Docker, Ollama on the host** — bind Ollama to all interfaces + and point `api_base` at the host gateway: + ```bash + # on the host + OLLAMA_HOST=0.0.0.0:11434 ollama serve + # Linux only: add the host-gateway alias on docker run + docker run -p 7687:7687 \ + --add-host=host.docker.internal:host-gateway \ + memgraph/memgraph-mage + ``` + Then in Cypher: `api_base: "http://host.docker.internal:11434"`. +3. **Both in Docker on a shared network** (most portable): + ```bash + docker network create mg_net + docker run -d --name ollama --network mg_net -v ollama:/root/.ollama ollama/ollama + docker exec ollama ollama pull nomic-embed-text + docker run -d --name memgraph --network mg_net -p 7687:7687 memgraph/memgraph-mage + ``` + Then in Cypher: `api_base: "http://ollama:11434"`. -| Name | Type | Default | Description | -|----------------------------|--------------|-------------------|----------------------------------------------------------------------------------------------------------| -| `model_name` | string | `"all-MiniLM-L6-v2"` | The name of the model to use for the embeddings computation, provided by the `sentence-transformers` library. | -| `dimension` | int | `384` | The dimension of the embeddings. | -| `max_seq_length` | int | `256` | The maximum sequence length. | + +Credentials live in the Memgraph process environment, not in Cypher. This +keeps API keys out of query logs and audit trails. If you need different +keys per graph or per tenant, run separate Memgraph instances with their +own environment. + + +### Examples + +**OpenAI — embed a list of strings:** + +```cypher +CALL embeddings.text( + ["hello world", "graph databases are fun"], + {model_name: "openai/text-embedding-3-small"} +) +YIELD success, embeddings, dimension +RETURN success, dimension, size(embeddings) AS n; +``` + +**OpenAI — smaller vectors via server‑side truncation:** + +```cypher +CALL embeddings.text( + ["hello"], + {model_name: "openai/text-embedding-3-small", dimensions: 768} +) +YIELD dimension +RETURN dimension; // -> 768 +``` + +**OpenAI — write back embeddings to nodes:** + +```cypher +MATCH (n:Doc) WITH collect(n) AS nodes +CALL embeddings.node_sentence(nodes, {model_name: "openai/text-embedding-3-small"}) +YIELD success, dimension +RETURN success, dimension; +``` + +**Ollama — embed against a local daemon:** + +```cypher +CALL embeddings.text( + ["hello world", "graph databases are fun"], + {model_name: "ollama/nomic-embed-text"} +) +YIELD success, embeddings, dimension +RETURN success, dimension, size(embeddings) AS n; +``` + +If Memgraph is in a container and Ollama is on the host, pass the reachable URL: + +```cypher +CALL embeddings.text( + ["hello world"], + {model_name: "ollama/nomic-embed-text", + api_base: "http://host.docker.internal:11434"} +) +YIELD success, dimension RETURN success, dimension; +``` + +**Cohere / Voyage — embedding queries rather than documents:** + +```cypher +CALL embeddings.text( + ["what is a graph database?"], + {model_name: "voyage/voyage-3", input_type: "query"} +) +YIELD success, embeddings, dimension +RETURN success, dimension; +``` + +**Inspect the active remote model:** + +```cypher +CALL embeddings.model_info({model_name: "openai/text-embedding-3-small"}) +YIELD info RETURN info; +// -> {model_name: "openai/text-embedding-3-small", dimension: 1536, max_sequence_length: null} +``` ## Example diff --git a/pages/advanced-algorithms/available-algorithms/gnn.mdx b/pages/advanced-algorithms/available-algorithms/gnn.mdx new file mode 100644 index 000000000..e1341f698 --- /dev/null +++ b/pages/advanced-algorithms/available-algorithms/gnn.mdx @@ -0,0 +1,285 @@ +--- +title: gnn +description: Export and import graph data in PyTorch Geometric (PyG) and TensorFlow GNN (TF-GNN) formats. Enables GNN training pipelines by converting Memgraph graphs to framework-native JSON representations and writing back inference results. +--- + +import { Callout } from 'nextra/components' +import { Cards } from 'nextra/components' +import GitHub from '/components/icons/GitHub' + +# gnn + +GNN integration module for Memgraph. Provides export/import procedures for +**[PyTorch Geometric (PyG)](https://pytorch-geometric.readthedocs.io/en/latest/)** and **[TensorFlow GNN (TF-GNN)](https://github.com/tensorflow/gnn)** formats. All +exports produce a single JSON string that can be deserialized on the client side +and fed into the respective framework. + +Typical workflow: + +1. **Export** – extract the graph (or a subgraph) from Memgraph into a + JSON representation that PyG or TF-GNN can consume directly. + +2. **Train / Infer** – use the exported data in your GNN pipeline outside + Memgraph. + +3. **Import** – write new nodes and relationships back into Memgraph from + the framework's output, or update existing nodes with inference results. + + + } + title="Source code" + href="https://github.com/memgraph/memgraph/blob/master/mage/python/gnn.py" + /> + + +| Trait | Value | +| ------------------- | ---------- | +| **Module type** | module | +| **Implementation** | Python | +| **Parallelism** | sequential | + +## Procedures + +### `pyg_export()` + +Exports the current graph to a JSON string in **PyTorch Geometric** format. + +The JSON payload contains: +- `edge_index` – source and destination index arrays. +- `x` – node feature matrix (when `node_property_names` is provided). +- `edge_attr` – edge feature matrix (when `edge_property_names` is provided). +- `y` – node labels (when `node_label_property` is provided). +- `num_nodes` – total number of nodes. +- `node_id_mapping` / `idx_to_node_id` – bidirectional mapping between + Memgraph internal IDs and PyG indices (used for write-back). +- `labels` – original node labels. +- `edge_types` – original relationship types. + +{

Input:

} + +- `node_property_names: List[string] (default = null)` ➡ Node properties to + include in the feature matrix `x`. Numeric properties are cast to floats; + list properties are flattened. +- `edge_property_names: List[string] (default = null)` ➡ Edge properties to + include in `edge_attr`. +- `node_label_property: string (default = null)` ➡ Node property to use as + the target label vector `y`. + +{

Output:

} + +- `json_data: string` ➡ A JSON string representing the graph in PyG format. + +{

Usage:

} + +Export features `feat` and edge attribute `weight`, with `class` as the +target label: + +```cypher +CALL gnn.pyg_export(["feat"], ["weight"], "class") +YIELD json_data +RETURN json_data; +``` + +Export with no features (topology only): + +```cypher +CALL gnn.pyg_export() +YIELD json_data +RETURN json_data; +``` + +--- + +### `pyg_import()` + +Imports data from a PyG JSON string into Memgraph. Supports two modes: + +- **Create mode** (default) – creates new nodes and relationships. +- **Update mode** (`update_existing = true`) – uses the `idx_to_node_id` + mapping in the JSON payload to find existing Memgraph nodes and sets + properties on them. This is the typical *export → inference → write-back* + workflow. + +{

Input:

} + +- `json_data: string` ➡ JSON string previously produced by `pyg_export()` (or + any compatible PyG-format JSON). +- `default_node_label: string (default = "PyGNode")` ➡ Label assigned to + created nodes when no label information is present in the JSON. +- `default_edge_type: string (default = "CONNECTS")` ➡ Relationship type + assigned to created relationships when no type information is present. +- `node_property_names: List[string] (default = null)` ➡ Names to assign + to individual feature columns when importing the feature matrix `x`. +- `edge_property_names: List[string] (default = null)` ➡ Names to assign + to individual edge-attribute columns when importing `edge_attr`. +- `update_existing: boolean (default = false)` ➡ When `true`, existing nodes + are updated instead of creating new ones. + +{

Output:

} + +- `nodes_created: integer` ➡ Number of nodes created (0 in update mode). +- `edges_created: integer` ➡ Number of relationships created (0 in update mode). +- `nodes_updated: integer` ➡ Number of existing nodes updated (0 in create mode). + +{

Usage:

} + +**Roundtrip example** – export from the graph and import as new nodes: + +```cypher +CALL gnn.pyg_export(["feat"], ["weight"], "class") +YIELD json_data +WITH json_data +CALL gnn.pyg_import(json_data, "Imported", "IMP", ["feat"], ["weight"]) +YIELD nodes_created, edges_created +RETURN nodes_created, edges_created; +``` + +**Write-back example** – update existing nodes with predictions after +inference: + +```cypher +CALL gnn.pyg_import($json_data, "Node", "EDGE", ["prediction"], null, true) +YIELD nodes_updated +RETURN nodes_updated; +``` + + +In update mode the procedure uses the `idx_to_node_id` mapping inside the JSON +payload to look up existing vertices by their Memgraph internal ID. Make sure +the JSON was originally exported from the same database. + + +--- + +### `tf_export()` + +Exports the current graph to a JSON string in **TF-GNN** format. + +The JSON payload contains: +- `schema` – describes node sets, edge sets and their feature schemas + (dtypes and shapes), matching the TF-GNN `GraphSchema` structure. +- `graph` – the actual graph data with feature values, sizes and adjacency + information. + +{

Input:

} + +- `node_property_names: List[string] (default = null)` ➡ Node properties to + include as node-set features. +- `edge_property_names: List[string] (default = null)` ➡ Edge properties to + include as edge-set features. +- `node_set_name: string (default = "node")` ➡ Name of the node set in the + TF-GNN schema. +- `edge_set_name: string (default = "edge")` ➡ Name of the edge set in the + TF-GNN schema. + +{

Output:

} + +- `json_data: string` ➡ A JSON string representing the graph in TF-GNN format. + +{

Usage:

} + +Export with node property `score` and edge property `weight`: + +```cypher +CALL gnn.tf_export(["score"], ["weight"]) +YIELD json_data +RETURN json_data; +``` + +Specify custom set names: + +```cypher +CALL gnn.tf_export(["score"], ["weight"], "items", "similarities") +YIELD json_data +RETURN json_data; +``` + +--- + +### `tf_import()` + +Imports data from a TF-GNN JSON string into Memgraph, creating new nodes +and relationships. + +{

Input:

} + +- `json_data: string` ➡ JSON string previously produced by `tf_export()` (or + any compatible TF-GNN-format JSON). +- `default_node_label: string (default = "TfGnnNode")` ➡ Label assigned to + created nodes when no label information is present. +- `default_edge_type: string (default = "CONNECTS")` ➡ Relationship type + assigned to created relationships when no type information is present. + +{

Output:

} + +- `nodes_created: integer` ➡ Number of nodes created. +- `edges_created: integer` ➡ Number of relationships created. + +{

Usage:

} + +**Roundtrip example** – export and re-import: + +```cypher +CALL gnn.tf_export(["score"], ["weight"]) +YIELD json_data +WITH json_data +CALL gnn.tf_import(json_data, "TfNode", "TF_EDGE") +YIELD nodes_created, edges_created +RETURN nodes_created, edges_created; +``` + +--- + +## Example + +The following end-to-end example shows how to move graph data through a PyG +training pipeline. + +**1. Create sample data:** + +```cypher +CREATE (a:Person {feat: [1.0, 2.0], age: 30, class: 0}) + -[:KNOWS {weight: 0.5}]-> + (b:Person {feat: [3.0, 4.0], age: 25, class: 1}) + -[:KNOWS {weight: 0.8}]-> + (c:Person {feat: [5.0, 6.0], age: 35, class: 0}); +``` + +**2. Export to PyG format:** + +```cypher +CALL gnn.pyg_export(["feat"], ["weight"], "class") +YIELD json_data +RETURN json_data; +``` + +**3. Use the JSON payload in Python (client-side):** + +```python +import json +import torch +from torch_geometric.data import Data + +# result is the json_data string returned by Memgraph +pyg_dict = json.loads(result) + +data = Data( + x=torch.tensor(pyg_dict["x"], dtype=torch.float), + edge_index=torch.tensor(pyg_dict["edge_index"], dtype=torch.long), + edge_attr=torch.tensor(pyg_dict["edge_attr"], dtype=torch.float), + y=torch.tensor(pyg_dict["y"], dtype=torch.long), +) +# Train your model ... +``` + +**4. Write predictions back to Memgraph:** + +After inference, update your JSON payload with the predictions and call +`pyg_import` with `update_existing` set to `true`: + +```cypher +CALL gnn.pyg_import($updated_json, "Person", "KNOWS", ["prediction"], null, true) +YIELD nodes_updated +RETURN nodes_updated; +``` diff --git a/pages/clustering/high-availability/best-practices.mdx b/pages/clustering/high-availability/best-practices.mdx index 516d5dd4b..0704bedaf 100644 --- a/pages/clustering/high-availability/best-practices.mdx +++ b/pages/clustering/high-availability/best-practices.mdx @@ -25,14 +25,10 @@ optimal availability they should be **physically separated**. ### Coordinator disk space -Ensure enough disk capacity for: - -- `--snapshot-retention-count + 1` snapshots -- Several WAL files - -A new snapshot is created before the oldest one is removed. Insufficient disk -space may cause snapshot creation to fail. This is particularly important in -**Kubernetes**, where volumes often have strict size limits. +Storage snapshots are **automatically disabled** on coordinator instances since +coordinators do not store user data. Coordinators still require disk capacity +for Raft metadata and internal logs. In **Kubernetes**, ensure volumes have +sufficient size limits for these files. ## Command-line flags @@ -113,39 +109,39 @@ coordinator’s RPC messages. **Example:** `--management-port=10000` -#### `--instance-health-check-frequency-sec` +#### `--instance-health-check-frequency-sec` (deprecated) -How often the coordinator pings data instances (default: 1 second). -Changing is usually unnecessary. - -**Example:** -`--instance-health-check-frequency-sec=1` - -#### `--instance-down-timeout-sec` - -How long to wait before marking an instance as down (default: 5 seconds). + +**Deprecated in Memgraph 3.10.** This startup flag is now ignored. Use the +`instance_health_check_frequency_sec` [coordinator runtime setting](#coordinator-runtime-settings) +instead. + -**Example:** -`--instance-down-timeout-sec=5` +#### `--instance-down-timeout-sec` (deprecated) + +**Deprecated in Memgraph 3.10.** This startup flag is now ignored. Use the +`instance_down_timeout_sec` [coordinator runtime setting](#coordinator-runtime-settings) +instead. + ### Health check behavior Coordinator health checks follow this pattern: -- A ping is sent every `--instance-health-check-frequency-sec`. +- A ping is sent every `instance_health_check_frequency_sec` seconds. - An instance is marked **down** only after - `--instance-down-timeout-sec` elapses without a response. + `instance_down_timeout_sec` elapses without a response. Requirements & recommendations: -- `down-timeout >= health-check-frequency` +- `instance_down_timeout_sec >= instance_health_check_frequency_sec` - Prefer using a multiplier: - **down-timeout = N × health-check-frequency**, with **N ≥ 2** + **instance_down_timeout_sec = N × instance_health_check_frequency_sec**, with **N ≥ 2** -**Example:** -`--instance-down-timeout-sec=5` -`--instance-health-check-frequency-sec=1` +**Example (defaults):** +`instance_down_timeout_sec=5` +`instance_health_check_frequency_sec=1` ## Environment variable configuration @@ -211,9 +207,39 @@ the command line argument. ## Coordinator runtime settings -There is a configuration option for specifying whether reads from the main are -enabled. The configuration value is by default false but can be changed in -run-time using the following query: +Coordinator runtime settings are Raft-replicated and can be changed on a live +cluster without downtime. Use `SET COORDINATOR SETTING` to modify a value and +`SHOW COORDINATOR SETTINGS` to inspect all current values. Changes propagate +automatically to every coordinator in the cluster. + +### `instance_health_check_frequency_sec` + +How often the coordinator pings data instances, in seconds. + +``` +SET COORDINATOR SETTING 'instance_health_check_frequency_sec' TO '1' ; +``` + +**Default:** `1` + +### `instance_down_timeout_sec` + +How long to wait (in seconds) before marking an instance as down. Must be +greater than or equal to `instance_health_check_frequency_sec`. + +``` +SET COORDINATOR SETTING 'instance_down_timeout_sec' TO '5' ; +``` + +**Default:** `5` + + +**Upgrade note (3.10):** Values previously set via the +`--instance-down-timeout-sec` and `--instance-health-check-frequency-sec` +startup flags are **not** automatically migrated. After upgrading, the settings +revert to their defaults (`5` and `1`). If you had customized these flags, run +`SET COORDINATOR SETTING` queries to re-apply your values. + ### `enabled_reads_on_main` diff --git a/pages/clustering/high-availability/ha-commands-reference.mdx b/pages/clustering/high-availability/ha-commands-reference.mdx index 728cbd6ae..add4b5672 100644 --- a/pages/clustering/high-availability/ha-commands-reference.mdx +++ b/pages/clustering/high-availability/ha-commands-reference.mdx @@ -145,11 +145,16 @@ REGISTER INSTANCE instanceName ( AS ASYNC | AS STRICT_SYNC ) ? WITH CONFIG { {

Behavior

} +- The operation is first committed to the Raft log and acknowledged by a + majority of coordinators. - Coordinator connects via `management_server` to verify liveness. - Coordinator begins periodic health checks. - Instance is automatically demoted to REPLICA. - Replication server is started on the data instance. -- Operation is persisted in Raft. +- If RPCs to the data instance fail (e.g., due to a transient network issue), + the registration still succeeds. The [reconciliation + loop](/clustering/high-availability/how-high-availability-works#how-the-reconciliation-loop-works) + automatically retries the RPCs. {

Replication mode rules

} @@ -192,7 +197,10 @@ UNREGISTER INSTANCE instanceName; - Do **not** unregister the MAIN instance; this may corrupt cluster state. - A healthy MAIN must exist during the operation. -- The instance is also removed from MAIN’s replica set. +- The instance is removed from the Raft state first. If the RPC to unregister + the replica from MAIN fails, the [reconciliation + loop](/clustering/high-availability/how-high-availability-works#how-the-reconciliation-loop-works) + automatically retries the operation. {

Example

} @@ -213,13 +221,17 @@ SET INSTANCE instanceName TO MAIN; {

Behavior

} +- The promotion is first committed to the Raft log and acknowledged by a + majority of coordinators. - All other registered instances become replicas of the new MAIN. -- Written to Raft log. +- RPCs (`PromoteToMainRpc`, `SwapAndUpdateUUID`) are sent to data instances on + a best-effort basis. If they fail, the [reconciliation + loop](/clustering/high-availability/how-high-availability-works#how-the-reconciliation-loop-works) + automatically retries them. {

Implications

} - Fails if a MAIN already exists. -- Fails if any instance is unavailable. {

Example

} @@ -238,8 +250,14 @@ DEMOTE INSTANCE instanceName; {

Behavior

} +- The role change is first committed to the Raft log and acknowledged by a + majority of coordinators. - MAIN becomes REPLICA. -- Written to Raft log. +- The `DemoteMainToReplicaRpc` is sent on a best-effort basis. If it fails, the + [reconciliation + loop](/clustering/high-availability/how-high-availability-works#how-the-reconciliation-loop-works) + automatically retries it. +- Returns an error if the instance is already a REPLICA. {

Implications

} @@ -312,6 +330,14 @@ SHOW REPLICATION LAG; - Useful during manual failover to evaluate risk of data loss. +## Error handling + +If a Raft log commit fails for any cluster operation (register, unregister, +promote, demote, add coordinator), the error message will indicate: + +> Writing to Raft log failed. Please retry the operation. + + ## Troubleshooting commands ### `FORCE RESET CLUSTER STATE` diff --git a/pages/clustering/high-availability/how-high-availability-works.mdx b/pages/clustering/high-availability/how-high-availability-works.mdx index 0a8cf04b9..682ff4540 100644 --- a/pages/clustering/high-availability/how-high-availability-works.mdx +++ b/pages/clustering/high-availability/how-high-availability-works.mdx @@ -90,6 +90,14 @@ added: - `--management-port` - used to get the **health state** of the data instance from the leader coordinator +When a data instance runs in HA mode (i.e. `--management-port` is set), the +`--init-file` and `--init-data-file` flags are **not supported**. The instance +will fail to start if either flag is provided. This is because the cluster's +leader coordinator drives role transitions (MAIN/REPLICA) and replication +setup, so executing arbitrary initialization queries on startup could conflict +with the cluster state. To bootstrap users or data in an HA cluster, run the +relevant queries through the MAIN after the cluster is formed. + ### Coordinator instance implementation The coordinator is a small orchestration instance which is shipped in the same @@ -106,10 +114,19 @@ role is COORDINATOR, the user needs to specify four flags: The COORDINATOR instance is a **very restricted instance**, and it will not respond to any queries that are not related to management of the cluster. -That means, you can not run any data queries on the coordinator directly (we -will talk more about routing data queries in the next sections). +That means, you cannot run any data queries on the coordinator directly (we +will talk more about routing data queries in the next sections). However, +system information queries such as `SHOW CONFIG`, `SHOW LICENSE INFO`, +`SHOW BUILD INFO` and `SHOW STORAGE INFO` are supported on coordinators, as +well as `SET DATABASE SETTING` and `RELOAD SSL`. +Since coordinators do not store user data, the following restrictions apply: +- **Snapshots are automatically disabled** on coordinators, even if + `--storage-snapshot-interval-sec` is set. +- **The `--init-file` and `--init-data-file` flags are not supported** on + coordinators (and likewise not supported on data instances in HA mode). The + instance will fail to start if either flag is provided. When deploying coordinators to servers, you can use the instance of almost any size. Instances of 4GiB or 8GiB will suffice since coordinators' job mainly @@ -158,7 +175,7 @@ All of the following messages were sent by the leader coordinator. | `DemoteMainToReplicaRpc` | Demote a Main after failover | Sent to the old MAIN in order to demote it to REPLICA. | | `RegisterReplicaOnMainRpc` | Instruct Main to accept replication from a Replica | Sent to the MAIN to register a REPLICA on the MAIN. | | `UnregisterReplicaRpc` | Remove Replica from Main | Sent to the MAIN to unregister a REPLICA from the MAIN. | -| `EnableWritingOnMainRpc` | Re-enable writes after Main restarts | Sent to the MAIN to enable writing on that MAIN. | +| `EnableWritingOnMainRpc` | Re-enable writes after Main restarts (deprecated) | Kept for backward compatibility (ISSU). No longer sent by coordinators — writing is implicitly enabled on promotion. | | `GetDatabaseHistoriesRpc` | Gather committed transaction counts during failover | Sent to all REPLICA instances in order to select a new MAIN during the failover process. | | `StateCheckRpc` | Health check ping (liveness) | Sent to all data instances for a liveness check. | | `SwapMainUUIDRpc` | Ensure Replica tracks the correct Main | Sent to REPLICA instances to set the UUID of the MAIN they should listen to. | @@ -235,7 +252,7 @@ in the cluster to ensure high availability, with timeouts. | `PromoteToMainReq` | Coordinator | Data instance | | | `RegisterReplicaOnMainReq` | Coordinator | Data instance | | | `UnregisterReplicaReq` | Coordinator | Data instance | | -| `EnableWritingOnMainReq` | Coordinator | Data instance | | +| `EnableWritingOnMainReq` | Coordinator | Data instance | deprecated | | `GetDatabaseHistoriesReq` | Coordinator | Data instance | | | `StateCheckReq` | Coordinator | Data instance | 5s | | `SwapMainUUIDReq` | Coordinator | Data instance | | @@ -277,15 +294,17 @@ other instances and starts accepting write queries. ### Instance health checks The coordinator performs health checks on each instance at a fixed interval, -configured with `--instance-health-check-frequency-sec`. An instance is not -considered down until it has failed to respond for the full duration specified -by `--instance-down-timeout-sec`. +configured with the `instance_health_check_frequency_sec` coordinator setting. +An instance is not considered down until it has failed to respond for the full +duration specified by the `instance_down_timeout_sec` coordinator setting. Both +settings can be changed at runtime using +[`SET COORDINATOR SETTING`](/clustering/high-availability/best-practices#coordinator-runtime-settings). **Example** -If you set: -- `--instance-health-check-frequency-sec=1` -- `--instance-down-timeout-sec=5` +With the default settings: +- `instance_health_check_frequency_sec=1` +- `instance_down_timeout_sec=5` …the coordinator will send a health check RPC (`StateCheckRpc`) every second. An instance is marked as down only after **five consecutive missed responses** (5 @@ -309,9 +328,9 @@ If a **REPLICA** fails to respond: 2. **Main instance fails to respond** If the **MAIN** instance fails to respond, two cases apply: -- **Down for less than** `--instance-down-timeout-sec` The instance is still +- **Down for less than** `instance_down_timeout_sec` The instance is still considered alive and will rejoin as MAIN when it responds again. -- **Down for longer than** `--instance-down-timeout-sec` The coordinator +- **Down for longer than** `instance_down_timeout_sec` The coordinator initiates the failover procedure. What the old MAIN becomes afterward depends on the outcome: - **Failover succeeds**: the old MAIN rejoins as a **REPLICA**. @@ -472,6 +491,36 @@ All state-changing operations are disabled on followers, including: These operations are permitted **only on the leader coordinator**. +## Raft-first operations and the reconciliation loop + +The coordinator follows a **Raft-first** pattern for all cluster operations +(registering, unregistering, promoting, demoting instances). This means every +state change is first committed to the Raft log and acknowledged by a majority +of coordinators **before** the operation returns success to the user. + +After the Raft commit, the coordinator sends RPCs to data instances (e.g., +`PromoteToMainRpc`, `DemoteMainToReplicaRpc`, `RegisterReplicaOnMainRpc`, +`UnregisterReplicaRpc`) on a **best-effort** basis. If an RPC fails due to a +transient network issue, the operation still succeeds from the user's +perspective because the Raft log is the single source of truth. + +### How the reconciliation loop works + +The coordinator leader runs a periodic **reconciliation loop** that +automatically detects and corrects discrepancies between the desired state (Raft +log) and the actual state of data instances. Specifically: + +- **Missing replicas on main**: If a replica exists in the Raft state but is not + registered on the current main instance, the reconciliation loop sends a + `RegisterReplicaOnMainRpc` to the main. +- **Stale replicas on main**: If the main instance reports a replica that no + longer exists in the Raft state, the reconciliation loop sends an + `UnregisterReplicaRpc` to remove it. + +This self-healing behavior means the cluster automatically recovers from +transient RPC failures without user intervention. Users only need to retry an +operation if the Raft commit itself fails. + ## Instance restarts ### Restarting data instances @@ -483,9 +532,9 @@ Both MAIN and REPLICA instances may fail and later restart. to follow. This synchronization happens automatically once the coordinator’s health check (“ping”) succeeds. -- When the **MAIN** instance restarts, it is initially prevented from accepting - write operations. Writes become allowed only after the coordinator confirms - the instance’s state and sends an `EnableWritingOnMainRpc` message. +- When the **MAIN** instance restarts, the coordinator confirms the instance’s + state through health checks. Writing is enabled once the + coordinator verifies the instance is healthy and its role is confirmed by sending `PromoteToMainRpc` to the data instance. This ensures that instances safely rejoin the cluster without causing inconsistencies. @@ -637,9 +686,9 @@ Thus, only STRICT_SYNC replicas can directly impact write availability. When the MAIN instance becomes unavailable, the failure is handled by the leader coordinator using two user-configured parameters: -- `--instance-health-check-frequency-sec`: how often health checks are sent -- `--instance-down-timeout-sec`: how long an instance must remain unresponsive - before it is considered down +- `instance_health_check_frequency_sec`: how often health checks are sent (configurable via [`SET COORDINATOR SETTING`](/clustering/high-availability/best-practices#coordinator-runtime-settings)) +- `instance_down_timeout_sec`: how long an instance must remain unresponsive + before it is considered down (configurable via [`SET COORDINATOR SETTING`](/clustering/high-availability/best-practices#coordinator-runtime-settings)) Once the coordinator gathers enough evidence that the MAIN is down, it begins a failover procedure using a small number of RPC messages. The exact time required diff --git a/pages/clustering/high-availability/querying-the-cluster-in-high-availability.mdx b/pages/clustering/high-availability/querying-the-cluster-in-high-availability.mdx index 3bfca7f5f..74eaa3b51 100644 --- a/pages/clustering/high-availability/querying-the-cluster-in-high-availability.mdx +++ b/pages/clustering/high-availability/querying-the-cluster-in-high-availability.mdx @@ -75,6 +75,41 @@ This ensures: - **Transparency:** Clients work seamlessly whether they connect to leaders or followers. +### Routing table TTL and refresh behavior + +Because routing is entirely client-side, the driver caches the routing table and +decides when to refresh it. There are two triggers: + +1. **TTL expiration.** Memgraph returns a TTL alongside the routing table, and + it is **hard-coded to 5 minutes**. Once the TTL expires, the driver fetches + a new routing table from a coordinator on the next request. + +2. **Driver-detected failure.** If the driver notices that the cached routing + table is wrong, it refreshes it before retrying: + + - `execute_read` refreshes the routing table when all read instances from the previous routing table become unavailable. + - `execute_write` refreshes the routing table when the **MAIN goes down**. + - `session.run` does **not** trigger a refresh on failure - it is up to the + application to handle the error and retry. + + + +The driver only refreshes the routing table for failures that match the query +type it dispatched. This means some topology changes stay hidden until the TTL +expires: + +- If the **set of REPLICAs changes** (a REPLICA is added, removed, or swapped) + and the driver is only issuing **write** queries, the routing table is not + refreshed. +- If the **MAIN changes** to an instance that was not part of the previously + cached routing table and the driver is only issuing **read** queries, the + routing table is not refreshed. + +In both cases, the routing table will eventually self-heal after the 5-minute +TTL expires. + + + ### Routing examples **WRITE query** using Bolt+routing - routed to MAIN: diff --git a/pages/clustering/high-availability/setup-ha-cluster-k8s.mdx b/pages/clustering/high-availability/setup-ha-cluster-k8s.mdx index 6c7c04ce4..4b322c6fe 100644 --- a/pages/clustering/high-availability/setup-ha-cluster-k8s.mdx +++ b/pages/clustering/high-availability/setup-ha-cluster-k8s.mdx @@ -254,6 +254,7 @@ from outside the cluster. Our HA supports out of the box following K8s resources - **NodePort** - exposes ports on each node (requires public node IPs). - **LoadBalancer** - one LoadBalancer per instance (highest cost). - **CommonLoadBalancer (coordinators only)** - single LB for all coordinators. +- **Gateway API** - uses Kubernetes Gateway API resources (Gateway + TCPRoute). Configured under `externalAccessConfig.gateway`. For coordinators, there is an additional option of using `CommonLoadBalancer`. In this scenario, there is one load balancer sitting in front of coordinators. @@ -266,12 +267,50 @@ The default Bolt port is opened on 7687 but you can change it by setting `ports. For more detailed IngressNginx setup, see [Use Memgraph HA chart with IngressNginx](#use-memgraph-ha-chart-with-ingressnginx). -Note however that Ingress Nginx is getting retired and one of alternatives is using resources like [TCPRoute/TLSRoute](https://doc.traefik.io/traefik/reference/routing-configuration/kubernetes/gateway-api/#tcp) with K8s -controllers like Envoy Gateway, Istio, Cilium, Traefik, Kong... For the detailed example on how to set-up -Envoy Gateway controller with Memgraph HA cluster, see [Use Memgraph HA chart with Envoy Gateway](#use-memgraph-ha-chart-with-envoy-gateway). +Note however that Ingress Nginx is getting retired and one of the alternatives is using the [Kubernetes Gateway API](https://gateway-api.sigs.k8s.io/) with +controllers like Envoy Gateway, Istio, Cilium, Traefik, or Kong. The HA chart has native Gateway API support — see [Use Memgraph HA chart with Gateway API](#use-memgraph-ha-chart-with-gateway-api). By default, the chart does **not** expose any external network services. +{

Per-instance external access annotations

} + +When using `LoadBalancer` or `NodePort` external access, you can set annotations +globally via `externalAccessConfig.dataInstance.annotations` and +`externalAccessConfig.coordinator.annotations`. These apply to every external +Service of that type. + +If you need different annotations per instance — for example, to assign unique +DNS hostnames via `external-dns` — use the `externalAccessAnnotations` field on +individual entries in `data[]` or `coordinators[]`. Per-instance annotations are +merged with the global annotations, and **per-instance values take precedence** +when the same key appears in both. + +```yaml +externalAccessConfig: + dataInstance: + serviceType: "LoadBalancer" + annotations: + service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing" + +data: + - id: "0" + externalAccessAnnotations: + external-dns.alpha.kubernetes.io/hostname: "data-0.memgraph.example.com" + args: + - "--management-port=10000" + - "--bolt-port=7687" + - id: "1" + externalAccessAnnotations: + external-dns.alpha.kubernetes.io/hostname: "data-1.memgraph.example.com" + args: + - "--management-port=10000" + - "--bolt-port=7687" +``` + +In this example, each data instance's external Service gets the shared +`aws-load-balancer-scheme` annotation plus its own unique `external-dns` +hostname. + ### Node affinity Memgraph HA deploys multiple pods, and you can control pod placement with @@ -372,159 +411,136 @@ Refer to the Memgraph HA [User API docs](/clustering/high-availability#user-api) for the full set of commands and usage patterns. -### Use Memgraph HA chart with Envoy Gateway +### Use Memgraph HA chart with Gateway API -Before configuring routes, a Gateway API controller must be installed. This guide demonstrates using Envoy Gateway. +The Memgraph HA Helm chart has native support for the [Kubernetes Gateway API](https://gateway-api.sigs.k8s.io/). When enabled, the chart automatically creates TCPRoute resources for each data and coordinator instance. You can either let the chart create its own Gateway or attach routes to a pre-existing one. -```bash -helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.2.4 -n envoy-gateway-system --create-namespace -``` -Next, we will create a `GatewayClass`. + +Gateway API is orthogonal to the `serviceType` external access options (IngressNginx, NodePort, LoadBalancer). The routes point at internal ClusterIP services that always exist, so you can use Gateway API alongside or instead of other external access methods. + + +#### Prerequisites + +Before enabling Gateway API in the chart, you need: + +1. **A Gateway API controller** installed in your cluster. Examples include [Envoy Gateway](https://gateway.envoyproxy.io/), [Istio](https://istio.io/), [Cilium](https://cilium.io/), [Traefik](https://traefik.io/), and [Kong](https://konghq.com/). This guide uses Envoy Gateway as an example: + + ```bash + helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.2.4 -n envoy-gateway-system --create-namespace + ``` + +2. **A GatewayClass resource** that references your controller. A GatewayClass is a cluster-scoped resource that defines which controller manages Gateways — each Gateway references a GatewayClass by name. The Helm chart does **not** create a GatewayClass; you must create one yourself or use one provided by your controller installation. For Envoy Gateway: + + ```yaml + apiVersion: gateway.networking.k8s.io/v1 + kind: GatewayClass + metadata: + name: eg + spec: + controllerName: gateway.envoyproxy.io/gatewayclass-controller + ``` + + +You must ensure the GatewayClass exists before enabling the gateway feature in the chart. If you create your own Gateway (Option 1 below), the chart requires `gatewayClassName` to reference an existing GatewayClass, and will fail with an error if it is not set. + + +#### Option 1: Chart-managed Gateway + +When you want the chart to create its own Gateway along with TCPRoute resources, set `externalAccessConfig.gateway.enabled` to `true` and provide the `gatewayClassName`: ```yaml -apiVersion: gateway.networking.k8s.io/v1 -kind: GatewayClass -metadata: - name: eg -spec: - controllerName: gateway.envoyproxy.io/gatewayclass-controller +externalAccessConfig: + gateway: + enabled: true + gatewayClassName: "eg" ``` -For this example we chose to create one `Gateway` for data instances and one for coordinator instances but you can choose -a different approach and use a single `Gateway` for both types of instances. +The chart will create: +- A **Gateway** (`gateway.networking.k8s.io/v1`) with TCP listeners auto-generated for each data and coordinator instance. +- A **TCPRoute** (`gateway.networking.k8s.io/v1alpha2`) per instance, routing traffic from the Gateway listener to the instance's Bolt port. -Data instances' gateway: +Data instance ports are assigned as `dataPortBase + array index` (default: 9000, 9001, ...) and coordinator ports as `coordinatorPortBase + coordinator id` (default: 9011, 9012, 9013). You can customize the base ports: ```yaml -apiVersion: gateway.networking.k8s.io/v1 -kind: Gateway -metadata: - name: memgraph-data-gateway - namespace: default -spec: - gatewayClassName: eg - listeners: - - name: data-0 - protocol: TCP - port: 9000 - allowedRoutes: - namespaces: - from: Same - - name: data-1 - protocol: TCP - port: 9001 - allowedRoutes: - namespaces: - from: Same +externalAccessConfig: + gateway: + enabled: true + gatewayClassName: "eg" + dataPortBase: 9000 + coordinatorPortBase: 9010 +``` ---- -apiVersion: gateway.networking.k8s.io/v1alpha2 -kind: TCPRoute -metadata: - name: data-0-route - namespace: default -spec: - parentRefs: - - name: memgraph-data-gateway - sectionName: data-0 - rules: - - backendRefs: - - name: memgraph-data-0 - port: 7687 +You can also set annotations and labels on the Gateway resource: ---- -apiVersion: gateway.networking.k8s.io/v1alpha2 -kind: TCPRoute -metadata: - name: data-1-route - namespace: default -spec: - parentRefs: - - name: memgraph-data-gateway - sectionName: data-1 - rules: - - backendRefs: - - name: memgraph-data-1 - port: 7687 +```yaml +externalAccessConfig: + gateway: + enabled: true + gatewayClassName: "eg" + annotations: + example.io/owner: "memgraph" + labels: + app: memgraph-ha +``` + +To install with a chart-managed Gateway: + +```bash +helm install memgraph-ha memgraph/memgraph-high-availability \ + --set env.MEMGRAPH_ENTERPRISE_LICENSE= \ + --set env.MEMGRAPH_ORGANIZATION_NAME= \ + --set externalAccessConfig.gateway.enabled=true \ + --set externalAccessConfig.gateway.gatewayClassName=eg ``` -Coordinator instances' gateway: +#### Option 2: Existing (external) Gateway + +When you already have a Gateway resource in your cluster (for example, a shared Gateway serving multiple services including Memgraph Lab), you can have the chart create only TCPRoute resources that attach to it: ```yaml -apiVersion: gateway.networking.k8s.io/v1 -kind: Gateway -metadata: - name: memgraph-coordinators-gateway - namespace: default -spec: - gatewayClassName: eg - listeners: - - name: coordinator-1 - protocol: TCP - port: 9011 - allowedRoutes: - namespaces: - from: Same - - name: coordinator-2 - protocol: TCP - port: 9012 - allowedRoutes: - namespaces: - from: Same - - name: coordinator-3 - protocol: TCP - port: 9013 - allowedRoutes: - namespaces: - from: Same +externalAccessConfig: + gateway: + enabled: true + existingGatewayName: "memgraph-gateway" +``` ---- -apiVersion: gateway.networking.k8s.io/v1alpha2 -kind: TCPRoute -metadata: - name: coordinator-1-route - namespace: default -spec: - parentRefs: - - name: memgraph-coordinators-gateway - sectionName: coordinator-1 - rules: - - backendRefs: - - name: memgraph-coordinator-1 - port: 7687 +In this mode, the chart skips Gateway creation and only creates TCPRoute resources. The `gatewayClassName` is not required. ---- -apiVersion: gateway.networking.k8s.io/v1alpha2 -kind: TCPRoute -metadata: - name: coordinator-2-route - namespace: default -spec: - parentRefs: - - name: memgraph-coordinators-gateway - sectionName: coordinator-2 - rules: - - backendRefs: - - name: memgraph-coordinator-2 - port: 7687 +If the existing Gateway is in a different namespace, specify it: ---- -apiVersion: gateway.networking.k8s.io/v1alpha2 -kind: TCPRoute -metadata: - name: coordinator-3-route - namespace: default -spec: - parentRefs: - - name: memgraph-coordinators-gateway - sectionName: coordinator-3 - rules: - - backendRefs: - - name: memgraph-coordinator-3 - port: 7687 +```yaml +externalAccessConfig: + gateway: + enabled: true + existingGatewayName: "memgraph-gateway" + existingGatewayNamespace: "gateway-system" +``` + +To install with an existing Gateway: + +```bash +helm install memgraph-ha memgraph/memgraph-high-availability \ + --set env.MEMGRAPH_ENTERPRISE_LICENSE= \ + --set env.MEMGRAPH_ORGANIZATION_NAME= \ + --set externalAccessConfig.gateway.enabled=true \ + --set externalAccessConfig.gateway.existingGatewayName=memgraph-gateway ``` -In the similar way you could configure `TLSRoute` instead of a `TCPRoute`. + +When using an existing Gateway, ensure it has listeners configured with the correct names and ports that match the TCPRoute `sectionName` references. The chart expects listener names in the format `data-{id}-bolt` for data instances and `coordinator-{id}-bolt` for coordinators. For example, the default HA setup (2 data instances, 3 coordinators) needs these listeners: + +- `data-0-bolt` on port 9000 +- `data-1-bolt` on port 9001 +- `coordinator-1-bolt` on port 9011 +- `coordinator-2-bolt` on port 9012 +- `coordinator-3-bolt` on port 9013 + +A standalone Gateway manifest with these pre-configured listeners is available in the [Helm charts repository](https://github.com/memgraph/helm-charts/blob/main/examples/gateway/gateway.yaml). + + + +**TCPRoute API version**: TCPRoute uses `v1alpha2`, which is the latest available API version. It is supported by Envoy Gateway and other major implementations but is not yet GA. Gateway and HTTPRoute are both GA (`v1`). + ### Use Memgraph HA chart with IngressNginx @@ -631,6 +647,38 @@ must also be set to `true`, as the uploader sidecar mounts the same PVC used for `s3://///`. +## Graceful termination + +When a pod is stopped (e.g., during upgrades, rescheduling, or scale-down), +Kubernetes sends `SIGTERM` and waits up to `terminationGracePeriodSeconds` +for the container to exit cleanly before forcefully killing it with `SIGKILL`. +The HA chart defaults this value to **30 seconds** for both data +and coordinator pods, configurable via: + +- `container.data.terminationGracePeriodSeconds` +- `container.coordinators.terminationGracePeriodSeconds` + +since `--storage-snapshot-on-exit` is explicitly set to false by default. + +### Using `--storage-snapshot-on-exit` with HA + +If you enable the [`--storage-snapshot-on-exit`](/database-management/configuration) +flag on data instances, Memgraph will attempt to create a full snapshot of the +database during shutdown. Snapshot creation time scales with dataset size and +can easily exceed the default grace period on larger deployments. + + +If `terminationGracePeriodSeconds` is shorter than the time needed to write the +on-exit snapshot, Kubernetes will `SIGKILL` the Memgraph process mid-write, +leaving the snapshot incomplete and defeating the purpose of the flag. + +When enabling `--storage-snapshot-on-exit`, set +`container.data.terminationGracePeriodSeconds` to a value that comfortably +covers the expected snapshot duration for your dataset. Benchmark the snapshot +time on a representative dataset and add a safety margin. + + + ## Monitoring Memgraph HA integrates with Kubernetes monitoring tools through: @@ -714,6 +762,161 @@ kubectl delete crd servicemonitors.monitoring.coreos.com kubectl delete crd thanosrulers.monitoring.coreos.com ``` +### Remote metrics and logs + +The HA chart supports optional remote observability: + +- `vmagentRemote` for shipping metrics with Prometheus `remote_write` +- `vectorRemote` sidecars for shipping Memgraph logs to Loki-compatible endpoints + +Prerequisites: + +- keep `prometheus.enabled: true` so `mg-exporter` is deployed +- if you only need remote shipping and not local scraping, set `prometheus.serviceMonitor.enabled: false` to avoid duplicate scraping +- configure `vectorRemote.data` and/or `vectorRemote.coordinators` depending on which pod roles should ship logs +- when `vectorRemote.enabled: true`, add `--monitoring-port=` and `--monitoring-address=0.0.0.0` to each instance `args` + +Example `values.yaml`: + +```yaml +prometheus: + enabled: true + namespace: monitoring + serviceMonitor: + enabled: false + +vmagentRemote: + enabled: true + namespace: monitoring + remoteWrite: + url: "https:///api/v1/write" + # Optional: only set basicAuth when your remote_write endpoint requires basic auth. + basicAuth: + secretName: monitoring-basic-auth + usernameKey: username + passwordKey: password + externalLabels: + cluster_id: "memgraph-testing-cluster-53" + service_name: "memgraph-ha" + cluster_env: "self-hosted-large-01" + +vectorRemote: + enabled: true + data: true + coordinators: true + websocketPort: 7444 + logsEndpoint: "https://" + # Optional: only set auth when your endpoint requires basic auth. + auth: + secretName: monitoring-basic-auth + usernameKey: username + passwordKey: password + extraLabels: + cluster_id: "memgraph-testing-cluster-53" + service_name: "memgraph-ha" + cluster_env: "self-hosted-large-01" + +data: + - id: "0" + args: + - "--management-port=10000" + - "--bolt-port=7687" + - "--monitoring-port=7444" + - "--monitoring-address=0.0.0.0" + - id: "1" + args: + - "--management-port=10000" + - "--bolt-port=7687" + - "--monitoring-port=7444" + - "--monitoring-address=0.0.0.0" + +coordinators: + - id: "1" + args: + - "--coordinator-id=1" + - "--coordinator-port=12000" + - "--management-port=10000" + - "--bolt-port=7687" + - "--monitoring-port=7444" + - "--monitoring-address=0.0.0.0" + - id: "2" + args: + - "--coordinator-id=2" + - "--coordinator-port=12000" + - "--management-port=10000" + - "--bolt-port=7687" + - "--monitoring-port=7444" + - "--monitoring-address=0.0.0.0" + - id: "3" + args: + - "--coordinator-id=3" + - "--coordinator-port=12000" + - "--management-port=10000" + - "--bolt-port=7687" + - "--monitoring-port=7444" + - "--monitoring-address=0.0.0.0" +``` + +Create credentials secret in the namespace where vmagent runs (usually `monitoring`): + +```bash +kubectl create secret generic monitoring-basic-auth -n monitoring \ + --from-literal=username='' \ + --from-literal=password='' +``` + +For HA Vector sidecars, create the same secret in the Memgraph release namespace as well: + +```bash +kubectl create secret generic monitoring-basic-auth -n \ + --from-literal=username='' \ + --from-literal=password='' +``` + +#### Kubernetes infrastructure metrics + +`vmagentRemote` can additionally scrape Kubernetes infrastructure metrics +(`kube-state-metrics`, `node-exporter`, `kubelet`) required by +`kube-prometheus-stack` Kubernetes and Node dashboards, and remote-write them +to your centralized monitoring cluster. + +Enable Kubernetes scraping by extending your existing `vmagentRemote` values: + +```yaml +vmagentRemote: + # ... existing fields (enabled, remoteWrite, externalLabels) ... + kubernetes: + enabled: true + kubeStateMetrics: + enabled: true + jobName: kube-state-metrics + targets: + - kube-prometheus-stack-kube-state-metrics.monitoring.svc.cluster.local:8080 + nodeExporter: + enabled: true + jobName: node-exporter + targets: + - kube-prometheus-stack-prometheus-node-exporter.monitoring.svc.cluster.local:9100 + kubelet: + enabled: true + jobName: kubelet + metricsPath: /metrics/cadvisor + apiServerAddress: kubernetes.default.svc:443 + insecureSkipVerify: false +``` + +Notes: + +- RBAC and `ServiceAccount` resources are created only when an enabled scrape + job requires Kubernetes API access (for example `kubelet.enabled=true` or + `nodeExporter.useKubernetesDiscovery=true`). +- Keep `jobName` values aligned with dashboard and recording-rule expectations + unless you also update those queries. +- Dashboards that rely on precomputed recording-rule series still require + rule evaluation in your monitoring stack. + +A ready-to-use example values file is available in the Helm charts repository: +[`examples/remote-monitoring/values-ha-k8s-metrics.yaml`](https://github.com/memgraph/helm-charts/blob/main/examples/remote-monitoring/values-ha-k8s-metrics.yaml). ## Configuration options @@ -758,6 +961,14 @@ and their default values. | `externalAccessConfig.coordinator.annotations` | Annotations for external services attached to coordinators. | `{}` | | `externalAccessConfig.dataInstance.serviceType` | IngressNginx, NodePort or LoadBalancer. By default, no external service will be created. | `""` | | `externalAccessConfig.dataInstance.annotations` | Annotations for external services attached to data instances. | `{}` | +| `externalAccessConfig.gateway.enabled` | Enable Gateway API external access. | `false` | +| `externalAccessConfig.gateway.gatewayClassName` | Name of a pre-existing GatewayClass. Required when creating a new Gateway. | `""` | +| `externalAccessConfig.gateway.existingGatewayName` | Name of an existing Gateway to attach routes to. Skips Gateway creation. | `""` | +| `externalAccessConfig.gateway.existingGatewayNamespace` | Namespace of the existing Gateway. Defaults to release namespace. | `""` | +| `externalAccessConfig.gateway.annotations` | Annotations for the Gateway resource. | `{}` | +| `externalAccessConfig.gateway.labels` | Labels for the Gateway resource. | `{}` | +| `externalAccessConfig.gateway.dataPortBase` | Base port for data instance Gateway listeners (`dataPortBase + index`). | `9000` | +| `externalAccessConfig.gateway.coordinatorPortBase` | Base port for coordinator Gateway listeners (`coordinatorPortBase + id`). | `9010` | | `headlessService.enabled` | Specifies whether headless services will be used inside K8s network on all instances. | `false` | | `ports.boltPort` | Bolt port used on coordinator and data instances. | `7687` | | `ports.managementPort` | Management port used on coordinator and data instances. | `10000` | @@ -782,8 +993,8 @@ and their default values. | `container.data.startupProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | | `container.data.startupProbe.failureThreshold` | Failure threshold for startup probe | `1440` | | `container.data.startupProbe.timeoutSeconds` | Timeout for probe | `10` | -| `container.data.startupProbe.periodSeconds` | Period seconds for startup probe | `5` | -| `container.coordinators.terminationGracePeriodSeconds` | Grace period for coordinator pod termination | `1800` | +| `container.data.startupProbe.periodSeconds` | Period seconds for startup probe | `10` | +| `container.data.terminationGracePeriodSeconds` | Grace period for data pod termination. Increase when `--storage-snapshot-on-exit` is enabled so the snapshot has time to finish. | `30` | | `container.coordinators.livenessProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as coordinator port. | `12000` | | `container.coordinators.livenessProbe.failureThreshold` | Failure threshold for liveness probe | `20` | | `container.coordinators.livenessProbe.timeoutSeconds` | Timeout for liveness probe | `10` | @@ -796,6 +1007,7 @@ and their default values. | `container.coordinators.startupProbe.failureThreshold` | Failure threshold for startup probe | `20` | | `container.coordinators.startupProbe.timeoutSeconds` | Timeout for probe | `10` | | `container.coordinators.startupProbe.periodSeconds` | Period seconds for startup probe | `10` | +| `container.coordinators.terminationGracePeriodSeconds` | Grace period for coordinators pod termination. | `30`. | | `data` | Configuration for data instances | See `data` section | | `coordinators` | Configuration for coordinator instances | See `coordinators` section | | `sysctlInitContainer.enabled` | Enable the init container to set sysctl parameters | `true` | @@ -818,6 +1030,38 @@ and their default values. | `prometheus.serviceMonitor.enabled` | If enabled, a `ServiceMonitor` object will be deployed. | `true` | | `prometheus.serviceMonitor.kubePrometheusStackReleaseName` | The release name under which `kube-prometheus-stack` chart is installed. | `kube-prometheus-stack` | | `prometheus.serviceMonitor.interval` | How often will Prometheus pull data from Memgraph's Prometheus exporter. | `15s` | +| `vmagentRemote.enabled` | Deploy a vmagent Deployment that scrapes mg-exporter and remote-writes to a Prometheus-compatible endpoint. | `false` | +| `vmagentRemote.namespace` | Namespace for the vmagent Deployment and its resources. Defaults to `prometheus.namespace` when empty. | `""` | +| `vmagentRemote.image.repository` | vmagent image repository. | `victoriametrics/vmagent` | +| `vmagentRemote.image.tag` | vmagent image tag. | `v1.139.0` | +| `vmagentRemote.image.pullPolicy` | vmagent image pull policy. | `IfNotPresent` | +| `vmagentRemote.remoteWrite.url` | Prometheus `remote_write` URL. Required when `vmagentRemote.enabled=true`. | `""` | +| `vmagentRemote.remoteWrite.basicAuth.secretName` | Kubernetes Secret holding basic-auth credentials for `remote_write`. When empty, basic auth is not configured. | `""` | +| `vmagentRemote.remoteWrite.basicAuth.usernameKey` | Key in the basic-auth Secret holding the username. | `username` | +| `vmagentRemote.remoteWrite.basicAuth.passwordKey` | Key in the basic-auth Secret holding the password. | `password` | +| `vmagentRemote.scrapeInterval` | Global `scrape_interval` applied to vmagent scrape jobs. | `15s` | +| `vmagentRemote.externalLabels` | External labels attached to every scraped sample before remote-write. | `{}` | +| `vmagentRemote.resources` | Resource requests/limits for the vmagent container. | `{}` | +| `vmagentRemote.httpPort` | vmagent local HTTP listen port for metrics/debug (the remote-write target is `remoteWrite.url`). | `8429` | +| `vmagentRemote.kubernetes.enabled` | Enable scraping of Kubernetes infrastructure metrics used by kube-prometheus dashboards. | `false` | +| `vmagentRemote.kubernetes.kubeStateMetrics.enabled` | Scrape `kube-state-metrics`. | `true` | +| `vmagentRemote.kubernetes.kubeStateMetrics.jobName` | Prometheus `job` label for kube-state-metrics. Keep aligned with dashboard/recording-rule expectations. | `kube-state-metrics` | +| `vmagentRemote.kubernetes.kubeStateMetrics.targets` | Static scrape targets for kube-state-metrics. | `[kube-prometheus-stack-kube-state-metrics.monitoring.svc.cluster.local:8080]` | +| `vmagentRemote.kubernetes.nodeExporter.enabled` | Scrape `node-exporter`. | `true` | +| `vmagentRemote.kubernetes.nodeExporter.jobName` | Prometheus `job` label for node-exporter. | `node-exporter` | +| `vmagentRemote.kubernetes.nodeExporter.useKubernetesDiscovery` | Discover node-exporter pods via Kubernetes SD so namespace/pod/node labels are present for recording rules. | `false` | +| `vmagentRemote.kubernetes.nodeExporter.podMetricsPort` | Pod port used by Kubernetes SD to match node-exporter pods. | `"9100"` | +| `vmagentRemote.kubernetes.nodeExporter.appNameLabel` | Expected value of `app.kubernetes.io/name` on node-exporter pods. | `prometheus-node-exporter` | +| `vmagentRemote.kubernetes.nodeExporter.appInstanceLabel` | Expected value of `app.kubernetes.io/instance` on node-exporter pods. | `kube-prometheus-stack-prometheus-node-exporter` | +| `vmagentRemote.kubernetes.nodeExporter.targets` | Static fallback targets for node-exporter when `useKubernetesDiscovery=false`. | `[kube-prometheus-stack-prometheus-node-exporter.monitoring.svc.cluster.local:9100]` | +| `vmagentRemote.kubernetes.kubelet.enabled` | Scrape kubelet metrics via the Kubernetes API server node proxy. | `true` | +| `vmagentRemote.kubernetes.kubelet.jobName` | Prometheus `job` label for kubelet. Keep as `kubelet` so kube-prometheus dashboards and rules still match. | `kubelet` | +| `vmagentRemote.kubernetes.kubelet.metricsPath` | Metrics path for the primary kubelet scrape (cAdvisor). | `/metrics/cadvisor` | +| `vmagentRemote.kubernetes.kubelet.additionalMetricsEnabled` | Enable a second kubelet scrape job for `/metrics` alongside the cAdvisor job. | `true` | +| `vmagentRemote.kubernetes.kubelet.additionalJobName` | Prometheus `job` label for the additional kubelet scrape. | `kubelet-metrics` | +| `vmagentRemote.kubernetes.kubelet.additionalMetricsPath` | Metrics path for the additional kubelet scrape. | `/metrics` | +| `vmagentRemote.kubernetes.kubelet.apiServerAddress` | Kubernetes API server address used to proxy kubelet scrapes. | `kubernetes.default.svc:443` | +| `vmagentRemote.kubernetes.kubelet.insecureSkipVerify` | Skip TLS verification of the kube-apiserver serving cert when scraping kubelet. | `false` | | `labels.coordinators.podLabels` | Enables you to set labels on a pod level for coordinators. | `{}` | | `labels.coordinators.statefulSetLabels` | Enables you to set labels on a stateful set level for coordinators. | `{}` | | `labels.coordinators.serviceLabels` | Enables you to set labels on a service level for coordinators. | `{}` | @@ -851,6 +1095,7 @@ following parameters: | Parameter | Description | Default | |---------------------------------------------|-----------------------------------------------------------------------------------------------------|-----------------------------------------| | `id` | ID of the instance | `0` for data, `1` for coordinators | +| `externalAccessAnnotations` | Per-instance annotations for the external access Service, merged with global annotations. | `{}` | | `args` | List of arguments for the instance | See `args` section | The `args` section contains a list of arguments for the instance. diff --git a/pages/clustering/replication/how-replication-works.mdx b/pages/clustering/replication/how-replication-works.mdx index b008f6f44..a36bb4970 100644 --- a/pages/clustering/replication/how-replication-works.mdx +++ b/pages/clustering/replication/how-replication-works.mdx @@ -117,6 +117,12 @@ received or a timeout is reached. If the REPLICA fails, MAIN instance will still the data and move forward. This behaviour does not block writes on REPLICA failure, and still ensures other REPLICAs to receive new data. +When a SYNC replica fails to confirm a transaction, Memgraph raises an error +that identifies the specific replica and the reason for the failure, for example: +`Failed to replicate to SYNC replica 'instance_1': replica is not reachable or not in sync with the main`. +The transaction is still committed on the MAIN instance and other alive replicas. +Failed replicas will be recovered automatically. + The following diagrams express the behavior of the MAIN instance in cases when SYNC REPLICA doesn't answer within the expected timeout. @@ -130,10 +136,15 @@ The STRICT_SYNC replication mode behaves very similarly to a SYNC mode except that MAIN **won't commit a transaction locally in a situation in which one of STRICT_SYNC replicas is down**. To achieve that, all instances run together a *two-phase commit* protocol which allows you such a synchronization. This -reduces the throughout but such a mode is super useful in a high-availability +reduces the throughput but such a mode is super useful in a high-availability scenario in which a failover is the most critical operation to support. Such a mode then allows you a failover **without the fear of experiencing a data loss**. +When a STRICT_SYNC replica fails to confirm a transaction, the transaction is +aborted on all instances. The error message identifies the specific replica and +the reason, for example: +`Failed to replicate to STRICT_SYNC replica 'instance_2': replica is not reachable or not in sync with the main`. + **STRICT_SYNC mode ensures consistency and partition tolerance (CP).** ### ASYNC replication mode diff --git a/pages/custom-query-modules/c/c-api.mdx b/pages/custom-query-modules/c/c-api.mdx index e1a6c31cd..919708e53 100644 --- a/pages/custom-query-modules/c/c-api.mdx +++ b/pages/custom-query-modules/c/c-api.mdx @@ -50,7 +50,7 @@ Memgraph in order to use them. | | Name | | -------------- | -------------- | -| enum| **[mgp_value_type](#enum-mgp-value-type)** {MGP_VALUE_TYPE_NULL, MGP_VALUE_TYPE_BOOL, MGP_VALUE_TYPE_INT, MGP_VALUE_TYPE_DOUBLE, MGP_VALUE_TYPE_STRING, MGP_VALUE_TYPE_LIST, MGP_VALUE_TYPE_MAP, MGP_VALUE_TYPE_VERTEX, MGP_VALUE_TYPE_EDGE, MGP_VALUE_TYPE_PATH, MGP_VALUE_TYPE_DATE, MGP_VALUE_TYPE_LOCAL_TIME, MGP_VALUE_TYPE_LOCAL_DATE_TIME, MGP_VALUE_TYPE_DURATION, MGP_VALUE_TYPE_ZONED_DATE_TIME}
All available types that can be stored in a mgp_value. | +| enum| **[mgp_value_type](#enum-mgp-value-type)** {MGP_VALUE_TYPE_NULL, MGP_VALUE_TYPE_BOOL, MGP_VALUE_TYPE_INT, MGP_VALUE_TYPE_DOUBLE, MGP_VALUE_TYPE_STRING, MGP_VALUE_TYPE_LIST, MGP_VALUE_TYPE_MAP, MGP_VALUE_TYPE_VERTEX, MGP_VALUE_TYPE_EDGE, MGP_VALUE_TYPE_PATH, MGP_VALUE_TYPE_DATE, MGP_VALUE_TYPE_LOCAL_TIME, MGP_VALUE_TYPE_LOCAL_DATE_TIME, MGP_VALUE_TYPE_DURATION, MGP_VALUE_TYPE_ZONED_DATE_TIME, MGP_VALUE_TYPE_POINT_2D, MGP_VALUE_TYPE_POINT_3D, MGP_VALUE_TYPE_ENUM}
All available types that can be stored in a mgp_value. | | typedef void(*)(struct mgp_list *, struct mgp_graph *, struct mgp_result *, struct mgp_memory *) | **[mgp_proc_cb](#typedef-mgp-proc-cb)**
Entry-point for a query module read procedure, invoked through openCypher. | | typedef void(*)(struct mgp_list *, struct mgp_graph *, struct mgp_memory *) | **[mgp_proc_initializer](#typedef-mgp-proc-initializer)**
Initialization point for a query module read procedure, invoked before procedure. | | typedef void(*)() | **[mgp_proc_cleanup](#typedef-mgp-proc-cleanup)**
Cleanup for a query module read procedure | @@ -82,6 +82,9 @@ Memgraph in order to use them. | enum [mgp_error](#variable-mgp-error) | **[mgp_value_make_local_date_time](#function-mgp-value-make-local-date-time)**(struct mgp_local_date_time * val, struct mgp_value ** result)
Create a mgp_value storing a mgp_local_date_time. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_make_duration](#function-mgp-value-make-duration)**(struct mgp_duration * val, struct mgp_value ** result)
Create a mgp_value storing a mgp_duration. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_make_zoned_date_time](#function-mgp-value-make-zoned-date-time)**(struct mgp_zoned_date_time * val, struct mgp_value ** result)
Create a mgp_value storing a mgp_zoned_date_time. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_value_make_point_2d](#function-mgp-value-make-point-2d)**(struct mgp_point_2d * val, struct mgp_value ** result)
Create a mgp_value storing a mgp_point_2d. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_value_make_point_3d](#function-mgp-value-make-point-3d)**(struct mgp_point_3d * val, struct mgp_value ** result)
Create a mgp_value storing a mgp_point_3d. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_value_make_enum](#function-mgp-value-make-enum)**(struct mgp_enum * val, struct mgp_value ** result)
Create a mgp_value storing a mgp_enum. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_type](#function-mgp-value-get-type)**(struct mgp_value * val, enum [mgp_value_type](#enum-mgp-value-type) * result)
Get the type of the value contained in mgp_value. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_is_null](#function-mgp-value-is-null)**(struct mgp_value * val, int * result)
Result is non-zero if the given mgp_value represents `null`. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_is_bool](#function-mgp-value-is-bool)**(struct mgp_value * val, int * result)
Result is non-zero if the given mgp_value stores a boolean. | @@ -98,6 +101,9 @@ Memgraph in order to use them. | enum [mgp_error](#variable-mgp-error) | **[mgp_value_is_local_date_time](#function-mgp-value-is-local-date-time)**(struct mgp_value * val, int * result)
Result is non-zero if the given mgp_value stores a local date-time. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_is_duration](#function-mgp-value-is-duration)**(struct mgp_value * val, int * result)
Result is non-zero if the given mgp_value stores a duration. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_is_zoned_date_time](#function-mgp-value-is-zoned-date-time)**(struct mgp_value * val, int * result)
Result is non-zero if the given mgp_value stores a zoned date-time. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_value_is_point_2d](#function-mgp-value-is-point-2d)**(struct mgp_value * val, int * result)
Result is non-zero if the given mgp_value stores a 2D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_value_is_point_3d](#function-mgp-value-is-point-3d)**(struct mgp_value * val, int * result)
Result is non-zero if the given mgp_value stores a 3D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_value_is_enum](#function-mgp-value-is-enum)**(struct mgp_value * val, int * result)
Result is non-zero if the given mgp_value stores an enum. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_bool](#function-mgp-value-get-bool)**(struct mgp_value * val, int * result)
Get the contained boolean value. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_int](#function-mgp-value-get-int)**(struct mgp_value * val, int64_t * result)
Get the contained integer. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_double](#function-mgp-value-get-double)**(struct mgp_value * val, double * result)
Get the contained double floating-point. | @@ -112,6 +118,9 @@ Memgraph in order to use them. | enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_local_date_time](#function-mgp-value-get-local-date-time)**(struct mgp_value * val, struct mgp_local_date_time ** result)
Get the contained local date-time. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_duration](#function-mgp-value-get-duration)**(struct mgp_value * val, struct mgp_duration ** result)
Get the contained duration. | | enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_zoned_date_time](#function-mgp-value-get-zoned-date-time)**(struct mgp_value * val, struct mgp_zoned_date_time ** result)
Get the contained zoned date-time. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_point_2d](#function-mgp-value-get-point-2d)**(struct mgp_value * val, struct mgp_point_2d ** result)
Get the contained 2D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_point_3d](#function-mgp-value-get-point-3d)**(struct mgp_value * val, struct mgp_point_3d ** result)
Get the contained 3D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_value_get_enum](#function-mgp-value-get-enum)**(struct mgp_value * val, struct mgp_enum ** result)
Get the contained enum. | | enum [mgp_error](#variable-mgp-error) | **[mgp_list_make_empty](#function-mgp-list-make-empty)**(size_t capacity, struct mgp_memory * memory, struct mgp_list ** result)
Create an empty list with given capacity. | | void | **[mgp_list_destroy](#function-mgp-list-destroy)**(struct mgp_list * list)
Free the memory used by the given mgp_list and contained elements. | | enum [mgp_error](#variable-mgp-error) | **[mgp_list_contains_deleted](#function-mgp-list-contains-deleted)**(struct mgp_list * list, int * result)
Result is non-zero if the given list contains any deleted values, otherwise 0. | @@ -285,6 +294,27 @@ Memgraph in order to use them. | enum [mgp_error](#variable-mgp-error) | **[mgp_duration_neg](#function-mgp-duration-neg)**(struct mgp_duration * dur, struct mgp_memory * memory, struct mgp_duration ** result)
Apply unary minus operator to the duration. | | enum [mgp_error](#variable-mgp-error) | **[mgp_duration_add](#function-mgp-duration-add)**(struct mgp_duration * first, struct mgp_duration * second, struct mgp_memory * memory, struct mgp_duration ** result)
Add two durations. | | enum [mgp_error](#variable-mgp-error) | **[mgp_duration_sub](#function-mgp-duration-sub)**(struct mgp_duration * first, struct mgp_duration * second, struct mgp_memory * memory, struct mgp_duration ** result)
Subtract two durations. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_2d_make](#function-mgp-point-2d-make)**(double x, double y, uint16_t srid, struct mgp_memory * memory, struct mgp_point_2d ** result)
Create a 2D point with given coordinates and SRID. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_2d_copy](#function-mgp-point-2d-copy)**(struct mgp_point_2d * point, struct mgp_memory * memory, struct mgp_point_2d ** result)
Copy a mgp_point_2d. | +| void | **[mgp_point_2d_destroy](#function-mgp-point-2d-destroy)**(struct mgp_point_2d * point)
Free the memory used by a mgp_point_2d. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_2d_equal](#function-mgp-point-2d-equal)**(struct mgp_point_2d * first, struct mgp_point_2d * second, int * result)
Result is non-zero if given 2D points are equal, otherwise 0. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_2d_get_x](#function-mgp-point-2d-get-x)**(struct mgp_point_2d * point, double * result)
Get the x coordinate of a 2D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_2d_get_y](#function-mgp-point-2d-get-y)**(struct mgp_point_2d * point, double * result)
Get the y coordinate of a 2D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_2d_get_srid](#function-mgp-point-2d-get-srid)**(struct mgp_point_2d * point, uint16_t * result)
Get the SRID of a 2D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_3d_make](#function-mgp-point-3d-make)**(double x, double y, double z, uint16_t srid, struct mgp_memory * memory, struct mgp_point_3d ** result)
Create a 3D point with given coordinates and SRID. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_3d_copy](#function-mgp-point-3d-copy)**(struct mgp_point_3d * point, struct mgp_memory * memory, struct mgp_point_3d ** result)
Copy a mgp_point_3d. | +| void | **[mgp_point_3d_destroy](#function-mgp-point-3d-destroy)**(struct mgp_point_3d * point)
Free the memory used by a mgp_point_3d. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_3d_equal](#function-mgp-point-3d-equal)**(struct mgp_point_3d * first, struct mgp_point_3d * second, int * result)
Result is non-zero if given 3D points are equal, otherwise 0. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_3d_get_x](#function-mgp-point-3d-get-x)**(struct mgp_point_3d * point, double * result)
Get the x coordinate of a 3D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_3d_get_y](#function-mgp-point-3d-get-y)**(struct mgp_point_3d * point, double * result)
Get the y coordinate of a 3D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_3d_get_z](#function-mgp-point-3d-get-z)**(struct mgp_point_3d * point, double * result)
Get the z coordinate of a 3D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_point_3d_get_srid](#function-mgp-point-3d-get-srid)**(struct mgp_point_3d * point, uint16_t * result)
Get the SRID of a 3D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_enum_make](#function-mgp-enum-make)**(const char * type_name, const char * value_name, struct mgp_memory * memory, struct mgp_enum ** result)
Create an enum value from type and value name strings. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_enum_copy](#function-mgp-enum-copy)**(struct mgp_enum * val, struct mgp_memory * memory, struct mgp_enum ** result)
Copy a mgp_enum. | +| void | **[mgp_enum_destroy](#function-mgp-enum-destroy)**(struct mgp_enum * val)
Free the memory used by a mgp_enum. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_enum_equal](#function-mgp-enum-equal)**(struct mgp_enum * first, struct mgp_enum * second, int * result)
Result is non-zero if given enum values are equal, otherwise 0. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_enum_get_type_name](#function-mgp-enum-get-type-name)**(struct mgp_enum * val, const char ** result)
Get the type name of the enum value. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_enum_get_value_name](#function-mgp-enum-get-value-name)**(struct mgp_enum * val, const char ** result)
Get the value name of the enum value. | | enum [mgp_error](#variable-mgp-error) | **[mgp_type_any](#function-mgp-type-any)**(struct mgp_type ** result)
Get the type representing any value that isn't `null`. | | enum [mgp_error](#variable-mgp-error) | **[mgp_type_bool](#function-mgp-type-bool)**(struct mgp_type ** result)
Get the type representing boolean values. | | enum [mgp_error](#variable-mgp-error) | **[mgp_type_string](#function-mgp-type-string)**(struct mgp_type ** result)
Get the type representing character string values. | @@ -300,6 +330,10 @@ Memgraph in order to use them. | enum [mgp_error](#variable-mgp-error) | **[mgp_type_local_time](#function-mgp-type-local-time)**(struct mgp_type ** result)
Get the type representing a local time. | | enum [mgp_error](#variable-mgp-error) | **[mgp_type_local_date_time](#function-mgp-type-local-date-time)**(struct mgp_type ** result)
Get the type representing a local date-time. | | enum [mgp_error](#variable-mgp-error) | **[mgp_type_duration](#function-mgp-type-duration)**(struct mgp_type ** result)
Get the type representing a duration. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_type_zoned_date_time](#function-mgp-type-zoned-date-time)**(struct mgp_type ** result)
Get the type representing a zoned date-time. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_type_point_2d](#function-mgp-type-point-2d)**(struct mgp_type ** result)
Get the type representing a 2D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_type_point_3d](#function-mgp-type-point-3d)**(struct mgp_type ** result)
Get the type representing a 3D point. | +| enum [mgp_error](#variable-mgp-error) | **[mgp_type_enum](#function-mgp-type-enum)**(struct mgp_type ** result)
Get the type representing an enum value. | | enum [mgp_error](#variable-mgp-error) | **[mgp_type_nullable](#function-mgp-type-nullable)**(struct mgp_type * type, struct mgp_type ** result)
Build a type representing either a `null` value or a value of given `type`. | | enum [mgp_error](#variable-mgp-error) | **[mgp_module_add_read_procedure](#function-mgp-module-add-read-procedure)**(struct mgp_module * module, const char * name, [mgp_proc_cb](#typedef-mgp-proc-cb) cb, struct mgp_proc ** result)
Register a read-only procedure to a module. | | enum [mgp_error](#variable-mgp-error) | **[mgp_module_add_write_procedure](#function-mgp-module-add-write-procedure)**(struct mgp_module * module, const char * name, [mgp_proc_cb](#typedef-mgp-proc-cb) cb, struct mgp_proc ** result)
Register a writeable procedure to a module. | @@ -624,6 +658,9 @@ All available types that can be stored in a mgp_value. | MGP_VALUE_TYPE_LOCAL_DATE_TIME | | MGP_VALUE_TYPE_DURATION | | MGP_VALUE_TYPE_ZONED_DATE_TIME | +| MGP_VALUE_TYPE_POINT_2D | +| MGP_VALUE_TYPE_POINT_3D | +| MGP_VALUE_TYPE_ENUM | ### typedef mgp_proc_cb [#typedef-mgp-proc-cb]### typedef mgp_proc_initializer [#typedef-mgp-proc-initializer]### typedef mgp_proc_cleanup [#typedef-mgp-proc-cleanup] ```cpp @@ -941,6 +978,62 @@ zoned date-time. `MGP_ERROR_UNABLE_TO_ALLOCATE` is returned if unable to allocat a `mgp_value`. +### mgp_value_make_point_2d [#function-mgp-value-make-point-2d] +```cpp +enum mgp_error mgp_value_make_point_2d( + struct mgp_point_2d * val, + struct mgp_value ** result +) +``` + +Create a `mgp_value` storing a `mgp_point_2d`. + +You need to free the instance through `mgp_value_destroy`. The ownership of the +2D point is transferred to the created `mgp_value` and destroying the +`mgp_value` will destroy the `mgp_point_2d`. Therefore, if a `mgp_value` is +successfully created you must not call `mgp_point_2d_destroy` on the given +point. `MGP_ERROR_UNABLE_TO_ALLOCATE` is returned if unable to allocate +a `mgp_value`. + + +### mgp_value_make_point_3d [#function-mgp-value-make-point-3d] +```cpp +enum mgp_error mgp_value_make_point_3d( + struct mgp_point_3d * val, + struct mgp_value ** result +) +``` + +Create a `mgp_value` storing a `mgp_point_3d`. + +You need to free the instance through `mgp_value_destroy`. The ownership of the +3D point is transferred to the created `mgp_value` and destroying the +`mgp_value` will destroy the `mgp_point_3d`. Therefore, if a `mgp_value` is +successfully created you must not call `mgp_point_3d_destroy` on the given +point. `MGP_ERROR_UNABLE_TO_ALLOCATE` is returned if unable to allocate +a `mgp_value`. + + +### mgp_value_make_enum [#function-mgp-value-make-enum] +```cpp +enum mgp_error mgp_value_make_enum( + struct mgp_enum * val, + struct mgp_value ** result +) +``` + +Create a `mgp_value` storing a `mgp_enum`. + +You need to free the instance through `mgp_value_destroy`. The ownership of the +enum is transferred to the created `mgp_value` and destroying the +`mgp_value` will destroy the `mgp_enum`. Therefore, if a `mgp_value` is +successfully created you must not call `mgp_enum_destroy` on the given +enum. `MGP_ERROR_UNABLE_TO_ALLOCATE` is returned if unable to allocate +a `mgp_value`. + +> **Note:** Enum values can be received as procedure input and inspected, but cannot be returned from procedures. + + ### mgp_value_get_type [#function-mgp-value-get-type] ```cpp enum mgp_error mgp_value_get_type( @@ -1148,6 +1241,45 @@ Result is non-zero if the given `mgp_value` stores a zoned date-time. Current implementation always returns without errors. +### mgp_value_is_point_2d [#function-mgp-value-is-point-2d] +```cpp +enum mgp_error mgp_value_is_point_2d( + struct mgp_value * val, + int * result +) +``` + +Result is non-zero if the given `mgp_value` stores a 2D point. + +Current implementation always returns without errors. + + +### mgp_value_is_point_3d [#function-mgp-value-is-point-3d] +```cpp +enum mgp_error mgp_value_is_point_3d( + struct mgp_value * val, + int * result +) +``` + +Result is non-zero if the given `mgp_value` stores a 3D point. + +Current implementation always returns without errors. + + +### mgp_value_is_enum [#function-mgp-value-is-enum] +```cpp +enum mgp_error mgp_value_is_enum( + struct mgp_value * val, + int * result +) +``` + +Result is non-zero if the given `mgp_value` stores an enum value. + +Current implementation always returns without errors. + + ### mgp_value_get_bool [#function-mgp-value-get-bool] ```cpp enum mgp_error mgp_value_get_bool( @@ -1330,6 +1462,45 @@ Get the contained zoned date-time. Result is undefined if `mgp_value` does not contain the expected type. Current implementation always returns without errors. +### mgp_value_get_point_2d [#function-mgp-value-get-point-2d] +```cpp +enum mgp_error mgp_value_get_point_2d( + struct mgp_value * val, + struct mgp_point_2d ** result +) +``` + +Get the contained 2D point. + +Result is undefined if `mgp_value` does not contain the expected type. Current implementation always returns without errors. + + +### mgp_value_get_point_3d [#function-mgp-value-get-point-3d] +```cpp +enum mgp_error mgp_value_get_point_3d( + struct mgp_value * val, + struct mgp_point_3d ** result +) +``` + +Get the contained 3D point. + +Result is undefined if `mgp_value` does not contain the expected type. Current implementation always returns without errors. + + +### mgp_value_get_enum [#function-mgp-value-get-enum] +```cpp +enum mgp_error mgp_value_get_enum( + struct mgp_value * val, + struct mgp_enum ** result +) +``` + +Get the contained enum value. + +Result is undefined if `mgp_value` does not contain the expected type. Current implementation always returns without errors. + + ### mgp_list_make_empty [#function-mgp-list-make-empty] ```cpp enum mgp_error mgp_list_make_empty( @@ -3532,6 +3703,287 @@ Subtract two durations. Resulting pointer must be freed with mgp_duration_destroy. Return MGP_ERROR_INVALID_ARGUMENT if the operation results in an invalid duration. Return MGP_ERROR_UNABLE_TO_ALLOCATE if unable to allocate a mgp_duration. +### mgp_point_2d_make [#function-mgp-point-2d-make] +```cpp +enum mgp_error mgp_point_2d_make( + double x, + double y, + uint16_t srid, + struct mgp_memory * memory, + struct mgp_point_2d ** result +) +``` + +Create a 2D point with given x and y coordinates and spatial reference identifier (SRID). + +Supported SRIDs: 4326 (WGS-84), 7203 (Cartesian 2D). Resulting point must be freed with `mgp_point_2d_destroy`. Return `MGP_ERROR_INVALID_ARGUMENT` if the SRID is not valid. Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate a `mgp_point_2d`. + + +### mgp_point_2d_copy [#function-mgp-point-2d-copy] +```cpp +enum mgp_error mgp_point_2d_copy( + struct mgp_point_2d * point, + struct mgp_memory * memory, + struct mgp_point_2d ** result +) +``` + +Copy a `mgp_point_2d`. + +Resulting pointer must be freed with `mgp_point_2d_destroy`. Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate a `mgp_point_2d`. + + +### mgp_point_2d_destroy [#function-mgp-point-2d-destroy] +```cpp +void mgp_point_2d_destroy( + struct mgp_point_2d * point +) +``` + +Free the memory used by a `mgp_point_2d`. + + +### mgp_point_2d_equal [#function-mgp-point-2d-equal] +```cpp +enum mgp_error mgp_point_2d_equal( + struct mgp_point_2d * first, + struct mgp_point_2d * second, + int * result +) +``` + +Result is non-zero if given 2D points are equal, otherwise 0. + +Current implementation always returns without errors. + + +### mgp_point_2d_get_x [#function-mgp-point-2d-get-x] +```cpp +enum mgp_error mgp_point_2d_get_x( + struct mgp_point_2d * point, + double * result +) +``` + +Get the x coordinate of a 2D point. + +Current implementation always returns without errors. + + +### mgp_point_2d_get_y [#function-mgp-point-2d-get-y] +```cpp +enum mgp_error mgp_point_2d_get_y( + struct mgp_point_2d * point, + double * result +) +``` + +Get the y coordinate of a 2D point. + +Current implementation always returns without errors. + + +### mgp_point_2d_get_srid [#function-mgp-point-2d-get-srid] +```cpp +enum mgp_error mgp_point_2d_get_srid( + struct mgp_point_2d * point, + uint16_t * result +) +``` + +Get the SRID (spatial reference identifier) of a 2D point. + +Current implementation always returns without errors. + + +### mgp_point_3d_make [#function-mgp-point-3d-make] +```cpp +enum mgp_error mgp_point_3d_make( + double x, + double y, + double z, + uint16_t srid, + struct mgp_memory * memory, + struct mgp_point_3d ** result +) +``` + +Create a 3D point with given x, y, and z coordinates and spatial reference identifier (SRID). + +Supported SRIDs: 4979 (WGS-84 3D), 9157 (Cartesian 3D). Resulting point must be freed with `mgp_point_3d_destroy`. Return `MGP_ERROR_INVALID_ARGUMENT` if the SRID is not valid. Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate a `mgp_point_3d`. + + +### mgp_point_3d_copy [#function-mgp-point-3d-copy] +```cpp +enum mgp_error mgp_point_3d_copy( + struct mgp_point_3d * point, + struct mgp_memory * memory, + struct mgp_point_3d ** result +) +``` + +Copy a `mgp_point_3d`. + +Resulting pointer must be freed with `mgp_point_3d_destroy`. Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate a `mgp_point_3d`. + + +### mgp_point_3d_destroy [#function-mgp-point-3d-destroy] +```cpp +void mgp_point_3d_destroy( + struct mgp_point_3d * point +) +``` + +Free the memory used by a `mgp_point_3d`. + + +### mgp_point_3d_equal [#function-mgp-point-3d-equal] +```cpp +enum mgp_error mgp_point_3d_equal( + struct mgp_point_3d * first, + struct mgp_point_3d * second, + int * result +) +``` + +Result is non-zero if given 3D points are equal, otherwise 0. + +Current implementation always returns without errors. + + +### mgp_point_3d_get_x [#function-mgp-point-3d-get-x] +```cpp +enum mgp_error mgp_point_3d_get_x( + struct mgp_point_3d * point, + double * result +) +``` + +Get the x coordinate of a 3D point. + +Current implementation always returns without errors. + + +### mgp_point_3d_get_y [#function-mgp-point-3d-get-y] +```cpp +enum mgp_error mgp_point_3d_get_y( + struct mgp_point_3d * point, + double * result +) +``` + +Get the y coordinate of a 3D point. + +Current implementation always returns without errors. + + +### mgp_point_3d_get_z [#function-mgp-point-3d-get-z] +```cpp +enum mgp_error mgp_point_3d_get_z( + struct mgp_point_3d * point, + double * result +) +``` + +Get the z coordinate of a 3D point. + +Current implementation always returns without errors. + + +### mgp_point_3d_get_srid [#function-mgp-point-3d-get-srid] +```cpp +enum mgp_error mgp_point_3d_get_srid( + struct mgp_point_3d * point, + uint16_t * result +) +``` + +Get the SRID (spatial reference identifier) of a 3D point. + +Current implementation always returns without errors. + + +### mgp_enum_make [#function-mgp-enum-make] +```cpp +enum mgp_error mgp_enum_make( + const char * type_name, + const char * value_name, + struct mgp_memory * memory, + struct mgp_enum ** result +) +``` + +Create an enum value from type and value name strings. + +Resulting enum must be freed with `mgp_enum_destroy`. Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate a `mgp_enum`. + +> **Note:** Enum values can be received as procedure input and inspected, but cannot be returned from procedures. + + +### mgp_enum_copy [#function-mgp-enum-copy] +```cpp +enum mgp_error mgp_enum_copy( + struct mgp_enum * val, + struct mgp_memory * memory, + struct mgp_enum ** result +) +``` + +Copy a `mgp_enum`. + +Resulting pointer must be freed with `mgp_enum_destroy`. Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate a `mgp_enum`. + + +### mgp_enum_destroy [#function-mgp-enum-destroy] +```cpp +void mgp_enum_destroy( + struct mgp_enum * val +) +``` + +Free the memory used by the given `mgp_enum`. + + +### mgp_enum_equal [#function-mgp-enum-equal] +```cpp +enum mgp_error mgp_enum_equal( + struct mgp_enum * first, + struct mgp_enum * second, + int * result +) +``` + +Result is non-zero if the given enum values are equal (same type name and value name), otherwise 0. + +Current implementation always returns without errors. + + +### mgp_enum_get_type_name [#function-mgp-enum-get-type-name] +```cpp +enum mgp_error mgp_enum_get_type_name( + struct mgp_enum * val, + const char ** result +) +``` + +Get the type name of the enum value (e.g. `"Status"` for `Status.Active`). + +Current implementation always returns without errors. + + +### mgp_enum_get_value_name [#function-mgp-enum-get-value-name] +```cpp +enum mgp_error mgp_enum_get_value_name( + struct mgp_enum * val, + const char ** result +) +``` + +Get the value name of the enum value (e.g. `"Active"` for `Status.Active`). + +Current implementation always returns without errors. + + ### mgp_type_any [#function-mgp-type-any] ```cpp enum mgp_error mgp_type_any( @@ -3719,6 +4171,56 @@ Get the type representing a duration. Return MGP_ERROR_UNABLE_TO_ALLOCATE if unable to allocate the new type. +### mgp_type_zoned_date_time [#function-mgp-type-zoned-date-time] +```cpp +enum mgp_error mgp_type_zoned_date_time( + struct mgp_type ** result +) +``` + +Get the type representing a zoned date-time. + +Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate the new type. + + +### mgp_type_point_2d [#function-mgp-type-point-2d] +```cpp +enum mgp_error mgp_type_point_2d( + struct mgp_type ** result +) +``` + +Get the type representing a 2D point. + +Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate the new type. + + +### mgp_type_point_3d [#function-mgp-type-point-3d] +```cpp +enum mgp_error mgp_type_point_3d( + struct mgp_type ** result +) +``` + +Get the type representing a 3D point. + +Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate the new type. + + +### mgp_type_enum [#function-mgp-type-enum] +```cpp +enum mgp_error mgp_type_enum( + struct mgp_type ** result +) +``` + +Get the type representing an enum value. + +Use this when declaring procedure parameters that accept enum values. Enum values can be received as input and inspected (type name, value name), but cannot be returned from procedures. + +Return `MGP_ERROR_UNABLE_TO_ALLOCATE` if unable to allocate the new type. + + ### mgp_type_nullable [#function-mgp-type-nullable] ```cpp enum mgp_error mgp_type_nullable( @@ -4240,7 +4742,10 @@ enum mgp_value_type { MGP_VALUE_TYPE_LOCAL_TIME, MGP_VALUE_TYPE_LOCAL_DATE_TIME, MGP_VALUE_TYPE_DURATION, - MGP_VALUE_TYPE_ZONED_DATE_TIME + MGP_VALUE_TYPE_ZONED_DATE_TIME, + MGP_VALUE_TYPE_POINT_2D, + MGP_VALUE_TYPE_POINT_3D, + MGP_VALUE_TYPE_ENUM }; void mgp_value_destroy(struct mgp_value *val); diff --git a/pages/custom-query-modules/cpp/cpp-api.md b/pages/custom-query-modules/cpp/cpp-api.md index ea19b0c09..a0c49e801 100644 --- a/pages/custom-query-modules/cpp/cpp-api.md +++ b/pages/custom-query-modules/cpp/cpp-api.md @@ -338,6 +338,18 @@ Inserts a value of given type under field `field_name`. void Insert(const char *field_name, const Duration value) ``` +```cpp + void Insert(const char *field_name, const Point2d &value) +``` + +```cpp + void Insert(const char *field_name, const Point3d &value) +``` + +```cpp + void Insert(const char *field_name, const Enum &value) +``` + ```cpp void Insert(const char *field_name, const Value &value) ``` @@ -431,6 +443,18 @@ Sets a return value of given type. void SetValue(const Duration &value) ``` +```cpp + void SetValue(const Point2d &value) +``` + +```cpp + void SetValue(const Point3d &value) +``` + +```cpp + void SetValue(const Enum &value) +``` + ##### SetErrorMessage Sets the given error message. @@ -1801,6 +1825,252 @@ Object is hashable using std::hash ``` +### Point2d + +Represents a 2D point with x and y coordinates and a spatial reference identifier (SRID). + +#### Constructors + +Creates a Point2d object from the copy of the given `mgp_point_2d`. + +```cpp +explicit Point2d(mgp_point_2d *ptr) +explicit Point2d(const mgp_point_2d *const_ptr) +``` + +Creates a Point2d object with the given `x`, `y` coordinates and `srid`. + +```cpp +Point2d(double x, double y, uint16_t srid) +``` + +Supported SRIDs: 4326 (WGS-84), 7203 (Cartesian 2D). + +Copy and move constructors: + +```cpp +Point2d(const Point2d &other) +Point2d(Point2d &&other) noexcept +``` + +#### Member functions + +| Name | Description | +| ------- | ------------------------------------------------- | +| `X` | Returns the x coordinate. | +| `Y` | Returns the y coordinate. | +| `Srid` | Returns the spatial reference identifier (SRID). | +| `ToString` | Returns the point's string representation. | + +##### X + +Returns the x coordinate of the point. + +```cpp +double X() const +``` + +##### Y + +Returns the y coordinate of the point. + +```cpp +double Y() const +``` + +##### Srid + +Returns the SRID (spatial reference identifier) of the point. + +```cpp +uint16_t Srid() const +``` + +##### ToString + +Returns the point's string representation. + +```cpp +std::string ToString() const +``` + +#### Operators + +| Name | Description | +| ------------ | -------------------- | +| `operator==` | comparison operator | +| `operator!=` | comparison operator | + +Object is hashable using + +```cpp +std::hash +``` + +### Point3d + +Represents a 3D point with x, y, and z coordinates and a spatial reference identifier (SRID). + +#### Constructors + +Creates a Point3d object from the copy of the given `mgp_point_3d`. + +```cpp +explicit Point3d(mgp_point_3d *ptr) +explicit Point3d(const mgp_point_3d *const_ptr) +``` + +Creates a Point3d object with the given `x`, `y`, `z` coordinates and `srid`. + +```cpp +Point3d(double x, double y, double z, uint16_t srid) +``` + +Supported SRIDs: 4979 (WGS-84 3D), 9157 (Cartesian 3D). + +Copy and move constructors: + +```cpp +Point3d(const Point3d &other) +Point3d(Point3d &&other) noexcept +``` + +#### Member functions + +| Name | Description | +| ------- | ------------------------------------------------- | +| `X` | Returns the x coordinate. | +| `Y` | Returns the y coordinate. | +| `Z` | Returns the z coordinate. | +| `Srid` | Returns the spatial reference identifier (SRID). | +| `ToString` | Returns the point's string representation. | + +##### X + +Returns the x coordinate of the point. + +```cpp +double X() const +``` + +##### Y + +Returns the y coordinate of the point. + +```cpp +double Y() const +``` + +##### Z + +Returns the z coordinate of the point. + +```cpp +double Z() const +``` + +##### Srid + +Returns the SRID (spatial reference identifier) of the point. + +```cpp +uint16_t Srid() const +``` + +##### ToString + +Returns the point's string representation. + +```cpp +std::string ToString() const +``` + +#### Operators + +| Name | Description | +| ------------ | -------------------- | +| `operator==` | comparison operator | +| `operator!=` | comparison operator | + +Object is hashable using + +```cpp +std::hash +``` + +### Enum + +Represents an enum value with a type name and value name. + +> **Note:** Enum values can be received as procedure input and inspected (type name, value name, equality), but cannot be returned from procedures. + +#### Constructors + +Creates an Enum object from the copy of the given `mgp_enum`. + +```cpp +explicit Enum(mgp_enum *ptr) +explicit Enum(const mgp_enum *const_ptr) +``` + +Creates an Enum from type name and value name strings. + +```cpp +Enum(std::string_view type_name, std::string_view value_name) +``` + +Copy and move constructors: + +```cpp +Enum(const Enum &other) +Enum(Enum &&other) noexcept +``` + +#### Member functions + +| Name | Description | +| ----------- | ------------------------------------------------ | +| `TypeName` | Returns the type name of the enum. | +| `ValueName` | Returns the value name of the enum. | +| `ToString` | Returns the enum's string representation. | + +##### TypeName + +Returns the type name of the enum (e.g. `"Status"` for `Status.Active`). + +```cpp +std::string_view TypeName() const +``` + +##### ValueName + +Returns the value name of the enum (e.g. `"Active"` for `Status.Active`). + +```cpp +std::string_view ValueName() const +``` + +##### ToString + +Returns the enum's string representation. + +```cpp +std::string ToString() const +``` + +#### Operators + +| Name | Description | +| ------------ | -------------------- | +| `operator==` | comparison operator | +| `operator!=` | comparison operator | + +Object is hashable using + +```cpp +std::hash +``` + ### Path A path is a data structure consisting of alternating nodes and relationships, with the start @@ -2313,6 +2583,22 @@ explicit Value(const Duration &value) explicit Value(Duration &&value) ``` +Spatial type constructors: + +```cpp +explicit Value(const Point2d &value) +explicit Value(Point2d &&value) +explicit Value(const Point3d &value) +explicit Value(Point3d &&value) +``` + +Enum type constructors: + +```cpp +explicit Value(const Enum &value) +explicit Value(Enum &&value) +``` + Copy and move constructors: ```cpp @@ -2428,6 +2714,21 @@ Duration ValueDuration() const Duration ValueDuration() ``` +```cpp +Point2d ValuePoint2d() const +Point2d ValuePoint2d() +``` + +```cpp +Point3d ValuePoint3d() const +Point3d ValuePoint3d() +``` + +```cpp +Enum ValueEnum() const +Enum ValueEnum() +``` + ##### Is[TYPE] Returns whether the value stored in the `Value` object is of the type in the call. @@ -2496,6 +2797,18 @@ bool IsZonedDateTime() const bool IsDuration() const ``` +```cpp +bool IsPoint2d() const +``` + +```cpp +bool IsPoint3d() const +``` + +```cpp +bool IsEnum() const +``` + ##### ToString Returns the value's string representation. It does this by finding the type of the object wrapped inside the Value object, calling its ToString() function or casting the object to string, depending on it's type. The table below shows the appropriate action for each type. @@ -2568,6 +2881,9 @@ The types are listed and described [in the reference guide](/fundamentals/data-t - `Type::LocalDateTime` - `Type::ZonedDateTime` - `Type::Duration` +- `Type::Point2d` +- `Type::Point3d` +- `Type::Enum` Additionally, operator<< is overloaded for Type enum, and usage of this operator will print the type represented by mgp::Type enum. diff --git a/pages/custom-query-modules/python/python-api.mdx b/pages/custom-query-modules/python/python-api.mdx index 2a8d90186..aea6c6c3e 100644 --- a/pages/custom-query-modules/python/python-api.mdx +++ b/pages/custom-query-modules/python/python-api.mdx @@ -1155,6 +1155,104 @@ def __init__(**kwargs) Initialize with name=value fields in kwargs. +## Point2d + +```python +class Point2d() +``` + +Type annotation marker for 2D spatial point values in procedure signatures. + +At runtime, 2D points are represented as objects with `x`, `y`, and `srid` properties. +This class is used solely in type annotations to declare that a procedure parameter or +return field expects a 2D point value. + +Supported SRIDs: 4326 (WGS-84), 7203 (Cartesian 2D). + +**Example usage** + +```python +import mgp + +@mgp.read_proc +def my_proc(ctx: mgp.ProcCtx, location: mgp.Point2d) -> mgp.Record(result=mgp.Point2d): + return mgp.Record(result=location) +``` + +## Point3d + +```python +class Point3d() +``` + +Type annotation marker for 3D spatial point values in procedure signatures. + +At runtime, 3D points are represented as objects with `x`, `y`, `z`, and `srid` properties. +This class is used solely in type annotations to declare that a procedure parameter or +return field expects a 3D point value. + +Supported SRIDs: 4979 (WGS-84 3D), 9157 (Cartesian 3D). + +**Example usage** + +```python +import mgp + +@mgp.read_proc +def my_proc(ctx: mgp.ProcCtx, location: mgp.Point3d) -> mgp.Record(result=mgp.Point3d): + return mgp.Record(result=location) +``` + +## ZonedDateTime + +```python +class ZonedDateTime() +``` + +Type annotation marker for zoned date-time values in procedure signatures. + +At runtime, zoned date-time values are represented as `datetime.datetime` objects with +`tzinfo` set. This class exists so that procedure signatures can distinguish zoned +date-times from local date-times (which are also `datetime.datetime` but without `tzinfo`). + +**Example usage** + +```python +import mgp + +@mgp.read_proc +def my_proc(ctx: mgp.ProcCtx, ts: mgp.ZonedDateTime) -> mgp.Record(result=mgp.ZonedDateTime): + return mgp.Record(result=ts) +``` + +## Enum + +```python +class Enum() +``` + +Represents an enum value with a type name and value name. + +Enum values can be received as procedure input and inspected, but cannot be returned +from procedures. + +| Property | Description | +| ------------ | -------------------------------------------------------------- | +| `type_name` | Returns the type name of the enum (e.g. `"Status"`). | +| `value_name` | Returns the value name of the enum (e.g. `"Active"`). | + +Enum objects support equality comparison and are hashable. + +**Example usage** + +```python +import mgp + +@mgp.read_proc +def inspect_enum(ctx: mgp.ProcCtx, val: mgp.Enum) -> mgp.Record(type_name=str, value_name=str): + return mgp.Record(type_name=val.type_name, value_name=val.value_name) +``` + ## Vertices Objects ```python diff --git a/pages/database-management/configuration.mdx b/pages/database-management/configuration.mdx index 795fa825d..70a005dd3 100644 --- a/pages/database-management/configuration.mdx +++ b/pages/database-management/configuration.mdx @@ -19,6 +19,15 @@ and key-value pairs that can be modified to suit your specific needs. Each configuration setting is in the form: `--setting-name=value`. + + +Boolean flags **must** use `=` syntax: `--flag=true` or `--flag=false`. Writing +`--flag false` (space-separated) does **not** work as expected — it sets the flag +to `true` and treats `false` as an unrecognized argument. You can also use +`--flag` to enable or `--noflag` to disable. + + + You can check the current configuration by using the following query: ```opencypher @@ -424,8 +433,8 @@ This section contains the list of flags that are used to configure highly availa | `--coordinator-id` | Raft server id on coordinator instance. | `[int32]` | | `--coordinator-port` | Raft server's port on coordinator instance. | `[uint32]` | | `--management-port` | Port on which replication instances receive messages from coordinator . | `[uint32]` | -| `--instance-health-check-frequency-sec=1` | The interval between two health checks that coordinator does on replication instances. | `[uint32]` | -| `--instance-down-timeout-sec=5 | Number of seconds that need to pass before replication instance is considered down. Must be greater or equal to the `--instance-health-check-frequency-sec`. | `[uint32]` | +| ~~`--instance-health-check-frequency-sec`~~ | **Deprecated in 3.10.** This flag is ignored. Use `SET COORDINATOR SETTING 'instance_health_check_frequency_sec' TO ''` instead. See [Coordinator runtime settings](/clustering/high-availability/best-practices#coordinator-runtime-settings). | `[uint32]` | +| ~~`--instance-down-timeout-sec`~~ | **Deprecated in 3.10.** This flag is ignored. Use `SET COORDINATOR SETTING 'instance_down_timeout_sec' TO ''` instead. See [Coordinator runtime settings](/clustering/high-availability/best-practices#coordinator-runtime-settings). | `[uint32]` | | `--nuraft-log-file` | Path to the file where NuRaft logs are saved. | `[string]` | | `--coordinator-hostname` | Coordinator's instance hostname. Used only in `SHOW INSTANCES` query. | `[string]` | @@ -516,6 +525,7 @@ This section contains the list of all other relevant flags used within Memgraph. | `--allow-load-csv=true` | Controls whether LOAD CSV clause is allowed in queries. | `[bool]` | | `--also-log-to-stderr=false` | Log messages go to stderr in addition to logfiles. | `[bool]` | | `--data-directory=/var/lib/memgraph` | Path to directory in which to save all permanent data. | `[string]` | +| `--data-dir-lock-acquisition-timeout-sec=30` | Timeout (in seconds) for retrying the acquisition of a file lock on the data directory during startup. When a Memgraph instance starts, it acquires a file lock on its data directory. If another process still holds the lock (e.g., a previous instance hasn't fully shut down, or lock release is delayed on a distributed storage system like CephFS), Memgraph will retry acquiring the lock until this timeout is reached. Set to `0` to fail immediately without retrying. | `[uint32]` | | `--data-recovery-on-startup=true` | Facilitates recovery of one or more individual databases and their contents during startup. Replaces `--storage-recover-on-startup` | `[bool]` | | `--debug-query-plans=false` | Enable DEBUG logging of potential query plans. | `[string]` | | `--delta-chain-cache-threshold=128` | The minimum number of deltas worth caching when rebuilding a certain object's state. Useful when executing parallel transactions dependent on changes of a frequently changed graph object, to lower CPU usage. Must be a positive non-zero integer. | `[uint64]` | @@ -523,8 +533,8 @@ This section contains the list of all other relevant flags used within Memgraph. | `--flag-file` | Path to the additional configuration file, overrides the default configuration settings. | `[string]` | | `--help` | Show help on all flags and exit. The default values is `false`. | `[bool]` | | `--help-xml` | Produce an XML version of help and exit. The default values is `false`. | `[bool]` | -| `--init-file` | Path to the CYPHERL file which contains queries that need to be executed before the Bolt server starts, such as creating users. | `[string]` | -| `--init-data-file` | Path to the CYPHERL file, which contains queries that need to be executed after the Bolt server starts. | `[string]` | +| `--init-file` | Path to the CYPHERL file which contains queries that need to be executed before the Bolt server starts, such as creating users. Not supported on [coordinator instances](/clustering/high-availability/how-high-availability-works#coordinator-instance-implementation) or on [data instances running in HA mode](/clustering/high-availability/how-high-availability-works#data-instance-implementation). | `[string]` | +| `--init-data-file` | Path to the CYPHERL file, which contains queries that need to be executed after the Bolt server starts. Not supported on [coordinator instances](/clustering/high-availability/how-high-availability-works#coordinator-instance-implementation) or on [data instances running in HA mode](/clustering/high-availability/how-high-availability-works#data-instance-implementation). | `[string]` | | `--isolation-level=SNAPSHOT_ISOLATION` | Isolation level used for the transactions. Allowed values: SNAPSHOT_ISOLATION, READ_COMMITTED, READ_UNCOMMITTED. | `[string]` | | `--log-file=/var/log/memgraph/memgraph.log` | Path to where the log should be stored. If set to an empty string (`--log-file=`), no logs will be saved. | `[string]` | | `--log-level=WARNING` | Minimum log level. Allowed values: TRACE, DEBUG, INFO, WARNING, ERROR, CRITICAL. | `[string]` | @@ -540,6 +550,7 @@ This section contains the list of all other relevant flags used within Memgraph. | `--replication-replica-check-frequency-sec` | The time duration in seconds between two replica checks/pings. If < 1, replicas will not be checked at all and the replica will never be recovered. The MAIN instance allocates a new thread for each REPLICA. | `[uint64]` | | `--replication-restore-state-on-startup=true` | Set to `true` when initializing an instance to restore the replication role and configuration upon restart. | `[bool]` | | `--schema-info-enabled=false` | Set to `true` to enable run-time schema info tracking. | `[bool]` | +| `--strict-flag-check=false` | If `true`, Memgraph will error and exit when suspicious positional arguments are detected (e.g. `--bool-flag false` instead of `--bool-flag=false`). If `false`, a warning is logged instead. | `[bool]` | | `--telemetry-enabled=true` | Set to true to enable telemetry. We collect information about the running system (CPU and memory information), information about the database runtime (vertex and edge counts and resource usage), and aggregated statistics about some features of the database (e.g. how many times a feature is used) to allow for an easier improvement of the product. | `[bool]` | ### Environment variables @@ -681,6 +692,12 @@ If an exception occurs during the execution of init script the queries will cont + + +The `--init-file` and `--init-data-file` flags are **not supported on coordinator instances** in a [high availability](/clustering/high-availability) setup, and are also **not supported on data instances running in HA mode** (i.e. when `--management-port` is set). The instance will fail to start if either flag is provided. To bootstrap users or data in an HA cluster, run the queries through the MAIN after the cluster is formed. + + + ### Use the `init-file` flag with Docker diff --git a/pages/database-management/ssl-encryption.mdx b/pages/database-management/ssl-encryption.mdx index a03a417a6..780b62c8e 100644 --- a/pages/database-management/ssl-encryption.mdx +++ b/pages/database-management/ssl-encryption.mdx @@ -3,6 +3,7 @@ title: SSL encryption description: Learn how to enable SSL encryption to secure data transmission and protect sensitive information. More security features are at your disposal in our documentation page. --- +import { Callout } from "nextra/components"; import { Tabs } from "nextra/components"; import { Steps } from "nextra/components"; @@ -145,6 +146,42 @@ WebSocket over SSL is currently not supported in Memgraph. +## Reload SSL certificates at runtime + +You can rotate SSL certificates without restarting Memgraph by using the +`RELOAD BOLT_SERVER TLS` Cypher command. This is useful in production +environments where certificate rotation is required (e.g., Let's Encrypt +renewals or compliance requirements) and downtime is not acceptable. + +To reload SSL certificates: + +1. Replace the certificate and key files on disk (at the paths originally + configured with `--bolt-cert-file` and `--bolt-key-file`). +2. Run the following command from any connected client: + +```cypher +RELOAD BOLT_SERVER TLS; +``` + +After a successful reload: +- **New connections** will use the updated certificate. +- **Existing connections** continue using the previous certificate until they + disconnect. + +If the reload fails (e.g., due to an invalid certificate or missing file), the +existing SSL configuration remains active and an error is returned. The server +continues to operate normally. + + +The `RELOAD BOLT_SERVER TLS` command cannot be executed inside an explicit +(multi-command) transaction. + + + +Running `RELOAD BOLT_SERVER TLS` on a Memgraph instance that was started +without SSL enabled will return an error. + + ## How to set up SSL encryption Memgraph uses SSL (Secure Sockets Layer) protocol for establishing an diff --git a/pages/fundamentals/indexes.mdx b/pages/fundamentals/indexes.mdx index 72079059b..63e5c9d58 100644 --- a/pages/fundamentals/indexes.mdx +++ b/pages/fundamentals/indexes.mdx @@ -377,6 +377,106 @@ DROP INDEX ON :Label(property1, property2); ``` +### Descending label-property index + +By default, a label-property index — single-property or composite — stores +entries in ascending order of the indexed properties. Queries that use +`ORDER BY … ASC` can then be served directly from the index, skipping the +sort step. If your workload frequently sorts in descending order (for example, +timestamps, scores, or any "newest/largest first" query), you can create a +descending index using the `WITH CONFIG` clause: + +```cypher +CREATE INDEX ON :Person(age) WITH CONFIG {"order": "DESC"}; +``` + +A descending index is used to eliminate the sort step for queries that order +the indexed property in descending order: + +```cypher +MATCH (n:Person) WHERE n.age > 30 RETURN n ORDER BY n.age DESC; +``` + +The benefit is especially large when the query also uses `LIMIT`. Without a +matching index, Memgraph must read all candidate rows, sort them, and then +return only the top *N*. With a matching descending index, rows are already +produced in the desired order, so execution can stop as soon as the first +*N* valid entries are yielded — a "top-N" query becomes O(*N*) index reads +instead of O(*all matches*) reads plus a sort: + +```cypher +MATCH (n:Person) RETURN n ORDER BY n.age DESC LIMIT 10; +``` + +Without the explicit `{"order": "DESC"}` config, the index defaults to +ascending order — both of the following are equivalent: + +```cypher +CREATE INDEX ON :Person(age); +CREATE INDEX ON :Person(age) WITH CONFIG {"order": "ASC"}; +``` + +#### Descending composite index + +The `{"order": "DESC"}` configuration applies equally to composite indices. In +that case, every column in the index is ordered in descending direction: + +```cypher +CREATE INDEX ON :Person(name, age) WITH CONFIG {"order": "DESC"}; +``` + +A descending composite index can eliminate the sort for queries where *all* +columns are ordered in the same (descending) direction: + +```cypher +MATCH (p:Person) WHERE p.name > '' RETURN p ORDER BY p.name DESC, p.age DESC; +``` + +Mixed-direction `ORDER BY` clauses (for example `ORDER BY p.name ASC, p.age +DESC`) are not eliminated by the index; use a regular `OrderBy` for those. + +#### Coexistence with ascending indices + +Ascending and descending indices on the same label and properties are +independent and can coexist. When both exist, Memgraph automatically picks the +one whose direction matches the query's `ORDER BY`: + +```cypher +CREATE INDEX ON :Person(age); +CREATE INDEX ON :Person(age) WITH CONFIG {"order": "DESC"}; + +MATCH (n:Person) WHERE n.age > 30 RETURN n ORDER BY n.age; // uses ASC index +MATCH (n:Person) WHERE n.age > 30 RETURN n ORDER BY n.age DESC; // uses DESC index +``` + +In `SHOW INDEX INFO`, descending indices are listed as `label+property (DESC)` +to distinguish them from ascending ones. + +#### Dropping descending indices + +`DROP INDEX ON :Label(property)` without a config drops both the ascending +and descending index (if they both exist) in a single operation. To drop only +one of them, pass the matching config: + +```cypher +DROP INDEX ON :Person(age) WITH CONFIG {"order": "DESC"}; +DROP INDEX ON :Person(age) WITH CONFIG {"order": "ASC"}; +``` + +The same applies to composite indices: + +```cypher +DROP INDEX ON :Person(name, age) WITH CONFIG {"order": "DESC"}; +``` + + + +Descending indices are currently supported only in the `IN_MEMORY_TRANSACTIONAL` +storage mode. Creating a descending index in `ON_DISK_TRANSACTIONAL` mode is +not yet supported. + + + ### Edge-type index diff --git a/pages/fundamentals/telemetry.mdx b/pages/fundamentals/telemetry.mdx index 337ea3a07..86d3dae84 100644 --- a/pages/fundamentals/telemetry.mdx +++ b/pages/fundamentals/telemetry.mdx @@ -64,7 +64,7 @@ available, the following data will be sent to and stored on Memgraph's servers. **High availability cluster information:** - The number of strict sync, sync and asynchronous replicas (retrieved from the current main). - The number of coordinators in the cluster. - - Configuration options: `instance_down_timeout_sec`, `instance_health_check_frequency_sec`, `enabled_reads_on_main`, `sync_failover_only`. + - Coordinator runtime settings: `instance_down_timeout_sec`, `instance_health_check_frequency_sec`, `enabled_reads_on_main`, `sync_failover_only`. **Running environment:** - Whether Memgraph is running in K8s or somewhere else. diff --git a/pages/getting-started/build-memgraph-from-source.mdx b/pages/getting-started/build-memgraph-from-source.mdx index e4ba7373b..e8e860884 100644 --- a/pages/getting-started/build-memgraph-from-source.mdx +++ b/pages/getting-started/build-memgraph-from-source.mdx @@ -203,9 +203,10 @@ To build Memgraph using this method, follow these steps: ```bash python3 -m venv env source env/bin/activate -pip install conan +pip install "conan>=2.26.0" conan profile detect conan config install conan_config +conan remote add memgraph-recipes "$(pwd)/conan_recipes" -t local-recipes-index --force ``` @@ -213,17 +214,9 @@ conan config install conan_config and build Memgraph. - `conan_config` is a subdirectory of the Memgraph repository which contains specific configuration files that tell `conan` how to build Memgraph and the libraries that link to it. - - -

Run the init script to fetch other libs required for the build

- -```bash -./init -``` - - - The `init` script fetches other libraries required for the build which are not yet provided - by `conan`. + - `conan_recipes` contains vendored Conan recipes with Memgraph-specific patches. Registering it + as a `local-recipes-index` remote allows `conan install --build=missing` to resolve them + without manual `conan export`.

Install conan dependencies

@@ -232,7 +225,7 @@ conan config install conan_config export MG_TOOLCHAIN_ROOT=/opt/toolchain-v7 conan install . \ --build=missing \ - -pr:h memgraph_template_profile \ + -pr:h memgraph_toolchain_v7 \ -pr:b memgraph_build_profile \ -s build_type=Release source build/generators/conanbuild.sh @@ -371,19 +364,19 @@ When build errors occur, there are some common issues that can be resolved by th 1. Remove the `build` directory and run the build again. -2. Re-run `./init`: Sometimes libraries that are configures by this script will be -upgraded/added/removed in the `master` branch, potentially causing future build errors. - -3. Re-run `conan install`: for simlar reasons as the previous step - libraries used in the build may -have been changed in `conanfile.py`. Unexpected linking errors can be caused by this. +2. Re-run `conan install`: libraries used in the build may have been changed in +`conanfile.py`. Unexpected linking errors can be caused by this. -4. Reinstall `conan_config`. As list of supported Linux distributions and architectures evolves, -and the libraries used in the build may change, the configuration may need to be updated to account -for specific build issues on some platforms. +3. Reinstall `conan_config`. As the list of supported Linux distributions and +architectures evolves, and the libraries used in the build may change, the +configuration may need to be updated to account for specific build issues on +some platforms. -5. Reinstall host dependencies, as these may sometimes change. +4. Reinstall host dependencies, as these may sometimes change. -6. Renaming or removing the conan cache directory (usually `~/.conan2`) in order to rebuild the -libraries and build tools from scratch can help to resolve build issues. +5. Renaming or removing the conan cache directory (usually `~/.conan2`) in order +to rebuild the libraries and build tools from scratch can help to resolve build +issues. -7. Open an issue on our [GitHub repository](https://github.com/memgraph/memgraph/issues) with the error message. +6. Open an issue on our [GitHub +repository](https://github.com/memgraph/memgraph/issues) with the error message. diff --git a/pages/getting-started/install-memgraph/kubernetes.mdx b/pages/getting-started/install-memgraph/kubernetes.mdx index 9a66c3a5b..c8794e603 100644 --- a/pages/getting-started/install-memgraph/kubernetes.mdx +++ b/pages/getting-started/install-memgraph/kubernetes.mdx @@ -265,6 +265,134 @@ kubectl delete crd servicemonitors.monitoring.coreos.com kubectl delete crd thanosrulers.monitoring.coreos.com ``` +#### Remote metrics and logs + +The standalone chart can also ship: + +- metrics to a remote Prometheus-compatible backend via `vmagentRemote` (`remote_write`) +- logs to a Loki-compatible backend via `vectorRemote` + +This is useful when your observability stack lives in a separate cluster or in a managed service. + +Prerequisites: + +- keep `prometheus.enabled: true` so `mg-exporter` is deployed +- for standalone deployments, enable Memgraph monitoring endpoints: + - `service.enableHttpMonitoring: true` + - `service.enableWebsocketMonitoring: true` +- when `vectorRemote.enabled: true`, add `--monitoring-port=` and `--monitoring-address=0.0.0.0` to `memgraphConfig` +- if you only need remote shipping and do not want duplicate scraping from kube-prometheus, set `prometheus.serviceMonitor.enabled: false` + +Example `values.yaml`: + +```yaml +prometheus: + enabled: true + namespace: monitoring + serviceMonitor: + enabled: false + +service: + enableHttpMonitoring: true + enableWebsocketMonitoring: true + +memgraphConfig: + - "--data-directory=/var/lib/memgraph/mg_data" + - "--also-log-to-stderr=true" + - "--monitoring-port=7444" + - "--monitoring-address=0.0.0.0" + +vmagentRemote: + enabled: true + namespace: monitoring + remoteWrite: + url: "https:///api/v1/write" + # Optional: only set basicAuth when your remote_write endpoint requires basic auth. + basicAuth: + secretName: monitoring-basic-auth + usernameKey: username + passwordKey: password + externalLabels: + cluster_id: "memgraph-standalone" + service_name: "memgraph" + cluster_env: "dev" + +vectorRemote: + enabled: true + logsEndpoint: "https://" + # Optional: only set auth when your endpoint requires basic auth. + auth: + secretName: monitoring-basic-auth + usernameKey: username + passwordKey: password + extraLabels: + cluster_id: "memgraph-standalone" + service_name: "memgraph" + cluster_env: "dev" + role: "standalone" +``` + +Create credentials secret in the namespace where vmagent runs (usually `monitoring`): + +```bash +kubectl create secret generic monitoring-basic-auth -n monitoring \ + --from-literal=username='' \ + --from-literal=password='' +``` + +For the standalone Vector sidecar, create the same secret in the Memgraph release namespace as well: + +```bash +kubectl create secret generic monitoring-basic-auth -n \ + --from-literal=username='' \ + --from-literal=password='' +``` + +##### Kubernetes infrastructure metrics + +`vmagentRemote` can additionally scrape Kubernetes infrastructure metrics +(`kube-state-metrics`, `node-exporter`, `kubelet`) required by +`kube-prometheus-stack` Kubernetes and Node dashboards, and remote-write them +to your centralized monitoring cluster. + +Enable Kubernetes scraping by extending your existing `vmagentRemote` values: + +```yaml +vmagentRemote: + # ... existing fields (enabled, remoteWrite, externalLabels) ... + kubernetes: + enabled: true + kubeStateMetrics: + enabled: true + jobName: kube-state-metrics + targets: + - kube-prometheus-stack-kube-state-metrics.monitoring.svc.cluster.local:8080 + nodeExporter: + enabled: true + jobName: node-exporter + targets: + - kube-prometheus-stack-prometheus-node-exporter.monitoring.svc.cluster.local:9100 + kubelet: + enabled: true + jobName: kubelet + metricsPath: /metrics/cadvisor + apiServerAddress: kubernetes.default.svc:443 + insecureSkipVerify: false +``` + +Notes: + +- RBAC and `ServiceAccount` resources are created only when an enabled scrape + job requires Kubernetes API access (for example `kubelet.enabled=true` or + `nodeExporter.useKubernetesDiscovery=true`). +- Keep `jobName` values aligned with dashboard and recording-rule expectations + unless you also update those queries. +- Dashboards that rely on precomputed recording-rule series still require + rule evaluation in your monitoring stack. + +A ready-to-use example values file is available in the Helm charts repository: +[`examples/remote-monitoring/values-standalone-k8s-metrics.yaml`](https://github.com/memgraph/helm-charts/blob/main/examples/remote-monitoring/values-standalone-k8s-metrics.yaml). + ### Node affinity The chart exposes the full Kubernetes `nodeAffinity` spec via the `nodeAffinity` @@ -339,111 +467,161 @@ Additional options like `externalTrafficPolicy`, `ipFamilyPolicy`, and ### Configuration options -The following table lists the configurable parameters of the Memgraph chart and their default values. - - -| Parameter | Description | Default | -| ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | ------------------------------ | -| `image.repository` | Memgraph Docker image repository | `docker.io/memgraph/memgraph` | -| `image.tag` | Specific tag for the Memgraph Docker image. Overrides the image tag whose default is chart version. | `3.1.0` | -| `image.pullPolicy` | Image pull policy | `IfNotPresent` | -| `env.MEMGRAPH_ENTERPRISE_LICENSE` | Memgraph enterprise license | `` | -| `env.MEMGRAPH_ORGANIZATION_NAME` | Organization name | `` | -| `memgraphUserId` | The user id that is hardcoded in Memgraph and Mage images | `101` | -| `memgraphGroupId` | The group id that is hardcoded in Memgraph and Mage images | `103` | -| `storage.libPVCSize` | Size of the storage PVC | `1Gi` | -| `storage.libStorageClassName` | The name of the storage class used for storing data. | `""` | -| `storage.libStorageAccessMode` | Access mode used for lib storage. | `ReadWriteOnce` | -| `storage.logPVCSize` | Size of the log PVC | `1Gi` | -| `storage.logStorageClassName` | The name of the storage class used for storing logs. | `""` | -| `storage.logStorageAccessMode` | Access mode used for log storage. | `ReadWriteOnce` | -| `externalAccess.coordinator.serviceType` | IngressNginx, NodePort, CommonLoadBalancer or LoadBalancer. By default, no external service will be created. | `""` | -| `externalAccess.coordinator.annotations` | Annotations for external services attached to coordinators. | `{}` | -| `externalAccess.dataInstance.serviceType` | IngressNginx, NodePort or LoadBalancer. By default, no external service will be created. | `""` | -| `externalAccess.dataInstance.annotations` | Annotations for external services attached to data instances. | `{}` | -| `headlessService.enabled` | Specifies whether headless services will be used inside K8s network on all instances. | `false` | -| `service.loadBalancer.enabled` | Defines if a Service intended for cluster-external access should be created. | `false` | -| `service.loadBalancer.type` | Kubernetes service type for the external access service. | `LoadBalancer` | -| `service.loadBalancer.enableBolt` | Enable Bolt protocol on the LoadBalancer service. | `true` | -| `service.loadBalancer.enableWebsocketMonitoring` | Enable WebSocket monitoring on the LoadBalancer service. | `false` | -| `service.loadBalancer.enableHttpMonitoring` | Enable HTTP monitoring on the LoadBalancer service. | `false` | -| `service.loadBalancer.annotations` | Annotations to add to the LoadBalancer service. | `{}` | -| `service.loadBalancer.labels` | Labels to add to the LoadBalancer service. | `{}` | -| `service.loadBalancer.externalTrafficPolicy` | Denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. | `null` | -| `service.loadBalancer.ipFamilyPolicy` | Represents the dual-stack-ness requested or required by this Service. | `null` | -| `service.loadBalancer.ipFamilies` | List of IP families (e.g. IPv4, IPv6) assigned to this service. | `null` | -| `service.loadBalancer.loadBalancerSourceRanges` | Restricts traffic through the cloud-provider load-balancer to the specified client IPs. | `null` | -| `ports.boltPort` | Bolt port used on coordinator and data instances. | `7687` | -| `ports.managementPort` | Management port used on coordinator and data instances. | `10000` | -| `ports.replicationPort` | Replication port used on data instances. | `20000` | -| `ports.coordinatorPort` | Coordinator port used on coordinators. | `12000` | -| `nodeAffinity` | Allows constraining on which nodes Pods can be scheduled on based on node labels | `{}` | -| `priorityClassName` | Defines which [priority class](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/) should be assigned to the Memgraph pods | `null` | -| `container.data.livenessProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | -| `container.data.livenessProbe.failureThreshold` | Failure threshold for liveness probe | `20` | -| `container.data.livenessProbe.timeoutSeconds` | Timeout for liveness probe | `10` | -| `container.data.livenessProbe.periodSeconds` | Period seconds for readiness probe | `5` | -| `container.data.readinessProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | -| `container.data.readinessProbe.failureThreshold` | Failure threshold for readiness probe | `20` | -| `container.data.readinessProbe.timeoutSeconds` | Timeout for readiness probe | `10` | -| `container.data.readinessProbe.periodSeconds` | Period seconds for readiness probe | `5` | -| `container.data.startupProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | -| `container.data.startupProbe.failureThreshold` | Failure threshold for startup probe | `1440` | -| `container.data.startupProbe.timeoutSeconds` | Timeout for probe | `10` | -| `container.data.startupProbe.periodSeconds` | Period seconds for startup probe | `10` | -| `container.data.terminationGracePeriodSeconds` | Grace period for data pod termination | `1800` | -| `container.coordinators.livenessProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `12000` | -| `container.coordinators.livenessProbe.failureThreshold` | Failure threshold for liveness probe | `20` | -| `container.coordinators.livenessProbe.timeoutSeconds` | Timeout for liveness probe | `10` | -| `container.coordinators.livenessProbe.periodSeconds` | Period seconds for readiness probe | `5` | -| `container.coordinators.readinessProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `12000` | -| `container.coordinators.readinessProbe.failureThreshold` | Failure threshold for readiness probe | `20` | -| `container.coordinators.readinessProbe.timeoutSeconds` | Timeout for readiness probe | `10` | -| `container.coordinators.readinessProbe.periodSeconds` | Period seconds for readiness probe | `5` | -| `container.coordinators.startupProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `12000` | -| `container.coordinators.startupProbe.failureThreshold` | Failure threshold for startup probe | `1440` | -| `container.coordinators.startupProbe.timeoutSeconds` | Timeout for probe | `10` | -| `container.coordinators.startupProbe.periodSeconds` | Period seconds for startup probe | `10` | -| `container.coordinators.terminationGracePeriodSeconds` | Grace period for coordinators pod termination | `1800` | -| `data` | Configuration for data instances | See `data` section | -| `coordinators` | Configuration for coordinator instances | See `coordinators` section | -| `sysctlInitContainer.enabled` | Enable the init container to set sysctl parameters | `true` | -| `sysctlInitContainer.maxMapCount` | Value for `vm.max_map_count` to be set by the init container | `262144` | -| `secrets.enabled` | Enable the use of Kubernetes secrets for Memgraph credentials | `false` | -| `secrets.name` | The name of the Kubernetes secret containing Memgraph credentials | `memgraph-secrets` | -| `secrets.userKey` | The key in the Kubernetes secret for the Memgraph user, the value is passed to the `MEMGRAPH_USER` env. | `USER` | -| `secrets.passwordKey` | The key in the Kubernetes secret for the Memgraph password, the value is passed to the `MEMGRAPH_PASSWORD`. | `PASSWORD` | -| `resources.coordinators` | CPU/Memory resource requests/limits. Left empty by default. | `{}` | -| `resources.data` | CPU/Memory resource requests/limits. Left empty by default. | `{}` | -| `prometheus.enabled` | If set to `true`, K8s resources representing Memgraph's Prometheus exporter will be deployed. | `false` | -| `prometheus.namespace` | The namespace in which `kube-prometheus-stack` and Memgraph's Prometheus exporter are installed. | `monitoring` | -| `prometheus.memgraphExporter.port` | The port on which Memgraph's Prometheus exporter is available. | `9115` | -| `prometheus.memgraphExporter.pullFrequencySeconds` | How often will Memgraph's Prometheus exporter pull data from Memgraph instances. | `5` | -| `prometheus.memgraphExporter.repository` | The repository where Memgraph's Prometheus exporter image is available. | `docker.io/memgraph/prometheus-exporter` | -| `prometheus.memgraphExporter.tag` | The tag of Memgraph's Prometheus exporter image. | `0.2.1` | -| `prometheus.serviceMonitor.enabled` | If enabled, a `ServiceMonitor` object will be deployed. | `true` | -| `prometheus.serviceMonitor.kubePrometheusStackReleaseName` | The release name under which `kube-prometheus-stack` chart is installed. | `kube-prometheus-stack` | -| `prometheus.serviceMonitor.interval` | How often will Prometheus pull data from Memgraph's Prometheus exporter. | `15s` | -| `labels.coordinators.podLabels` | Enables you to set labels on a pod level. | `{}` | -| `labels.coordinators.statefulSetLabels` | Enables you to set labels on a stateful set level. | `{}` | -| `labels.coordinators.serviceLabels` | Enables you to set labels on a service level. | `{}` | -| `updateStrategy.type` | Update strategy for StatefulSets. Possible values are `RollingUpdate` and `OnDelete` | `RollingUpdate` | -| `extraEnv.data` | Env variables that users can define and are applied to data instances | `[]` | -| `extraEnv.coordinators` | Env variables that users can define and are applied to coordinators | `[]` | -| `initContainers.data` | Init containers that users can define that will be applied to data instances. | `[]` | -| `initContainers.coordinators` | Init containers that users can define that will be applied to coordinators. | `[]` | -| `tolerations.coordinators` | Applied to a coordinator pod and allows the pod to be scheduled on nodes with matching taints. | `[]` | -| `tolerations.data` | Applied to a data pod and allows the pod to be scheduled on nodes with matching taints. | `[]` | - - -For the `data` and `coordinators` sections, each item in the list has the following parameters: - -| Parameter | Description | Default | -|---------------------------------------------|-----------------------------------------------------------------------------------------------------|-----------------------------------------| -| `id` | ID of the instance | `0` for data, `1` for coordinators | -| `args` | List of arguments for the instance | See `args` section | - -The `args` section contains a list of arguments for the instance. +The following table lists the configurable parameters of the Memgraph standalone chart and their default values. + + +| Parameter | Description | Default | +| ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | +| `image.repository` | Memgraph Docker image repository. | `docker.io/memgraph/memgraph` | +| `image.tag` | Specific tag for the Memgraph Docker image. Overrides the image tag whose default is chart's app version. | `""` (chart's app version) | +| `image.pullPolicy` | Image pull policy. | `IfNotPresent` | +| `useImagePullSecrets` | Whether to use image pull secrets defined under `imagePullSecrets`. | `false` | +| `imagePullSecrets` | List of image pull secrets. | `[{name: regcred}]` | +| `replicaCount` | Number of Memgraph StatefulSet replicas. | `1` | +| `nodeAffinity` | `spec.affinity.nodeAffinity` field for the Memgraph Pod. | `{}` | +| `priorityClassName` | Indicates the importance of the Pod relative to other Pods. | `null` | +| `nodeSelector` | Node selector for Pod scheduling. | `{}` | +| `tolerations` | Pod tolerations. | `[]` | +| `service.type` | Kubernetes service type (`ClusterIP`, `NodePort`, `LoadBalancer`). | `ClusterIP` | +| `service.enableBolt` | Enable the Bolt port on the main service. | `true` | +| `service.boltPort` | Bolt port. If changed, update ports in probes as well. | `7687` | +| `service.enableWebsocketMonitoring` | Enable the websocket monitoring port on the main service. | `false` | +| `service.websocketPortMonitoring` | Websocket monitoring port. | `7444` | +| `service.enableHttpMonitoring` | Enable the HTTP monitoring port on the main service. | `false` | +| `service.httpPortMonitoring` | HTTP monitoring port. | `9091` | +| `service.annotations` | Annotations to add to the main service. | `{}` | +| `service.labels` | Labels to add to the main service. | `{}` | +| `service.loadBalancer.enabled` | Create an additional Service (default type `LoadBalancer`) for external access using the same ports as the main service. | `false` | +| `service.loadBalancer.type` | Type of the additional external-access service. | `LoadBalancer` | +| `service.loadBalancer.enableBolt` | Expose the Bolt port through the additional service. | `true` | +| `service.loadBalancer.enableWebsocketMonitoring` | Expose the websocket monitoring port through the additional service. | `false` | +| `service.loadBalancer.enableHttpMonitoring` | Expose the HTTP monitoring port through the additional service. | `false` | +| `service.loadBalancer.annotations` | Annotations for the additional service. | `{}` | +| `service.loadBalancer.labels` | Labels for the additional service. | `{}` | +| `service.loadBalancer.externalTrafficPolicy` | `externalTrafficPolicy` for the additional service. | `null` | +| `service.loadBalancer.ipFamilyPolicy` | `ipFamilyPolicy` for the additional service. | `null` | +| `service.loadBalancer.ipFamilies` | `ipFamilies` for the additional service. | `null` | +| `service.loadBalancer.loadBalancerSourceRanges` | Restrict traffic to the specified client IP ranges (only applies if the cloud provider supports it). | `null` | +| `persistentVolumeClaim.createStorageClaim` | Create a PVC for data storage. If `false`, use `existingClaim` or `storageVolumeName`. | `true` | +| `persistentVolumeClaim.storageClassName` | Storage class for the data PVC. Use a `Retain` reclaim policy to preserve data when the release is deleted. | `local-path` | +| `persistentVolumeClaim.storageSize` | Size of the data PVC. Must be at least 4x the maximum dataset size to accommodate snapshots and WAL files. | `10Gi` | +| `persistentVolumeClaim.libStorageAccessMode` | Access mode for the data PVC. | `ReadWriteOnce` | +| `persistentVolumeClaim.existingClaim` | Name of an existing PVC to use when `createStorageClaim` is `false`. | `memgraph-0` | +| `persistentVolumeClaim.storageVolumeName` | Existing volume to back the PVC. | `""` | +| `persistentVolumeClaim.createLogStorage` | Create a PVC for logs. If `false`, logs are only written to stdout/stderr. | `true` | +| `persistentVolumeClaim.logStorageClassName` | Storage class for the log PVC. | `local-path` | +| `persistentVolumeClaim.logStorageSize` | Size of the log PVC. | `1Gi` | +| `persistentVolumeClaim.createUserClaim` | Create a dynamic PVC for user files (configs, certificates, etc.). | `false` | +| `persistentVolumeClaim.userStorageClassName` | Storage class for the user PVC. | `""` | +| `persistentVolumeClaim.userStorageSize` | Size of the user PVC. | `1Gi` | +| `persistentVolumeClaim.userStorageAccessMode` | Access mode for the user PVC. | `ReadWriteOnce` | +| `persistentVolumeClaim.userMountPath` | Mount path for the user PVC inside the container. | `""` | +| `persistentVolumeClaim.createCoreDumpsClaim` | Create a PVC for core dumps. | `false` | +| `persistentVolumeClaim.coreDumpsStorageClassName` | Storage class for the core dumps PVC. | `""` | +| `persistentVolumeClaim.coreDumpsStorageSize` | Size of the core dumps PVC. | `10Gi` | +| `persistentVolumeClaim.coreDumpsMountPath` | Mount path for the core dumps PVC. | `/var/core/memgraph` | +| `storageClass.create` | Create a new StorageClass for data and logs. | `false` | +| `storageClass.name` | Name of the created StorageClass. | `memgraph-generic-storage-class` | +| `storageClass.provisioner` | StorageClass provisioner (change for production: AWS `ebs.csi.aws.com`, GCP `pd.csi.storage.gke.io`, Azure `disk.csi.azure.com`). | `k8s.io/minikube-hostpath` | +| `storageClass.storageType` | StorageClass storage type (e.g. `gp2`, `pd-standard`, `StandardSSD_LRS`). | `hostPath` | +| `storageClass.fsType` | Filesystem type for the StorageClass. | `ext4` | +| `storageClass.reclaimPolicy` | Reclaim policy for the StorageClass. | `Retain` | +| `storageClass.volumeBindingMode` | Volume binding mode for the StorageClass. | `Immediate` | +| `memgraphConfig` | List of Memgraph CLI flags passed at startup. See [configuration settings](/database-management/configuration). | `["--data-directory=/var/lib/memgraph/mg_data", "--also-log-to-stderr=true"]` | +| `memgraphUserId` | The user ID hardcoded in Memgraph and MAGE images. | `101` | +| `memgraphGroupId` | The group ID hardcoded in Memgraph and MAGE images. | `103` | +| `secrets.enabled` | Enable the use of Kubernetes secrets for Memgraph credentials. | `false` | +| `secrets.name` | Name of the Kubernetes secret containing Memgraph credentials. | `memgraph-secrets` | +| `secrets.userKey` | Key in the secret whose value is passed to the `MEMGRAPH_USER` environment variable. | `USER` | +| `secrets.passwordKey` | Key in the secret whose value is passed to the `MEMGRAPH_PASSWORD` environment variable. | `PASSWORD` | +| `memgraphEnterpriseLicense` | Memgraph Enterprise license key. | `""` | +| `memgraphOrganizationName` | Organization name associated with the Enterprise license. | `""` | +| `statefulSetAnnotations` | Annotations to add to the StatefulSet. | `{}` | +| `podAnnotations` | Annotations to add to the Pod. | `{}` | +| `extraEnv` | Additional environment variables passed to the Memgraph container. | `[]` | +| `resources` | CPU/Memory resource requests/limits. Left empty by default. | `{}` | +| `serviceAccount.create` | Whether a service account should be created. | `true` | +| `serviceAccount.annotations` | Annotations to add to the service account. | `{}` | +| `serviceAccount.name` | Name of the service account to use. If empty and `create` is `true`, a name is generated. | `""` | +| `container.terminationGracePeriodSeconds` | Grace period for Pod termination. Increase this when `--storage-snapshot-on-exit` is enabled so the on-exit snapshot has time to finish before `SIGKILL`. | `1800` | +| `container.readinessProbe.tcpSocket.port` | Port used for the readiness TCP probe. Keep aligned with `service.boltPort`. | `7687` | +| `container.readinessProbe.failureThreshold` | Failure threshold for the readiness probe. | `20` | +| `container.readinessProbe.timeoutSeconds` | Timeout (seconds) for the readiness probe. | `10` | +| `container.readinessProbe.periodSeconds` | Period (seconds) for the readiness probe. | `5` | +| `container.readinessProbe.initialDelaySeconds` | Initial delay (seconds) for the readiness probe. | `0` | +| `container.livenessProbe.tcpSocket.port` | Port used for the liveness TCP probe. Keep aligned with `service.boltPort`. | `7687` | +| `container.livenessProbe.failureThreshold` | Failure threshold for the liveness probe. | `20` | +| `container.livenessProbe.timeoutSeconds` | Timeout (seconds) for the liveness probe. | `10` | +| `container.livenessProbe.periodSeconds` | Period (seconds) for the liveness probe. | `5` | +| `container.livenessProbe.initialDelaySeconds` | Initial delay (seconds) for the liveness probe. | `0` | +| `container.startupProbe.tcpSocket.port` | Port used for the startup TCP probe. Keep aligned with `service.boltPort`. | `7687` | +| `container.startupProbe.failureThreshold` | Failure threshold for the startup probe. Increase if recovery takes longer than 2 hours. | `1440` | +| `container.startupProbe.timeoutSeconds` | Timeout (seconds) for the startup probe. | `1` | +| `container.startupProbe.periodSeconds` | Period (seconds) for the startup probe. | `5` | +| `container.startupProbe.initialDelaySeconds` | Initial delay (seconds) for the startup probe. | `0` | +| `customQueryModules` | List of custom query modules mounted into the Memgraph container and loaded at startup. Each item requires `volume` (ConfigMap name) and `file`. | `[]` | +| `lifecycleHooks` | Container [lifecycle hooks](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/) for the Memgraph container. | `{}` | +| `extraVolumes` | Additional volumes mounted into the Pod. | `[]` | +| `extraVolumeMounts` | Additional volume mounts for the Memgraph container. | `[]` | +| `sysctlInitContainer.enabled` | Enable the init container that sets `vm.max_map_count`. | `true` | +| `sysctlInitContainer.maxMapCount` | Value to set for `vm.max_map_count`. | `524289` | +| `sysctlInitContainer.image.repository` | Image repository for the sysctl init container. | `docker.io/library/busybox` | +| `sysctlInitContainer.image.tag` | Image tag for the sysctl init container. | `latest` | +| `sysctlInitContainer.image.pullPolicy` | Image pull policy for the sysctl init container. | `IfNotPresent` | +| `initContainers` | Additional init containers to add to the Memgraph Pod. | `[]` | +| `prometheus.enabled` | Deploy Kubernetes resources for Memgraph's Prometheus exporter. | `false` | +| `prometheus.namespace` | Namespace where the exporter and `kube-prometheus-stack` resources are installed. | `monitoring` | +| `prometheus.memgraphExporter.port` | Port on which Memgraph's Prometheus exporter is exposed. | `9115` | +| `prometheus.memgraphExporter.pullFrequencySeconds` | How often the exporter pulls data from Memgraph. | `5` | +| `prometheus.memgraphExporter.repository` | Image repository for Memgraph's Prometheus exporter. | `docker.io/memgraph/prometheus-exporter` | +| `prometheus.memgraphExporter.tag` | Image tag for Memgraph's Prometheus exporter. | `0.2.1` | +| `prometheus.serviceMonitor.enabled` | Deploy a `ServiceMonitor` object for Prometheus scraping. | `true` | +| `prometheus.serviceMonitor.kubePrometheusStackReleaseName` | Release name under which `kube-prometheus-stack` is installed. | `kube-prometheus-stack` | +| `prometheus.serviceMonitor.interval` | How often Prometheus pulls data from Memgraph's exporter. | `15s` | +| `vmagentRemote.enabled` | Deploy a vmagent Deployment that scrapes mg-exporter and remote-writes to a Prometheus-compatible endpoint. | `false` | +| `vmagentRemote.namespace` | Namespace for the vmagent Deployment and its resources. Defaults to `prometheus.namespace` when empty. | `""` | +| `vmagentRemote.image.repository` | vmagent image repository. | `victoriametrics/vmagent` | +| `vmagentRemote.image.tag` | vmagent image tag. | `v1.139.0` | +| `vmagentRemote.image.pullPolicy` | vmagent image pull policy. | `IfNotPresent` | +| `vmagentRemote.remoteWrite.url` | Prometheus `remote_write` URL. Required when `vmagentRemote.enabled=true`. | `""` | +| `vmagentRemote.remoteWrite.basicAuth.secretName` | Kubernetes Secret holding basic-auth credentials for `remote_write`. When empty, basic auth is not configured. | `""` | +| `vmagentRemote.remoteWrite.basicAuth.usernameKey` | Key in the basic-auth Secret holding the username. | `username` | +| `vmagentRemote.remoteWrite.basicAuth.passwordKey` | Key in the basic-auth Secret holding the password. | `password` | +| `vmagentRemote.scrapeInterval` | Global `scrape_interval` applied to vmagent scrape jobs. | `15s` | +| `vmagentRemote.externalLabels` | External labels attached to every scraped sample before remote-write. | `{}` | +| `vmagentRemote.resources` | Resource requests/limits for the vmagent container. | `{}` | +| `vmagentRemote.httpPort` | vmagent local HTTP listen port for metrics/debug (the remote-write target is `remoteWrite.url`). | `8429` | +| `vmagentRemote.kubernetes.enabled` | Enable scraping of Kubernetes infrastructure metrics used by kube-prometheus dashboards. | `false` | +| `vmagentRemote.kubernetes.kubeStateMetrics.enabled` | Scrape `kube-state-metrics`. | `true` | +| `vmagentRemote.kubernetes.kubeStateMetrics.jobName` | Prometheus `job` label for kube-state-metrics. Keep aligned with dashboard/recording-rule expectations. | `kube-state-metrics` | +| `vmagentRemote.kubernetes.kubeStateMetrics.targets` | Static scrape targets for kube-state-metrics. | `[kube-prometheus-stack-kube-state-metrics.monitoring.svc.cluster.local:8080]` | +| `vmagentRemote.kubernetes.nodeExporter.enabled` | Scrape `node-exporter`. | `true` | +| `vmagentRemote.kubernetes.nodeExporter.jobName` | Prometheus `job` label for node-exporter. | `node-exporter` | +| `vmagentRemote.kubernetes.nodeExporter.useKubernetesDiscovery` | Discover node-exporter pods via Kubernetes SD so namespace/pod/node labels are present for recording rules. | `false` | +| `vmagentRemote.kubernetes.nodeExporter.podMetricsPort` | Pod port used by Kubernetes SD to match node-exporter pods. | `"9100"` | +| `vmagentRemote.kubernetes.nodeExporter.appNameLabel` | Expected value of `app.kubernetes.io/name` on node-exporter pods. | `prometheus-node-exporter` | +| `vmagentRemote.kubernetes.nodeExporter.appInstanceLabel` | Expected value of `app.kubernetes.io/instance` on node-exporter pods. | `kube-prometheus-stack-prometheus-node-exporter` | +| `vmagentRemote.kubernetes.nodeExporter.targets` | Static fallback targets for node-exporter when `useKubernetesDiscovery=false`. | `[kube-prometheus-stack-prometheus-node-exporter.monitoring.svc.cluster.local:9100]` | +| `vmagentRemote.kubernetes.kubelet.enabled` | Scrape kubelet metrics via the Kubernetes API server node proxy. | `true` | +| `vmagentRemote.kubernetes.kubelet.jobName` | Prometheus `job` label for kubelet. Keep as `kubelet` so kube-prometheus dashboards and recording rules continue to match. | `kubelet` | +| `vmagentRemote.kubernetes.kubelet.metricsPath` | Metrics path for the primary kubelet scrape (cAdvisor). | `/metrics/cadvisor` | +| `vmagentRemote.kubernetes.kubelet.additionalMetricsEnabled` | Enable a second kubelet scrape job for `/metrics` alongside the cAdvisor job. | `true` | +| `vmagentRemote.kubernetes.kubelet.additionalJobName` | Prometheus `job` label for the additional kubelet scrape. | `kubelet-metrics` | +| `vmagentRemote.kubernetes.kubelet.additionalMetricsPath` | Metrics path for the additional kubelet scrape. | `/metrics` | +| `vmagentRemote.kubernetes.kubelet.apiServerAddress` | Kubernetes API server address used to proxy kubelet scrapes. | `kubernetes.default.svc:443` | +| `vmagentRemote.kubernetes.kubelet.insecureSkipVerify` | Skip TLS verification of the kube-apiserver serving cert when scraping kubelet. | `false` | + + +If you enable the [`--storage-snapshot-on-exit`](/database-management/configuration) +flag via `memgraphConfig`, Memgraph creates a full snapshot of the database +during shutdown. Snapshot creation time scales with dataset size and can +exceed the default 30-minute `container.terminationGracePeriodSeconds`. + +If the grace period is shorter than the time needed to write the on-exit +snapshot, Kubernetes will `SIGKILL` Memgraph mid-write, leaving the snapshot +incomplete. Benchmark the snapshot time on a representative dataset and set +`container.terminationGracePeriodSeconds` to comfortably cover it. + For all available database settings, refer to the [configuration settings docs](/database-management/configuration). @@ -474,6 +652,81 @@ Or you can modify a `values.yaml` file and override the desired values: helm install memgraph/memgraph-lab -f values.yaml ``` +#### Gateway API support + +The Memgraph Lab Helm chart supports the [Kubernetes Gateway API](https://gateway-api.sigs.k8s.io/) for external access. When enabled, the chart creates an HTTPRoute resource to route HTTP(S) traffic to Memgraph Lab. You can either let the chart create its own Gateway or attach the route to a pre-existing one. + + +Before enabling Gateway API, you must have a Gateway API controller (e.g., Envoy Gateway, Istio, Cilium) installed in your cluster, and a **GatewayClass** resource must exist. The chart does not create a GatewayClass — you must create it yourself or use one provided by your controller installation. See the [HA chart Gateway API prerequisites](/clustering/high-availability/setup-ha-cluster-k8s#prerequisites-1) for detailed setup instructions. + + +**Chart-managed Gateway** + +To let the chart create a Gateway with an HTTPRoute: + +```yaml +gateway: + enabled: true + gatewayClassName: "eg" + listeners: + - name: lab-http + port: 80 + protocol: HTTP +``` + +For HTTPS with TLS termination: + +```yaml +gateway: + enabled: true + gatewayClassName: "eg" + listeners: + - name: lab-https + port: 443 + protocol: HTTPS + tls: + certificateRefs: + - name: lab-tls-secret +``` + +**Existing (external) Gateway** + +To attach an HTTPRoute to a pre-existing Gateway (for example, a shared Gateway that also serves the HA chart): + +```yaml +gateway: + enabled: true + existingGatewayName: "memgraph-gateway" +``` + +When the existing Gateway uses different listener names than the chart defaults, use `httpRoute.sectionNames` to specify which listener names the route should attach to: + +```yaml +gateway: + enabled: true + existingGatewayName: "memgraph-gateway" + httpRoute: + sectionNames: + - lab-http +``` + +You can also configure host-based routing with `httpRoute.hostnames`: + +```yaml +gateway: + enabled: true + existingGatewayName: "memgraph-gateway" + httpRoute: + sectionNames: + - lab-http + hostnames: + - lab.example.com +``` + + +A standalone Gateway manifest with pre-configured listeners for both Lab and HA is available in the [Helm charts repository](https://github.com/memgraph/helm-charts/blob/main/examples/gateway/gateway.yaml). Deploy it with `kubectl apply -f gateway.yaml` before installing the charts with `existingGatewayName`. + + #### Configuration options The following table lists the configurable parameters of the Memgraph Lab chart @@ -498,6 +751,16 @@ and their default values. | `secrets.enabled` | Enable the use of Kubernetes secrets. Will be injected as env variables. | `false` | | `secrets.name` | The name of the Kubernetes secret that will be used. | `memgraph-secrets` | | `secrets.keys` | Keys from the `secrets.name` that will be stored as env variables inside the pod. | `[]` | +| `gateway.enabled` | Enable Gateway API external access. | `false` | +| `gateway.gatewayClassName` | Name of a pre-existing GatewayClass. Required when creating a new Gateway. | `""` | +| `gateway.existingGatewayName`| Name of an existing Gateway to attach routes to. Skips Gateway creation. | `""` | +| `gateway.existingGatewayNamespace` | Namespace of the existing Gateway. Defaults to release namespace. | `""` | +| `gateway.annotations` | Annotations for the Gateway resource. | `{}` | +| `gateway.labels` | Labels for the Gateway resource. | `{}` | +| `gateway.listeners` | List of Gateway listeners with `name`, `port`, `protocol`, and optional `tls` configuration. | `[{name: lab-http, port: 80, protocol: HTTP}]` | +| `gateway.httpRoute.sectionNames` | Listener names to attach to on the Gateway. If empty, derived from `listeners[].name`. | `[]` | +| `gateway.httpRoute.hostnames`| Hostnames for the HTTPRoute. | `[]` | +| `gateway.httpRoute.matches` | HTTPRoute match rules. | `[{path: {type: PathPrefix, value: /}}]` | Memgraph Lab can be further configured with environment variables in your `values.yaml` file. @@ -528,6 +791,3 @@ when running queries over 65 seconds. Refer to the [Memgraph Lab documentation](/memgraph-lab/configuration) for details on how to configure Memgraph Lab. - - -kube diff --git a/pages/help-center/errors/high-availability.mdx b/pages/help-center/errors/high-availability.mdx index 97094df29..9df6096e3 100644 --- a/pages/help-center/errors/high-availability.mdx +++ b/pages/help-center/errors/high-availability.mdx @@ -10,20 +10,29 @@ import {CommunityLinks} from '/components/social-card/CommunityLinks' ## Errors -1. [At least one SYNC replica has not confirmed...](#error-1) - -### Troubleshooting SYNC replica not being confirmed [#error-1] - -If you're connecting to the cluster and encounter the error message **"At least one -SYNC replica has not confirmed..."** when writing to the main instance, several -issues could be causing this. Below are the possible reasons and how to resolve -them: - -1. Network isn't correctly configured between MAIN and REPLICAs -> Check if - hostnames/IPs can be reached. -2. Storage on replica isn't clean -> If you used your replica instances before -connecting them in the cluster, MAIN won't be able to successfully register -replica instance. Delete data directory of data instances and try to reconnect -the cluster again. +1. [Failed to replicate to SYNC/STRICT_SYNC replica...](#error-1) + +### Troubleshooting replication failure errors [#error-1] + +If you're writing to the main instance and encounter an error message like +**"Failed to replicate to SYNC replica 'instance_1': replica is not reachable or +not in sync with the main"**, several issues could be causing this. + +The error message identifies exactly which replicas failed, the replication mode +(SYNC or STRICT_SYNC), and the specific failure reason. For a full list of +possible failure reasons, see the +[replication errors reference](/help-center/errors/replication#error-4). + +Below are common causes and how to resolve them: + +1. **Network isn't correctly configured between MAIN and REPLICAs** — Check if + hostnames/IPs can be reached from the MAIN instance. +2. **Replica is behind MAIN** — It is possible that the replica is behind MAIN and that the recovery of the replica is in progress. Wait for a bit until the replica catches up with MAIN. If the replica is registered as a SYNC one, the transaction will still get eventually committed. +3. **Replica has diverged from MAIN** — If the error indicates the replica has + diverged, manual recovery or a force sync may be needed. See the + [force sync documentation](/clustering/high-availability/how-high-availability-works#replication-scenarios). +4. **RPC timeout** — If the error mentions an RPC timeout, the replica may be + overloaded or the network latency is too high. Consider adjusting the + `deltas_batch_progress_size` coordinator setting. diff --git a/pages/help-center/errors/replication.mdx b/pages/help-center/errors/replication.mdx index 31d101155..48f5ae31c 100644 --- a/pages/help-center/errors/replication.mdx +++ b/pages/help-center/errors/replication.mdx @@ -19,7 +19,7 @@ import {CommunityLinks} from '/components/social-card/CommunityLinks' 3. [Write query forbidden on the replica!](#error-2) 4. [Query forbidden on the replica!](#error-2) 5. [Replication clause not allowed in multicommand transactions.](#error-3) -6. [Replication Exception: At least one SYNC replica has not confirmed committing last transaction. Check the status of the replicas using 'SHOW REPLICAS' query.](#error-4) +6. [Failed to replicate to SYNC/STRICT_SYNC replica 'replica_name': reason.](#error-4) ## Warning @@ -44,11 +44,52 @@ MAIN is the only source of truth. System queries cannot be executed inside a multicommand/explicit transaction. -## At least one of the SYNC replicas has not confirmed committing [#error-4] - -This is a spurious error. While it is true that a SYNC replica has failed to -commit, it does not mean MAIN has not committed. -NOTE: Future releases will move this error under a notification instead. +## Replication failure error [#error-4] + +When a transaction fails to replicate to one or more SYNC or STRICT_SYNC +replicas, Memgraph returns an error that identifies exactly which replicas +failed and why. The error message follows this format: + +``` +Failed to replicate to SYNC replica 'instance_1': . +Replica will be recovered automatically. +Transaction is still committed on the main instance and other alive replicas. +``` + +When multiple replicas fail, each failure is listed with its replica name, +replication mode, and specific reason: + +``` +Failed to replicate to SYNC replicas 'instance_1' and 'instance_2': . +Replicas will be recovered automatically. +Transaction is still committed on the main instance and other alive replicas. +``` + +The possible failure reasons are: + +| Reason | Description | +|--------|-------------| +| `replica is not reachable or not in sync with the main` | The replica is down, behind, or the connection could not be established. | +| `failed to obtain RPC lock (another transaction is in progress)` | Another replication transaction is already in progress on that replica. | +| `RPC communication error` | A generic error occurred during the RPC call. | +| `replica has diverged from main` | The replica's data has diverged and manual recovery may be needed. | +| `RPC timeout while replicating` | The MAIN timed out waiting for a response from the replica. | + +The error message also indicates the transaction outcome: +- **"Transaction is still committed on the main instance and other alive replicas."** — + This appears for SYNC replicas. The transaction succeeded on MAIN despite the + replica failure. +- **"Transaction was aborted on all instances."** — This appears for STRICT_SYNC + replicas. The two-phase commit protocol ensures that if any STRICT_SYNC replica + fails, the transaction is rolled back everywhere. + + +Failed replicas will be recovered automatically. Check the status of replicas +using the `SHOW REPLICAS` query. + + +If the failure reason is an RPC timeout, the error message includes additional +guidance about adjusting the `deltas_batch_progress_size` coordinator setting. ## Snapshots are disabled for replicas [#warning-1] diff --git a/pages/release-notes.mdx b/pages/release-notes.mdx index 73cbc6db6..0b5bb5409 100644 --- a/pages/release-notes.mdx +++ b/pages/release-notes.mdx @@ -46,6 +46,228 @@ guide. ## 🚀 Latest release +### Memgraph v3.10.0 - May 6th, 2026 + +{

⚠️ Breaking changes

} + +- Coordinators ignore periodic storage snapshots (no user data to snapshot), and + `--init-file` / `--init-data-file` are rejected on coordinators and on data + instances that join an HA cluster (`--management-port` set). Remove those + flags from coordinator and HA data-instance configs before upgrading, and run + any bootstrap Cypher against the MAIN after the cluster is formed instead of + relying on startup init files. + [#4035](https://github.com/memgraph/memgraph/pull/4035) +- Fine-grained label and edge-type permissions now support explicit `DENY` + semantics (including `DENY ALL`/`*`) which take precedence over `GRANT`. The + `UPDATE` permission is split into granular capabilities: `SET LABEL`, `REMOVE + LABEL`, `SET PROPERTY`, `CREATE EDGE`, `DELETE EDGE`. Existing grants that + relied on `GRANT NOTHING` / `REVOKE NOTHING` must be migrated to `DENY`. + [#3789](https://github.com/memgraph/memgraph/pull/3789) +- `SHOW STORAGE INFO` output renames three fields and adds one new field: + `disk_usage` → `global_disk_usage`, `memory_tracked` → `global_memory_tracked`, + `allocation_limit` → `global_runtime_allocation_limit`, and adds + `global_license_allocation_limit`. Update any tooling or scripts that parse + these field names. + [#3999](https://github.com/memgraph/memgraph/pull/3999) + +{

✨ New features

} + +- Coordinators now accept read-only system queries (`SHOW CONFIG`, `SHOW LICENSE + INFO`, `SHOW BUILD INFO`, `SHOW STORAGE INFO`) plus `SET DATABASE SETTING` and + `RELOAD SSL`, so you can inspect configuration or reload TLS from a + coordinator without opening a session to a data instance. + [#4035](https://github.com/memgraph/memgraph/pull/4035) +- Added `SHOW QUERY CALLABLE MAPPINGS` to list all query procedure and function + aliases with their original source names and types (`procedure` / `function`), + so you can audit callable mappings without digging through configuration. + [#4014](https://github.com/memgraph/memgraph/pull/4014) +- Added `WHERE` clause support directly after `YIELD` in procedure calls: + `CALL mg.procedures() YIELD * WHERE name = 'mg.procedures' RETURN name`. This + lets you filter procedure results inline instead of wrapping the call in a + `WITH … WHERE` subquery. + [#4010](https://github.com/memgraph/memgraph/pull/4010) +- Memgraph now warns (or errors with `--strict-flag-check`) when unexpected + positional arguments appear after startup flag parsing, catching common + mistakes like `--flag false` (which silently ignores `false` and sets the flag + to `true`). Default behavior is strict in development builds; release builds + emit a warning. + [#4009](https://github.com/memgraph/memgraph/pull/4009) +- Added `--data-dir-lock-acquisition-timeout-sec` to retry acquiring the + data-directory file lock for a configurable duration before giving up. This + prevents spurious startup failures when a pod is force-killed and the old lock + has not been released yet (e.g. on CephFS or other distributed storage with + propagation delay). [#4021](https://github.com/memgraph/memgraph/pull/4021) +- Replication error messages now identify exactly which replica failed and why + (replica name, replication mode, failure reason) instead of generic "at least + one SYNC/STRICT_SYNC replica has not confirmed" messages, making it faster to + pinpoint replication issues. + [#3985](https://github.com/memgraph/memgraph/pull/3985) +- Added `RELOAD BOLT_SERVER TLS` Cypher command to reload SSL/TLS certificates + at runtime without restarting Memgraph. Existing connections keep the old + certificate; new connections use the reloaded one. Useful for Let's Encrypt + renewals and compliance certificate rotations. + [#3962](https://github.com/memgraph/memgraph/pull/3962) +- `WITH CONFIG` maps (e.g. `CREATE VECTOR INDEX … WITH CONFIG {…}`) now accept + query parameters as values, so you can pass `{dimension: $dim, capacity: $cap}` + instead of hard-coding values in the config literal. + [#3959](https://github.com/memgraph/memgraph/pull/3959) +- `instance_down_timeout_sec` and `instance_health_check_frequency_sec` are now + Raft-replicated coordinator runtime settings changeable via + `SET COORDINATOR SETTING` without restarting the cluster. The old + `--instance-down-timeout-sec` and `--instance-health-check-frequency-sec` + flags are deprecated and ignored. + [#3949](https://github.com/memgraph/memgraph/pull/3949) +- HA coordinator cluster operations (register, unregister, promote, demote) + now follow a Raft-first pattern: the state is committed to the Raft log and + acknowledged by a majority before returning success. RPCs to data instances + are retried automatically by the reconciliation loop, so transient network + issues no longer require manual intervention. + [#3942](https://github.com/memgraph/memgraph/pull/3942) + [#3922](https://github.com/memgraph/memgraph/pull/3922) +- Point2D, Point3D, and Enum types are now fully supported in the MGP C, C++, + and Python query module APIs. Query modules can create, copy, compare, and + inspect these types as `mgp_value`, fixing crashes in `schema.node_type_properties()` + on graphs that use spatial or enum properties. + [#3980](https://github.com/memgraph/memgraph/pull/3980) +- Added `schema.node_type_properties` and `schema.rel_type_properties` config + argument to match the Neo4j specification, improving compatibility with the + Neo4j ODBC BI Connector. Also adds the `apoc.version` mapping. + [#4000](https://github.com/memgraph/memgraph/pull/4000) +- Added server-side descriptions: annotate labels, edge types, properties, and + databases with human-readable strings that are persisted and visible in + `SHOW SCHEMA INFO`. Useful for documenting your graph schema directly in the + database. + [#3894](https://github.com/memgraph/memgraph/pull/3894) +- Vector edge indexes now use the single-store pattern (storing only a + `VectorIndexId` in edge properties rather than duplicating vector data), + matching vertex vector indexes. Snapshot serialization is backward-compatible. + [#3929](https://github.com/memgraph/memgraph/pull/3929) +- Added GNN import/export module to MAGE: export the graph to PyTorch Geometric + (PyG) or TensorFlow GNN (TF-GNN) JSON format, run your GNN pipeline + externally, then write predictions back with `pyg_import` / `tf_import`. + [#3803](https://github.com/memgraph/memgraph/pull/3803) +- Added Kerberos SSO authentication module: clients can authenticate using + Kerberos service tickets via GSSAPI, with role mapping through LDAP group + membership or static principal-to-role configuration. + [#3916](https://github.com/memgraph/memgraph/pull/3916) +- Added descending (DESC) label-property index support. `ORDER BY … DESC` + queries can now be served directly by a DESC index, eliminating the sort. + Create with `CREATE INDEX ON :Label(prop) WITH CONFIG {"order": "DESC"}`. + [#3996](https://github.com/memgraph/memgraph/pull/3996) + +{

🛠️ Improvements

} + +- Text index search performance is significantly improved: GIDs are now returned + directly from the Rust layer instead of full JSON documents, and Tantivy + searchers are pinned per transaction for snapshot-consistent reads. + [#3963](https://github.com/memgraph/memgraph/pull/3963) +- Upgraded Tantivy to 0.25, bringing ~30% faster text index imports and ~20% + better search throughput under contention. + [#3927](https://github.com/memgraph/memgraph/pull/3927) +- `ORDER BY` is now eliminated when an index scan already provides the required + ascending order, avoiding an unnecessary sort. With `LIMIT`, this means the + planner stops after N rows instead of sorting the entire result set first. + [#3950](https://github.com/memgraph/memgraph/pull/3950) +- Added `AI_PLATFORM` license type that enforces memory limits only on graph + memory, leaving vector index memory unconstrained up to the system + `--memory-limit`. Useful when running large vector workloads alongside graph + data without hitting the overall allocation cap. + [#3999](https://github.com/memgraph/memgraph/pull/3999) +- Most third-party build dependencies are now managed through Conan 2 packages, + making builds more reproducible and the contributor setup easier. See the + updated [build-from-source guide](/getting-started/build-memgraph-from-source) + for changes to the build workflow. + [#4002](https://github.com/memgraph/memgraph/pull/4002) +- Additional third-party libraries (gflags, NuRaft, Pulsar client, RocksDB, + usearch) are now managed as Conan recipes, removing the `init` script + and the `libs/` directory from the build process. + [#4023](https://github.com/memgraph/memgraph/pull/4023) +- `SHOW INDEX INFO`, `SHOW CONSTRAINT INFO`, `SHOW NODE LABELS`, and + `SHOW EDGE TYPES` no longer open a storage transaction, reducing lock + contention and improving responsiveness on busy instances. + [#4006](https://github.com/memgraph/memgraph/pull/4006) + +{

🐞 Bug fixes

} + +- Fixed a potential OOM crash when dropping a large vector index: the internal + transfer from indexed format back to property store now manages peak memory + usage so the operation completes cleanly even under tight memory limits. + [#4001](https://github.com/memgraph/memgraph/pull/4001) +- `RELOAD BOLT_SERVER TLS` is now supported on coordinator instances, so you can + rotate TLS certificates cluster-wide without opening a session to each data + instance separately. + [#4003](https://github.com/memgraph/memgraph/pull/4003) +- Fixed `refactor.merge_nodes` discarding relationship properties: all + properties on merged relationships are now correctly copied to the new + relationship. + [#3948](https://github.com/memgraph/memgraph/pull/3948) +- Fixed `ORDER BY` failing for list values; Cypher list ordering semantics + (lexicographic / dictionary order) are now applied correctly. + [#3877](https://github.com/memgraph/memgraph/pull/3877) +- Fixed the `link_prediction` MAGE module returning incorrect ROC-AUC scores + when all labels belong to a single class (returns a neutral 0.5 AUC instead + of erroring or producing inconsistent results). + [#3845](https://github.com/memgraph/memgraph/pull/3845) +- Fixed a use-after-free crash on replicas receiving a snapshot: `Clear()` now + also discards pending GC deltas, preventing stale pointers from being + dereferenced on the next transaction commit. + [#4013](https://github.com/memgraph/memgraph/pull/4013) +- Fixed a crash in HA coordinators caused by a dangling pointer in + `ReplicationInstanceConnector`; concurrent coordinator operations are now safe + and cluster membership changes complete without instability. + [#3922](https://github.com/memgraph/memgraph/pull/3922) +- Fixed a bug where `std::thread::hardware_concurrency()` returning 0 on some + machines caused RPC and communication servers to be created with zero worker + threads, leading to silent failures. A safe fallback of 2 is now used. + [#3982](https://github.com/memgraph/memgraph/pull/3982) +- Fixed replication network channels not being cleanly destroyed during + shutdown, which could block shutdown or leave sockets open. + [#4005](https://github.com/memgraph/memgraph/pull/4005) +- Fixed a crash that could occur when calling `SetValueForce` on storage + failures during a scheduled license revalidation. + [#4004](https://github.com/memgraph/memgraph/pull/4004) +- Fixed a TCP segmentation bug in SLK multi-file stream processing: when a file + transition arrives in a fragmented TCP segment, `CheckStreamStatus` now + validates metadata completeness before signalling `NEW_FILE`, preventing + incorrect stream state. + [#3925](https://github.com/memgraph/memgraph/pull/3925) +- Fixed a crash if the telemetry server fails to initialize: the exception is + now caught, logged, and the instance shuts down gracefully instead of + terminating abruptly. + [#4026](https://github.com/memgraph/memgraph/pull/4026) +- Transient disk failures when opening WAL or snapshot output files are now + handled without crashing; failed snapshots are skipped and retried at the next + scheduled interval (WAL failures remain fatal). + [#4025](https://github.com/memgraph/memgraph/pull/4025) +- Fixed log rotation being skipped after instance restarts: spdlog in-memory + rotation state is now re-initialized on startup so daily log files are + rotated correctly even after frequent restarts. + [#4019](https://github.com/memgraph/memgraph/pull/4019) +- Fixed a crash when calling `std::abort` while using an async logger: removing + the `spdlog::shutdown()` call before abort eliminates a window where the + logger could segfault, making core dumps more useful for diagnosis. + [#4034](https://github.com/memgraph/memgraph/pull/4034) +- Fixed a network descriptor leak in the epoll listener and a potential crash + if `epoll_create1` fails; the latter now throws an exception instead of + crashing the database. + [#4032](https://github.com/memgraph/memgraph/pull/4032) +- Fixed fine-grained access control roles leaking across databases: permission + checks now evaluate only the roles that apply to the currently active + database, preventing unintended privilege escalation for users with + per-database roles. + [#4042](https://github.com/memgraph/memgraph/pull/4042) +- Fixed a replication lag counter overflow that could occur when a REPLICA + temporarily had more committed transactions than the new MAIN. Replicas in + this transient state are now excluded from the routing table until they + converge. + [#4040](https://github.com/memgraph/memgraph/pull/4040) + +### Lab v3.10.0 - May 6th, 2026 + + + +## Previous releases + ### Memgraph v3.9.0 {

⚠️ Breaking changes

} @@ -286,8 +508,6 @@ guide. -## Previous releases - ### Memgraph v3.8.1 - February 17th, 2026 {

🐞 Bug fixes

} @@ -622,8 +842,6 @@ triggers with `SECURITY DEFINER`. Triggers now report the definer's identity correctly for auditing and role-based logic. [#3768](https://github.com/memgraph/memgraph/pull/3768) -## Previous releases - ### Lab v3.8.0 - February 11th, 2026 @@ -692,7 +910,6 @@ correctly for auditing and role-based logic. - Version bump to v3.7.1 to match the latest Memgraph version. - ### Lab v3.7.1 - December 19th, 2025 diff --git a/skills/check-release-milestone/SKILL.md b/skills/check-release-milestone/SKILL.md index 552ec92bd..da2c42a19 100644 --- a/skills/check-release-milestone/SKILL.md +++ b/skills/check-release-milestone/SKILL.md @@ -1,11 +1,11 @@ --- name: check_before_release -description: Run before every release to ensure all memgraph PRs have changelog entries, docs pages where required, that all changed text is free of spelling and grammar issues, and that existing documentation cross-links the new content. Use when preparing a release branch, before merging into the main branch, or when asked to "check before release". +description: Run before every release to ensure all merged memgraph PRs have a docs label assigned, have changelog entries, have docs pages where required, that all changed text is free of spelling and grammar issues, and that existing documentation cross-links the new content. Use when preparing a release branch, before merging into the main branch, or when asked to "check before release". Also use standalone when asked to audit docs labels on a milestone. --- # Check before release -Run this check before every release to find memgraph PRs that are missing from the changelog or that have no documentation page despite being labeled "Docs needed", and to catch spelling/grammar issues in changed documentation. +Run this check before every release to audit docs labels on every merged PR, find PRs that are missing from the changelog or that have no documentation page despite being labeled "Docs needed", and to catch spelling/grammar issues in changed documentation. ## When to use @@ -16,36 +16,87 @@ Run this check before every release to find memgraph PRs that are missing from t ## Assumptions - Memgraph PRs that need docs are labeled **"Docs needed"** or **"Docs - changelog only"**. -- The release documentation PR (in `memgraph/documentation`) contains two lists: - - **Memgraph PRs Docs Needed** – memgraph PR numbers with their corresponding doc PRs (e.g. `memgraph#3801 → #1555`). - - **Release Notes Required** – memgraph PR numbers that must appear in the changelog. +- Only consider **merged** PRs (i.e. `merged_at != null`). PRs that are closed without merging must be ignored entirely — no changelog entry, no doc page, not in the tracking list. +- The release documentation PR (in `memgraph/documentation`) uses the following **Docs Integration Tracking** format (three plain checklist sections — no tables): + +``` +#### Breaking changes PRs + +- [ ] https://github.com/memgraph/memgraph/pull/NNNN — short title @author + +#### Docs needed (Memgraph PR → Docs PR) + +- [ ] https://github.com/memgraph/memgraph/pull/NNNN → https://github.com/memgraph/documentation/pull/MMMM @author +- [ ] https://github.com/memgraph/memgraph/pull/NNNN → no doc PR yet @author + +#### Changelog (all PRs requiring a changelog entry) + +- [ ] https://github.com/memgraph/memgraph/pull/NNNN @author +``` + +Rules for this format: +- **Breaking changes PRs** — every merged PR with the `breaking` label (one line each). +- **Docs needed** — every merged PR labeled `Docs needed`. Each line ends with the doc PR link (or `no doc PR yet` if none exists) and the code PR author's GitHub handle. +- **Changelog** — every merged PR labeled `Docs needed` **or** `Docs - changelog only`. This section deliberately duplicates `Docs needed` PRs — it tracks the changelog entry independently of the doc page. Each line ends with the code PR author's handle. +- Tick `[x]` when the item is complete. GitHub renders `[ ]` as a clickable checkbox so reviewers can tick without editing markdown. ## Steps -1. **Identify versions** +1. **Docs label audit** + - Fetch every merged PR in the milestone: + ``` + gh api "repos/memgraph/memgraph/issues?milestone=&state=closed&per_page=100" \ + --jq '[.[] | select(.pull_request != null and .pull_request.merged_at != null)] | + map({number, title, author: .user.login, + docs_labels: [.labels[].name | select(startswith("Docs"))]})[]' + ``` + - For each merged PR classify its docs label state: + - **OK** — exactly one of: `Docs needed`, `Docs - changelog only`, `Docs unnecessary`. + - **Missing label** — no docs label at all. + - **Questionable** — the assigned label appears inconsistent with the PR content (e.g. a pure CI/test PR carrying `Docs needed`, or a user-facing feature carrying `Docs unnecessary`). + - Apply these heuristics to spot questionable labels: + - PRs whose title starts with `test:`, `testing:`, `tests:`, `ci:`, `infra:`, or `refactor:` and whose labels include only `infrastructure`, `tests`, `benchmarking`, or `Code improvements` almost always warrant `Docs unnecessary`. + - PRs labeled `feature` that carry `Docs unnecessary` should be scrutinised — only internal-only features (no new flags, commands, or user-visible behavior) are legitimately `Docs unnecessary`. + - Build-system / packaging PRs (Conan, CMake, Docker CI changes) are normally `Docs unnecessary` unless they change how end-users build or install Memgraph. + - For PRs where the label is missing or questionable, fetch the PR body for context: + ``` + gh pr view --repo memgraph/memgraph --json title,body,labels,author + ``` + - Produce a **canvas** (read and follow `~/.cursor/skills-cursor/canvas/SKILL.md`) showing: + - Summary stats: total merged, count per label, count needing attention. + - An "Issues requiring action" section listing PRs with missing or questionable labels, with a recommended label and a one-line reason. + - A filterable table of all merged PRs with their current docs label highlighted. + - **Fix or flag:** For PRs missing a label, recommend the correct one. For questionable ones, surface them to the reviewer — do not change the label unilaterally unless it is obviously wrong (e.g. no docs label at all on a pure test PR). + +2. **Identify versions** - Current release (e.g. `3.9`) and previous one (e.g. `3.8.0`). - Memgraph milestone for the release (e.g. `mg-v3.9.0`, milestone 43). - The open documentation release PR (e.g. `memgraph/documentation#1530`). -2. **Get the authoritative lists from the docs PR** - - From the PR description, extract: - - Every **Memgraph PRs Docs Needed** line: memgraph PR # and linked doc PR #. - - Every **Release Notes Required** line: memgraph PR # (and title if present). +3. **Get the authoritative lists from the docs PR** + - Fetch the PR body via `gh api repos/memgraph/documentation/pulls/ --jq .body`. + - From the **Docs Integration Tracking** section, extract: + - Every **Docs needed** line: memgraph PR # and linked doc PR # (or "no doc PR yet"). + - Every **Changelog** line: memgraph PR # (all PRs that need a changelog entry). + - Every **Breaking changes PRs** line: memgraph PR # marked as breaking. + - Cross-check against the memgraph milestone (closed + merged only) to catch any PRs the docs PR has not yet listed. -3. **Changelog check** +4. **Changelog check** - Open `pages/release-notes.mdx` and locate the section for the new release (e.g. `### Memgraph v3.9.0`). - For each PR in **Release Notes Required**, confirm it appears in that section (e.g. as `[#NNNN](https://github.com/memgraph/memgraph/pull/NNNN)` or equivalent). - Optionally cross-check the memgraph milestone: merged PRs with user-visible work (e.g. **Docs needed**, **Docs - changelog only**) that are absent from that release section should also be treated as **missing from changelog**, even if they were never added to **Release Notes Required** on the docs PR. - List any **missing from changelog** with PR numbers (and titles if known). - **If anything is missing from the changelog:** follow **`skills/write-changelog-item/SKILL.md`** end-to-end for each gap. That skill defines how to write the item (benefit-focused bullet, markdown, PR link, breaking vs non-breaking) and requires updating `pages/release-notes.mdx` plus keeping the open documentation release PR description aligned (**Release Notes Required**, and **Memgraph PRs Docs Needed** when a doc PR exists). Do not use a different format or skip the release PR body update. + **If anything is missing from the changelog:** follow **`skills/write-changelog-item/SKILL.md`** end-to-end for each gap. That skill defines how to write the item (benefit-focused bullet, markdown, PR link, breaking vs non-breaking) and requires updating `pages/release-notes.mdx` plus keeping the open documentation release PR description aligned. When updating the docs PR body, use the **Docs Integration Tracking** checklist format described in the Assumptions section — tick `[x]` on the relevant **Changelog** line and, if the PR is also "Docs needed", on the **Docs needed** line once a doc PR exists. Do not use a different format or skip the release PR body update. + + **One item at a time:** When proposing or drafting changelog text, handle **a single PR / single bullet** per turn. Give the full proposed entry (wording, PR link, breaking vs non-breaking if relevant) and any file or release-PR body edits that go with that one item, then **stop and wait for reviewer feedback** before moving to the next missing item. Do not batch multiple proposed changelog bullets or multiple PR remediations in one message — the reviewer must be able to approve, edit, or reject each entry on its own. -4. **Docs page check** - - For each memgraph PR listed under **Memgraph PRs Docs Needed**, confirm the description links to a documentation PR (e.g. `→ #1555`). - - Optionally fetch the milestone or PR list from `github.com/memgraph/memgraph/milestone/` and find any PR with label "Docs needed" that is **not** mentioned in the docs PR's "Memgraph PRs Docs Needed" list. - - List any **docs page missing**: PRs labeled "Docs needed" (or that clearly need a dedicated docs page) with no corresponding doc PR in the release docs PR. +5. **Docs page check** + - For each memgraph PR in the **Docs needed** section, confirm the line links to a documentation PR (not `no doc PR yet`). + - Fetch the documentation milestone (e.g. `memgraph/documentation/milestone/`) and cross-reference: any doc PR in the milestone that maps to a memgraph "Docs needed" PR should be reflected on the docs PR's tracking list. + - List any **docs page missing**: merged memgraph PRs labeled "Docs needed" with no corresponding doc PR on the tracking list. -5. **Grammar and spelling check** +6. **Grammar and spelling check** - Run `git diff main...HEAD -- '*.mdx' '*.md'` (or the appropriate base branch) to get all text changes on the release branch. - Review every added or modified line for: - **Spelling mistakes** (typos, wrong words). @@ -58,7 +109,7 @@ Run this check before every release to find memgraph PRs that are missing from t - Do **not** rewrite for style or restructure paragraphs. Only fix clear errors: misspellings, missing/wrong articles, broken grammar, and wrong word forms. - Apply fixes directly, then list what was changed in the report. -6. **Cross-linking integration check** +7. **Cross-linking integration check** - Run `git diff main...HEAD --name-only -- '*.mdx' '*.md'` to get every file changed on the release branch. - For each changed file, identify the **key new concepts** it introduces (new pages, new sections, new Cypher commands, new configuration, renamed behavior). These are the integration targets. - Scan the existing documentation on `main` for pages that discuss the same topic area but do **not** yet link to the new content. Efficient approach: @@ -72,15 +123,18 @@ Run this check before every release to find memgraph PRs that are missing from t - For each proposal, state: **file**, **location** (section or line range), **what to add** (sentence or callout), and **why** (what reader journey it fixes). - After user approval, apply the edits directly. -7. **Report** - - **Not in changelog:** list memgraph PR numbers (and titles if helpful). For each, state that remediation is **`write_changelog_item`** per `skills/write-changelog-item/SKILL.md` (unless the user asked for report-only). Include PRs found only via milestone cross-check, not only those on **Release Notes Required**. - - **Docs page missing:** list of memgraph PR/issue numbers with "Docs needed" and no doc PR linked in the release docs PR; briefly note what's missing (e.g. "TLS .pem-only behavior"). +8. **Report** + - **Docs label issues:** list every merged PR with a missing or questionable label, the recommended label, and a one-line reason. The canvas from step 1 serves as the primary deliverable for this section. + - **Not in changelog:** summarize all gaps (PR numbers and titles). When remediating, apply the **one item at a time** rule from step 4. For each gap, state that remediation follows **`write_changelog_item`** per `skills/write-changelog-item/SKILL.md` (unless the user asked for report-only). Include PRs found only via milestone cross-check. + - **Docs page missing:** list merged memgraph PRs labeled "Docs needed" with no doc PR on the tracking list; briefly note what's missing (e.g. "TLS .pem-only behavior"). + - **Docs PR tracking list gaps:** if any merged "Docs needed" or "Docs - changelog only" PR is absent from the **Docs Integration Tracking** checklist on the release docs PR, add it using the checklist format from the Assumptions section (unticked `[ ]`, correct section, author handle). - **Spelling/grammar fixes:** list each fix with file name and a short before → after summary. - **Cross-link improvements:** list each proposed (or applied) edit with file, location, what was added, and why. - - If all lists are empty, state that the release is clear for changelog, docs, grammar, and cross-links. + - If all lists are empty, state that the release is clear for label audit, changelog, docs, grammar, and cross-links. ## References +- **Canvas skill:** `~/.cursor/skills-cursor/canvas/SKILL.md` — required for producing the label audit canvas in step 1. - **Missing changelog items:** `skills/write-changelog-item/SKILL.md` — authoritative procedure to add entries and sync the docs release PR. - Memgraph commits (since last release): `https://github.com/memgraph/memgraph/commits/master/` - Memgraph milestone (e.g. 3.9): `https://github.com/memgraph/memgraph/milestone/43?closed=1` @@ -89,6 +143,10 @@ Run this check before every release to find memgraph PRs that are missing from t ## Notes +- **Label audit scope:** Step 1 can be run standalone (without the full release check) when asked to "audit docs labels" or "check labels on milestone". When run standalone, produce only the canvas and a brief chat summary — skip steps 2–8. +- **Changelog workflow:** Multiple missing entries are common; never merge several proposed bullets into one reviewer-facing message. One PR → one proposed changelog item → pause for review → next PR. +- **Merged PRs only:** always filter the milestone by `merged_at != null` (use `gh api "repos/memgraph/memgraph/issues?milestone=&state=closed" --jq '[.[] | select(.pull_request != null and .pull_request.merged_at != null)]'`). Closed-without-merge PRs must be ignored entirely. - PRs labeled **"Docs unnecessary"** (e.g. CI, tests, internal tooling) are excluded; no changelog or docs page required. - If work landed in a different PR than originally planned, the changelog should reference the **merged** PR that shipped the behavior; if user-visible behavior still has no line in `release-notes.mdx`, treat it as a gap and use **write_changelog_item**. - The grammar check scope is the diff against `main` (or the base branch). Existing text that was not touched in this release is out of scope. +- When writing the local tracking file (`.release-tracking/vX.Y.Z.md`), keep it in sync with the docs PR body — both should reflect the same checklist state. diff --git a/skills/new-release-branch/SKILL.md b/skills/new-release-branch/SKILL.md new file mode 100644 index 000000000..111de0317 --- /dev/null +++ b/skills/new-release-branch/SKILL.md @@ -0,0 +1,170 @@ +--- +name: new-release-branch +description: >- + Create a new release branch for Memgraph documentation with the initial + release-notes scaffold commit and PR. Use when the user wants to start a new + release, create a release branch, or prepare release notes for a new version. +--- + +# New Release Branch + +Automates the first commit of a new Memgraph documentation release branch: +creates the branch, scaffolds the release-notes titles, and opens a PR. + +## Step 1 — Gather information + +Ask the user (use `AskQuestion` if available, otherwise ask conversationally): + +1. **Branch name** — exact name, e.g. `release/3.9`. +2. **Memgraph version + date** — e.g. `v3.9.0 - March 25th, 2026`. +3. **Lab version + date** — e.g. `v3.9.0 - March 25th, 2026`. +4. **GitHub milestone URL** — for the PR description, e.g. + `https://github.com/memgraph/memgraph/milestone/44`. + +## Step 2 — Create the branch + +```bash +cd +git checkout main && git pull +git checkout -b +``` + +## Step 3 — Edit `pages/release-notes.mdx` + +The file has this high-level structure: + +``` +## 🚀 Latest release + +### Memgraph - +...content... + +### Lab - + + + +## Previous releases + +### Memgraph ... +``` + +Make **two** changes (use `StrReplace` or equivalent): + +### 3a — Add new empty titles above the current latest + +Insert immediately after the line `## 🚀 Latest release`: + +``` +### Memgraph - + +### Lab - + + +``` + +### 3b — Move previous latest Memgraph + Lab into "Previous releases" + +Take the **entire previous latest block** — from the old +`### Memgraph ` heading through the old +`### Lab ` + its `` line — and +move it so it appears right after the `## Previous releases` heading (before +whatever was previously the first entry there). + +The result should look like: + +``` +## 🚀 Latest release + +### Memgraph - + +### Lab - + + + +## Previous releases + +### Memgraph - +...all previous content... + +### Lab - + + + +### Memgraph ... +``` + +**Important:** If there are multiple Memgraph patch releases under Latest +(e.g. v3.8.1 and v3.8.0), move *all* of them together with the Lab entry. + +## Step 4 — Commit and push + +```bash +git add pages/release-notes.mdx +git commit -m "Add Memgraph and Lab release note titles" +git push -u origin +``` + +## Step 5 — Create the PR + +Use `gh pr create`. The title should be: + +``` +Memgraph +``` + +Use this exact body template (fill in the placeholders): + +~~~ +Make sure to do: +* [ ] update sitemap +* [ ] update direct download links + +Milestones +* Memgraph -> + +### Docs Integration Tracking + +#### Breaking changes PRs + +- [ ] https://github.com/memgraph/memgraph/pull/NNNN — short title @author + +#### Docs needed (Memgraph PR → Docs PR) + +- [ ] https://github.com/memgraph/memgraph/pull/NNNN → https://github.com/memgraph/documentation/pull/MMMM @author +- [ ] https://github.com/memgraph/memgraph/pull/NNNN → no doc PR yet @author + +#### Changelog (all PRs requiring a changelog entry) + +- [ ] https://github.com/memgraph/memgraph/pull/NNNN @author +~~~ + +Replace `` with the actual URL provided by the user. +The `NNNN` / `MMMM` / `@author` / `short title` placeholders are filled in manually +as PRs are merged — remove the example lines and replace them with real entries. +Rules for each section: +- **Breaking changes PRs** — one line per merged PR carrying the `breaking` label. +- **Docs needed** — one line per merged PR labeled `Docs needed`; end with the doc PR link + (or `no doc PR yet`) and the code PR author's `@handle`. Tick `[x]` once the doc PR is merged. +- **Changelog** — one line per merged PR labeled `Docs needed` **or** `Docs - changelog only`; + end with the author's `@handle`. This section deliberately duplicates `Docs needed` entries — + it tracks the changelog entry independently of the doc page. Tick `[x]` once the entry lands + in `release-notes.mdx`. + +## Step 6 — Update the milestone description + +Add a `DOCS ->` link back to the documentation PR in the GitHub milestone +description. Extract the milestone number from the milestone URL +(e.g. `49` from `https://github.com/memgraph/memgraph/milestone/49`) and the +PR number from the PR URL created in Step 5. + +```bash +gh api repos/memgraph/memgraph/milestones/ \ + --method PATCH \ + -f description='DOCS -> https://github.com/memgraph/documentation/pull/' +``` + +## Step 7 — Report back + +Print the PR URL and remind the user to: +- Fill in the PR/author placeholders in the PR description. +- Update the sitemap and direct download links before merging.