Skip to content

Commit 41b4f36

Browse files
committed
refactoring
1 parent 4150c3c commit 41b4f36

File tree

67 files changed

+904
-477
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

67 files changed

+904
-477
lines changed

.editorconfig

Lines changed: 464 additions & 0 deletions
Large diffs are not rendered by default.

AGENTS.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,12 @@
11
# Repository Guidelines
22

3-
# Rules to follow
3+
## Rules to follow
44
- Always run `dotnet build GraphRag.slnx` (or the relevant project) before executing any `dotnet test` command.
55
- Default to the latest available versions (e.g., Apache AGE `latest`) when selecting dependencies, per user request ("тобі треба latest").
66
- Do not create or rely on fake database stores (e.g., `FakePostgresGraphStore`); all tests must use real connectors/backing services.
7+
- Keep default prompts in static C# classes; do not rely on prompt files under `prompts/` for built-in templates.
8+
- Register language models through Microsoft.Extensions.AI keyed services; avoid bespoke `LanguageModelConfig` providers.
9+
- Always run `dotnet format GraphRag.slnx` before finishing work.
710

811
# Conversations
912
any resulting updates to agents.md should go under the section "## Rules to follow"

Directory.Packages.props

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
44
<PackageVersion Include="Microsoft.Azure.Cosmos" Version="3.54.0" />
55
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="9.0.10" />
6+
<PackageVersion Include="Microsoft.Extensions.Caching.Memory" Version="9.0.10" />
67
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="9.0.10" />
78
<PackageVersion Include="Microsoft.Extensions.Logging" Version="9.0.10" />
89
<PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="9.0.10" />
@@ -20,4 +21,4 @@
2021
<PackageVersion Include="xunit" Version="2.9.3" />
2122
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />
2223
</ItemGroup>
23-
</Project>
24+
</Project>

README.md

Lines changed: 43 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,13 +93,55 @@ graphrag/
9393

9494
---
9595

96+
## Pipeline Cache
97+
98+
GraphRag exposes the `IPipelineCache` abstraction. To use the built-in in-memory cache, register it alongside the standard ASP.NET Core services:
99+
100+
```csharp
101+
using GraphRag.Cache;
102+
103+
builder.Services.AddMemoryCache();
104+
builder.Services.AddSingleton<IPipelineCache, MemoryPipelineCache>();
105+
```
106+
107+
Prefer a different backend? Implement `IPipelineCache` yourself and register it through DI—the pipeline will pick up your custom cache automatically.
108+
109+
---
110+
111+
## Language Model Registration
112+
113+
GraphRAG delegates language-model configuration to [Microsoft.Extensions.AI](https://learn.microsoft.com/dotnet/ai/overview). Register keyed clients for every `ModelId` you reference in configuration—pick any string key that matches your config:
114+
115+
```csharp
116+
using Azure;
117+
using Azure.AI.OpenAI;
118+
using GraphRag.Config;
119+
using Microsoft.Extensions.AI;
120+
121+
var openAi = new OpenAIClient(new Uri(endpoint), new AzureKeyCredential(key));
122+
const string chatModelId = "chat_model";
123+
const string embeddingModelId = "embedding_model";
124+
125+
builder.Services.AddKeyedSingleton<IChatClient>(
126+
chatModelId,
127+
_ => openAi.GetChatClient(chatDeployment));
128+
129+
builder.Services.AddKeyedSingleton<IEmbeddingGenerator<string, Embedding>>(
130+
embeddingModelId,
131+
_ => openAi.GetEmbeddingClient(embeddingDeployment));
132+
```
133+
134+
Rate limits, retries, and other policies should be configured when you create these clients (for example by wrapping them with `Polly` handlers). `GraphRagConfig.Models` simply tracks the set of model keys that have been registered so overrides can validate references.
135+
136+
---
137+
96138
## Indexing, Querying, and Prompt Tuning Alignment
97139

98140
The .NET port mirrors the [GraphRAG indexing architecture](https://microsoft.github.io/graphrag/index/overview/) and its query workflows so downstream applications retain parity with the Python reference implementation.
99141

100142
- **Indexing overview.** Workflows such as `extract_graph`, `create_communities`, and `community_summaries` map 1:1 to the [default data flow](https://microsoft.github.io/graphrag/index/default_dataflow/) and persist the same tables (`text_units`, `entities`, `relationships`, `communities`, `community_reports`, `covariates`). The new prompt template loader honours manual or auto-tuned prompts before falling back to the stock templates in `prompts/`.
101143
- **Query capabilities.** The query pipeline retains global search, local search, drift search, and question generation semantics described in the [GraphRAG query overview](https://microsoft.github.io/graphrag/query/overview/). Each orchestrator continues to assemble context from the indexed tables so you can reference [global](https://microsoft.github.io/graphrag/query/global_search/) or [local](https://microsoft.github.io/graphrag/query/local_search/) narratives interchangeably.
102-
- **Prompt tuning.** GraphRAG’s [manual](https://microsoft.github.io/graphrag/prompt_tuning/manual_prompt_tuning/) and [auto](https://microsoft.github.io/graphrag/prompt_tuning/auto_prompt_tuning/) strategies are surfaced through `GraphRagConfig.PromptTuning`. Store custom templates under `prompts/` or point `PromptTuning.Manual.Directory`/`PromptTuning.Auto.Directory` at your tuning outputs. Files follow the stage keys documented in `docs/indexing-and-query.md` (for example, `index/community_reports/system.txt` overrides the community summary system prompt). Templates can use placeholders such as `{{max_length}}`, `{{max_entities}}`, and `{{entities}}`.
144+
- **Prompt tuning.** GraphRAG’s [manual](https://microsoft.github.io/graphrag/prompt_tuning/manual_prompt_tuning/) and [auto](https://microsoft.github.io/graphrag/prompt_tuning/auto_prompt_tuning/) strategies are surfaced through `GraphRagConfig.PromptTuning`. Store custom templates under `prompts/` or point `PromptTuning.Manual.Directory`/`PromptTuning.Auto.Directory` at your tuning outputs. You can also skip files entirely by assigning inline text (multi-line or prefixed with `inline:`) to workflow prompt properties. Stage keys and placeholders are documented in `docs/indexing-and-query.md`.
103145

104146
See [`docs/indexing-and-query.md`](docs/indexing-and-query.md) for a deeper mapping between the .NET workflows and the research publications underpinning GraphRAG.
105147

docs/indexing-and-query.md

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,37 @@ GraphRAG for .NET keeps feature parity with the Python reference project describ
1313
- **Storage schema.** Tables share the column layout described under [index outputs](https://microsoft.github.io/graphrag/index/outputs/). The new strongly-typed records (`CommunityRecord`, `CovariateRecord`, etc.) mirror the JSON representation used by the Python implementation.
1414
- **Cluster configuration.** `GraphRagConfig.ClusterGraph` exposes the same knobs as the Python `cluster_graph` settings, enabling largest-component filtering and deterministic seeding.
1515

16+
## Language Model Registration
17+
18+
Workflows resolve language models from the DI container via [Microsoft.Extensions.AI](https://learn.microsoft.com/dotnet/ai/overview). Register keyed services for every `ModelId` you plan to reference:
19+
20+
```csharp
21+
using Azure;
22+
using Azure.AI.OpenAI;
23+
using GraphRag.Config;
24+
using Microsoft.Extensions.AI;
25+
26+
var openAi = new OpenAIClient(new Uri(endpoint), new AzureKeyCredential(key));
27+
const string chatModelId = "chat_model";
28+
const string embeddingModelId = "embedding_model";
29+
30+
services.AddKeyedSingleton<IChatClient>(chatModelId, _ => openAi.GetChatClient(chatDeployment));
31+
services.AddKeyedSingleton<IEmbeddingGenerator<string, Embedding>>(embeddingModelId, _ => openAi.GetEmbeddingClient(embeddingDeployment));
32+
```
33+
34+
Configure retries, rate limits, and logging when you construct the concrete clients. `GraphRagConfig.Models` simply records the set of registered keys so configuration overrides can validate references.
35+
36+
## Pipeline Cache
37+
38+
`IPipelineCache` is intentionally infrastructure-neutral. To mirror ASP.NET Core's in-memory behaviour, register the built-in cache services alongside the provided adapter:
39+
40+
```csharp
41+
services.AddMemoryCache();
42+
services.AddSingleton<IPipelineCache, MemoryPipelineCache>();
43+
```
44+
45+
Need Redis or something else? Implement `IPipelineCache` yourself and register it through DI; the pipeline will automatically consume your custom cache.
46+
1647
## Query Capabilities
1748

1849
The query layer ports the orchestrators documented in the [GraphRAG query overview](https://microsoft.github.io/graphrag/query/overview/):
@@ -34,6 +65,7 @@ Manual and auto prompt tuning are both available without code changes:
3465
2. **Auto tuning** integrates the outputs documented in [auto prompt tuning](https://microsoft.github.io/graphrag/prompt_tuning/auto_prompt_tuning/).
3566
- Point `GraphRagConfig.PromptTuning.Auto.Directory` at the folder containing the generated prompts and set `Enabled = true`.
3667
- The runtime prefers explicit paths from workflow configs, then manual overrides, then auto-tuned files, and finally the built-in defaults in `prompts/`.
68+
3. **Inline overrides** can be injected directly from code: set `ExtractGraphConfig.SystemPrompt`, `ExtractGraphConfig.Prompt`, or the equivalent properties to either a multi-line string or a value prefixed with `inline:`. Inline values bypass template file lookups and are used as-is.
3769

3870
### Stage Keys and Placeholders
3971

@@ -51,7 +83,7 @@ Placeholders are replaced at runtime with values drawn from workflow configurati
5183
- `{{max_length}}``CommunityReportsConfig.MaxLength`
5284
- `{{entities}}` → bullet list of entity titles and descriptions
5385

54-
If a template is omitted, the runtime falls back to the built-in prompts stored under `prompts/` and bundled with the repository.
86+
If a template is omitted, the runtime falls back to the built-in prompts defined in `GraphRagPromptLibrary`.
5587

5688
## Integration Tests
5789

src/ManagedCode.GraphRag.CosmosDb/CosmosGraphStore.cs

Lines changed: 7 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,32 +1,18 @@
1-
using System;
2-
using System.Collections.Generic;
3-
using System.Linq;
41
using System.Runtime.CompilerServices;
5-
using System.Threading;
6-
using System.Threading.Tasks;
72
using GraphRag.Graphs;
83
using Microsoft.Azure.Cosmos;
94
using Microsoft.Azure.Cosmos.Linq;
105
using Microsoft.Extensions.Logging;
116

127
namespace GraphRag.Storage.Cosmos;
138

14-
public sealed class CosmosGraphStore : IGraphStore
9+
public sealed class CosmosGraphStore(CosmosClient client, string databaseId, string nodesContainerId, string edgesContainerId, ILogger<CosmosGraphStore> logger) : IGraphStore
1510
{
16-
private readonly CosmosClient _client;
17-
private readonly string _databaseId;
18-
private readonly string _nodesContainerId;
19-
private readonly string _edgesContainerId;
20-
private readonly ILogger<CosmosGraphStore> _logger;
21-
22-
public CosmosGraphStore(CosmosClient client, string databaseId, string nodesContainerId, string edgesContainerId, ILogger<CosmosGraphStore> logger)
23-
{
24-
_client = client ?? throw new ArgumentNullException(nameof(client));
25-
_databaseId = databaseId ?? throw new ArgumentNullException(nameof(databaseId));
26-
_nodesContainerId = nodesContainerId ?? throw new ArgumentNullException(nameof(nodesContainerId));
27-
_edgesContainerId = edgesContainerId ?? throw new ArgumentNullException(nameof(edgesContainerId));
28-
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
29-
}
11+
private readonly CosmosClient _client = client ?? throw new ArgumentNullException(nameof(client));
12+
private readonly string _databaseId = databaseId ?? throw new ArgumentNullException(nameof(databaseId));
13+
private readonly string _nodesContainerId = nodesContainerId ?? throw new ArgumentNullException(nameof(nodesContainerId));
14+
private readonly string _edgesContainerId = edgesContainerId ?? throw new ArgumentNullException(nameof(edgesContainerId));
15+
private readonly ILogger<CosmosGraphStore> _logger = logger ?? throw new ArgumentNullException(nameof(logger));
3016

3117
public async Task InitializeAsync(CancellationToken cancellationToken = default)
3218
{
@@ -60,7 +46,7 @@ public async Task UpsertRelationshipAsync(string sourceId, string targetId, stri
6046
public IAsyncEnumerable<GraphRelationship> GetOutgoingRelationshipsAsync(string sourceId, CancellationToken cancellationToken = default)
6147
{
6248
ArgumentException.ThrowIfNullOrWhiteSpace(sourceId);
63-
return Fetch();
49+
return Fetch(cancellationToken);
6450

6551
async IAsyncEnumerable<GraphRelationship> Fetch([EnumeratorCancellation] CancellationToken token = default)
6652
{

src/ManagedCode.GraphRag.CosmosDb/ServiceCollectionExtensions.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
using System;
21
using System.Text.Json;
32
using GraphRag.Graphs;
43
using Microsoft.Azure.Cosmos;

src/ManagedCode.GraphRag.CosmosDb/SystemTextJsonCosmosSerializer.cs

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,21 @@
1-
using System.IO;
2-
using System.Text;
31
using System.Text.Json;
42
using Microsoft.Azure.Cosmos;
53

64
namespace GraphRag.Storage.Cosmos;
75

8-
internal sealed class SystemTextJsonCosmosSerializer : CosmosSerializer
6+
internal sealed class SystemTextJsonCosmosSerializer(JsonSerializerOptions? options = null) : CosmosSerializer
97
{
108
private static readonly JsonSerializerOptions DefaultOptions = new(JsonSerializerDefaults.Web)
119
{
1210
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
1311
WriteIndented = false
1412
};
1513

16-
private readonly JsonSerializerOptions _options;
17-
18-
public SystemTextJsonCosmosSerializer(JsonSerializerOptions? options = null)
19-
{
20-
_options = options ?? DefaultOptions;
21-
}
14+
private readonly JsonSerializerOptions _options = options ?? DefaultOptions;
2215

2316
public override T FromStream<T>(Stream stream)
2417
{
25-
if (stream is null)
26-
{
27-
throw new ArgumentNullException(nameof(stream));
28-
}
18+
ArgumentNullException.ThrowIfNull(stream);
2919

3020
if (typeof(T) == typeof(Stream))
3121
{

src/ManagedCode.GraphRag.Neo4j/Neo4jGraphStore.cs

Lines changed: 4 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,20 @@
1-
using System;
2-
using System.Collections.Generic;
3-
using System.Linq;
41
using System.Runtime.CompilerServices;
5-
using System.Threading;
6-
using System.Threading.Tasks;
72
using GraphRag.Graphs;
83
using Microsoft.Extensions.Logging;
94
using Neo4j.Driver;
105

116
namespace GraphRag.Storage.Neo4j;
127

13-
public sealed class Neo4jGraphStore : IGraphStore, IAsyncDisposable
8+
public sealed class Neo4jGraphStore(IDriver driver, ILogger<Neo4jGraphStore> logger) : IGraphStore, IAsyncDisposable
149
{
15-
private readonly IDriver _driver;
16-
private readonly ILogger<Neo4jGraphStore> _logger;
10+
private readonly IDriver _driver = driver ?? throw new ArgumentNullException(nameof(driver));
11+
private readonly ILogger<Neo4jGraphStore> _logger = logger ?? throw new ArgumentNullException(nameof(logger));
1712

1813
public Neo4jGraphStore(string uri, string username, string password, ILogger<Neo4jGraphStore> logger)
1914
: this(GraphDatabase.Driver(uri, AuthTokens.Basic(username, password)), logger)
2015
{
2116
}
2217

23-
public Neo4jGraphStore(IDriver driver, ILogger<Neo4jGraphStore> logger)
24-
{
25-
_driver = driver ?? throw new ArgumentNullException(nameof(driver));
26-
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
27-
}
28-
2918
public async Task InitializeAsync(CancellationToken cancellationToken = default)
3019
{
3120
cancellationToken.ThrowIfCancellationRequested();
@@ -64,7 +53,7 @@ public IAsyncEnumerable<GraphRelationship> GetOutgoingRelationshipsAsync(string
6453
{
6554
ArgumentException.ThrowIfNullOrWhiteSpace(sourceId);
6655
cancellationToken.ThrowIfCancellationRequested();
67-
return Fetch();
56+
return Fetch(cancellationToken);
6857

6958
async IAsyncEnumerable<GraphRelationship> Fetch([EnumeratorCancellation] CancellationToken token = default)
7059
{

src/ManagedCode.GraphRag.Neo4j/ServiceCollectionExtensions.cs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
using System;
21
using GraphRag.Graphs;
32
using Microsoft.Extensions.DependencyInjection;
43
using Microsoft.Extensions.Logging;

0 commit comments

Comments
 (0)