Skip to content

Commit 91fcd8c

Browse files
committed
tests and more work
1 parent 4aac60a commit 91fcd8c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+876
-523
lines changed

.editorconfig

Lines changed: 26 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -86,15 +86,24 @@ csharp_style_namespace_declarations = file_scoped:warning
8686
# Brace settings - ALWAYS use braces, even for single-line blocks
8787
csharp_prefer_braces = true:warning
8888

89-
# Expression-bodied members (modern C#)
90-
csharp_style_expression_bodied_methods = when_on_single_line:suggestion
91-
csharp_style_expression_bodied_constructors = false:suggestion
92-
csharp_style_expression_bodied_operators = when_on_single_line:suggestion
93-
csharp_style_expression_bodied_properties = true:suggestion
94-
csharp_style_expression_bodied_indexers = true:suggestion
95-
csharp_style_expression_bodied_accessors = true:suggestion
96-
csharp_style_expression_bodied_lambdas = true:suggestion
97-
csharp_style_expression_bodied_local_functions = when_on_single_line:suggestion
89+
# Expression-bodied members are style-only noise for this library.
90+
csharp_style_expression_bodied_methods = false:none
91+
csharp_style_expression_bodied_constructors = false:none
92+
csharp_style_expression_bodied_operators = false:none
93+
csharp_style_expression_bodied_properties = false:none
94+
csharp_style_expression_bodied_indexers = false:none
95+
csharp_style_expression_bodied_accessors = false:none
96+
csharp_style_expression_bodied_lambdas = false:none
97+
csharp_style_expression_bodied_local_functions = false:none
98+
dotnet_diagnostic.IDE0021.severity = none
99+
dotnet_diagnostic.IDE0022.severity = none
100+
dotnet_diagnostic.IDE0023.severity = none
101+
dotnet_diagnostic.IDE0024.severity = none
102+
dotnet_diagnostic.IDE0025.severity = none
103+
dotnet_diagnostic.IDE0026.severity = none
104+
dotnet_diagnostic.IDE0053.severity = none
105+
dotnet_diagnostic.IDE0061.severity = none
106+
resharper_convert_to_expression_body_highlighting = none
98107

99108
# Pattern matching (C# 7+)
100109
csharp_style_pattern_matching_over_is_with_cast_check = true:warning
@@ -484,3 +493,11 @@ indent_style = unset
484493
insert_final_newline = false
485494
tab_width = unset
486495
trim_trailing_whitespace = false
496+
497+
[src/MarkdownLd.Kb/Ai/*.cs]
498+
# These are public root-namespace AI contracts kept in subfolders for file organization.
499+
resharper_check_namespace_highlighting = none
500+
501+
[src/MarkdownLd.Kb/Models/*.cs]
502+
# These are public root-namespace models kept in subfolders for file organization.
503+
resharper_check_namespace_highlighting = none

.github/workflows/release.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,10 +70,10 @@ jobs:
7070
run: dotnet build ${{ env.SOLUTION }} --configuration Release --no-restore
7171

7272
- name: Test
73-
run: dotnet test --solution ${{ env.SOLUTION }} --configuration Release --no-build --verbosity normal
73+
run: dotnet test --solution ${{ env.SOLUTION }} --configuration Release --verbosity normal
7474

7575
- name: Pack NuGet packages
76-
run: dotnet pack ${{ env.SOLUTION }} --configuration Release --no-build -p:IncludeSymbols=false -p:SymbolPackageFormat=snupkg --output ${{ env.ARTIFACTS_DIR }}
76+
run: dotnet pack ${{ env.SOLUTION }} --configuration Release -p:IncludeSymbols=false -p:SymbolPackageFormat=snupkg --output ${{ env.ARTIFACTS_DIR }}
7777

7878
- name: Upload artifacts
7979
uses: actions/upload-artifact@v4

.github/workflows/validation.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,16 +39,16 @@ jobs:
3939
run: dotnet build MarkdownLd.Kb.slnx --configuration Release --no-restore
4040

4141
- name: Test with coverage
42-
run: dotnet test --solution MarkdownLd.Kb.slnx --configuration Release --no-build --verbosity normal --coverlet --coverlet-output-format cobertura --coverlet-include '[ManagedCode.MarkdownLd.Kb]*' --results-directory TestResults/CoverletMtpFiltered
42+
run: dotnet test --solution MarkdownLd.Kb.slnx --configuration Release --verbosity normal -- --coverage --coverage-output-format cobertura --coverage-output "${{ github.workspace }}/TestResults/TUnitCoverage/coverage.cobertura.xml" --coverage-settings "${{ github.workspace }}/CodeCoverage.runsettings"
4343

4444
- name: Pack
45-
run: dotnet pack MarkdownLd.Kb.slnx --configuration Release --no-build -p:IncludeSymbols=false -p:SymbolPackageFormat=snupkg --output ./artifacts
45+
run: dotnet pack MarkdownLd.Kb.slnx --configuration Release -p:IncludeSymbols=false -p:SymbolPackageFormat=snupkg --output ./artifacts
4646

4747
- name: Upload coverage artifact
4848
uses: actions/upload-artifact@v4
4949
with:
5050
name: coverage
51-
path: TestResults/CoverletMtpFiltered/**/coverage.cobertura.xml
51+
path: TestResults/TUnitCoverage/coverage.cobertura.xml
5252
retention-days: 5
5353

5454
- name: Upload package artifact

AGENTS.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,8 @@ Target capabilities:
6262
- Tests must verify the real Markdown -> graph -> query/search flow, including success, malformed input, and empty/no-match paths where relevant.
6363
- Tests must not use mocks, stubs, or fakes, except one local test implementation of `Microsoft.Extensions.AI.IChatClient` used to prove the LLM extraction boundary without network access.
6464
- Use TUnit for tests and Shouldly for assertions.
65+
- Fallbacks are forbidden. Do not silently substitute generic/default behaviour when parsing, extraction, graph building, query execution, or AI extraction fails; fail explicitly or skip the invalid fact with a caller-visible test.
66+
- Legacy or leftover old code is forbidden. Do not keep deprecated duplicate paths, compatibility shims, or unused old implementations; remove them immediately unless an ADR documents a temporary migration path.
6567

6668
## Global Skills
6769

@@ -85,9 +87,9 @@ List only the skills this solution actually uses.
8587

8688
- `restore`: `dotnet restore MarkdownLd.Kb.slnx`
8789
- `build`: `dotnet build MarkdownLd.Kb.slnx --no-restore`
88-
- `test`: `dotnet test MarkdownLd.Kb.slnx --no-build`
90+
- `test`: `dotnet test MarkdownLd.Kb.slnx --configuration Release`
8991
- `format`: `dotnet format MarkdownLd.Kb.slnx --verify-no-changes`
90-
- `coverage`: `dotnet test MarkdownLd.Kb.slnx --no-build --coverlet --coverlet-output-format cobertura --coverlet-include '[ManagedCode.MarkdownLd.Kb]*' --results-directory TestResults/CoverletMtpFiltered`
92+
- `coverage`: `dotnet test MarkdownLd.Kb.slnx --configuration Release -- --coverage --coverage-output-format cobertura --coverage-output "$PWD/TestResults/TUnitCoverage/coverage.cobertura.xml" --coverage-settings "$PWD/CodeCoverage.runsettings"`
9193

9294
`.NET` runner policy:
9395

@@ -96,7 +98,7 @@ List only the skills this solution actually uses.
9698
- Test framework: TUnit.
9799
- Assertion library: Shouldly.
98100
- Test runner model: TUnit through `dotnet test` / Microsoft.Testing.Platform-compatible execution.
99-
- Coverage: `coverlet.MTP` through `dotnet test --coverlet` with the production assembly include filter.
101+
- Coverage: `Microsoft.Testing.Extensions.CodeCoverage` through TUnit / Microsoft Testing Platform with `CodeCoverage.runsettings`.
100102

101103
### Project AGENTS Policy
102104

Directory.Packages.props

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,13 @@
44
</PropertyGroup>
55

66
<ItemGroup>
7-
<PackageVersion Include="coverlet.collector" Version="8.0.1" />
8-
<PackageVersion Include="coverlet.MTP" Version="8.0.1" />
97
<PackageVersion Include="dotNetRdf" Version="3.5.1" />
108
<PackageVersion Include="DotNet.ReproducibleBuilds" Version="1.2.25" />
119
<PackageVersion Include="Markdig" Version="1.1.2" />
1210
<PackageVersion Include="Microsoft.Extensions.AI" Version="10.4.1" />
1311
<PackageVersion Include="Microsoft.Extensions.AI.Abstractions" Version="10.4.1" />
1412
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.4.0" />
13+
<PackageVersion Include="Microsoft.Testing.Extensions.CodeCoverage" Version="18.6.2" />
1514
<PackageVersion Include="Shouldly" Version="4.3.0" />
1615
<PackageVersion Include="TUnit" Version="1.31.0" />
1716
<PackageVersion Include="YamlDotNet" Version="17.0.1" />

README.md

Lines changed: 62 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,9 @@ The pipeline extracts:
2424

2525
- article identity, title, summary, dates, tags, authors, and topics from YAML front matter
2626
- heading sections and document identity from Markdown
27-
- wikilinks such as `[[RDF]]`
2827
- Markdown links such as `[SPARQL](https://www.w3.org/TR/sparql11-query/)`
29-
- assertion arrows such as `article --mentions--> RDF`
28+
- optional wikilinks such as `[[RDF]]`
29+
- optional assertion arrows such as `article --mentions--> RDF`
3030
- optional LLM-produced entities and assertions through `Microsoft.Extensions.AI.IChatClient`
3131

3232
## Install
@@ -48,11 +48,9 @@ using ManagedCode.MarkdownLd.Kb.Pipeline;
4848

4949
internal static class MinimalGraphDemo
5050
{
51-
private const string BaseUriText = "https://kb.example/";
52-
private const string ArticlePath = "content/zero-cost-knowledge-graph.md";
5351
private const string SearchTerm = "rdf";
54-
private const string SubjectKey = "subject";
55-
private const string RdfEntityUri = "https://kb.example/id/rdf";
52+
private const string NameKey = "name";
53+
private const string RdfLabel = "RDF";
5654

5755
private const string ArticleMarkdown = """
5856
---
@@ -62,46 +60,38 @@ tags:
6260
- markdown
6361
- rdf
6462
author:
65-
- label: Ada Lovelace
66-
type: schema:Person
67-
entity_hints:
68-
- label: RDF
69-
type: schema:Thing
70-
sameAs:
71-
- https://www.w3.org/RDF/
63+
- Ada Lovelace
7264
---
7365
# Zero Cost Knowledge Graph
7466
75-
Markdown-LD Knowledge Bank links [[RDF]] and [SPARQL](https://www.w3.org/TR/sparql11-query/).
76-
article --mentions--> RDF
77-
RDF --sameas--> https://www.w3.org/RDF/
67+
Markdown-LD Knowledge Bank links [RDF](https://www.w3.org/RDF/) and [SPARQL](https://www.w3.org/TR/sparql11-query/).
7868
""";
7969

80-
private const string AskQuery = """
70+
private const string SelectFactsQuery = """
8171
PREFIX schema: <https://schema.org/>
82-
ASK WHERE {
83-
<https://kb.example/zero-cost-knowledge-graph/> schema:name "Zero Cost Knowledge Graph" ;
84-
schema:keywords "markdown" ;
85-
schema:mentions <https://kb.example/id/rdf> .
86-
<https://kb.example/id/rdf> schema:sameAs <https://www.w3.org/RDF/> .
72+
SELECT ?article ?entity ?name WHERE {
73+
?article a schema:Article ;
74+
schema:name "Zero Cost Knowledge Graph" ;
75+
schema:keywords "markdown" ;
76+
schema:mentions ?entity .
77+
?entity schema:name ?name ;
78+
schema:sameAs <https://www.w3.org/RDF/> .
8779
}
8880
""";
8981

9082
public static async Task RunAsync()
9183
{
92-
var pipeline = new MarkdownKnowledgePipeline(new Uri(BaseUriText));
84+
var pipeline = new MarkdownKnowledgePipeline();
9385

94-
var result = await pipeline.BuildAsync([
95-
new MarkdownSourceDocument(ArticlePath, ArticleMarkdown),
96-
]);
86+
var result = await pipeline.BuildFromMarkdownAsync(ArticleMarkdown);
9787

98-
var graphHasExpectedFacts = await result.Graph.ExecuteAskAsync(AskQuery);
88+
var graphRows = await result.Graph.ExecuteSelectAsync(SelectFactsQuery);
9989
var search = await result.Graph.SearchAsync(SearchTerm);
10090

101-
Console.WriteLine(graphHasExpectedFacts);
91+
Console.WriteLine(graphRows.Rows.Count);
10292
Console.WriteLine(search.Rows.Any(row =>
103-
row.Values.TryGetValue(SubjectKey, out var subject) &&
104-
subject == RdfEntityUri));
93+
row.Values.TryGetValue(NameKey, out var name) &&
94+
name == RdfLabel));
10595
}
10696
}
10797
```
@@ -113,14 +103,13 @@ using ManagedCode.MarkdownLd.Kb.Pipeline;
113103

114104
internal static class FileGraphDemo
115105
{
116-
private const string BaseUriText = "https://kb.example/";
117106
private const string FilePath = "/absolute/path/to/content/article.md";
118107
private const string DirectoryPath = "/absolute/path/to/content";
119108
private const string MarkdownSearchPattern = "*.md";
120109

121110
public static async Task RunAsync()
122111
{
123-
var pipeline = new MarkdownKnowledgePipeline(new Uri(BaseUriText));
112+
var pipeline = new MarkdownKnowledgePipeline();
124113

125114
var singleFile = await pipeline.BuildFromFileAsync(FilePath);
126115
var directory = await pipeline.BuildFromDirectoryAsync(
@@ -135,17 +124,24 @@ internal static class FileGraphDemo
135124

136125
`KnowledgeSourceDocumentConverter` supports Markdown and other text-like knowledge inputs: `.md`, `.markdown`, `.mdx`, `.txt`, `.text`, `.log`, `.csv`, `.json`, `.jsonl`, `.yaml`, and `.yml`.
137126

127+
You do not need to pass a base URI for normal use. Document identity is resolved in this order:
128+
129+
- `canonicalUrl` or `canonical_url` in Markdown front matter
130+
- the file path, normalized the same way as the upstream project: `content/notes/rdf.md` becomes a stable document IRI
131+
- the generated inline document path when `BuildFromMarkdownAsync` is called without a path
132+
133+
The library uses `urn:managedcode:markdown-ld-kb:/` as an internal default base URI only to create valid RDF IRIs when the Markdown does not provide a canonical URL. Pass `new MarkdownKnowledgePipeline(new Uri("https://your-domain/"))` only when you want generated document/entity IRIs to live under your own domain.
134+
138135
## Optional AI Extraction
139136

140-
The core library depends on `Microsoft.Extensions.AI.IChatClient`, not on a provider-specific SDK. Your host application owns the concrete provider, credentials, model choice, and optional Microsoft Agent Framework orchestration.
137+
Optional AI extraction enriches the deterministic Markdown graph with entities and assertions returned by an injected `Microsoft.Extensions.AI.IChatClient`. The package stays provider-neutral: it does not reference OpenAI, Azure OpenAI, Anthropic, or any other model-specific SDK. If no chat client is provided, the pipeline still runs fully locally and builds the graph from Markdown/front matter/link extraction only.
141138

142139
```csharp
143140
using ManagedCode.MarkdownLd.Kb.Pipeline;
144141
using Microsoft.Extensions.AI;
145142

146143
internal static class AiGraphDemo
147144
{
148-
private const string BaseUriText = "https://kb.example/";
149145
private const string ArticlePath = "content/entity-extraction.md";
150146

151147
private const string ArticleMarkdown = """
@@ -160,28 +156,28 @@ The article mentions Markdown-LD Knowledge Bank, SPARQL, RDF, and entity extract
160156
private const string AskQuery = """
161157
PREFIX schema: <https://schema.org/>
162158
ASK WHERE {
163-
<https://kb.example/entity-extraction/> schema:mentions ?entity .
159+
?article a schema:Article ;
160+
schema:name "Entity Extraction RDF Pipeline" ;
161+
schema:mentions ?entity .
164162
?entity schema:name ?name .
165163
}
166164
""";
167165

168166
public static async Task RunAsync(IChatClient chatClient)
169167
{
170-
var pipeline = new MarkdownKnowledgePipeline(
171-
new Uri(BaseUriText),
172-
chatClient);
168+
var pipeline = new MarkdownKnowledgePipeline(chatClient: chatClient);
173169

174-
var result = await pipeline.BuildAsync([
175-
new MarkdownSourceDocument(ArticlePath, ArticleMarkdown),
176-
]);
170+
var result = await pipeline.BuildFromMarkdownAsync(
171+
ArticleMarkdown,
172+
path: ArticlePath);
177173

178174
var hasAiFacts = await result.Graph.ExecuteAskAsync(AskQuery);
179175
Console.WriteLine(hasAiFacts);
180176
}
181177
}
182178
```
183179

184-
The built-in chat extractor requests structured output through `GetResponseAsync<T>()` and normalizes the returned entity/assertion payload before graph construction. Tests use one local non-network `IChatClient` implementation so the full flow is covered without a live model.
180+
The built-in chat extractor requests structured output through `GetResponseAsync<T>()`, normalizes the returned entity/assertion payload, merges it with deterministic facts, and then builds the same in-memory RDF graph used by search and SPARQL. Tests use one local non-network `IChatClient` implementation so the full extraction-to-graph flow is covered without a live model.
185181

186182
## Query And Export
187183

@@ -195,7 +191,8 @@ PREFIX schema: <https://schema.org/>
195191
SELECT ?article ?title WHERE {
196192
?article a schema:Article ;
197193
schema:name ?title ;
198-
schema:mentions <https://kb.example/id/rdf> .
194+
schema:mentions ?entity .
195+
?entity schema:name "RDF" .
199196
}
200197
LIMIT 100
201198
""";
@@ -226,6 +223,20 @@ LIMIT 100
226223

227224
SPARQL execution is intentionally read-only. `SELECT` and `ASK` are allowed; mutation forms such as `INSERT`, `DELETE`, `LOAD`, `CLEAR`, `DROP`, and `CREATE` are rejected before execution.
228225

226+
## Thread Safety
227+
228+
`KnowledgeGraph` is safe for shared in-memory read/write use through its public API. Search, read-only SPARQL, and serialization run under a read lock; `MergeAsync` snapshots a built graph and merges it under a write lock.
229+
230+
Use this when many workers convert Markdown independently and publish their results into one graph:
231+
232+
```csharp
233+
var shared = await pipeline.BuildFromMarkdownAsync(string.Empty);
234+
var next = await pipeline.BuildFromMarkdownAsync(markdown, path: "content/note.md");
235+
236+
await shared.Graph.MergeAsync(next.Graph);
237+
var rows = await shared.Graph.SearchAsync("rdf");
238+
```
239+
229240
## Markdown Conventions
230241

231242
```markdown
@@ -237,25 +248,16 @@ tags:
237248
- markdown
238249
- rdf
239250
author:
240-
- label: Ada Lovelace
241-
type: schema:Person
251+
- Ada Lovelace
242252
about:
243253
- Knowledge Graph
244-
entity_hints:
245-
- label: SPARQL
246-
type: schema:Thing
247-
sameAs:
248-
- https://www.w3.org/TR/sparql11-query/
249254
---
250255
# Markdown-LD Knowledge Bank
251256

252-
Use [[RDF]] and [SPARQL](https://www.w3.org/TR/sparql11-query/).
253-
article --mentions--> RDF
254-
RDF --sameas--> https://www.w3.org/RDF/
255-
SPARQL --relatedTo--> RDF
257+
Use [RDF](https://www.w3.org/RDF/) and [SPARQL](https://www.w3.org/TR/sparql11-query/).
256258
```
257259

258-
Useful predicate forms:
260+
Optional advanced predicate forms:
259261

260262
- `mentions` becomes `schema:mentions`
261263
- `about` becomes `schema:about`
@@ -294,14 +296,15 @@ The original repository is kept as a read-only submodule under `external/lqdev-m
294296
```bash
295297
dotnet restore MarkdownLd.Kb.slnx
296298
dotnet build MarkdownLd.Kb.slnx --configuration Release --no-restore
297-
dotnet test --solution MarkdownLd.Kb.slnx --configuration Release --no-build
299+
dotnet test --solution MarkdownLd.Kb.slnx --configuration Release
298300
dotnet format MarkdownLd.Kb.slnx --verify-no-changes
299-
dotnet test --solution MarkdownLd.Kb.slnx --configuration Release --no-build --coverlet --coverlet-output-format cobertura --coverlet-include '[ManagedCode.MarkdownLd.Kb]*' --results-directory TestResults/CoverletMtpFiltered
301+
dotnet test --solution MarkdownLd.Kb.slnx --configuration Release -- --coverage --coverage-output-format cobertura --coverage-output "$PWD/TestResults/TUnitCoverage/coverage.cobertura.xml" --coverage-settings "$PWD/CodeCoverage.runsettings"
300302
```
301303

302304
Current verification baseline:
303305

304-
- tests: 67 passed, 0 failed
305-
- line coverage: 95.83%
306+
- tests: 69 passed, 0 failed
307+
- line coverage: 96.06%
308+
- branch coverage: 85.22%
306309
- target framework: .NET 10
307310
- package version: 0.0.1

docs/ADR/ADR-0001-rdf-sparql-library.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -164,9 +164,9 @@ Mitigations:
164164
### Test commands
165165

166166
- build: `dotnet build MarkdownLd.Kb.slnx --no-restore`
167-
- test: `dotnet test MarkdownLd.Kb.slnx --no-build`
167+
- test: `dotnet test MarkdownLd.Kb.slnx --configuration Release`
168168
- format: `dotnet format MarkdownLd.Kb.slnx --verify-no-changes`
169-
- coverage: `dotnet test --solution MarkdownLd.Kb.slnx --no-build --coverlet --coverlet-output-format cobertura --coverlet-include '[ManagedCode.MarkdownLd.Kb]*' --results-directory TestResults/CoverletMtpFiltered`
169+
- coverage: `dotnet test --solution MarkdownLd.Kb.slnx --configuration Release -- --coverage --coverage-output-format cobertura --coverage-output "$PWD/TestResults/TUnitCoverage/coverage.cobertura.xml" --coverage-settings "$PWD/CodeCoverage.runsettings"`
170170

171171
### New or changed tests
172172

docs/ADR/ADR-0002-llm-extraction-ichatclient.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -112,9 +112,9 @@ Mitigations:
112112
### Test commands
113113

114114
- build: `dotnet build MarkdownLd.Kb.slnx --no-restore`
115-
- test: `dotnet test MarkdownLd.Kb.slnx --no-build`
115+
- test: `dotnet test MarkdownLd.Kb.slnx --configuration Release`
116116
- format: `dotnet format MarkdownLd.Kb.slnx --verify-no-changes`
117-
- coverage: `dotnet test --solution MarkdownLd.Kb.slnx --no-build --coverlet --coverlet-output-format cobertura --coverlet-include '[ManagedCode.MarkdownLd.Kb]*' --results-directory TestResults/CoverletMtpFiltered`
117+
- coverage: `dotnet test --solution MarkdownLd.Kb.slnx --configuration Release -- --coverage --coverage-output-format cobertura --coverage-output "$PWD/TestResults/TUnitCoverage/coverage.cobertura.xml" --coverage-settings "$PWD/CodeCoverage.runsettings"`
118118

119119
## References
120120

0 commit comments

Comments
 (0)