Skip to content

Commit 98953f2

Browse files
wisedev-pstachwisedev-pstach
authored
Merge pull request #10 from wisedev-code/0.3.4-release,-knowledge-base
feat: docs for knowledge base
2 parents ff5c630 + d607e87 commit 98953f2

5 files changed

Lines changed: 842 additions & 0 deletions

File tree

‎src/assets/docs/docs-index.json‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,26 @@
105105
}
106106
]
107107
},
108+
{
109+
"title": "Knowledge base",
110+
"children": [
111+
{
112+
"title": "Overview",
113+
"path": "examples/agent-knowledge-overview"
114+
},
115+
{
116+
"title": "With File source",
117+
"path": "examples/example-agent-knowledge-file"
118+
},
119+
{
120+
"title": "With Web source",
121+
"path": "examples/example-agent-knowledge-web"
122+
},
123+
{
124+
"title": "With Mcp source",
125+
"path": "examples/example-agent-knowledge-mcp"
126+
}
127+
]},
108128
{
109129
"title":"Flow",
110130
"children":[

‎src/assets/docs/examples/agent-knowledge-overview.md‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Knowledge Base Feature Overview
2+
3+
## Introduction
4+
5+
The Knowledge Base feature in MaIN.NET 0.5.0 enables agents to access and utilize external information sources intelligently. This feature allows agents to provide accurate, context-aware responses by leveraging persisted knowledge from files, web sources, and MCP (Model Context Protocol) servers.
6+
7+
## How Knowledge Base Works
8+
9+
### Core Concept
10+
The knowledge base system creates an `index.json` file that contains a set of sources with assigned tags. When an agent receives a query, it can:
11+
12+
1. **Evaluate Query Relevance** - Determine if external knowledge is needed
13+
2. **Tag Matching** - Find relevant knowledge sources based on tags
14+
3. **Source Selection** - Choose appropriate knowledge items (1-4 tags maximum)
15+
4. **Context Integration** - Use selected knowledge to enhance responses
16+
17+
### Knowledge Storage Structure
18+
Knowledge is persisted as an index with the following structure:
19+
- **Name**: Identifier for the knowledge item
20+
- **Type**: File, URL, Text, or MCP
21+
- **Value**: Actual content or reference
22+
- **Tags**: Array of searchable keywords
23+
24+
## Knowledge Decision Process
25+
26+
The `AnswerCommandHandler` implements intelligent knowledge usage through three modes:
27+
28+
### KnowledgeUsage.UseMemory
29+
Uses traditional chat memory without external knowledge sources.
30+
31+
### KnowledgeUsage.UseKnowledge
32+
Intelligently decides whether to use knowledge based on query analysis:
33+
- Evaluates if the question requires external knowledge
34+
- Uses knowledge unless certain the question needs only basic facts (like "What is 2+2?" or "Capital of France?")
35+
- When in doubt, defaults to using external knowledge
36+
37+
### KnowledgeUsage.AlwaysUseKnowledge
38+
Always processes queries through the knowledge base system.
39+
40+
## Knowledge Types Supported
41+
42+
### File Sources (KnowledgeItemType.File)
43+
- Local files (Markdown, text, etc.)
44+
- Added to `ChatMemoryOptions.FilesData`
45+
- Example: Company documentation, policies, manuals
46+
47+
### Web Sources (KnowledgeItemType.Url)
48+
- Web pages and online resources
49+
- Added to `ChatMemoryOptions.WebUrls`
50+
- Example: Online tutorials, documentation sites
51+
52+
### Text Sources (KnowledgeItemType.Text)
53+
- Direct text content
54+
- Added to `ChatMemoryOptions.TextData`
55+
- Example: Structured data, snippets
56+
57+
### MCP Sources (KnowledgeItemType.Mcp)
58+
- Model Context Protocol servers
59+
- Handled through `mcpService.Prompt()`
60+
- Example: GitHub integration, filesystem access, web search
61+
62+
## Tag-Based Retrieval System
63+
64+
The system uses JSON-serialized tag matching:
65+
66+
1. **Index Creation**: Available knowledge sources with their tags
67+
2. **Query Analysis**: LLM determines relevant tags for the user query
68+
3. **Source Filtering**: Finds knowledge items where tags intersect with query tags or match item names
69+
4. **Context Building**: Selected sources are integrated into the chat context
70+
71+
## KnowledgeBuilder API
72+
73+
The `KnowledgeBuilder` provides a fluent API for constructing knowledge bases:
74+
75+
### File Sources
76+
```csharp
77+
KnowledgeBuilder.Instance
78+
.AddFile("filename", "./path/to/file.md", tags: ["tag1", "tag2"])
79+
```
80+
81+
### Web Sources
82+
```csharp
83+
KnowledgeBuilder.Instance
84+
.AddUrl("source_name", "https://example.com", tags: ["web", "tutorial"])
85+
```
86+
87+
### MCP Sources
88+
```csharp
89+
KnowledgeBuilder.Instance
90+
.AddMcp(new Mcp
91+
{
92+
Name = "ServerName",
93+
Command = "npx",
94+
Arguments = ["server-package"],
95+
Backend = BackendType.OpenAi,
96+
Model = "gpt-4"
97+
}, ["mcp", "external"])
98+
```
99+
100+
## Integration with Agents
101+
102+
Knowledge bases integrate seamlessly with MaIN.NET agents:
103+
104+
```csharp
105+
var context = await AIHub.Agent()
106+
.WithModel("your-model")
107+
.WithKnowledge(KnowledgeBuilder.Instance
108+
.AddFile(...)
109+
.AddUrl(...)
110+
.AddMcp(...))
111+
.WithSteps(StepBuilder.Instance
112+
.AnswerUseKnowledge()
113+
.Build())
114+
.CreateAsync();
115+
```
116+
117+
## Performance and Limitations
118+
119+
- **Tag Limit**: Returns 1-4 tags maximum per query to maintain focus
120+
- **MCP Limitation**: Cannot combine responses from multiple MCP servers in one request
121+
- **Grammar Control**: Uses structured grammars for decision-making and tag selection
122+
- **Memory Integration**: Knowledge sources are added to chat memory for processing
123+
124+
## Notification System
125+
126+
The system provides progress notifications showing:
127+
- Selected knowledge items (Name|Type format)
128+
- Model being used for processing
129+
- Real-time updates during knowledge processing
130+
131+
This knowledge base implementation provides a lightweight yet powerful way to enhance agent capabilities with external information sources while maintaining performance and accuracy.

‎src/assets/docs/examples/example-agent-knowledge-file.md‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# Agent with Knowledge File Example
2+
3+
## Overview
4+
5+
This example demonstrates how to create an agent that can access and query local file-based knowledge sources. The agent acts as a company assistant for "TechVibe Solutions," helping employees find answers about company information stored in multiple markdown files.
6+
7+
## Code Example
8+
9+
```csharp
10+
using Examples.Utils;
11+
using MaIN.Core.Hub;
12+
using MaIN.Core.Hub.Utils;
13+
using MaIN.Domain.Entities.Agents.AgentSource;
14+
15+
namespace Examples.Agents;
16+
17+
public class AgentWithKnowledgeFileExample : IExample
18+
{
19+
public async Task Start()
20+
{
21+
Console.WriteLine("Agent with knowledge base example");
22+
AIHub.Extensions.DisableLLamaLogs();
23+
24+
var context = await AIHub.Agent()
25+
.WithModel("gemma3:4b")
26+
.WithInitialPrompt("""
27+
You are a helpful assistant that answers questions about a company. Try to
28+
help employees find answers to their questions. Company you work for is TechVibe Solutions.
29+
""")
30+
.WithKnowledge(KnowledgeBuilder.Instance
31+
.AddFile("people.md", "./Files/Knowledge/people.md",
32+
tags: ["workers", "employees", "company"])
33+
.AddFile("organization.md", "./Files/Knowledge/organization.md",
34+
tags:["company structure", "company policy", "company culture", "company overview"])
35+
.AddFile("events.md", "./Files/Knowledge/events.md",
36+
tags: ["company events", "company calendar", "company agenda"])
37+
.AddFile("office_layout.md", "./Files/Knowledge/office_layout.md",
38+
tags: ["company layout", "company facilities", "company environment", "office items", "supplies"]))
39+
.WithSteps(StepBuilder.Instance
40+
.AnswerUseKnowledge()
41+
.Build())
42+
.CreateAsync();
43+
44+
var result = await context
45+
.ProcessAsync("Hey! Where I can find some printer paper?");
46+
Console.WriteLine(result.Message.Content);
47+
}
48+
}
49+
```
50+
51+
## Key Components
52+
53+
### Model Configuration
54+
- **Model**: `gemma3:4b` - A lightweight local model suitable for company assistance tasks
55+
- **Initial Prompt**: Establishes the agent's role as a TechVibe Solutions company assistant
56+
57+
### Knowledge Sources
58+
59+
The example demonstrates four different file-based knowledge sources:
60+
61+
#### 1. People Directory (`people.md`)
62+
- **Purpose**: Employee information and contact details
63+
- **Tags**: `["workers", "employees", "company"]`
64+
- **Use Cases**: Finding colleagues, contact information, organizational structure
65+
66+
#### 2. Organization Information (`organization.md`)
67+
- **Purpose**: Company structure, policies, and culture documentation
68+
- **Tags**: `["company structure", "company policy", "company culture", "company overview"]`
69+
- **Use Cases**: Understanding company hierarchy, policies, cultural guidelines
70+
71+
#### 3. Events Calendar (`events.md`)
72+
- **Purpose**: Company events and calendar information
73+
- **Tags**: `["company events", "company calendar", "company agenda"]`
74+
- **Use Cases**: Upcoming meetings, company events, important dates
75+
76+
#### 4. Office Layout (`office_layout.md`)
77+
- **Purpose**: Physical office information, facilities, and supplies
78+
- **Tags**: `["company layout", "company facilities", "company environment", "office items", "supplies"]`
79+
- **Use Cases**: Finding office resources, navigation, facility information
80+
81+
### Step Configuration
82+
- Uses `AnswerUseKnowledge()` which intelligently determines when to access knowledge sources
83+
- The system evaluates each query to decide if external knowledge is needed
84+
85+
## Example Query Flow
86+
87+
When a user asks: **"Hey! Where I can find some printer paper?"**
88+
89+
1. **Query Analysis**: The system analyzes the question and identifies relevant tags
90+
2. **Knowledge Matching**: Matches tags like "office items" and "supplies" to `office_layout.md`
91+
3. **Content Retrieval**: Loads the relevant file content into memory
92+
4. **Response Generation**: Uses the file content to provide specific location information
93+
94+
## File Structure Requirements
95+
96+
The example expects the following file structure:
97+
```
98+
./Files/Knowledge/
99+
├── people.md
100+
├── organization.md
101+
├── events.md
102+
└── office_layout.md
103+
```
104+
105+
## Use Cases
106+
107+
This pattern is ideal for:
108+
109+
### Corporate Knowledge Bases
110+
- Employee handbooks
111+
- Policy documentation
112+
- Organizational charts
113+
- Office information
114+
115+
### Documentation Systems
116+
- Technical documentation
117+
- User manuals
118+
- FAQ collections
119+
- Standard operating procedures
120+
121+
### Educational Content
122+
- Course materials
123+
- Reference guides
124+
- Learning resources
125+
- Curriculum information
126+
127+
## Best Practices
128+
129+
### Tag Selection
130+
- Use specific, descriptive tags that match how users might ask questions
131+
- Include both broad categories (`"company"`) and specific terms (`"supplies"`)
132+
- Consider synonyms and alternative ways users might phrase queries
133+
134+
### File Organization
135+
- Keep files focused on specific topics
136+
- Use clear, descriptive filenames
137+
- Maintain consistent formatting within files
138+
- Regular updates to keep information current
139+
140+
### Model Selection
141+
- Local models like `gemma3:4b` work well for company-specific knowledge
142+
- Consider model size vs. response quality based on your needs
143+
- Test with your specific knowledge content to ensure good performance
144+
145+
## Integration Tips
146+
147+
### Error Handling
148+
Consider adding error handling for missing files:
149+
```csharp
150+
// Verify files exist before creating agent
151+
var filePaths = new[] {
152+
"./Files/Knowledge/people.md",
153+
"./Files/Knowledge/organization.md",
154+
// ... other files
155+
};
156+
157+
foreach (var path in filePaths)
158+
{
159+
if (!File.Exists(path))
160+
throw new FileNotFoundException($"Knowledge file not found: {path}");
161+
}
162+
```
163+
164+
### Dynamic Content
165+
For frequently updated content, consider:
166+
- Implementing file watchers to detect changes
167+
- Periodic knowledge base rebuilding
168+
- Version control integration for content management
169+
170+
This example provides a solid foundation for building file-based knowledge systems that can scale to handle extensive corporate or domain-specific information repositories.

0 commit comments

Comments
 (0)