[BUG] Adding project with existing files does not build knowledge graph

[lemon1324]

Bug Description

I have a collection of .md files in a folder, and the files have cross-reference links and appropriate frontmatter. Running basic-memory project add "notes" [location], follower by basic-memory project default "notes" and basic-memory sync reports no errors.
However, directly inspecting memory.db shows that all of the files

1. have type "file", even though the frontmatter says "note"
2. have NULL permalink
3. have no relations, despite links in the text.

Running Claude desktop and specifically asking it to use the MCP tool to create a new note results in only that entry in the database having correct metadata, type, and permalink.

Based on your bug description and our investigation, here's a complete Steps To Reproduce section:

Steps To Reproduce

Steps to reproduce the behavior:

1. Install version 0.14.2
2. Create a folder with existing .md files containing:
   • Proper frontmatter with type: note and permalink: example-permalink
   • Cross-reference links using [[filename.md]] or [[folder/filename.md]] syntax
3. Run command basic-memory project add "notes" /path/to/existing/folder
4. Run command basic-memory project default "notes"
5. Run command basic-memory sync
6. Verify sync status with basic-memory status (reports success)
7. Ask Claude Desktop to make a new note using the MCP tool.
8. Inspect SQLite database directly

Expected Behavior

All files should have type: "note", proper permalinks, and detected relations

Actual Behavior

All pre-existing files show entity_type: "file", permalink: null, no relations
Only the MCP-added file shows entity_type: "note", and has correct metadata and permalink, and relations.

Additional Notes

• Files are readable via read_note but not searchable/traversable
• sync_status reports "All projects synced successfully" despite indexing problems
• Knowledge graph features (build_context, relationship detection) non-functional for existing files

Environment

• OS: Windows 11 Home, 10.0.26100
• Python version: 3.12.4
• Basic Memory version: 0.14.2
• Installation method: uv
• Claude Desktop version (if applicable): 0.11.6

Additional Context

• basic-memory-sync.log

Possible Solution

This seems like a mismatch between actions taken by write_note when adding a file vs actions taken by add project and sync commands from the command line. It seems like adding the database logic from write_note to the logic for initial ingestion of a manually-added file should resolve the issue, assuming the database logic doesn't depend directly on any extra data passed by Claude in the write_note function.

[phernandez]

@lemon1324 its interesting that the notes that have no entity_metadata and permalink are of type text/plain. That would explain why basic-memory did not try and parse the doc and find the info in the frontmatter or create a permalink (which it writes to frontmatter).

The code tries to guess the mime-type of the file. It looks for a *.md extension and also uses the python mimetypes lib to try and guess the content type:

basic-memory/src/basic_memory/services/file_service.py

Line 261 in 6344861

def content_type(self, path: FilePath) -> str:

it seems it decided these were text/plain. I'm guessing this is a Windows issue. We'll look at it.

[phernandez]

@claude what do you think of the python mimetypes lib not finding the text/markdown mime type?

[claude]

Claude finished @phernandez's task —— View job

---

Analyzing Python mimetypes issue

• Read file_service.py code to understand mime type detection
• Analyze why mimetypes lib might fail on Windows
• Provide insights on root cause and solutions

Root Cause Analysis

The issue is in the content_type() method in file_service.py:261-281. Here's what's happening:

def content_type(self, path: FilePath) -> str:
    # ...
    mime_type, _ = mimetypes.guess_type(full_path.name)
    # ...
    content_type = mime_type or "text/plain"  # Falls back to text/plain if None
    return content_type

Problem: Python's mimetypes.guess_type() returns None for .md files on Windows because the system doesn't have a default MIME type mapping for markdown files. This causes the method to fall back to "text/plain".

Impact: The is_markdown() method checks self.content_type(path) == "text/markdown", which fails for Windows users, so their files aren't processed as markdown.

Potential Solutions

1. Explicit file extension mapping (Recommended):

def content_type(self, path: FilePath) -> str:
    path_obj = self.base_path / path if isinstance(path, str) else path
    full_path = path_obj if path_obj.is_absolute() else self.base_path / path_obj
    
    # Check for known extensions first
    if full_path.suffix.lower() == ".md":
        return "text/markdown"
    if full_path.suffix == ".canvas":
        return "application/json"
    
    # Fall back to mimetypes for other files
    mime_type, _ = mimetypes.guess_type(full_path.name)
    return mime_type or "text/plain"

2. Initialize mimetypes with common types:

# In __init__ or module level
mimetypes.add_type('text/markdown', '.md')
mimetypes.add_type('text/markdown', '.markdown')

3. Use a more robust MIME type detection library like python-magic, though this adds a dependency.

The first solution is cleanest since you already have special handling for .canvas files. It avoids Windows registry dependencies and ensures consistent behavior across platforms.

[phernandez]

I think this was related to the watch service file sync not reloading until the service was restarted. This is fixed with issue #156