Locus - Multi-Tenant File Storage Pool System

A high-performance, multi-tenant file storage pool system for .NET targeting netstandard2.0 with SQLite-based metadata management.

Overview

Locus is designed as a file queue system that provides:

Multi-tenant isolation - Each tenant has isolated storage space with enable/disable controls
Queue-based processing - Files are processed as a queue with automatic retry on failure
Unlimited storage expansion - Dynamically mount multiple storage volumes
High concurrency - Thread-safe operations with per-tenant SQLite databases and active-data caching
Automatic management - System handles directory structure, file placement, and cleanup
Directory-level quota control - Configurable file count limits per directory

Key Concepts

File Queue System

Locus is not a traditional file system where users specify file paths. Instead:

Write - User provides file content, system generates and returns a fileKey
Process - Workers fetch next pending file from queue for processing
Complete/Retry - Mark file as completed (deleted) or failed (retry)

Users never need to know:

Which storage volume holds the file
Which directory the file is in
How files are distributed across volumes

System-Generated File Keys

// Get tenant context
var tenant = await tenantManager.GetTenantAsync("tenant-001", ct);

// Write a file - system generates unique fileKey
// Optional: provide original file name to preserve extension
string fileKey = await storagePool.WriteFileAsync(tenant, fileStream, "invoice.pdf", ct);
// Returns: "f7b3c9d2-4a1e-4f8b-9c3d-2e1a4b5c6d7e"
// Physical file: ./storage/vol-001/tenant-001/f7/b3/f7b3c9d2...pdf ✅

// Without original file name (backward compatible)
string fileKey2 = await storagePool.WriteFileAsync(tenant, fileStream, null, ct);
// Physical file: ./storage/vol-001/tenant-001/a1/b2/a1b2c3d4... (no extension)

// Read file content directly
using var stream = await storagePool.ReadFileAsync(tenant, fileKey, ct);

// Get file basic information
var fileInfo = await storagePool.GetFileInfoAsync(tenant, fileKey, ct);
// Returns: FileInfo { FileKey, FileSize, CreatedAt, Status }

// Get detailed location (for diagnostics)
var location = await storagePool.GetFileLocationAsync(tenant, fileKey, ct);
// Returns: FileLocation { FileKey, VolumeId, PhysicalPath, Status, RetryCount, ... }

Architecture

flowchart LR
    Client["Client / Worker"] --> Pool["StoragePool / IStoragePool"]
    Watcher["FileWatcher auto-import"] --> Pool
    Pool --> Tenant["Tenant validation"]
    Pool --> Projection["Projection lookup"]
    Projection --> Cache["In-memory active-data cache"]
    Cache -. "warm on first access or restart" .-> Sqlite["Per-tenant SQLite projections<br/>metadata.db / quotas.db"]
    Pool --> Volume["Storage volumes<br/>physical file bytes"]
    Pool --> Journal["Per-tenant journal files<br/>queue.log / state / cursor / snapshot"]
    Journal --> Projector["QueueEventProjectionService"]
    Projector --> Sqlite
    Projector --> Cache
    Cleanup["Cleanup / recovery services"] --> Projection
    Cleanup --> Volume
    Cleanup --> Journal

Read Path

sequenceDiagram
    participant Caller
    participant StoragePool
    participant ProjectionStore
    participant Cache as In-Memory Cache
    participant SQLite as SQLite Projection
    participant Volume as Storage Volume

    Caller->>StoragePool: ReadFileAsync(tenant, fileKey)
    StoragePool->>ProjectionStore: GetProjectedFileAsync(tenantId, fileKey)
    ProjectionStore->>Cache: Get metadata

    alt Cache already warm
        Cache-->>ProjectionStore: FileMetadata
    else First tenant access or process restart
        Cache->>SQLite: Load active projection rows
        SQLite-->>Cache: Active file metadata
        Cache-->>ProjectionStore: FileMetadata
    end

    ProjectionStore-->>StoragePool: FileMetadata
    StoragePool->>Volume: ReadAsync(physicalPath)
    Volume-->>StoragePool: Stream
    StoragePool-->>Caller: Stream

Key Design Decisions

Unified API: IStoragePool combines file storage and queue processing in one interface
Per-Tenant SQLite: Each tenant has an isolated subdirectory with metadata.db and quotas.db
Active-Data Caching: Only cache files in Pending/Processing/Failed states
Completed Files: Move through Completed / DeleteRequested and are reaped by background cleanup
Atomic Quota Operations: Lock-free CAS counters + Write-Behind timer ensure concurrency safety
Startup Volume Configuration: Storage volumes are configured at startup and managed internally

Durable Queue and Recovery Model

Locus separates file content, queue-state durability, and queryable projections into different layers:

Physical files on storage volumes hold the actual bytes
Per-tenant queue.log stores durable queue-state transitions such as Accepted, ProcessingStarted, ProcessingFailed, ProcessingTimedOut, ProcessingCompleted, DeleteRequested, DeleteSucceeded, and DeadLettered
Per-tenant SQLite projections (metadata.db and quotas.db) provide the current queryable state
In-memory caches keep hot-path reads, leasing, and quota checks fast

This means SQLite is still operationally important, but it is no longer the only durable truth:

file bytes are durable on the storage volumes
queue-state transitions are durable in queue.log
SQLite stores rebuildable local projections used by normal reads, scheduling, cleanup, and reconciliation

Direct file reads should therefore be understood as:

metadata lookup is memory-first in the current process
SQLite is the projection persistence and rebuild source, not the file-content store
file bytes are always read from the mounted storage volume, never from SQLite blobs

For a full lifecycle walkthrough, including orphan recovery, startup rebuild, timeout reset, and delete reaping, see docs/storage-lifecycle-overview.md.

Quota reconciliation is now treated as an explicit maintenance operation rather than a normal scheduled runtime feature. The manual maintenance APIs on IStorageCleanupService remain available for operator repair flows and recovery tooling.

Queue Journal Growth and Compaction

queue.log is append-only during normal operation, so it grows as files move through the queue lifecycle. To avoid unbounded growth, journal compaction is enabled by default.

Compaction only runs when all of the following are true:

The queue projector has caught up to the current tenant journal tail
The processed byte range since the current base offset is at least MinBytesBeforeCompaction
A projection snapshot has been saved so rebuild can continue from snapshot + journal tail replay

Default behavior:

EnableCompaction = true
EnableAutomaticSnapshots = true
MinBytesBeforeCompaction = 32 MB in the current sample configuration
MinBytesBeforeAutomaticSnapshot = 8 MB

When compaction runs, the already-projected prefix of queue.log is trimmed away. If the entire file has already been projected, the tenant journal is truncated to an empty file and its base/tail offsets advance.

The sample appsettings file currently favors high-throughput image ingestion: AckMode = Async, JournalFormat = BinaryV1, larger projection batches, and ForceFlushAfterWrite = false on the first sample volume. Deployments that need stronger "success returned means flushed" semantics should evaluate AckMode = Durable or Balanced, and set critical volumes to ForceFlushAfterWrite = true.

Configuration Reference

Runtime options are bound from the Locus section. Keep these files aligned when option shapes change:

src/Locus/appsettings.sample.json - package-level sample
samples/Locus.Sample.Console/appsettings.json - runnable sample
docs/appsettings-sample-reference.md - field-by-field reference

The current sample includes the major runtime surfaces:

MetadataRepository, StoragePool, and QueueEventJournal tune write-behind persistence, timeout reclaim, journal ACK behavior, snapshots, projection, and compaction.
Sqlite controls WAL mode, synchronous behavior, cache size, busy timeout, and checkpoint policy.
RetryPolicy, Volumes, Tenants, and FileWatchers define retry cadence, physical storage, tenant bootstrap, and directory import behavior.
OrphanRecoveryOptions and CleanupOptions cover startup/periodic recovery, timeout reset, completed-file reaping, dead-letter handling, retired-volume metadata handling, invalid database backup cleanup, database optimization, and junk-file cleanup.

CleanupOptions.CleanupJunkFiles enables a background recursive sweep for common system files such as Thumbs.db, .DS_Store, and desktop.ini. JunkFileCleanupInterval controls the minimum time between those heavier volume scans, independent from the normal status cleanup cadence.

Core APIs

IStoragePool - Unified Storage and Queue Management

Note: IStoragePool provides a unified interface that combines file storage operations with queue-based processing. Storage volumes are configured at startup and managed internally.

public interface IStoragePool
{
    // ===== File Storage Operations =====

    // Write file → returns system-generated fileKey
    // Optional: provide originalFileName (e.g., "invoice.pdf") to preserve file extension
    Task<string> WriteFileAsync(ITenantContext tenant, Stream content, string? originalFileName, CancellationToken ct);

    // Read file by fileKey
    Task<Stream> ReadFileAsync(ITenantContext tenant, string fileKey, CancellationToken ct);

    // Get file basic information
    Task<FileInfo?> GetFileInfoAsync(ITenantContext tenant, string fileKey, CancellationToken ct);

    // Get file location (for diagnostics)
    Task<FileLocation?> GetFileLocationAsync(ITenantContext tenant, string fileKey, CancellationToken ct);

    // ===== Queue-Based Processing =====

    // Get next pending file (thread-safe, no duplicates)
    Task<FileLocation?> GetNextFileForProcessingAsync(ITenantContext tenant, CancellationToken ct);

    // Get batch of pending files
    Task<IEnumerable<FileLocation>> GetNextBatchForProcessingAsync(
        ITenantContext tenant, int batchSize, CancellationToken ct);

    // Mark file as completed → deletes file and metadata (requires lease token)
    Task MarkAsCompletedAsync(string fileKey, DateTime expectedProcessingStartTimeUtc, CancellationToken ct);

    // Mark file as failed → returns to queue for retry (requires lease token)
    Task MarkAsFailedAsync(string fileKey, DateTime expectedProcessingStartTimeUtc, string errorMessage, CancellationToken ct);

    // Get current file status
    Task<FileProcessingStatus> GetFileStatusAsync(string fileKey, CancellationToken ct);

    // ===== Capacity Management =====

    // Get total capacity across all volumes
    Task<long> GetTotalCapacityAsync(CancellationToken ct);

    // Get available space across all volumes
    Task<long> GetAvailableSpaceAsync(CancellationToken ct);
}

ITenantManager - Multi-Tenant Management

public interface ITenantManager
{
    // Get tenant context (auto-create if enabled)
    Task<ITenantContext> GetTenantAsync(string tenantId, CancellationToken ct);

    // Check if tenant is enabled
    Task<bool> IsTenantEnabledAsync(string tenantId, CancellationToken ct);

    // Create new tenant
    Task CreateTenantAsync(string tenantId, CancellationToken ct);

    // Enable/Disable tenant
    Task EnableTenantAsync(string tenantId, CancellationToken ct);
    Task DisableTenantAsync(string tenantId, CancellationToken ct);

    // Get all tenants
    Task<IEnumerable<ITenantContext>> GetAllTenantsAsync(CancellationToken ct);
}

Usage Example

Basic File Queue Processing

// Get tenant context
var tenant = await tenantManager.GetTenantAsync("tenant-001", ct);

// Producer: Write files to the queue
// Provide original file names to preserve extensions
var fileKey1 = await storagePool.WriteFileAsync(tenant, stream1, "document.pdf", ct);
var fileKey2 = await storagePool.WriteFileAsync(tenant, stream2, "invoice.xlsx", ct);
// Files are automatically queued as "Pending" status

// Consumer: Process files from the queue (10 concurrent workers)
var tasks = Enumerable.Range(0, 10).Select(async threadId =>
{
    while (true)
    {
        // Get next file (thread-safe, no duplicates across threads)
        var file = await storagePool.GetNextFileForProcessingAsync(tenant, ct);
        if (file == null) break; // No more files

        try
        {
            // Read and process file
            using var stream = await storagePool.ReadFileAsync(tenant, file.FileKey, ct);
            await ProcessFileAsync(stream);

            // Success: mark as completed (deletes file and metadata)
            var expectedProcessingStartTimeUtc = file.ProcessingStartTime
                ?? throw new InvalidOperationException("Missing processing start time.");
            await storagePool.MarkAsCompletedAsync(file.FileKey, expectedProcessingStartTimeUtc, ct);
        }
        catch (Exception ex)
        {
            // Failure: return to queue for retry
            var expectedProcessingStartTimeUtc = file.ProcessingStartTime
                ?? throw new InvalidOperationException("Missing processing start time.");
            await storagePool.MarkAsFailedAsync(file.FileKey, expectedProcessingStartTimeUtc, ex.Message, ct);
            // File will be retried based on retry policy
        }
    }
});

await Task.WhenAll(tasks);

Batch Processing

// Get tenant context
var tenant = await tenantManager.GetTenantAsync("tenant-001", ct);

// Process files in batches
while (true)
{
    // Get batch of 100 files
    var batch = await storagePool.GetNextBatchForProcessingAsync(tenant, 100, ct);
    if (!batch.Any()) break;

    // Process batch in parallel
    await Parallel.ForEachAsync(batch, ct, async (file, token) =>
    {
        try
        {
            using var stream = await storagePool.ReadFileAsync(tenant, file.FileKey, token);
            await ProcessFileAsync(stream);
            var expectedProcessingStartTimeUtc = file.ProcessingStartTime
                ?? throw new InvalidOperationException("Missing processing start time.");
            await storagePool.MarkAsCompletedAsync(file.FileKey, expectedProcessingStartTimeUtc, token);
        }
        catch (Exception ex)
        {
            var expectedProcessingStartTimeUtc = file.ProcessingStartTime
                ?? throw new InvalidOperationException("Missing processing start time.");
            await storagePool.MarkAsFailedAsync(file.FileKey, expectedProcessingStartTimeUtc, ex.Message, token);
        }
    });
}

Performance

Benchmark Results

Performance benchmarks run on Intel Core i5-9400 CPU 2.90GHz (Coffee Lake), 6 cores, .NET 10.0.0, Windows 11:

Write Throughput (Single-Threaded, 100 KB file)

File Size	Mean Time	Allocated	Notes
100 KB	1.074 ms	7.34 KB	End-to-end: quota check + disk write + metadata
1 MB	1.296 ms	7.34 KB	I/O dominated
10 MB	4.736 ms	12.55 KB	I/O dominated

Concurrent Write Scalability (1 MB per write)

Concurrency	Mean Time	StdDev	Allocated	Notes
1 writer (baseline)	2.839 ms	0.350 ms	77.04 KB	Single-threaded baseline
10 concurrent writers	26.368 ms	6.849 ms	273.52 KB	All 10 writes in parallel
50 concurrent writers	113.463 ms	10.608 ms	701.02 KB	All 50 writes in parallel
100 concurrent writers	228.952 ms	11.859 ms	1163.76 KB	All 100 writes in parallel

Metadata Operations (Write-Behind + `_pendingKeys` Index)

Operation	Mean Time	Allocated	Notes
AddOrUpdate single file	1.878 μs	2.4 KB	⚡ Memory-first, async SQLite persistence
Get file metadata (cache hit)	40.91 ns	72 B	⚡ ConcurrentDictionary lookup
Get non-existent file (returns null)	34.87 ns	0 B	Cache miss → null, no SQLite fallback
Batch insert 100 files	237.9 μs	63.4 KB	~2.4 μs per file
Get next pending file (100-file pool)	2.975 μs	1.4 KB	⚡ O(n_pending) scan via `_pendingKeys`

Directory Quota Operations (Lock-Free CAS)

Operation	Mean Time	Allocated	Notes
Check can add (no limit)	141.25 ns	176 B	⚡ AtomicQuotaState hot path
Check can add (with limit)	139.07 ns	176 B	⚡ Lock-free CAS comparison
Increment file count	94.80 ns	72 B	⚡ Lock-free CAS, near-zero contention
Decrement file count	231.05 ns	248 B	Increment + decrement pair
Set directory limit	2.916 ms	142.2 KB	SQLite write (persisted immediately)
Get file count	145.41 ns	232 B	⚡ AtomicQuotaState read

Volume Health & Space (TTL Cached)

Operation	Mean Time	Notes
IsHealthy	17.88 ns	⚡ 30s TTL cache (no disk I/O)
AvailableSpace	22.33 ns	⚡ 30s TTL cache
TotalCapacity	22.32 ns	⚡ 30s TTL cache

Tenant Management

Operation	Mean Time	Allocated	Notes
Create tenant	2.589 ms	7.0 KB	JSON write + directory creation
Get tenant (cache hit)	81.95 ns	104 B	⚡ 5-minute in-memory cache
Get tenant (cache miss)	24.23 μs	1.28 KB	JSON file read + parse
Get tenant (auto-create)	2.116 ms	9.3 KB	Create + load
Check tenant enabled (cache hit)	89.86 ns	104 B	⚡ Cache hit
Enable tenant	3.060 ms	21.1 KB	JSON update + cache invalidation
Disable tenant	1.864 ms	14.3 KB	JSON update + cache invalidation

End-to-End Concurrent Operations

Last updated: 2026-02-28 (BenchmarkDotNet v0.15.8, .NET 10.0.0, IterationCount=10, WarmupCount=3)

Operation	threadCount	Mean Time	Allocated	Notes
Core path: 100 concurrent writes (no Task.Run)	—	77.599 ms	26243.21 KB	Core path, no Task.Run
10 concurrent reads (pure read)	—	0.653 ms	104.10 KB	Read-only, no setup
Core path: 10 concurrent reads (no Task.Run)	—	2.128 ms	104.43 KB	Core path, no Task.Run
10 concurrent reads (write+read)	—	8.802 ms	3132.49 KB	Includes pre-write setup + parallel reads
Mixed read/write (20 ops)	—	7.840 ms	4417.77 KB	10 writes + 10 reads concurrent
Concurrent writes	10	3.149 ms	2158.13 KB	10 simultaneous writes
Concurrent writes	50	13.551 ms	10720.19 KB	50 simultaneous writes
Concurrent writes	100	26.945 ms	22256.45 KB	100 simultaneous writes

ConcurrentReads_PureRead benchmark is now included to report read-only throughput separately from setup cost.

Key Findings:

⚡ Directory quota CAS: 94.80 ns per increment — lock-free atomic operations (vs. ~200 μs with SemaphoreSlim)
⚡ Volume health/space: 17–22 ns — 30-second TTL cache eliminates one disk I/O per write
⚡ Metadata write-behind: 1.878 μs per file — memory-first, SQLite persistence is async
⚡ Metadata cache hit: 40.91 ns — pure ConcurrentDictionary lookup
⚡ Tenant cache: 82–90 ns — 5-minute cache keeps tenant lookups near-zero cost
✅ 100 KB write: ~1.1 ms end-to-end (quota check + disk write + metadata)
✅ 100-concurrent writes (1 MB): 229 ms total for 100 simultaneous writes, StdDev 5.2%

Architecture-Sensitive Update (2026-04-09)

The benchmark suite was updated after the cleanup lifecycle changed from delete-only handling to the default dead-letter disposition, and after DeadLettered became part of the active metadata lifecycle.

Benchmark	Parameters	Mean	StdDev	Allocated
Cleanup status path (`MoveToDeadLetter`)	1200 processing + 800 permanently failed	848.0 ms	21.56 ms	19.28 MB
Cleanup status path (`Delete`)	1200 processing + 800 permanently failed	559.8 ms	76.37 ms	13.97 MB
Cold-start active-index load (`StartupLoadBatchSize=512`)	20000 active rows, includes `DeadLettered`	114.6 ms	22.38 ms	29.94 MB
Cold-start active-index load (`StartupLoadBatchSize=2000`)	20000 active rows, includes `DeadLettered`	111.9 ms	14.62 ms	29.55 MB

These numbers were collected on 2026-04-09 with BenchmarkDotNet v0.15.8, .NET 10.0.5, Windows 11 25H2, on an Intel Core Ultra 9 185H host, and the runs were executed outside the sandbox as requested.

Running Benchmarks

cd tests/Locus.Benchmarks
dotnet run -c Release

# Run specific benchmark class
dotnet run -c Release --filter "Locus.Benchmarks.MetadataRepositoryBenchmarks*"
dotnet run -c Release --filter "Locus.Benchmarks.DirectoryQuotaBenchmarks*"
dotnet run -c Release --filter "Locus.Benchmarks.TenantManagerBenchmarks*"
dotnet run -c Release --filter "Locus.Benchmarks.StoragePoolWriteThroughputBenchmarks*"
dotnet run -c Release --filter "Locus.Benchmarks.StoragePoolConcurrencyBenchmarks*"
dotnet run -c Release --filter "Locus.Benchmarks.VolumeHealthCheckBenchmarks*"

See Benchmark README for detailed analysis.

Project Structure

Locus/
├── src/
│   ├── Locus.Core/              # Core abstractions and interfaces
│   │   ├── Abstractions/
│   │   │   ├── IStoragePool.cs
│   │   │   ├── IFileScheduler.cs
│   │   │   ├── ITenantManager.cs
│   │   │   ├── IStorageVolume.cs
│   │   │   ├── IDirectoryQuotaManager.cs
│   │   │   ├── ITenantQuotaManager.cs
│   │   │   └── IStorageCleanupService.cs
│   │   ├── Models/
│   │   │   ├── FileLocation.cs
│   │   │   ├── FileProcessingStatus.cs
│   │   │   ├── TenantStatus.cs
│   │   │   ├── FileRetryPolicy.cs
│   │   │   ├── DirectoryQuotaConfig.cs
│   │   │   └── CleanupStatistics.cs
│   │   └── Exceptions/
│   │       ├── TenantDisabledException.cs
│   │       ├── TenantNotFoundException.cs
│   │       ├── DirectoryQuotaExceededException.cs
│   │       ├── NoFilesAvailableException.cs
│   │       └── InsufficientStorageException.cs
│   ├── Locus.FileSystem/        # Local file system implementation
│   │   ├── LocalFileSystemVolume.cs
│   │   └── FileSystemPathSanitizer.cs
│   ├── Locus.Storage/           # Storage pool and metadata management
│   │   ├── StoragePool.cs
│   │   ├── FileScheduler.cs
│   │   ├── DirectoryQuotaManager.cs
│   │   ├── TenantQuotaManager.cs
│   │   ├── StorageCleanupService.cs
│   │   └── Data/
│   │       ├── FileMetadata.cs
│   │       ├── DirectoryQuota.cs
│   │       ├── MetadataRepository.cs
│   │       └── DirectoryQuotaRepository.cs
│   ├── Locus.MultiTenant/       # Multi-tenant isolation
│   │   ├── TenantManager.cs
│   │   ├── TenantContext.cs
│   │   └── Data/
│   │       └── TenantMetadata.cs
│   └── Locus/                   # Main library (aggregates all components)
│       ├── LocusBuilder.cs
│       └── ServiceCollectionExtensions.cs
├── tests/
│   ├── Locus.FileSystem.Tests/  # 51 tests ✅
│   ├── Locus.Storage.Tests/     # 179 tests ✅
│   ├── Locus.MultiTenant.Tests/ # 12 tests ✅
│   ├── Locus.IntegrationTests/  # 10 tests ✅
│   └── Locus.Benchmarks/        # Performance benchmarks
│       ├── MetadataRepositoryBenchmarks.cs
│       ├── DirectoryQuotaBenchmarks.cs
│       ├── TenantManagerBenchmarks.cs
│       └── ConcurrentOperationsBenchmarks.cs
└── samples/
    └── Locus.Sample.Console/

Verification

The solution is covered by focused xUnit projects and BenchmarkDotNet harnesses:

tests/Locus.FileSystem.Tests
tests/Locus.Storage.Tests
tests/Locus.MultiTenant.Tests
tests/Locus.IntegrationTests
tests/Locus.Benchmarks

Use dotnet test Locus.sln --no-build after a successful build, or run a focused project such as dotnet test tests/Locus.Storage.Tests/Locus.Storage.Tests.csproj --no-restore while working on storage regressions.

Implementation Status

Locus is implemented as a single public package surface over the core modules, with the console sample and configuration reference kept in the repository:

Runtime Core:

Solution and project structure with central package management
IStoragePool, IFileScheduler, ITenantManager, FileWatcher, cleanup, and quota abstractions
LocalFileSystemVolume, path sanitization, volume health checks, and cross-platform path handling
Per-tenant metadata and quota persistence using SQLite with WAL-oriented defaults

Queue and Recovery:

Durable per-tenant queue.log with binary journal format support
Background projection to metadata/quota state, automatic snapshots, and compaction
Startup database health checks and corrupted database recovery
Processing timeout recovery, orphan recovery, retired-volume metadata handling, and dead-letter lifecycle

Operations and Configuration:

LocusBuilder and appsettings binding through the Locus section
Background cleanup for completed files, timed-out processing rows, permanently failed rows, invalid database backups, and junk files
FileWatcher auto-import with single-tenant and multi-tenant modes
Sample console application with current appsettings defaults

Build Commands

# Build the solution
dotnet build

# Build in Release mode
dotnet build -c Release

# Run tests
dotnet test

# Pack NuGet package (includes all components)
dotnet pack src/Locus/Locus.csproj -c Release

Documentation

Core Documentation

CLAUDE.md - Complete implementation guidelines, architecture decisions, API references, and FileWatcher usage guide
docs/appsettings-sample-reference.md - Current appsettings field reference with JSONC-style comments
docs/storage-lifecycle-overview.md - End-to-end lifecycle, projection, recovery, and cleanup flow
docs/queue-journal-observability.md - Queue journal metrics, state files, and recovery signals

Sample Projects

Locus.Sample.Console - Complete working example with appsettings.json configuration

Key Features

📦 Multi-Tenant Storage

Tenant isolation with enable/disable controls
Per-tenant SQLite databases in isolated subdirectories
Active-data caching for high concurrency
File extension preservation - Original file extensions are preserved in physical storage

🔄 File Queue Processing

System-generated file keys
Automatic retry on failure with exponential backoff
Processing status tracking (Pending -> Processing -> Completed/Failed/PermanentlyFailed/DeadLettered)
Durable queue journal, projection snapshots, and compaction

📁 FileWatcher Auto-Import

Multi-tenant mode with automatic directory creation
Configurable polling intervals and concurrency
Post-import actions (Delete/Move/Keep)

🧹 Automatic Cleanup

Completed-file reaping and empty directory cleanup
Junk-file cleanup for Thumbs.db, .DS_Store, and desktop.ini
Timeout detection and reset
Orphan recovery and orphaned metadata cleanup
Failed file retention, dead-letter handling, and retired-volume metadata policies

🔧 Storage Management

Dynamic volume mounting/unmounting
Automatic volume expansion
Load balancing across volumes
Directory-level quota control

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 199 Commits
.github/workflows		.github/workflows
docs		docs
samples/Locus.Sample.Console		samples/Locus.Sample.Console
src		src
tests		tests
.gitignore		.gitignore
AGENTS.md		AGENTS.md
CLAUDE.md		CLAUDE.md
Directory.Packages.props		Directory.Packages.props
Locus.sln		Locus.sln
README.md		README.md
common.props		common.props

Folders and files

Latest commit

History

Repository files navigation

Locus - Multi-Tenant File Storage Pool System

Overview

Key Concepts

File Queue System

System-Generated File Keys

Architecture

Read Path

Key Design Decisions

Durable Queue and Recovery Model

Queue Journal Growth and Compaction

Configuration Reference

Core APIs

IStoragePool - Unified Storage and Queue Management

ITenantManager - Multi-Tenant Management

Usage Example

Basic File Queue Processing

Batch Processing

Performance

Benchmark Results

Write Throughput (Single-Threaded, 100 KB file)

Concurrent Write Scalability (1 MB per write)

Metadata Operations (Write-Behind + _pendingKeys Index)

Directory Quota Operations (Lock-Free CAS)

Volume Health & Space (TTL Cached)

Tenant Management

End-to-End Concurrent Operations

Architecture-Sensitive Update (2026-04-09)

Running Benchmarks

Project Structure

Verification

Implementation Status

Build Commands

Documentation

Core Documentation

Sample Projects

Key Features

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases 32

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Metadata Operations (Write-Behind + `_pendingKeys` Index)

Packages