Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
310 commits
Select commit Hold shift + click to select a range
8757bfc
Fix WorkerProxyStore::inner_store blocking FastSlowStore downcast
rejuvenile Mar 10, 2026
19de0b3
Eviction age/size logging + fix worker mDNS hostname
rejuvenile Mar 10, 2026
cf801c2
Fix spawn_upload_to_remote missing output directory file blobs
rejuvenile Mar 10, 2026
2da59a4
Race peer-to-peer transfer against server fetch in WorkerProxyStore
rejuvenile Mar 10, 2026
d46f12c
Fix racing to only activate on worker side (race_peers flag)
rejuvenile Mar 10, 2026
41ac67f
Perf + fix missing blob race: parallel GetTree, skip FindMissing on C…
rejuvenile Mar 10, 2026
63f1245
Fix race log levels: server wins were at debug!, invisible at info level
rejuvenile Mar 10, 2026
ee85fdc
Reduce EvictingMap contention: bound hardlink concurrency, chunk has_…
rejuvenile Mar 10, 2026
bbbe8ac
Add FindMissingBlobs logging to cas_server
rejuvenile Mar 10, 2026
287856a
Route BatchReadBlobs through peers: locality-aware parallel batch dis…
rejuvenile Mar 10, 2026
08d2fc4
Round-robin peer selection, best-effort retries, increase ByteStream …
rejuvenile Mar 10, 2026
d86480d
Worker CPU load reporting and load-aware scheduling
rejuvenile Mar 11, 2026
328943b
Fix GetTree BFS ordering, paging, and page_size=0 handling
rejuvenile Mar 11, 2026
1d9979d
Input fetch optimizations, blob eviction fix, logging, directory cach…
rejuvenile Mar 11, 2026
7b093e8
Pipelined fetch+hardlink, diagnostic logging, raise peer hints to 16384
rejuvenile Mar 11, 2026
a8db953
Concurrent blob fetch: launch all missing fetches upfront, stream har…
rejuvenile Mar 11, 2026
da9063a
Fix directory cache stripping execute bit, promote load-aware log to …
rejuvenile Mar 11, 2026
0171a11
Add CPU-load-aware scheduling tests
rejuvenile Mar 11, 2026
8430879
Directory cache scheduler routing, fix EPERM on cached executables
rejuvenile Mar 11, 2026
e52789f
Directory cache subtree caching: reuse cached subtrees via symlinks
rejuvenile Mar 11, 2026
8a1d472
Fix directory cache rename EPERM on macOS
rejuvenile Mar 11, 2026
b37e56d
Delta-encoded subtree reporting, always-set file permissions, schedul…
rejuvenile Mar 11, 2026
81e87ae
Weighted subtree coverage scoring: score workers by cached file bytes…
rejuvenile Mar 11, 2026
21c3473
Fix EPERM creating output dirs: use hardlinks not symlinks for subtre…
rejuvenile Mar 11, 2026
07e41f2
Add path diagnostics for output directory EPERM: log mode, type for e…
rejuvenile Mar 11, 2026
99b56a0
Fix EPERM creating output dirs: resolve directory symlinks during har…
rejuvenile Mar 11, 2026
c23acfa
Fix EPERM: replace symlinks-to-read-only-dirs with writable shallow c…
rejuvenile Mar 11, 2026
d7b6717
Fix EPERM writing outputs: verify parent dir is writable after create…
rejuvenile Mar 11, 2026
3abbd44
Fix EEXIST race and EPERM on executable files in cached directory trees
rejuvenile Mar 11, 2026
2da97bb
Add cache format version to directory cache, auto-wipe stale entries
rejuvenile Mar 11, 2026
1f3d2e9
Fix EPERM on shell scripts: default file mode 0o555 instead of 0o444
rejuvenile Mar 11, 2026
f8158df
Bump directory cache format version to 3
rejuvenile Mar 11, 2026
6914052
Fix EPERM in directory cache: set 0o555 on all files, not just is_exe…
rejuvenile Mar 11, 2026
1734026
Add .sh permission diagnostics and fix rustc_lib BUILD globs
rejuvenile Mar 11, 2026
a505a7c
Fix EPERM on shell scripts: directory cache must set 0o555 on all files
rejuvenile Mar 11, 2026
a47774d
Fix CAS inode corruption: directory cache cleanup must not chmod files
rejuvenile Mar 11, 2026
d198902
Fix directory cache: handle zero-byte files (don't hardlink from CAS)
rejuvenile Mar 11, 2026
9478a83
Fix LRU eviction ordering at startup + diagnostic logging
rejuvenile Mar 11, 2026
1173748
Fix subtree race condition: fall back to download when cached subtree…
rejuvenile Mar 11, 2026
59156d6
DirectoryCache perf: spawn_blocking for filesystem ops, parallel blob…
rejuvenile Mar 12, 2026
f08be3c
Use clonefile(2) on macOS for O(1) directory tree cloning
rejuvenile Mar 12, 2026
a5b5b0c
Fix clonefile build on macOS: unsafe extern, path qualifications
rejuvenile Mar 12, 2026
f22c0f9
Skip set_readonly on macOS (CoW makes it unnecessary), parallelize su…
rejuvenile Mar 12, 2026
107c683
Fix: cfg-gate set_readonly_and_calculate_size in cache loading path
rejuvenile Mar 12, 2026
eb06532
Overlap subtree clones with file downloads using tokio::join!
rejuvenile Mar 12, 2026
d3c1a9a
Blend file count with byte size in subtree locality scoring
rejuvenile Mar 12, 2026
9df8dde
Add macOS F_RDADVISE, micro-prefetch in read loop, batch page cache w…
rejuvenile Mar 12, 2026
7d8aa8a
Decouple FastSlowStore writes, add direct-use directory cache mode
rejuvenile Mar 12, 2026
2d770d9
Fix has_with_results to check in-flight slow writes
rejuvenile Mar 12, 2026
3982d28
Disable direct-use directory cache mode (incompatible with Bazel)
rejuvenile Mar 12, 2026
13fcc0c
Add CloneMethod tracking to hardlink_directory_tree and directory cache
rejuvenile Mar 12, 2026
31869ea
Increase concurrent file downloads from 32 to 64
rejuvenile Mar 12, 2026
732bd1c
Add QUIC/HTTP3 transport, TCP socket buffer tuning, GrpcStore dual-tr…
rejuvenile Mar 12, 2026
3dd9efc
Fix QuicChannel to reuse QUIC connections across RPCs
rejuvenile Mar 12, 2026
c95c2d7
Fix QUIC channel sharing: use Arc<Mutex<H3Channel>> so clones share c…
rejuvenile Mar 12, 2026
7edabaa
Replace load average with instantaneous CPU utilization sampling
rejuvenile Mar 12, 2026
90dadb0
Fix worker process death on server restart
rejuvenile Mar 12, 2026
119bded
Fix batch locality routing: WorkerProxyStore was invisible to downcas…
rejuvenile Mar 12, 2026
822db1b
Increase connection-level flow control windows to 128 MiB
rejuvenile Mar 12, 2026
1f86153
Add fuzzy directory cache matching scaffolding
rejuvenile Mar 12, 2026
91d69af
Enable QUIC/HTTP3 on all three communication paths
rejuvenile Mar 12, 2026
dd17e23
Wire fuzzy directory cache matching into construction path
rejuvenile Mar 12, 2026
5c3b1dd
Fix unsafe extern block for Rust 2024 edition
rejuvenile Mar 12, 2026
0114caa
Add parallel chunked ByteStream read for large blobs
rejuvenile Mar 12, 2026
636bf10
Pin output blob digests during background upload to prevent eviction
rejuvenile Mar 12, 2026
fbf6e55
Fix WorkerProxyStore QUIC timeout on worker CAS proxy
rejuvenile Mar 12, 2026
2b1102d
Revert "Fix WorkerProxyStore QUIC timeout on worker CAS proxy"
rejuvenile Mar 12, 2026
2be29d7
Add QUIC keepalives and send locality snapshot immediately on connect
rejuvenile Mar 12, 2026
e020168
Fix emplace_file race when concurrent writes target same key
rejuvenile Mar 12, 2026
d83df70
Resolve QUIC hostnames to IPv4 to fix server-to-worker proxy timeouts
rejuvenile Mar 12, 2026
8733966
Fix worker QUIC server guard dropped immediately after startup
rejuvenile Mar 12, 2026
c424e51
Fix test compilation and unused_mut warning from guard refactor
rejuvenile Mar 12, 2026
54b5865
Include tree-expanded file digests in worker locality hints
rejuvenile Mar 12, 2026
3cdf970
Increase default parallel ByteStream chunk count from 8 to 64
rejuvenile Mar 12, 2026
89b2d87
Change Minimum platform property from u64 to f64
rejuvenile Mar 12, 2026
345d30b
Reject NaN, Infinity, and negative Minimum property values
rejuvenile Mar 12, 2026
6c2d34d
Skip ByteStream uploads when server already has the blob
rejuvenile Mar 13, 2026
149f3b5
Fix early-close channel error when server skips ByteStream upload
rejuvenile Mar 13, 2026
d4c92f7
Add heterogeneous core (P/E) load reporting and scheduling
rejuvenile Mar 18, 2026
695414f
Skip cache-affinity winner when worker load_score exceeds 150
rejuvenile Mar 18, 2026
0fce813
Set QoS to USER_INITIATED on macOS workers for P-core preference
rejuvenile Mar 19, 2026
3a2fdfa
Fix broken subtree symlinks and lower cache-affinity load cutoff
rejuvenile Mar 19, 2026
4e55b3b
Validate all files in symlinked subtrees, not just directory existence
rejuvenile Mar 19, 2026
4e8baee
Revert subtree symlink validation in DirectoryCache
rejuvenile Mar 19, 2026
81a02c6
Redact env vars from debug log and add AC NotFound timing
rejuvenile Mar 20, 2026
669208c
Fix TLS crypto provider for HTTP listeners
rejuvenile Mar 20, 2026
1f0f255
Add TLS, mTLS, require_tls, and review fixes
rejuvenile Mar 20, 2026
417baca
Install rustls crypto provider globally and add scheduler integration…
rejuvenile Mar 20, 2026
19d7e20
Fix zero-digest files missing from worker execution directories
rejuvenile Mar 23, 2026
8b3b966
Fix compilation after v1.0.0 rebase: API changes and test updates
rejuvenile Mar 23, 2026
2298f18
Fix DirectoryCache zero-digest files and add tests
rejuvenile Mar 23, 2026
5e87ee3
Fix DirectoryCache direct_use_mode default to false
rejuvenile Mar 23, 2026
0807153
Resolve absolute symlinks in output upload instead of uploading raw t…
rejuvenile Mar 23, 2026
77a6c42
Add diagnostic logging for missing small blob investigation
rejuvenile Mar 24, 2026
a4fa27f
Add pprof HTTP endpoint for CPU profiling (server + worker)
rejuvenile Mar 24, 2026
74bece7
Fix pprof server startup: enter tokio runtime context
rejuvenile Mar 24, 2026
0ed239c
Fix pprof-rs empty flamegraph on macOS workers
rejuvenile Mar 24, 2026
344e29f
Add auto-capture CPU profiling with threshold trigger
rejuvenile Mar 24, 2026
48bef04
Lower auto-capture CPU threshold to 5% of total cores
rejuvenile Mar 24, 2026
33fdf85
Increase buf_channel default capacity from 64 to 256 slots
rejuvenile Mar 24, 2026
5526150
Eliminate unnecessary Bytes copies in write hot path
rejuvenile Mar 24, 2026
d05956e
Eliminate double-copy in FastSlowStore streaming update path
rejuvenile Mar 24, 2026
aead2f1
Tune QUIC transport: BBR congestion control, 1ms ACK delay, 30s idle
rejuvenile Mar 24, 2026
7008d32
Fix memory pinning and uncapped pre-allocation from perf review
rejuvenile Mar 24, 2026
be02c57
Fix Duration::new panic on macOS in CPU usage tracking
rejuvenile Mar 24, 2026
afb5221
Increase FilesystemStore read_buffer_size from 256KiB to 3MiB
rejuvenile Mar 24, 2026
5b73e23
Scatter-gather MemoryStore: store data as chunk chain, zero copies
rejuvenile Mar 24, 2026
d9ad6bc
Reduce buf_channel capacity from 256 to 24 slots
rejuvenile Mar 24, 2026
4cf510b
Add disable_otlp config option, default true
rejuvenile Mar 24, 2026
9be4235
Add release_max_level_info to compile out debug/trace spans
rejuvenile Mar 24, 2026
c013e3b
Add non-blocking stdout logging via tracing-appender
rejuvenile Mar 24, 2026
8883a2e
Increase QUIC ACK delay from 1ms to 5ms to fix H3 frame errors
rejuvenile Mar 25, 2026
9da4de4
Add warn/error logging for failed file materialization in workers
rejuvenile Mar 25, 2026
babbadc
Add comprehensive observability for load debugging
rejuvenile Mar 25, 2026
c706470
Add comprehensive write path and eviction logging
rejuvenile Mar 25, 2026
8e18e28
Untrack .cargo/config.toml — machine-local build settings
rejuvenile Mar 25, 2026
294c526
Gate tokio RuntimeMetrics behind #[cfg(tokio_unstable)]
rejuvenile Mar 25, 2026
8ac6d99
Add BlobsInStableStorage foundation: proto, tracking, bounded pins
rejuvenile Mar 25, 2026
9f26278
Implement BlobsInStableStorage: batching, worker handling, bug fixes
rejuvenile Mar 25, 2026
c87055b
Mirror Bazel direct uploads to random worker for OOM redundancy
rejuvenile Mar 25, 2026
c38b614
Fix mirroring review issues: avoid re-read, size gate, batch threshold
rejuvenile Mar 25, 2026
0f1baec
Add eviction reason to every EvictingMap eviction log
rejuvenile Mar 25, 2026
be5b7f3
Optimize scheduler: scores cache, Arc<str> endpoints, partial sort
rejuvenile Mar 25, 2026
abc849b
Reduce critical path latency and eliminate needless allocations
rejuvenile Mar 25, 2026
e4cb878
Fix test for lazy tree resolution behavior
rejuvenile Mar 25, 2026
3f990f3
Fix remaining TODOs: completeness pins, monitor, log levels
rejuvenile Mar 26, 2026
63453de
Add batch pin_keys to EvictingMap for single-lock pinning
rejuvenile Mar 26, 2026
3c6eb81
Pin verified digests immediately in existence check loop
rejuvenile Mar 26, 2026
9a09257
Increase QUIC Tower buffer from 1024 to 8192 for mirror bursts
rejuvenile Mar 26, 2026
08cd682
Eliminate insert_callbacks double-lock in EvictingMap
rejuvenile Mar 26, 2026
eb5ec40
Move eviction to background task for dramatically reduced lock holds
rejuvenile Mar 26, 2026
dfe5480
Shard EvictingMap into 64 independent shards for ~64x contention redu…
rejuvenile Mar 26, 2026
718d8be
Fix sharded pin_keys to use batch method, document metrics limitation
rejuvenile Mar 26, 2026
f69aaf8
Fix has_with_results to check fast store, downgrade CRITICAL to warn
rejuvenile Mar 26, 2026
4e249f6
Skip redundant CAS insert when blob already exists (avoid unref rename)
rejuvenile Mar 26, 2026
ff655f9
Gate CAS dedup optimization on content_is_immutable
rejuvenile Mar 26, 2026
a83c8d6
Increase TCP socket buffers from 4MB to 8MB (match QUIC, cover 10GbE …
rejuvenile Mar 26, 2026
c903685
Add POSIX_FADV_SEQUENTIAL on write file descriptors
rejuvenile Mar 26, 2026
a3673dd
Worker proxy store, blob backfill, Box::pin stack overflow fix, and t…
rejuvenile Mar 26, 2026
7b83a80
Gate trace-level log assertion on debug_assertions
rejuvenile Mar 26, 2026
52432aa
Fix flaky execute_peer_sharing_test BlobsAvailable count assertion
rejuvenile Mar 26, 2026
3315434
Increase worker CAS message size to 64MiB, downgrade ByteStream empty…
rejuvenile Mar 27, 2026
52ae7ea
Add tokio-epoll-uring fork as submodule with metadata ops
rejuvenile Mar 27, 2026
809b0c1
Wire io_uring metadata ops into fs.rs (hard_link, rename, mkdir, unli…
rejuvenile Mar 27, 2026
e036307
Add io_uring batch submission and hard_link_batch
rejuvenile Mar 27, 2026
70e07de
Hybrid hardlink_directory_tree: readdir + io_uring batch submission
rejuvenile Mar 27, 2026
cf1e849
Fix review findings: single spawn_blocking, dirs in collection, batch…
rejuvenile Mar 27, 2026
985bf3f
Downgrade redirect/NotFound ByteStream errors to info, review fixes
rejuvenile Mar 28, 2026
8cb0ffa
Fix 7 confirmed audit bugs
rejuvenile Mar 28, 2026
ab4d770
Performance: remove diagnostics, downgrade hot-path logging, tune chu…
rejuvenile Mar 28, 2026
c16de5f
Downgrade remaining hot-path info!() to debug!() (MemoryStore, ByteSt…
rejuvenile Mar 28, 2026
07d5fbc
Optimize BlobLocalityMap: passthrough hasher + Vec-based EndpointList
rejuvenile Mar 28, 2026
3329450
Harden DigestHasher: write_usize no-op, Arc::ptr_eq fast path in upsert
rejuvenile Mar 28, 2026
b03e571
Zero-copy ByteStream Write codec: bypass tonic's BytesMut reassembly
rejuvenile Mar 28, 2026
05e6337
Replace unsafe get_unchecked_mut with safe get_mut in ZeroCopyWriteSt…
rejuvenile Mar 28, 2026
32ac149
Add stall dump file rotation: keep newest 10, clean up after each dump
rejuvenile Mar 29, 2026
f04ab3c
Zero-copy BatchUpdateBlobs, eu-stack timeout, macOS watchdog, operation
rejuvenile Mar 29, 2026
a1a03a4
Refactor: extract shared bytestream_write from duplicated write handlers
rejuvenile Mar 30, 2026
3f70979
Tree resolution dedup/negative-cache, fix locality map false positive…
rejuvenile Mar 30, 2026
d685eb1
Parallelize batch_flush_loop: 8 concurrent BatchUpdateBlobs RPCs
rejuvenile Mar 30, 2026
de61b45
ExistenceCacheStore update_oneshot + batch has() in BatchUpdateBlobs
rejuvenile Mar 30, 2026
b5f7cd6
Drain-then-fire batching: eliminate coalesce delays everywhere
rejuvenile Mar 30, 2026
b5f259c
Eliminate scheduler PeerHint/String clone overhead (45% CPU → ~0%)
rejuvenile Mar 30, 2026
a134963
Add info-level action lifecycle logging on workers
rejuvenile Mar 30, 2026
a115bcd
Zero-copy ByteStream Read encoding: eliminate ~3MiB memcpy per chunk
rejuvenile Mar 30, 2026
2d72ad9
Fix write stalls, tree resolution storms, stale peer refs, io_uring e…
rejuvenile Mar 31, 2026
f99ed9e
Fix worker reconnect on scheduler eviction / server restart
rejuvenile Mar 31, 2026
bcb1dd9
Server-side disconnect on evicted worker, remove dead client-side check
rejuvenile Mar 31, 2026
5b0cb9e
Fix store fallthrough, NotFound broadening, and cleanup ref_count leak
rejuvenile Mar 31, 2026
e2bcab7
Add server-side QUIC keepalives, increase idle timeout 30s→60s
rejuvenile Mar 31, 2026
09debf0
Set worker QUIC idle timeout to 60s to match server and client
rejuvenile Mar 31, 2026
b64944a
Optimizations: pre-resolved tree, targeted prefetch, adaptive fetch, …
rejuvenile Apr 2, 2026
94c8d6f
Fix worker eviction on ExecutionComplete after Completed
rejuvenile Apr 3, 2026
8f6a153
Raise prefetch blob threshold 1MiB→4MiB for better batch coverage
rejuvenile Apr 3, 2026
4c0b249
Fix scores cache thrashing, premature has() TOCTOU, symlink path norm…
rejuvenile Apr 3, 2026
252d519
Fix normalize_relative_path: preserve leading .. instead of dropping
rejuvenile Apr 3, 2026
2341fcc
Re-implement server-side cache warming correctly
rejuvenile Apr 3, 2026
416ca05
Filter cache warming to blobs <64KB (MemoryStore threshold)
rejuvenile Apr 3, 2026
c2853a6
Cache warming: runtime threshold, only warm missing blobs
rejuvenile Apr 4, 2026
d2f0c2c
QUIC connection pool, SO_REUSEPORT, opportunistic batching, mirror se…
rejuvenile Apr 4, 2026
ad8b4ac
Graceful shutdown flush, ByteStream write timeout, review fixes
rejuvenile Apr 4, 2026
5838fcc
Enable QUIC jumbo MTU discovery (1200→8500 bytes)
rejuvenile Apr 5, 2026
b5f0ffc
Raise QUIC MTU discovery upper bound to 8952
rejuvenile Apr 5, 2026
68eee64
TLS support for worker-to-server and server-to-worker mirror connections
rejuvenile Apr 5, 2026
c1e134a
Blob pinning in FastSlowStore + IPv6 dual-stack worker CAS bind
rejuvenile Apr 5, 2026
bdb1b92
Write dedup: has() check + in-flight coalescing in ByteStream and Bat…
rejuvenile Apr 5, 2026
a89dd0c
Worker CAS server uses read-only slow store to prevent mirror write-b…
rejuvenile Apr 5, 2026
9fdcc08
Mirror write pinning, write-back-on-disconnect, and coalescing error …
rejuvenile Apr 5, 2026
8b57fdc
Fix share_failed_slow_writes panic: use constructor instead of Arc::g…
rejuvenile Apr 5, 2026
f1943f0
Cooperative signal stack dumper + io_uring write timing breakdown
rejuvenile Apr 6, 2026
10f37ca
FuturesUnordered + pipeline depth 512 for io_uring writes
rejuvenile Apr 6, 2026
66ee021
Drain ready completions eagerly with now_or_never()
rejuvenile Apr 6, 2026
cbcc047
Set buf_channel, io_uring ring size, and write pipeline depth all to 512
rejuvenile Apr 6, 2026
c9f97d2
Raise concurrency limits: VerifyStore 256, ConnectionManager 256, Bat…
rejuvenile Apr 6, 2026
41238f4
Raise buf_channel, ring size, and write pipeline depth to 1024
rejuvenile Apr 6, 2026
7e05951
Pipeline io_uring reads: FuturesUnordered + depth 1024
rejuvenile Apr 6, 2026
a3d65ed
Instrument read pipeline: queue_ms vs io_ms + in_flight/pending_send …
rejuvenile Apr 6, 2026
8c574b2
Switch to spawn_blocking for all file I/O, 1024 blocking threads
rejuvenile Apr 6, 2026
90ba56a
Reduce bridge channel depth from 1024 to 8 (OOM risk)
rejuvenile Apr 6, 2026
77dd905
Fix: io_uring open_file skipped seek, causing parallel read corruption
rejuvenile Apr 6, 2026
9a61293
Add optional concurrency limiter for large file reads
rejuvenile Apr 6, 2026
3aa3fbe
Extend fs_io_bench with realistic concurrent read benchmarks
rejuvenile Apr 6, 2026
a23c961
Apply review feedback: threshold 4MiB, remove Arc wrapper
rejuvenile Apr 6, 2026
65a7d3f
Update tokio-epoll-uring: lock-free MPSC batch submission
rejuvenile Apr 6, 2026
211e7b9
Hybrid I/O, dual TCP+QUIC transport, and transport benchmarks
rejuvenile Apr 7, 2026
ccf7104
Writev coalescing, streaming mirror, unlimited mirror size
rejuvenile Apr 7, 2026
ee01cdf
Apply review feedback: 1MiB coalesce target, drain-until-empty
rejuvenile Apr 7, 2026
1ff3305
Validate digest on idle stream resume to prevent data cross-contamina…
rejuvenile Apr 8, 2026
40f7111
Tee streaming: mirror CAS blobs to workers simultaneously with store …
rejuvenile Apr 8, 2026
7a71d30
Fix cross-platform build: restrict mold linker to Linux
rejuvenile Apr 8, 2026
23e08ea
Fix Redis TOCTOU race: retry GETRANGE after EXISTS confirms key exists
rejuvenile Apr 8, 2026
369f3d6
Fix mirror tee blocking store writes: 100ms timeout on mirror send
rejuvenile Apr 8, 2026
eca7cfc
Skip mirror tee for worker uploads — workers already have the blob
rejuvenile Apr 8, 2026
7d5da89
Mirror-to-memory-only, parallel BFS tree resolution, tolerant GetTree
rejuvenile Apr 8, 2026
b49e0c6
Add std-directory regression test and bug-fix-test-first rule
rejuvenile Apr 9, 2026
99df8a0
Fix GetTree BFS position assignment bug: use digest-based parsing
rejuvenile Apr 9, 2026
7715f52
Mirror memory cap, DirectoryCache startup cleanup, review fixes
rejuvenile Apr 10, 2026
86e35c3
Fix Redis timeout silent data corruption, add read-path verification
rejuvenile Apr 10, 2026
e53c2fe
Fix flaky test_sentinel_connect_with_bad_master
rejuvenile Apr 10, 2026
bd1fa8c
Detect and skip partial/corrupt MemoryStore entries in FastSlowStore
rejuvenile Apr 11, 2026
c6f7df8
MemoryStore: return error on incomplete read, not silent Ok
rejuvenile Apr 11, 2026
c97c7e3
StreamingBlob Phase 1, design docs, latency analysis
rejuvenile Apr 11, 2026
622ace5
Fix parallel_chunk_count doc: default is 64, not 8
rejuvenile Apr 12, 2026
57ed619
MemoryStore: add diagnostic logging for partial read investigation
rejuvenile Apr 12, 2026
70e5c2d
FastSlowStore: add diagnostic logging for bytes_written mismatches
rejuvenile Apr 12, 2026
6026e75
Streaming pipeline, partial-read fix, locality scoring, zstd compression
rejuvenile Apr 12, 2026
2276bc1
Add streaming_read_while_write to OldByteStreamConfig
rejuvenile Apr 12, 2026
54f7192
Stream populate data to waiters instead of blocking on OnceCell
rejuvenile Apr 13, 2026
eb326c6
True concurrent streaming for populate waiters
rejuvenile Apr 13, 2026
599c652
Fix streaming populate for large blobs, add earliest_chunk_idx accessor
rejuvenile Apr 13, 2026
a6b7bdf
Add streaming populate logging and zero-byte fallback
rejuvenile Apr 13, 2026
1f8067b
Fix streaming populate: skip buffer when populate already completed
rejuvenile Apr 13, 2026
9730089
Fix spurious StreamingBlobWriter in copy_slow_to_fast
rejuvenile Apr 13, 2026
b806c03
Latency optimizations: parallel shards, skip has(), single-fetch AC, …
rejuvenile Apr 13, 2026
33c136b
QUIC: reduce idle timeout 60s→15s, keepalive 5s→2s for faster reconne…
rejuvenile Apr 13, 2026
1378a82
Graceful shutdown, streaming blob fix, MokaEvictingMap
rejuvenile Apr 14, 2026
e3ca6c5
Redis pipelining: batch STRLEN+EXISTS into single pipeline
rejuvenile Apr 14, 2026
8617a81
Migrate FilesystemStore and scheduler to MokaEvictingMap
rejuvenile Apr 14, 2026
5756dfe
Fix MokaEvictingMap review issues: atime ordering, weigher, unpin
rejuvenile Apr 14, 2026
44d7f59
Batch GetTree BFS reads: pipelined GETRANGE across store chain
rejuvenile Apr 14, 2026
f2fec8d
Pipeline RENAME+PUBLISH, batch BatchReadBlobs via store chain
rejuvenile Apr 14, 2026
4449e92
Review fixes: pipeline chunking, BatchReadBlobs size cap, cache fix
rejuvenile Apr 14, 2026
0a7a49f
Phase 4: Remove old EvictingMap/ShardedEvictingMap dead code
rejuvenile Apr 14, 2026
724330c
MokaEvictingMap tests, VerifyStore docs, Expiry investigation
rejuvenile Apr 14, 2026
6339eef
MemoryStore + FilesystemStore batch_get_part_unchunked overrides
rejuvenile Apr 14, 2026
1dd6b51
Fix dual_transport: skip QUIC endpoints in TCP ConnectionManager
rejuvenile Apr 14, 2026
236c402
GetTree cache, batch ExistenceCache insert, get_many docs
rejuvenile Apr 14, 2026
55b45ac
Subtree cache, coalescing, Arc wrapping, insert_many fix, CLAUDE.md
rejuvenile Apr 14, 2026
bdb015a
GetTree tests, Arc zero-copy cache insert
rejuvenile Apr 14, 2026
a590f9e
Downgrade hot-path info! to debug! (12K lines/sec → ~200)
rejuvenile Apr 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[build]
rustflags = ["-C", "target-cpu=native", "-C", "force-frame-pointers=yes", "--cfg", "tokio_unstable"]

[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "target-cpu=native", "-C", "link-arg=-fuse-ld=mold", "-C", "force-frame-pointers=yes", "--cfg", "tokio_unstable"]

[profile.release]
lto = "thin"
codegen-units = 8
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,5 @@ nativelink.bazelrc
*.log
buck-out/
nativelink_config.schema.json
.cargo/config.toml
.cargo/config.toml
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "tokio-epoll-uring"]
path = tokio-epoll-uring
url = forgejo@optimus.m0n0.space:rejuvenile/tokio-epoll-uring.git
96 changes: 96 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# NativeLink Rust Style & Conventions

## Imports
- Order: `core::` → `std::` → external crates (alphabetical) → internal `nativelink-*` crates
- Group proto imports by module, not alphabetically
- Use `use crate::...` for same-crate modules

## Error Handling
- `make_err!(Code::..., "message")` for internal errors; `make_input_err!("message")` for bad input
- `error_if!(condition, "message")` for early validation returns
- `.err_tip(|| "context")` to chain diagnostic context onto Results
- Never panic or unwrap in library code; always return `Result<T, Error>`
- Use `{:?}` for debug formatting of upstream errors in messages

## Logging (tracing)
- `use tracing::{debug, error, info, trace, warn};`
- Structured fields: `%` for Display, `?` for Debug — `info!(%key, ?value, "message")`
- `info!` for state transitions, transfer completions with throughput/duration
- `warn!` for performance anomalies (slow ops, contention, early evictions)
- `trace!` for hot-path / repetitive loops; avoid logging inside tight loops
- Messages: lowercase, no trailing period, describe **why** not what

## Async & Concurrency
- `#[async_trait]` on trait definitions and impls
- `Pin<&Self>` for `StoreDriver` trait methods
- `spawn_blocking()` for CPU-bound or sync filesystem work; avoid async recursion
- `tokio::join!` for fixed concurrent work; `FuturesUnordered` for variable-count
- `parking_lot::Mutex` for sync contexts; never hold locks across `.await`

## Config (serde)
- `#[serde(deny_unknown_fields)]` on config structs
- `#[serde(default)]` or `#[serde(default = "fn_name")]` for optional fields
- `#[serde(deserialize_with = "convert_string_with_shellexpand")]` for paths
- `#[serde(rename_all = "snake_case")]` on enums

## Metrics
- `#[derive(MetricsComponent)]` on public structs
- `#[metric(help = "...")]` on fields; `#[metric(group = "...")]` for nesting

## Naming & Formatting
- Functions: `snake_case`; types: `PascalCase`; constants: `UPPER_SNAKE_CASE`
- ~100 char soft line limit; readability over rigid length
- Blank line between logical sections; single blank line between items
- `Cow<'_, T>` in hot paths to avoid allocation

## Comments
- `///` doc comments on public items explain **why** and show examples
- `//` inline comments only for non-obvious logic or workarounds
- `TODO(...)` with issue number when possible for known issues

## Feature Gates
- `#[cfg(feature = "...")]` at definition site
- `#[cfg(target_os = "...")]` for OS-specific code (Linux vs macOS)

## Tests
- **Test-first development**: when implementing any new feature, write tests first
(unit, integration, and cross-component interaction tests). Verify they fail before
implementing the feature, then make them pass. Include fakes/mocks for
hardware-interaction tests where needed.
- **Bug fixes require a failing test first**: when fixing a bug, write a test that
reproduces the failure, verify it fails, then implement the fix and show the test
passes. Never fix a bug without a regression test.
- Integration tests in `tests/` directory; minimal inline `#[cfg(test)]` modules
- Use `nativelink-macro` test harness (`#[nativelink_test]`)

## Change Process
- **Chesterton's Fence**: before modifying or removing any behavior, always check
`git log`, `git blame`, and `git log -S` to understand *why* the code exists.
If a commit message or comment explains the reason, evaluate whether that reason
still applies before making the change.

## Code Review
- **Before committing any change**, send the changes to a code review agent and a
performance review agent. Work to obtain their sign-off before committing. Fix
any issues they identify. Only commit after both reviews pass with no blocking
issues.

## Git Journal
- **Journal all git operations**: append every `git commit`, `git push`, `git revert`,
`git stash`, and any other state-changing git command to `.claude/git-journal.md`
in the working directory. Each entry should include the timestamp, command, and a
one-line description. This prevents losing track of what was done across context
compressions.

## Working Directory Discipline
- **Always verify `pwd` before git operations.** Agent worktrees (`.claude/worktrees/`)
have separate git branches. Commits in a worktree do NOT go to `main`. The Bash tool
may silently `cd` into a worktree after an agent runs. Always `cd /path/to/nativelink`
before any `git commit`, `git push`, or `git status`.
- **Never use `git stash pop`** — it can cause merge conflicts that `git checkout --` resolves
by reverting uncommitted edits. Use `git stash apply` + `git stash drop` separately.
- **Commit early, commit often.** After each logical change compiles, commit immediately.
Don't accumulate multiple uncommitted changes across a session — context compression
or worktree confusion can lose them.
- **After editing files, verify with `git diff --stat HEAD`** that the expected changes
appear before moving on to the next task.
Loading