Skip to content

Commit 1bcbce2

Browse files
feat: Improve stability of file search discovery in background (#431)
* feat: Improve scan runtime performance and RSS * chore: Comprehensive randomized fuzzy mutation testing It's such a shame that we keep commiting regressions that are leading ot the missed events from the file watcher. I want to minimize this to 0. * chore: Update docs for - chore: Comprehensive randomized fuzzy mutation testing * chore: Update docs for - chore: Update docs for - chore: Comprehensive randomized fuzzy mutation testing * test: Verbose stress test output
1 parent 573a783 commit 1bcbce2

44 files changed

Lines changed: 5156 additions & 1656 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/rust.yml

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ jobs:
1919
strategy:
2020
matrix:
2121
os: [ubuntu-latest, macos-latest]
22+
# Guard against deadlocks in the shared-picker / watcher teardown
23+
# path: a stuck test would otherwise consume a full 6h CI slot.
24+
timeout-minutes: 10
2225
steps:
2326
- uses: actions/checkout@v5
2427

@@ -39,6 +42,58 @@ jobs:
3942
- name: Run tests
4043
run: cargo test --features zlob --workspace --exclude fff-nvim
4144

45+
stress-test:
46+
name: Stress Test (Watcher + Git)
47+
runs-on: ${{ matrix.os }}
48+
strategy:
49+
# Keep going after one OS fails so we can see whether a bug
50+
# reproduces everywhere or is platform-specific.
51+
fail-fast: false
52+
matrix:
53+
os: [ubuntu-latest, macos-latest, windows-latest]
54+
# Long-running; don't let a stuck watcher thread burn a full CI
55+
# timeout. Two scenarios should finish well under this limit.
56+
timeout-minutes: 20
57+
steps:
58+
- uses: actions/checkout@v5
59+
60+
- name: Install Zig
61+
uses: goto-bus-stop/setup-zig@v2
62+
with:
63+
version: 0.16.0
64+
65+
- name: Install Rust
66+
uses: actions-rust-lang/setup-rust-toolchain@v1.15.4
67+
with:
68+
cache: true
69+
cache-on-failure: true
70+
cache-key: "v1-rust-stress-${{ matrix.os }}"
71+
components: rustfmt, clippy
72+
73+
- name: Stress test (seeded / deterministic)
74+
shell: bash
75+
run: make test-stress-seeded
76+
env:
77+
FFF_STRESS_CASES: "3"
78+
FFF_STRESS_MIN_OPS: "30"
79+
FFF_STRESS_MAX_OPS: "50"
80+
81+
- name: Stress test (random / fuzzy)
82+
shell: bash
83+
run: make test-stress-random
84+
env:
85+
FFF_STRESS_CASES: "5"
86+
FFF_STRESS_MIN_OPS: "30"
87+
FFF_STRESS_MAX_OPS: "60"
88+
89+
- name: Upload proptest regressions on failure
90+
if: failure()
91+
uses: actions/upload-artifact@v4
92+
with:
93+
name: proptest-regressions-${{ matrix.os }}
94+
path: crates/fff-core/tests/fuzz_git_watcher_stress.proptest-regressions
95+
if-no-files-found: ignore
96+
4297
fmt:
4398
name: cargo fmt
4499
runs-on: ubuntu-latest

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,6 @@ scripts/benchmark-results/
2424
*.dylib
2525
*.so
2626
*.dll
27+
28+
# Instruments traces
29+
*.trace/

AGENTS.md

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# To Clankers
2+
3+
This repository contains **FFF.nvim (Fast File Finder)**, a high-performance file picker for Neovim inspired by blink.cmp's fuzzy matching technology. It's NOT a completion plugin, but rather a standalone file finder with advanced fuzzy search and frecency scoring. The project aims to be the drop-in replacement for telescope, fzf-lua, snacks.picker and similar plugins, focusing on speed, accuracy search and usability features.
4+
5+
## Development Commands
6+
7+
### Building
8+
9+
- `cargo build --release` - Build the Rust fuzzy matcher and file picker
10+
11+
### Testing and Development Tools
12+
13+
This project does not have a traditional test suite. Testing is done through:
14+
15+
- Create e2e local test file for Neovim: Load any Lua test file with `nvim -l <test_file>`
16+
- Write inline rust unit tests for any functionality that is standalone and scoped within a single function
17+
18+
### Code Quality
19+
20+
- `cargo fmt` - Format Rust code (follows standard conventions)
21+
- `make lint` - Rust linting and code analysis
22+
- `make format` - Format all code
23+
- `make test` - Run unit tests (limited coverage, primarily integration testing)
24+
25+
When doing code make sure to REDUCE SIZE OF COMMENTS. This is very important. Every comment should be concise 1-2 liner maximum 4 lines if describes really extensive and unnatural concept.
26+
27+
### Important coding rules
28+
29+
- Do not add doc comments to the private structs and functions.
30+
- Do not make public structs if something can be private
31+
32+
33+
## Architecture
34+
35+
Everything that is performance critical happens in rust world, everything that is neovim specific happens in the lua code.
36+
37+
There are 3 main components:
38+
39+
- Rust binary with the global file picker state containing index of all files
40+
- Background thread with the file system watcher that updates the index in real time
41+
- Lua UI layer that renders the picker, handles user input, and calls the rust functions via FFI
42+
43+
There are 2 databases:
44+
45+
- Frecency database (LMDB) that tracks file access patterns for scoring
46+
- Query history database used to track the user's previous search queries
47+
48+
### Key Files
49+
50+
- `lua/fff.lua` - Entry point, delegates to main.lua
51+
- `lua/fff/main.lua` - Public API (find_files, search, change_directory)
52+
- `lua/fff/core.lua` - Initialization, autocmds, global state management
53+
- `lua/fff/picker_ui.lua` - UI rendering, layout calculation, keymaps
54+
- `lua/fff/file_picker/preview.lua` - File preview with syntax highlighting
55+
- `lua/fff/file_picker/image.lua` - Image preview (snacks.nvim integration)
56+
- `lua/fff/conf.lua` - Default config
57+
- `lua/fff/rust/init.lua` - Loads compiled Rust shared library
58+
59+
**Rust Side:**
60+
61+
- `lua/fff/rust/lib.rs` - FFI bindings, global state (FILE_PICKER, FRECENCY)
62+
- `lua/fff/rust/file_picker.rs` - Core FilePicker struct, indexing, background watcher
63+
- `lua/fff/rust/frecency.rs` - Frecency database (LMDB) and scoring
64+
- `lua/fff/rust/query_tracker.rs` - Search query history tracking
65+
- `lua/fff/rust/score.rs` - Fuzzy match scoring with frizbee integration
66+
- `lua/fff/rust/git.rs` - Git status caching and repository detection
67+
- `lua/fff/rust/background_watcher.rs` - File system watcher thread
68+
69+
### Scoring Algorithm
70+
71+
Located at the score.rs file
72+
73+
### Build System
74+
75+
- `Cargo.toml` - Rust dependencies and build configuration (package name: `fff_nvim`)
76+
- `rust-toolchain.toml` - Specifies Rust nightly toolchain with required components
77+
- `Cross.toml` - Cross-compilation settings using Zig for Linux targets
78+
- **CI/CD Workflows**:
79+
- `.github/workflows/rust.yml` - Rust testing, formatting, and clippy checks
80+
- `.github/workflows/release.yaml` - Automated multi-platform builds
81+
- `.github/workflows/stylua.yaml` - Lua code formatting validation
82+
- `.github/workflows/nix.yml` - Nix build validation
83+
- **Cross-compilation Support**: Uses `cross` tool with Zig backend for efficient cross-compilation
84+
85+
## Development Notes
86+
87+
### Working with Rust Code
88+
89+
- Prefer struct methods over functions
90+
- If there is more than 2 impls in the file - create new file
91+
- Smaller concise comments over giant comment blocks
92+
- Do not add doc comments to the private functions/structs
93+
- Be very careful around locking and better double check with the human if something is going to require potentially long lock on a mutex/rwlock
94+
95+
### Working with lua code
96+
97+
- Document the types of public functions in every module
98+
- Use `vim.validate()` for validating user inputs in public functions
99+
- Try to reuse as much of existing functions as possible
100+
- When working on new features for the UI **IT IS EXTREMELY IMPORTANT** to keep the core functionality of navigating between files, selecting, and seeing the preview working as is. NEVER break anything from the core UI functionality, only add new features on top of the current UI.
101+
- When making a large chunk of code make lua test that opens neovim at `~/dev/lightsource` and opens the picker to test the ui functionality across the actual code.
102+
- When adding a new highlights or any new shortcuts and configurable UI options add them to the neovim config. AND IMPORTANT: update the README.md with the new configuration options.
103+
104+
### UI rendering
105+
106+
When working on the UI changeds IT IS EXTREMELY important for you to test it for both prompt_position="bottom" and prompt_position="top" as the rendering logic is different for both of them in both rust and lua world. When the prompt is positioed in the bottom everything should work the same way as the top but would be reversed in order. (though navigation is same for both)
107+
108+
## Top level API that can not introduce breaking changes under any circumstance
109+
110+
Top level rust, lua, C, and bun APIs can not be changed under any circumstance

CLAUDE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
AGENTS.md

0 commit comments

Comments
 (0)