fix(scanner): keep the AOB scanner and SEH probe ASan-clean

[tkhquang]

Summary

The AOB scanner (scan_readable_regions) and the SEH-guarded probe read deliberately read arbitrary mapped process memory. Under MSVC AddressSanitizer those in-bounds, never-faulting reads land on memory ASan has poisoned for its own bookkeeping and are reported as false-positive overflows. The scenario cannot occur in production, where the scanner targets a separate, non-instrumented process.

This excludes only the deliberate foreign-memory readers from ASan, entirely under #if defined(__SANITIZE_ADDRESS__), so release and non-ASan builds are byte-for-byte unchanged. The full suite passes under the msvc-debug-asan preset with the scanner exercised, and the MinGW path is unchanged.

Rationale, rejected alternatives, and the pattern for new foreign-memory primitives are in docs/misc/asan-memory-scanner.md.

Summary by CodeRabbit

• Documentation

  • Added documentation explaining AddressSanitizer compatibility and false-positive mitigation strategies.
  • Updated documentation clarifying memory scanner behavior under AddressSanitizer instrumentation.

• Bug Fixes

  • Implemented AddressSanitizer-specific handling to prevent false-positive memory access reports when using the AOB memory scanner in instrumented builds.
  • Release builds remain unchanged.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7a4dbb59-4cc8-47b7-bc65-e7e0d6585677

📥 Commits

Reviewing files that changed from the base of the PR and between 597b0a6 and 0f2622b.

📒 Files selected for processing (4)

• AGENTS.md
• docs/misc/asan-memory-scanner.md
• src/memory.cpp
• src/scanner.cpp

---

📝 Walkthrough

Walkthrough

This PR adds AddressSanitizer (ASan) false-positive mitigation for the AOB memory scanner and SEH-guarded memory readers. The implementation uses preprocessor guards, compiler attributes, and platform-specific intrinsics to bypass ASan instrumentation in code paths that deliberately read foreign process memory.

Changes

AddressSanitizer Mitigation for Foreign-Memory Reads

Layer / File(s)	Summary
ASan mitigation documentation and strategy docs/misc/asan-memory-scanner.md, AGENTS.md	Introduces new documentation explaining why the AOB scanner's deliberate foreign-memory reads trigger ASan false positives, documents the implemented mitigations (preprocessor guards, __movsb intrinsic, inline byte scanning), describes rejected alternatives, and provides guidance for future foreign-memory primitives.
Memory module ASan support for MSVC src/memory.cpp	Conditionally includes <intrin.h> under MSVC+ASan, and modifies seh_read_bytes to use __movsb instead of std::memcpy when under AddressSanitizer, avoiding libc interceptor instrumentation in the fault-handler copy path.
Scanner ASan hotpath protection and byte-search refactoring src/scanner.cpp	Defines DMK_NO_SANITIZE_ADDRESS macro for MSVC+ASan, annotates AVX2 verification and raw pattern-search functions to disable ASan load instrumentation, introduces scan_for_byte helper that performs manual byte iteration under ASan instead of calling memchr, and updates the main search loop to use the new helper.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• tkhquang/DetourModKit#91: Modifies src/scanner.cpp scan-path byte reads and Memory::seh_read fault-probing, which are the same code paths targeted by this PR's ASan mitigation.
• tkhquang/DetourModKit#57: This PR applies DMK_NO_SANITIZE_ADDRESS to the AVX2 pattern verification helper, which was added in the retrieved PR.
• tkhquang/DetourModKit#87: The retrieved PR's new seh_read_chain* primitives directly use the seh_read_bytes code path modified in this PR for ASan support.

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly summarizes the main change: keeping the AOB scanner and SEH probe clean from AddressSanitizer false positives, which is the core objective across all modified files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}