Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
6f4826d
Update documentation formatting and add decision log for hashing arch…
peter-lawrey Nov 21, 2025
1f389dc
Refactor access method visibility and improve variable declarations i…
peter-lawrey Nov 21, 2025
cc483cb
Add documentation for AI agents and project guidelines
peter-lawrey Nov 21, 2025
7822884
Refactor access method visibility and improve variable declarations i…
peter-lawrey Nov 21, 2025
ccc07d2
Update JDK profile in POM to target JDK 17
peter-lawrey Nov 21, 2025
c7b9a3d
Apply checkstyle adjustments
peter-lawrey Nov 12, 2025
a8490b1
Add MurmurHash_3 implementation for efficient hashing
peter-lawrey Nov 14, 2025
e6031d8
Rename MurmurHash3 references to MurmurHash_3 for consistency
peter-lawrey Nov 14, 2025
8d93bc2
Updated documentation
peter-lawrey Nov 14, 2025
86874e7
Enhance documentation with British English conventions and formatting…
peter-lawrey Nov 14, 2025
d6c2e84
Update documentation
peter-lawrey Nov 16, 2025
ed9d22a
Checkpoint
peter-lawrey Nov 18, 2025
d9828c9
Refactor access modifiers in hashing methods for consistency and clarity
peter-lawrey Nov 19, 2025
020ca07
Code Analysis fixes
peter-lawrey Nov 21, 2025
a60f311
Remove unnecessary blank lines in various classes for improved code r…
peter-lawrey Nov 24, 2025
db0e56a
Remove unnecessary blank lines in various classes for improved code r…
peter-lawrey Nov 24, 2025
177fed3
Remove JDK activation from quality profile in pom.xml
peter-lawrey Nov 24, 2025
1a10d21
Revert adding final local var
peter-lawrey Nov 28, 2025
f4935b2
Merge branch 'develop' into adv/develop
peter-lawrey Apr 29, 2026
8bce5de
Revert low-value churn; fix doc bugs and Javadoc placement
peter-lawrey Apr 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 169 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# Guidance for AI agents, bots, and humans contributing to Chronicle Software's OpenHFT projects

LLM-based agents can accelerate development only if they respect our house rules. This file tells you:

* how to run and verify the build;
* what *not* to comment;
* when to open pull requests.

## Language & character-set policy

| Requirement | Rationale |
|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|
| **British English** spelling (`organisation`, `licence`, *not* `organization`, `license`) except technical US spellings like `synchronized` | Keeps wording consistent with Chronicle's London HQ and existing docs. See the University of Oxford style guide for reference. |
| **ISO-8859-1** (code-points 0-255), except in string literals. Avoid smart quotes, non-breaking spaces and accented characters. | ISO-8859-1 survives every toolchain Chronicle uses, incl. low-latency binary wire formats that expect the 8th bit to be 0. |
| If a symbol is not available in ISO-8859-1, use a textual form such as `micro-second`, `>=`, `:alpha:`, `:yes:`. This is the preferred approach and Unicode must not be inserted. | Extended or '8-bit ASCII' variants are *not* portable and are therefore disallowed. |

## Javadoc guidelines

**Goal:** Every Javadoc block should add information you cannot glean from the method signature alone. Anything else is
noise and slows readers down.

| Do | Don't |
|----|-------|
| State *behavioural contracts*, edge-cases, thread-safety guarantees, units, performance characteristics and checked exceptions. | Restate the obvious ("Gets the value", "Sets the name"). |
| Keep the first sentence short; it becomes the summary line in aggregated docs. | Duplicate parameter names/ types unless more explanation is needed. |
| Prefer `@param` for *constraints* and `@throws` for *conditions*, following Oracle's style guide. | Pad comments to reach a line-length target. |
| Remove or rewrite autogenerated Javadoc for trivial getters/setters. | Leave stale comments that now contradict the code. |

The principle that Javadoc should only explain what is *not* manifest from the signature is well-established in the
wider Java community.

## Build & test commands

Agents must verify that the project still compiles and all unit tests pass before opening a PR:

```bash
# From repo root
mvn -q verify
```

## Commit-message & PR etiquette

1. **Subject line <= 72 chars**, imperative mood: Fix roll-cycle offset in `ExcerptAppender`.
2. Reference the JIRA/GitHub issue if it exists.
3. In *body*: *root cause -> fix -> measurable impact* (latency, allocation, etc.). Use ASCII bullet points.
4. **Run `mvn verify`** again after rebasing.

## What to ask the reviewers

* *Is this AsciiDoc documentation precise enough for a clean-room re-implementation?*
* Does the Javadoc explain the code's *why* and *how* that a junior developer would not be expected to work out?
* Are the documentation, tests and code updated together so the change is clear?
* Does the commit point back to the relevant requirement or decision tag?
* Would an example or small diagram help future maintainers?

## Project requirements

See the [Decision Log](src/main/docs/decision-log.adoc) for the latest project decisions.
See the [Project Requirements](src/main/docs/project-requirements.adoc) for details on project requirements.

## Elevating the Workflow with Real-Time Documentation

Building upon our existing Iterative Workflow, the newest recommendation is to emphasise *real-time updates* to documentation.
Ensure the relevant `.adoc` files are updated when features, requirements, implementation details, or tests change.
This tight loop informs the AI accurately and creates immediate clarity for all team members.

### Benefits of Real-Time Documentation

* **Confidence in documentation**: Accurate docs prevent miscommunications that derail real-world outcomes.
* **Reduced drift**: Real-time updates keep requirements, tests and code aligned.
* **Faster feedback**: AI can quickly highlight inconsistencies when everything is in sync.
* **Better quality**: Frequent checks align the implementation with the specified behaviour.
* **Smoother onboarding**: Up-to-date AsciiDoc clarifies the system for new developers.

### Best Practices

* **Maintain Sync**: Keep documentation (AsciiDoc), tests, and code synchronised in version control. Changes in one area should prompt reviews and potential updates in the others.
* **Doc-First for New Work**: For *new* features or requirements, aim to update documentation first, then use AI to help produce or refine corresponding code and tests. For refactoring or initial bootstrapping, updates might flow from code/tests back to documentation, which should then be reviewed and finalised.
* **Small Commits**: Each commit should ideally relate to a single requirement or coherent change, making reviews easier for humans and AI analysis tools.
* **Team Buy-In**: Encourage everyone to review AI outputs critically and contribute to maintaining the synchronicity of all artefacts.

## AI Agent Guidelines

When using AI agents to assist with development, please adhere to the following guidelines:

* **Respect the Language & Character-set Policy**: Ensure all AI-generated content follows the British English and ISO-8859-1 guidelines outlined above.
* **Focus on Clarity**: AI-generated documentation should be clear and concise and add value beyond what is already present in the code or existing documentation.
* **Avoid Redundancy**: Do not generate content that duplicates existing documentation or code comments unless it provides additional context or clarification.
* **Review AI Outputs**: Always review AI-generated content for accuracy, relevance, and adherence to the project's documentation standards before committing it to the repository.

## Company-Wide Tagging

This section records **company-wide** decisions that apply to *all* Chronicle projects. All identifiers use the `<Scope>-<Tag>-xxx` prefix. The `xxx` digits are unique within the same Scope even if the tags are different. Component-specific decisions live in their xxx-decision-log.adoc files.

### Tag Taxonomy (Nine-Box Framework)

To improve traceability, we adopt the Nine-Box taxonomy for requirement and decision identifiers. These tags are used in addition to the existing ALL prefix, which remains reserved for global decisions across every project.

.Adopt a Nine-Box Requirement Taxonomy

|Tag | Scope | Typical examples |
|----|-------|------------------|
|FN |Functional user-visible behaviour | Message routing, business rules |
|NF-P |Non-functional - Performance | Latency budgets, throughput targets |
|NF-S |Non-functional - Security | Authentication method, TLS version |
|NF-O |Non-functional - Operability | Logging, monitoring, health checks |
|TEST |Test / QA obligations | Chaos scenarios, benchmarking rigs |
|DOC |Documentation obligations | Sequence diagrams, user guides |
|OPS |Operational / DevOps concerns | Helm values, deployment checklist |
|UX |Operator or end-user experience | CLI ergonomics, dashboard layouts |
|RISK |Compliance / risk controls | GDPR retention, audit trail |

`ALL-*` stays global, case-exact tags. Pick one primary tag if multiple apply.

### Decision Record Template

```asciidoc
=== [Identifier] Title of Decision

Date:: YYYY-MM-DD
Context::
* What is the issue that this decision addresses?
* What are the driving forces, constraints, and requirements?
Decision Statement::
* What is the change that is being proposed or was decided?
Alternatives Considered::
* [Alternative 1 Name/Type]:
** *Description:* Brief description of the alternative.
** *Pros:* ...
** *Cons:* ...
* [Alternative 2 Name/Type]:
** *Description:* Brief description of the alternative.
** *Pros:* ...
** *Cons:* ...
Rationale for Decision::
* Why was the chosen decision selected?
* How does it address the context and outweigh the cons of alternatives?
Impact & Consequences::
* What are the positive and negative consequences of this decision?
* How does this decision affect the system, developers, users, or operations?
- What are the trade-offs made?
Notes/Links::
** (Optional: Links to relevant issues, discussions, documentation, proof-of-concepts)
```

## Asciidoc formatting guidelines

### List Indentation

Do not rely on indentation for list items in AsciiDoc documents. Use the following pattern instead:

```asciidoc
section:: Top Level Section
* first level
** nested level
```

### Emphasis and Bold Text

In AsciiDoc, an underscore `_` is _emphasis_; `*text*` is *bold*.

## Zero-Allocation Hashing module specifics

- Follow repository `AGENTS.md` as the base rules; this section adds module notes. Durable docs live in `src/main/docs/` with the landing page at `README.adoc`.
- Module purpose: zero-allocation hashing primitives and utilities for bytes/arrays/buffers with stable, cross-platform output.
- Build commands: full build `mvn -q clean verify`; quality gates `mvn -Pquality verify` (requires JDK 11+).
- Quality gates: keep Checkstyle/SpotBugs clean; preserve deterministic hash outputs across platforms; avoid hidden allocations in hot paths.
- Documentation: maintain Nine-Box IDs in `src/main/docs/specifications.adoc`/`project-requirements` if added, and link decisions/tests accordingly; British English, ASCII/ISO-8859-1, `:source-highlighter: rouge`.
- Guardrails: changes to hashing outputs are breaking; document any algorithm/version bumps; call out platform-specific behaviour in `unsafe-and-platform-notes.adoc`.
218 changes: 218 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Zero-Allocation Hashing is a Java library providing fast, non-cryptographic hash functions that allocate zero objects during hash computation. It implements multiple algorithms (CityHash, FarmHash, MurmurHash3, xxHash, XXH3, wyHash, MetroHash) for hashing byte sequences from various sources (arrays, buffers, CharSequence, raw memory). The corresponding Java class for MurmurHash3 is `MurmurHash_3`.

Target: Java 8+ (supports JDK 8, 11, 17, 21+)

## Build & Test Commands

```bash
# Run full build with tests
mvn verify

# Run tests only
mvn test

# Run specific test class
mvn test -Dtest=LongHashFunctionTest

# Run specific test method
mvn test -Dtest=LongHashFunctionTest#testHashBytes

# Clean build
mvn clean install

# Generate javadoc
mvn javadoc:javadoc

# Run quality profile (Checkstyle + SpotBugs); requires JDK 11+
mvn -Pquality verify
```

## Architecture & Key Concepts

### Core Abstractions

**`LongHashFunction`** (src/main/java/net/openhft/hashing/LongHashFunction.java)
- Primary facade for 64-bit hashes
- Factory methods: `city_1_1()`, `farmNa()`, `farmUo()`, `murmur_3()`, `xx()`, `xx3()`, `xx128low()`, `wy_3()`, `metro()`
- All instances are immutable and thread-safe
- Seeds are baked into instances at construction time

**`LongTupleHashFunction`** (src/main/java/net/openhft/hashing/LongTupleHashFunction.java)
- Multi-word hash results (128-bit and beyond)
- Uses reusable `long[]` buffers to maintain zero-allocation guarantee
- Caller must manage/reuse result arrays

**`Access<T>`** (src/main/java/net/openhft/hashing/Access.java)
- Strategy pattern abstracting byte sequence reading from different sources
- Implementations: `UnsafeAccess` (heap arrays), `ByteBufferAccess`, `CharSequenceAccess`, `CompactLatin1CharSequenceAccess`
- Handles byte-order normalisation via `Access.byteOrder(input, desiredOrder)`
- Algorithms are written once against `Access` interface, work with all input types

**Byte Order Handling**
- All algorithms normalise to Little-Endian internally (ADR-002)
- Ensures cross-platform deterministic results (x86 vs s390x)
- Performance penalty on Big-Endian platforms due to byte-swapping

### Memory Access

**`UnsafeAccess`** (src/main/java/net/openhft/hashing/UnsafeAccess.java)
- Wraps `sun.misc.Unsafe` for zero-copy memory reads
- Enables "type punning" (reading bytes as longs) efficiently
- Requires `--add-opens java.base/jdk.internal.misc=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED` on Java 9+

**String Hashing Runtime Adaptation** (src/main/java/net/openhft/hashing/Util.java)
- `VALID_STRING_HASH` selects correct strategy at JVM initialisation
- Handles: HotSpot (pre-compact, compact strings), OpenJ9, Zing, unknown VMs
- Uses reflection to access internal `String.value` field (ADR-004)
- Fallback: `UnknownJvmStringHash` for unrecognized JVMs

### Algorithm Implementations

All live in package-private classes with factory methods exposed via `LongHashFunction`:
- `CityAndFarmHash_1_1` - CityHash64 v1.1, FarmHash NA/UO
- `MurmurHash_3` - 64-bit and 128-bit variants
- `XxHash` - XXH64
- `XXH3` - XXH3 64-bit and 128-bit
- `WyHash` - wyHash v3
- `MetroHash` - metrohash64_2

## Project-Specific Guidelines

### Language & Character Set (CRITICAL)
- Use **British English**: "organisation", "licence", "optimisation", "normalise", "initialise" (NOT "organization", "license", "optimization")
- Technical US spellings allowed: "synchronized", "byte"
- **ISO-8859-1 only** (code-points 0-255) except in string literals
- NO smart quotes, non-breaking spaces, accented characters in code/docs
- Use textual forms: "micro-second", ">=", ":alpha:" instead of Unicode

### Javadoc Standards
**DO:**
- Document behavioural contracts, edge-cases, thread-safety, units, performance characteristics
- Explain constraints via `@param`, conditions via `@throws`
- Keep first sentence short (becomes summary line)

**DON'T:**
- Restate the obvious ("Gets the value", "Sets the name")
- Duplicate parameter names/types without additional explanation
- Leave autogenerated Javadoc for trivial getters/setters

### Code Conventions
- Zero-allocation in steady state (one-time allocation during class loading permitted)
- Validate array offsets/lengths via `Util.checkArrayOffs` to prevent crashes
- Assume raw memory addresses are pre-validated by caller
- No ThreadLocal usage (containerisation requirement)
- Immutable, stateless hash function instances

### Documentation Workflow (IMPORTANT)
- Keep AsciiDoc files synchronized with code/tests
- Update `src/main/docs/*.adoc` when changing features, requirements, or implementations
- See `AGENTS.md` for company-wide documentation standards
- Reference decisions in commit messages (e.g., "Implements ADR-003")

### Testing Requirements
- Thoroughly tested on LTS JDKs: 8, 11, 17, 21
- Test both Little-Endian and Big-Endian platforms
- JDK 9+ runs tests twice: with and without `-XX:-CompactStrings` (see pom.xml profiles)
- All tests use JUnit 4 (supports Java 7+)

### Commit & PR Guidelines
- Subject line <= 72 chars, imperative mood: "Fix roll-cycle offset in ExcerptAppender"
- Body format: root cause -> fix -> measurable impact
- Run `mvn verify` before committing
- Reference JIRA/GitHub issues
- Use ASCII bullet points in commit bodies
- Main branch for PRs: **ea** (not master)

## File Structure

```
src/main/java/net/openhft/hashing/
Access.java - Core abstraction for byte sequence reading
LongHashFunction.java - Primary 64-bit hash facade
LongTupleHashFunction.java - Multi-word hash facade
DualHashFunction.java - Bridges 128-bit -> 64-bit views
UnsafeAccess.java - sun.misc.Unsafe wrapper

[Algorithm Implementations]
CityAndFarmHash_1_1.java
MurmurHash_3.java
XxHash.java
XXH3.java
WyHash.java
MetroHash.java

[Memory Access Strategies]
ByteBufferAccess.java
CharSequenceAccess.java
CompactLatin1CharSequenceAccess.java

[String Hashing Adapters]
StringHash.java
ModernHotSpotStringHash.java
ModernCompactStringHash.java
HotSpotPrior7u6StringHash.java
UnknownJvmStringHash.java

[Utilities]
Util.java - Runtime detection, validation
Primitives.java - Byte-order normalization
Maths.java - Low-level arithmetic helpers

src/main/docs/
project-requirements.adoc - Functional/non-functional requirements
decision-log.adoc - Architecture Decision Records (ADRs)
architecture-overview.adoc - System architecture details
algorithm-profiles.adoc - Per-algorithm characteristics
testing-strategy.adoc - Test coverage approach
```

## Common Development Patterns

### Adding a New Hash Algorithm
1. Create package-private class implementing the algorithm
2. Extend `LongHashFunction` or `LongTupleHashFunction`
3. Use `Access<T>` abstraction for all memory reads
4. Normalize to Little-Endian via `Access.byteOrder(input, LITTLE_ENDIAN)`
5. Add factory methods to `LongHashFunction` facade
6. Add tests validating against reference implementation
7. Update `algorithm-profiles.adoc` with characteristics

### Implementing Custom Access Strategy
```java
// Example from Access.java documentation
class Pair {
long first, second;

static final long pairDataOffset =
theUnsafe.objectFieldOffset(Pair.class.getDeclaredField("first"));

static long hashPair(Pair pair, LongHashFunction hashFunction) {
return hashFunction.hash(pair, Access.unsafe(), pairDataOffset, 16L);
}
}
```

### Running on Java 9+
Requires JVM flags for internal API access:
```bash
--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
--add-opens java.base/sun.nio.ch=ALL-UNNAMED
```

## Key Decisions

See `src/main/docs/decision-log.adoc` for the canonical ADR list (ADR-001 through ADR-006). Reference ADR IDs in commit messages where relevant.

## References

- Javadoc: https://javadoc.io/doc/net.openhft/zero-allocation-hashing/latest
- GitHub: https://github.com/OpenHFT/Zero-Allocation-Hashing
- Issues: https://github.com/OpenHFT/Zero-Allocation-Hashing/issues
- Release Notes: https://chronicle.software/release-notes/
- Company Guidelines: See `AGENTS.md` for Chronicle Software standards
5 changes: 4 additions & 1 deletion README.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
== Zero-Allocation Hashing
= Zero-Allocation Hashing
:toc:
:pp: ++
:lang: en-GB
:source-highlighter: rouge

Chronicle Software

Expand Down
Loading
Loading