Skip to content

docs(hydro_lang): restructure reference docs around sliced blocks and mutable singleton references#3018

Merged
shadaj merged 1 commit into
mainfrom
sandbox-8ea7aeba-3bae-48f6-b27e-cdd1ad101c56
Jul 16, 2026
Merged

docs(hydro_lang): restructure reference docs around sliced blocks and mutable singleton references#3018
shadaj merged 1 commit into
mainfrom
sandbox-8ea7aeba-3bae-48f6-b27e-cdd1ad101c56

Conversation

@shadaj

@shadaj shadaj commented Jul 9, 2026

Copy link
Copy Markdown
Member

Reorganize the "Core Framework" reference docs into a new hierarchy and
rewrite content to reflect current Hydro APIs (sliced! blocks,
by_ref/by_mut reference handles, BoundedValue keyed singletons)
instead of the legacy .tick()-era APIs.

New structure under docs/docs/hydro/reference/:

  • introduction/: Introduction (keeps /docs/hydro/reference/ URL via
    slug), new "Live Collections" concept page (placed first, as the most
    core concept), and "Dataflow Programming" (refreshed to emphasize that
    Hydro looks like analytics frameworks but models request/response
    services)
  • correctness/ ("Safety and Correctness"): section index; "Bounded and
    Unbounded Types" rewritten to decouple boundedness from ticks (Bounded =
    complete, contents immediately available; unbounded→bounded conversion
    via sliced! hooks); "Eventual Determinism" trimmed to the semantic
    model; new flagship "Non-Determinism and nondet!" page covering guard
    mechanics, locally-resolved vs forwarded guards, multiple NonDet
    parameters as independently observable forms of non-determinism (Paxos
    example), proof-style comment guidance, nondet_ naming and
    # Non-Determinism rustdoc conventions, simulator tie-in, and
    user-defined-function caveats
  • streaming-data/ ("Streaming Data"): section index (request/response
    framing); "Streams" (refreshed); "Keyed Streams" (hidden doctest
    harnesses migrated to sliced!); new "Keyed Singletons" page focused on
    the BoundedValue variant (one immutable value per request key;
    first(), fold_early_stop, map-based request→response, slicing as a
    batch of new entries) with a new request/response animation
  • state-management/ ("State Management"): new section index laying out
    the state philosophy (declarative aggregations → sliced+by_ref reads →
    use::state+by_mut for multi-input imperative state); "Singletons and
    Optionals" written from placeholder; new "Keyed State" page (Unbounded
    KeyedSingleton, per-key aggregations, join/lookup patterns); "Slice
    Blocks" refreshed (leads with the by_ref snapshot-reading idiom); new
    "Slice Hooks" reference (use / use::atomic / use::state /
    use::state_null semantics per collection type, guarantees, unslicing);
    new "References and Mutations" page (by_ref/by_mut handles,
    boundedness requirement, code-order access groups, multi-input
    shared-state pattern)
  • atomic-collections.mdx: "Atomic Collections" promoted to a top-level
    doc, de-tick-ified wording, modern sim test API in examples

Also update all internal links (learn/, dfir/, locations/),
regenerate category files and sidebar positions, and update hydro.run doc
URLs in hydro_lang rustdoc comments to the new paths.

Validation: full docusaurus build passes (no broken links); all mdtest
doctests pass (62 passing, remainder intentionally no_run/ignore); no
legacy APIs (.tick()/.batch(&tick)/.snapshot(&tick)/.all_ticks()/
.latest()) appear in any visible example code.

Co-authored-by: Infinity 🤖 infinity@hydro.run

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 9, 2026

Copy link
Copy Markdown

Deploying hydro with  Cloudflare Pages  Cloudflare Pages

Latest commit: bfe9115
Status: ✅  Deploy successful!
Preview URL: https://219d8ba3.hydroflow.pages.dev
Branch Preview URL: https://sandbox-8ea7aeba-3bae-48f6-b.hydroflow.pages.dev

View logs

shadaj added a commit that referenced this pull request Jul 9, 2026
… mutable singleton references

Reorganize the "Core Framework" reference docs into a new hierarchy and
rewrite content to reflect current Hydro APIs (`sliced!` blocks,
`by_ref`/`by_mut` reference handles, `BoundedValue` keyed singletons)
instead of the legacy `.tick()`-era APIs.

New structure under `docs/docs/hydro/reference/`:

- `introduction/`: Introduction (keeps `/docs/hydro/reference/` URL via
  slug), new "Live Collections" concept page (placed first, as the most
  core concept), and "Dataflow Programming" (refreshed to emphasize that
  Hydro looks like analytics frameworks but models request/response
  services)
- `correctness/` ("Safety and Correctness"): section index; "Bounded and
  Unbounded Types" rewritten to decouple boundedness from ticks (Bounded =
  complete, contents immediately available; unbounded→bounded conversion
  via `sliced!` hooks); "Eventual Determinism" trimmed to the semantic
  model, with the guarantee correctly scoped to safe APIs ("Hydro's type
  system is focused on eventual determinism"); new flagship
  "Non-Determinism and `nondet!`" page covering guard mechanics,
  locally-resolved vs forwarded guards, multiple `NonDet` parameters as
  independently observable forms of non-determinism (Paxos example),
  proof-style comment guidance, `nondet_` naming and `# Non-Determinism`
  rustdoc conventions, simulator tie-in, and user-defined-function caveats
- `streaming-data/` ("Streaming Data"): section index (request/response
  framing); "Streams" (refreshed); "Keyed Streams" (hidden doctest
  harnesses migrated to `sliced!`); new "Keyed Singletons" page focused on
  the `BoundedValue` variant (one immutable value per request key;
  `first()`, `fold_early_stop`, map-based request→response, slicing as a
  batch of new entries), with a boundedness table covering all five bound
  variants (including MonotonicValue/MonotonicKeys, and Unbounded keys
  being added *and removed*) and a new request/response animation
- `state-management/` ("State Management"): new section index laying out
  the state philosophy (declarative aggregations → sliced+`by_ref` reads →
  `use::state`+`by_mut` for multi-input imperative state); "Singletons and
  Optionals" written from placeholder; new "Keyed State" page (Unbounded
  `KeyedSingleton`, per-key aggregations, join/lookup patterns); "Slice
  Blocks" refreshed (leads with the `by_ref` snapshot-reading idiom); new
  "Slice Hooks" reference (`use` / `use::atomic` / `use::state` /
  `use::state_null` semantics per collection type, guarantees, unslicing);
  new "References and Mutations" page (`by_ref`/`by_mut` handles,
  boundedness requirement, code-order access groups, multi-input
  shared-state pattern)
- `atomic-collections.mdx`: "Atomic Collections" promoted to a top-level
  doc, de-tick-ified wording, modern sim test API in examples

Also update all internal links (`learn/`, `dfir/`, `locations/`),
regenerate category files and sidebar positions, update hydro.run doc
URLs in `hydro_lang` rustdoc comments to the new paths, and align the
single-counter tutorial's eventual-determinism phrasing with the
reference docs.

Validation: full docusaurus build passes (no broken links); all mdtest
doctests pass (62 passing, remainder intentionally no_run/ignore); no
legacy APIs (`.tick()`/`.batch(&tick)`/`.snapshot(&tick)`/`.all_ticks()`/
`.latest()`) appear in any visible example code.

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
PR: #3018
@shadaj
shadaj force-pushed the sandbox-8ea7aeba-3bae-48f6-b27e-cdd1ad101c56 branch from 7a11e99 to c195ac6 Compare July 9, 2026 23:27
@shadaj
shadaj marked this pull request as ready for review July 9, 2026 23:31
@shadaj
shadaj requested a review from a team July 9, 2026 23:31
@luckyworkama

Copy link
Copy Markdown
Contributor

under streaming data we have "streams, keyed streams, keyed singletons", then under the next section, state management, it starts with "singletons".

Something feels wrong about that order

@shadaj

shadaj commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

under streaming data we have "streams, keyed streams, keyed singletons", then under the next section, state management, it starts with "singletons".

Something feels wrong about that order

So the tricky part is keyed singletons can be used in both "stream mode" (ValueBounded) and "state mode" (Unbounded, MonotoneValue, etc). So we kinda have to talk about them twice in the two different places. But open to better titles.

Comment thread docs/docs/hydro/reference/introduction/live-collections.md Outdated
shadaj added a commit that referenced this pull request Jul 10, 2026
… mutable singleton references

Reorganize the "Core Framework" reference docs into a new hierarchy and
rewrite content to reflect current Hydro APIs (`sliced!` blocks,
`by_ref`/`by_mut` reference handles, `BoundedValue` keyed singletons)
instead of the legacy `.tick()`-era APIs.

New structure under `docs/docs/hydro/reference/`:

- `introduction/`: Introduction (keeps `/docs/hydro/reference/` URL via
  slug), new "Live Collections" concept page (placed first, as the most
  core concept), and "Dataflow Programming" (refreshed to emphasize that
  Hydro looks like analytics frameworks but models request/response
  services)
- `correctness/` ("Safety and Correctness"): section index; "Bounded and
  Unbounded Types" rewritten to decouple boundedness from ticks (Bounded =
  complete, contents immediately available; unbounded→bounded conversion
  via `sliced!` hooks); "Eventual Determinism" trimmed to the semantic
  model, with the guarantee correctly scoped to safe APIs ("Hydro's type
  system is focused on eventual determinism"); new flagship
  "Non-Determinism and `nondet!`" page covering guard mechanics,
  locally-resolved vs forwarded guards, multiple `NonDet` parameters as
  independently observable forms of non-determinism (Paxos example),
  proof-style comment guidance, `nondet_` naming and `# Non-Determinism`
  rustdoc conventions, simulator tie-in, and user-defined-function caveats
- `streaming-data/` ("Streaming Data"): section index (request/response
  framing); "Streams" (refreshed); "Keyed Streams" (hidden doctest
  harnesses migrated to `sliced!`); new "Keyed Singletons" page focused on
  the `BoundedValue` variant (one immutable value per request key;
  `first()`, `fold_early_stop`, map-based request→response, slicing as a
  batch of new entries), with a boundedness table covering all five bound
  variants (including MonotonicValue/MonotonicKeys, and Unbounded keys
  being added *and removed*) and a new request/response animation
- `state-management/` ("State Management"): new section index laying out
  the state philosophy (declarative aggregations → sliced+`by_ref` reads →
  `use::state`+`by_mut` for multi-input imperative state); "Singletons and
  Optionals" written from placeholder; new "Keyed State" page (Unbounded
  `KeyedSingleton`, per-key aggregations, join/lookup patterns); "Slice
  Blocks" refreshed (leads with the `by_ref` snapshot-reading idiom); new
  "Slice Hooks" reference (`use` / `use::atomic` / `use::state` /
  `use::state_null` semantics per collection type, guarantees, unslicing);
  new "References and Mutations" page (`by_ref`/`by_mut` handles,
  boundedness requirement, code-order access groups, multi-input
  shared-state pattern)
- `atomic-collections.mdx`: "Atomic Collections" promoted to a top-level
  doc, de-tick-ified wording, modern sim test API in examples

Also update all internal links (`learn/`, `dfir/`, `locations/`),
regenerate category files and sidebar positions, update hydro.run doc
URLs in `hydro_lang` rustdoc comments to the new paths, and align the
single-counter tutorial's eventual-determinism phrasing with the
reference docs.

Validation: full docusaurus build passes (no broken links); all mdtest
doctests pass (62 passing, remainder intentionally no_run/ignore); no
legacy APIs (`.tick()`/`.batch(&tick)`/`.snapshot(&tick)`/`.all_ticks()`/
`.latest()`) appear in any visible example code.

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
PR: #3018
@shadaj
shadaj force-pushed the sandbox-8ea7aeba-3bae-48f6-b27e-cdd1ad101c56 branch from c195ac6 to de3da64 Compare July 10, 2026 20:36
shadaj added a commit that referenced this pull request Jul 10, 2026
… mutable singleton references

Reorganize the "Core Framework" reference docs into a new hierarchy and
rewrite content to reflect current Hydro APIs (`sliced!` blocks,
`by_ref`/`by_mut` reference handles, `BoundedValue` keyed singletons)
instead of the legacy `.tick()`-era APIs.

New structure under `docs/docs/hydro/reference/`:

- `introduction/`: Introduction (keeps `/docs/hydro/reference/` URL via
  slug), new "Live Collections" concept page (placed first, as the most
  core concept), and "Dataflow Programming" (refreshed to emphasize that
  Hydro looks like analytics frameworks but models request/response
  services)
- `correctness/` ("Safety and Correctness"): section index; "Bounded and
  Unbounded Types" rewritten to decouple boundedness from ticks (Bounded =
  complete, contents immediately available; unbounded→bounded conversion
  via `sliced!` hooks); "Eventual Determinism" trimmed to the semantic
  model, with the guarantee correctly scoped to safe APIs ("Hydro's type
  system is focused on eventual determinism"); new flagship
  "Non-Determinism and `nondet!`" page covering guard mechanics,
  locally-resolved vs forwarded guards, multiple `NonDet` parameters as
  independently observable forms of non-determinism (Paxos example),
  proof-style comment guidance, `nondet_` naming and `# Non-Determinism`
  rustdoc conventions, simulator tie-in, and user-defined-function caveats
- `streaming-data/` ("Streaming Data"): section index (request/response
  framing); "Streams" (refreshed); "Keyed Streams" (hidden doctest
  harnesses migrated to `sliced!`); new "Keyed Singletons" page focused on
  the `BoundedValue` variant (one immutable value per request key;
  `first()`, `fold_early_stop`, map-based request→response, slicing as a
  batch of new entries), with a boundedness table covering all five bound
  variants (including MonotonicValue/MonotonicKeys, and Unbounded keys
  being added *and removed*) and a new request/response animation
- `state-management/` ("State Management"): new section index laying out
  the state philosophy (declarative aggregations → sliced+`by_ref` reads →
  `use::state`+`by_mut` for multi-input imperative state); "Singletons and
  Optionals" written from placeholder; new "Keyed State" page (Unbounded
  `KeyedSingleton`, per-key aggregations, join/lookup patterns); "Slice
  Blocks" refreshed (leads with the `by_ref` snapshot-reading idiom); new
  "Slice Hooks" reference (`use` / `use::atomic` / `use::state` /
  `use::state_null` semantics per collection type, guarantees, unslicing);
  new "References and Mutations" page (`by_ref`/`by_mut` handles,
  boundedness requirement, code-order access groups, multi-input
  shared-state pattern)
- `atomic-collections.mdx`: "Atomic Collections" promoted to a top-level
  doc, de-tick-ified wording, modern sim test API in examples

Also update all internal links (`learn/`, `dfir/`, `locations/`),
regenerate category files and sidebar positions, update hydro.run doc
URLs in `hydro_lang` rustdoc comments to the new paths, and align the
single-counter tutorial's eventual-determinism phrasing with the
reference docs.

Validation: full docusaurus build passes (no broken links); all mdtest
doctests pass (62 passing, remainder intentionally no_run/ignore); no
legacy APIs (`.tick()`/`.batch(&tick)`/`.snapshot(&tick)`/`.all_ticks()`/
`.latest()`) appear in any visible example code.

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
PR: #3018
@shadaj
shadaj force-pushed the sandbox-8ea7aeba-3bae-48f6-b27e-cdd1ad101c56 branch from de3da64 to e5f4fbf Compare July 10, 2026 20:40
@shadaj

shadaj commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

under streaming data we have "streams, keyed streams, keyed singletons", then under the next section, state management, it starts with "singletons".
Something feels wrong about that order

So the tricky part is keyed singletons can be used in both "stream mode" (ValueBounded) and "state mode" (Unbounded, MonotoneValue, etc). So we kinda have to talk about them twice in the two different places. But open to better titles.

I renamed that page to Streaming Keyed Singletons.

shadaj added a commit that referenced this pull request Jul 16, 2026
… mutable singleton references

Reorganize the "Core Framework" reference docs into a new hierarchy and
rewrite content to reflect current Hydro APIs (`sliced!` blocks,
`by_ref`/`by_mut` reference handles, `BoundedValue` keyed singletons)
instead of the legacy `.tick()`-era APIs.

New structure under `docs/docs/hydro/reference/`:

- `introduction/`: Introduction (keeps `/docs/hydro/reference/` URL via
  slug), new "Live Collections" concept page (placed first, as the most
  core concept), and "Dataflow Programming" (refreshed to emphasize that
  Hydro looks like analytics frameworks but models request/response
  services)
- `correctness/` ("Safety and Correctness"): section index; "Bounded and
  Unbounded Types" rewritten to decouple boundedness from ticks (Bounded =
  complete, contents immediately available; unbounded→bounded conversion
  via `sliced!` hooks); "Eventual Determinism" trimmed to the semantic
  model, with the guarantee correctly scoped to safe APIs ("Hydro's type
  system is focused on eventual determinism"); new flagship
  "Non-Determinism and `nondet!`" page covering guard mechanics,
  locally-resolved vs forwarded guards, multiple `NonDet` parameters as
  independently observable forms of non-determinism (Paxos example),
  proof-style comment guidance, `nondet_` naming and `# Non-Determinism`
  rustdoc conventions, simulator tie-in, and user-defined-function caveats
- `streaming-data/` ("Streaming Data"): section index (request/response
  framing); "Streams" (refreshed); "Keyed Streams" (hidden doctest
  harnesses migrated to `sliced!`); new "Keyed Singletons" page focused on
  the `BoundedValue` variant (one immutable value per request key;
  `first()`, `fold_early_stop`, map-based request→response, slicing as a
  batch of new entries), with a boundedness table covering all five bound
  variants (including MonotonicValue/MonotonicKeys, and Unbounded keys
  being added *and removed*) and a new request/response animation
- `state-management/` ("State Management"): new section index laying out
  the state philosophy (declarative aggregations → sliced+`by_ref` reads →
  `use::state`+`by_mut` for multi-input imperative state); "Singletons and
  Optionals" written from placeholder; new "Keyed State" page (Unbounded
  `KeyedSingleton`, per-key aggregations, join/lookup patterns); "Slice
  Blocks" refreshed (leads with the `by_ref` snapshot-reading idiom); new
  "Slice Hooks" reference (`use` / `use::atomic` / `use::state` /
  `use::state_null` semantics per collection type, guarantees, unslicing);
  new "References and Mutations" page (`by_ref`/`by_mut` handles,
  boundedness requirement, code-order access groups, multi-input
  shared-state pattern)
- `atomic-collections.mdx`: "Atomic Collections" promoted to a top-level
  doc, de-tick-ified wording, modern sim test API in examples

Also update all internal links (`learn/`, `dfir/`, `locations/`),
regenerate category files and sidebar positions, update hydro.run doc
URLs in `hydro_lang` rustdoc comments to the new paths, and align the
single-counter tutorial's eventual-determinism phrasing with the
reference docs.

Validation: full docusaurus build passes (no broken links); all mdtest
doctests pass (62 passing, remainder intentionally no_run/ignore); no
legacy APIs (`.tick()`/`.batch(&tick)`/`.snapshot(&tick)`/`.all_ticks()`/
`.latest()`) appear in any visible example code.

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
PR: #3018
@shadaj
shadaj force-pushed the sandbox-8ea7aeba-3bae-48f6-b27e-cdd1ad101c56 branch from e5f4fbf to 19b05f1 Compare July 16, 2026 16:53
@shadaj
shadaj requested a review from akainth015 July 16, 2026 16:54
… mutable singleton references

Reorganize the "Core Framework" reference docs into a new hierarchy and
rewrite content to reflect current Hydro APIs (`sliced!` blocks,
`by_ref`/`by_mut` reference handles, `BoundedValue` keyed singletons)
instead of the legacy `.tick()`-era APIs.

New structure under `docs/docs/hydro/reference/`:

- `introduction/`: Introduction (keeps `/docs/hydro/reference/` URL via
  slug), new "Live Collections" concept page (placed first, as the most
  core concept), and "Dataflow Programming" (refreshed to emphasize that
  Hydro looks like analytics frameworks but models request/response
  services)
- `correctness/` ("Safety and Correctness"): section index; "Bounded and
  Unbounded Types" rewritten to decouple boundedness from ticks (Bounded =
  complete, contents immediately available; unbounded→bounded conversion
  via `sliced!` hooks); "Eventual Determinism" trimmed to the semantic
  model, with the guarantee correctly scoped to safe APIs ("Hydro's type
  system is focused on eventual determinism"); new flagship
  "Non-Determinism and `nondet!`" page covering guard mechanics,
  locally-resolved vs forwarded guards, multiple `NonDet` parameters as
  independently observable forms of non-determinism (Paxos example),
  proof-style comment guidance, `nondet_` naming and `# Non-Determinism`
  rustdoc conventions, simulator tie-in, and user-defined-function caveats
- `streaming-data/` ("Streaming Data"): section index (request/response
  framing); "Streams" (refreshed); "Keyed Streams" (hidden doctest
  harnesses migrated to `sliced!`); new "Keyed Singletons" page focused on
  the `BoundedValue` variant (one immutable value per request key;
  `first()`, `fold_early_stop`, map-based request→response, slicing as a
  batch of new entries), with a boundedness table covering all five bound
  variants (including MonotonicValue/MonotonicKeys, and Unbounded keys
  being added *and removed*) and a new request/response animation
- `state-management/` ("State Management"): new section index laying out
  the state philosophy (declarative aggregations → sliced+`by_ref` reads →
  `use::state`+`by_mut` for multi-input imperative state); "Singletons and
  Optionals" written from placeholder; new "Keyed State" page (Unbounded
  `KeyedSingleton`, per-key aggregations, join/lookup patterns); "Slice
  Blocks" refreshed (leads with the `by_ref` snapshot-reading idiom); new
  "Slice Hooks" reference (`use` / `use::atomic` / `use::state` /
  `use::state_null` semantics per collection type, guarantees, unslicing);
  new "References and Mutations" page (`by_ref`/`by_mut` handles,
  boundedness requirement, code-order access groups, multi-input
  shared-state pattern)
- `atomic-collections.mdx`: "Atomic Collections" promoted to a top-level
  doc, de-tick-ified wording, modern sim test API in examples

Also update all internal links (`learn/`, `dfir/`, `locations/`),
regenerate category files and sidebar positions, update hydro.run doc
URLs in `hydro_lang` rustdoc comments to the new paths, and align the
single-counter tutorial's eventual-determinism phrasing with the
reference docs.

Validation: full docusaurus build passes (no broken links); all mdtest
doctests pass (62 passing, remainder intentionally no_run/ignore); no
legacy APIs (`.tick()`/`.batch(&tick)`/`.snapshot(&tick)`/`.all_ticks()`/
`.latest()`) appear in any visible example code.

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
PR: #3018
@shadaj
shadaj force-pushed the sandbox-8ea7aeba-3bae-48f6-b27e-cdd1ad101c56 branch from 19b05f1 to bfe9115 Compare July 16, 2026 17:17
@shadaj
shadaj merged commit 84bff7b into main Jul 16, 2026
19 checks passed
@shadaj
shadaj deleted the sandbox-8ea7aeba-3bae-48f6-b27e-cdd1ad101c56 branch July 16, 2026 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants