| title | Glossary | ||
|---|---|---|---|
| description | One-stop place to resolve any pg_hardstorage-specific term you encounter elsewhere in the docs. | ||
| tags |
|
The terms a reader is likely to hit while learning pg_hardstorage, in alphabetical order. Each entry is one to three sentences plus a "see also" link to the page that goes deeper. Cross-references between entries use the exact term as it appears here.
If a term is missing, the support page has the right place to file the gap.
The opt-in privilege mode for the LLM helper that allows
execute_command after a successful preview_command in
the same turn, with explicit user confirmation. Off by
default. See the
LLM safety stack.
The chunk-encryption cipher shipping today — a random 96-bit nonce per chunk. AES-256-GCM-SIV (RFC 8452, nonce-misuse-resistant) is the planned default once a validated implementation lands; the Go standard library does not yet ship it, and BoringCrypto (FIPS) does not expose it either. See envelope encryption.
The long-lived pg_hardstorage agent process that does
the actual work — base backup, WAL streaming, chunk
upload, manifest commit. Co-located with the DB host or
remote, talking to PG over libpq. One agent per host
multiplexes every deployment on that host. See
architecture tour § three execution modes.
PG's per-segment WAL archive hook. pg_hardstorage offers
a thin shim (pg_hardstorage wal push %p) for setups that
want classical archiving alongside streaming; both
shims feed the same content-addressed chunk store. See
wal pipeline.
The PG 15+ replacement for archive_command, where a
shared library handles archive callbacks.
pg_hardstorage ships pg_hardstorage_archive (~200 LOC of
C) for the optional double-archiving path.
An Ed25519-signed claim (agent keyring) attached to a backup manifest, or a cosign-signed claim attached to a release artefact, optionally anchored to a transparency log. Both backups and release binaries carry attestations. See audit chain.
The append-only, hash-chained Merkle log of every
significant event (backup committed, restore started, KMS
rotated, LLM session opened, …). Periodic anchors land in a
transparency log (self-hosted today; external Rekor anchoring
is roadmap). Verifiable post-hoc via
pg_hardstorage audit verify-chain. See
audit chain.
The mechanism the chunker uses to keep slow stages from
buffer-bombing memory: each pipeline stage carries a
bounded MemBudget; if storage is slower than chunking,
the chunker blocks rather than allocating. See the
resilience section of the
architecture tour.
One point-in-time recoverable artefact for a deployment — a manifest plus the chunks and WAL it references. See the SPEC's Vocabulary table.
The "Backup Data Encryption Key" — a 256-bit random key
generated per backup, wrapped by the deployment's
KEK, and stored in manifest.json.encryption.wrapped_dek.
See envelope encryption.
The PG replication-protocol command pg_hardstorage uses to
take physical base backups. Streams tar files per
tablespace; the agent feeds them through the chunker
pipeline. Optional INCREMENTAL <prior-manifest> for
PG 17 incremental support. See
wal pipeline.
A WAL streaming mode where Region-B's agent streams from Region-A's repo (not from PG), keeping primary load independent of region count. Selected for multi-region deployments. See wal pipeline.
The repository layout in which every chunk is keyed by its plaintext SHA-256. Two backups that share a 4 KiB region share one chunk; deleting one backup never invalidates another. See content-addressed storage.
The agent's startup hook that writes its own cgroup with
memory.max (default 70% of host) so approaching limits
triggers chunker backpressure rather than the kernel
OOM-killing us mid-pg_backup_stop. Linux only. See the
resilience section of the
architecture tour.
A content-addressed unit of repository storage. Plaintext
SHA-256 is the key; on-disk object is
[1B version | 1B compression | 1B encryption | 12B nonce | payload].
See
content-addressed storage.
The per-backend-host failure-rate watchdog that opens (and stops sending requests) on sustained errors, ramps back up gradually after a cool-off. Prevents a flaky region from starving healthy ones. See the resilience section of the architecture tour.
A per-deployment label — Public / Internal / Confidential
/ Restricted — that drives retention floor, encryption
requirement, and allowed regions. Set via
pg_hardstorage classify. See
data-residency how-to.
The orchestration layer: schedules, RBAC, fleet view, audit, verifier. In-process for single-host (embedded mode); a separate runtime for fleets. Intentionally thin, to avoid the central-throughput choke pgBackRest hits at scale. See three execution modes.
The Kubernetes-native lease primitive pg_hardstorage uses for leader election in K8s topologies. No second coordination service needed. See coordination without etcd.
The Sigstore CLI used to sign release artefacts (keyless). Backup manifests are signed separately with the agent's Ed25519 keyring, and that signature is verified on every manifest read.
The async copy of every committed manifest + its chunks to a repository owned by a different cloud account, with an explicit ACL boundary. Used for M&A and partner-data scenarios. See the SPEC's enterprise features § data lifecycle.
"Distributed Configuration Store" — Patroni's term for
etcd / Consul / Zookeeper. pg_hardstorage reuses
Patroni's existing DCS (writing under
/pg_hardstorage/<deployment>/...) instead of standing up
a second one. See
coordination without etcd.
The control plane's overdue-backup alert: if no successful
backup happens in N×scheduled_interval (default 2×),
raise backup_overdue through every configured Sink.
Same for WAL silence.
"Data Encryption Key." In pg_hardstorage's three-layer envelope, the DEK is the per-backup random key (the BDEK). Chunks use further per-chunk keys derived from the BDEK via HKDF. See envelope encryption.
A logical PostgreSQL service we back up — one Patroni
cluster, one standalone primary, one CNPG Cluster. Replaces
the word stanza from pgBackRest. Bound to ≥ 1 agents
for HA.
The default verifier sandbox: Docker postgres:<major>
container with tmpfs scratch, used by verify --full to
exercise restore + pg_verifybackup + pg_amcheck. See
verify-sandbox tradeoffs.
pg_hardstorage doctor — the single-command UX for "is
everything OK?". Surfaces health checks per deployment
with plain-English remediation suggestions you run
yourself. See the
doctor CLI reference.
The WAL streaming mode where two replication slots on two
different nodes (typically primary + replica) feed the
same content-addressed store concurrently. CAS dedup
makes duplicate chunks free; either stream can fail
without RPO impact. Auto-selected at ≥ 50 TB or
availability=high. See
wal pipeline.
The single-process execution mode where agent + minimal
control plane live in one binary, with bookkeeping in
small JSON files under <state>/bookkeeping/. The
default for single-host deployments; restarting the binary
is the entire HA story. See
three execution modes.
The three-layer key hierarchy: a KEK wraps a BDEK which derives per-chunk keys via HKDF. Crypto-shred works because destroying the KEK makes every DEK unrecoverable. See envelope encryption.
A signed, self-contained tarball produced by
pg_hardstorage llm export-session (or the audit
export-bundle command) containing every prompt, tool
call, response, executed command, and a Merkle proof.
Independently verifiable post-hoc; the artefact a regulator
sees in a post-incident review. See
audit evidence bundles.
The content-defined chunking algorithm pg_hardstorage uses (gear-hash, 4 KiB / 64 KiB / 256 KiB parameters), with forced splits at PG's 8 KiB page boundaries for heap and index files. Keeps dedup ratios high across small page changes. See content-addressed storage.
The pg-hardstorage-fips build flavour
(GOEXPERIMENT=boringcrypto) — every crypto/* call
routes through Google's FIPS 140-2 validated module.
Refuses to start if crypto/tls reports non-FIPS.
--fips-strict panics on any non-FIPS plugin. See
FIPS variant.
The opt-in alternative verifier sandbox — KVM-isolated microVMs instead of Docker containers, for stronger isolation when verifying untrusted backups. Linux + KVM only. See firecracker sandbox.
The wire contract pg_hardstorage uses to host Tier-2
plugins: a one-shot stdio JSON-RPC protocol
(pg_hardstorage.plugin.v1) — the host launches the plugin
executable per operation and exchanges line-delimited JSON
over stdin/stdout. Crash-isolated, language-agnostic. See
Tier-2 protocol.
The repo-root magic file ({"version":1,"id":"...","tenants":[...]})
that marks a directory or bucket prefix as a
pg_hardstorage repository. Read on every connect; refuses
to operate without it.
"Just-in-time" — time-bound elevated tokens issued for
break-glass restore operations, auto-expiring, audit-
stamped. Surfaced via pg_hardstorage jit issue. See
the jit CLI reference.
"Key Encryption Key." The long-lived per-tenant key, held in KMS, that wraps the BDEK of every backup taken under that tenant. Destroying the KEK is the crypto-shred primitive. See envelope encryption.
The URL-shaped reference to a KEK
(aws-kms://arn:..., gcp-kms://..., vault-transit://...,
local:default, etc.). Stored in
manifest.json.encryption.kek_ref. Schemes documented in
the auto-generated KEKRef reference page.
The single primary in a Patroni-managed cluster, elected via the DCS lease. pg_hardstorage's agent watches the Patroni REST API for leader changes and reconnects to the new leader on failover. See Patroni failover deep-dive.
A flag that suspends deletion regardless of retention
policy — set via pg_hardstorage hold add, removable only
by an actor with the right RBAC verb, every change
audit-logged. See
legal hold.
pg_hardstorage llm — the grounded chat surface that
reads cluster state, runbooks, and audit log; suggests
commands; (in opt-in mode) executes them after preview +
confirmation. Read-only by default. See
LLM safety stack.
The plugin tier for chat-completion backends (OpenAI, Bedrock, Vertex, Ollama, llama.cpp, Hugging Face, …). Tier-1 in-tree, Tier-2 external via stdio JSON-RPC. See the LLM provider contract.
The per-backup JSON document — manifest.json — that
declares the backup's identity, LSNs, files, chunks, WAL
required, encryption envelope, and attestation.
Independently verifiable via the embedded public key.
Stored twice (canonical + replica prefix) for redundancy.
pg_hardstorage llm --mcp-server — the Model Context
Protocol stdio (or TCP) server that exposes the LLM
helper's tool surface to any MCP-aware client (Continue,
Cursor, Zed, Goose, Cline, …). See the SPEC's
LLM helper § MCP.
See audit chain.
The CLI flag that auto-launches the LLM helper with the failed command's context already loaded after a mutating command fails. The 3am-operator entry point.
Patroni 3.0+'s replication-slot management feature. When
a slot is declared as permanent, Patroni recreates it on
the new leader after failover with restart_lsn
propagated from the old leader. pg_hardstorage's
preferred slot-continuity strategy. See
Patroni failover deep-dive.
The PG-bundled tool that walks heap and index pages to
detect corruption. Run as part of full-verify alongside
pg_verifybackup.
The PG 17 tool that flattens an incremental chain into a single full data dir. pg_hardstorage's restore wraps it when the selected target is an incremental.
The PG-bundled tool a rejoining old primary uses to
fast-forward to a common ancestor LSN with the new primary
after a Patroni failover. pg_hardstorage's slot-recovery
logic accounts for pg_rewind removing WAL from the
rewound node.
The PG-bundled tool that re-checks file checksums against
a manifest. Run after every restore (and as fast verify
after every backup). Restore success is gated on this
unless --skip-verify is explicitly acknowledged.
The PG SQL function that converts an LSN to a WAL filename.
Used by the backup orchestrator to confirm that the
stop_lsn of a base backup is in our WAL store before
manifest commit.
The chunker's forced split at every 8 KiB page boundary
inside heap and index files (base/, indexes). Keeps
dedup high: a single page changing in a 1 GB heap touches
exactly one chunk, not two. See
content-addressed storage.
The popular PG cluster-manager pg_hardstorage integrates
with via REST + DCS awareness, permanent_slots, dual-
slot, and sync-target modes. See
Patroni failover deep-dive.
A pluggable extension point for storage, source,
encryption, compression, renderer, sink, and LLM provider.
Tier-1 plugins are first-party and compiled into the
binary; Tier-2 plugins are third-party, ship as separate
binaries, and are discovered on $HSPLUGIN_PATH via
stdio JSON-RPC. See
Tier-1 vs Tier-2.
See replication slot.
The Sigstore transparency log that pg_hardstorage anchors its audit chain (and optional manifest signatures) into. Public, verifiable, immutable. See audit chain.
The synchronous, command-scoped output plugin tier — takes
typed Event values and writes bytes to a Writer
(stdout, stderr, file). Built-ins: text, json,
ndjson, yaml, template. See the
Renderer contract.
A non-primary PG node managed by Patroni. pg_hardstorage auto-routes backups to a Patroni replica at deployments ≥ 5 TB to keep primary I/O free.
The redundant copy of a manifest written at
manifests/_replicas/<id>.manifest.json. Cheap insurance
against single-key corruption. Recoverable via
pg_hardstorage repair manifest.
The PG protocol (replication=database connection
parameter) used for BASE_BACKUP, START_REPLICATION,
and IDENTIFY_SYSTEM. pg_hardstorage's entire data plane
runs over this — the reason the agent needs no SSH or OS
access to the database host. See
wal pipeline.
A persistent server-side cursor that tells PG to retain
WAL until a specific consumer acknowledges it.
pg_hardstorage uses pg_hardstorage_<deployment> as the
default name. See
architecture tour § slot semantics.
The destination where chunks, manifests, and WAL live —
e.g. s3://acme-pg-backups/. One repo can hold many
deployments. See the SPEC's
Vocabulary table.
The syslog severity standard pg_hardstorage's event severities map onto: emerg(0) / alert(1) / crit(2) / error(3) / warning(4) / notice(5) / info(6) / debug(7). Renderers and Sinks both use this mapping.
"Repository Key Encryption Key" — a synonym for KEK when emphasising that it's held at the repository level. See envelope encryption.
"Recovery Point Objective" — the maximum acceptable data
loss measured in time. pg_hardstorage exports
pg_hardstorage_rpo_seconds{deployment} and surfaces SLO
violations through SLO-as-code. See
SLO as code.
"Recovery Time Objective" — the maximum acceptable restore duration. The agent runs a 30-second throughput probe before a long restore and prints the projected RTO in the preview block.
A < 1-page Markdown playbook for a named disaster
scenario (R1–R7). Shipped with the binary, addressable
from doctor when relevant, customisable per deployment.
See the runbook index.
The periodic chunk-rehash job (pg_hardstorage repair scrub) that reads chunks back, decrypts and re-hashes
them, and surfaces mismatches as verify.scrub_mismatch.
Detects bit-rot at rest. See
scrub-and-heal how-to.
The user / group provisioning standard pg_hardstorage implements at v1.0 for auto-provisioning and de-provisioning of human users from the IdP.
The RFC 5424-aligned numeric severity attached to every
Event. Sinks declare a severity floor; the active
renderer doesn't filter (always renders what the user
asked the command to do). See RFC 5424.
The execution mode where the agent runs as a sidecar container next to PG in the same Pod (CloudNativePG, Zalando, Crunchy patterns). Talks to PG over the local Unix socket. See three execution modes.
The asynchronous, system-scoped output plugin tier — takes
typed Event values and fans them out to external systems
on its own schedule (Slack, Jira, PagerDuty, syslog,
webhook, …). See the
Sink contract.
A versioned, declarative YAML file under
/usr/share/pg_hardstorage/skills/ that defines an LLM
behaviour (restore wizard, incident responder, …).
Hot-reloadable, signed, RBAC-scoped, golden-tested
against a pinned model checkpoint per release. See the
SPEC's
LLM helper § skills.
"Service Level Objective" — a declarative per-deployment RPO and RTO target. pg_hardstorage compares observed metrics against the SLO and raises alerts on violation. See SLO as code.
"Supply chain Levels for Software Artifacts." pg_hardstorage's release artefacts ship with SLSA Level 3 build provenance attestations. See SLSA L3 provenance.
The plugin tier for backup sources — streaming
BASE_BACKUP, PG 17 incremental, snapshot
(ZFS / Btrfs / LVM / cloud-volume). The agent picks a
source based on the deployment profile. See the
Source contract.
The PG replication-protocol command that begins WAL streaming from a slot at a given LSN. pg_hardstorage invokes it for every connection to the primary (or replica in offload mode).
pgBackRest's term for a logical PG service. In pg_hardstorage we use Deployment.
The URL prefix that selects a storage backend —
s3://, gcs://, azblob://, sftp://, scp://,
file://, etc. Auto-generated reference page lists all
built-in schemes.
The structured locator carried inside every Event —
{Tenant, Deployment, BackupID, Timeline, …} — used by
Sinks for ticket dedup (e.g. recurring failures update one
Jira ticket instead of spawning fifty).
The tiny parent process (< 5 MB RSS) that fork-execs the
agent worker, watches via Unix-socket heartbeat, captures
crash bundles on death. systemd Restart=always is
layered on top for double-coverage.
The opt-in WAL mode (wal_mode: synchronous) where the
agent advertises itself as a synchronous_standby_names
candidate. PG waits for our flush ACK before commit;
RPO = 0 at the cost of write latency. See
wal pipeline.
An isolation boundary — one customer in SaaS, or a logical
zone like prod/dev in single-org. Each tenant has its
own KEK; single-org users get a default tenant they never
see. Crypto-shred operates at tenant granularity.
A first-party plugin compiled into the binary. Self-
registers via init(); one signed binary is easier to
audit, FIPS-build, and ship. See
Tier-1 vs Tier-2.
A third-party plugin shipped as a separate binary,
discovered on $HSPLUGIN_PATH, hosted via one-shot stdio
JSON-RPC. Crash-isolated, language-agnostic.
PG's monotonically-increasing identifier for a recovery branch. Promotion increments the TLI; any WAL the old primary had not replicated is forked. Manifests record the timeline they ended on; PITR walks the timeline- history files. See Patroni failover deep-dive.
The .history file PG produces on each promotion that
records the parent TLI and switch LSN. pg_hardstorage
captures every .history file into the repo at first
sight; PITR across a failover boundary depends on it.
A public, append-only log (Rekor by default) that anchors audit chain hashes externally so tampering becomes detectable from outside the system being audited. See audit chain.
Source-side encryption of PostgreSQL heap files, indexes,
the control file, and WAL at rest, performed by a PG fork
(CYBERTEC PGEE, pg_tde, EDB TDE) using a key the operator
holds. Orthogonal to repo-side
envelope encryption:
TDE protects bytes-at-rest on the SOURCE PG's disks, envelope
encryption protects bytes-at-rest in the REPO; both layers
can be active simultaneously. pg_hardstorage handles TDE
deployments via a single per-deployment tde.enabled: true
flag — see TDE awareness.
The terminal user interface — pg_hardstorage ui (and the
LLM chat surface) — that renders fleet view, progress
bars, and chat conversations directly in the terminal.
The subsystem that runs pg_verifybackup and (in full
mode) actually restores into a Docker / Firecracker /
k8s-Job sandbox to exercise restore. See
verify-sandbox tradeoffs.
PostgreSQL's "Write-Ahead Log" — the durability primitive that makes PITR possible. pg_hardstorage streams WAL continuously via the replication protocol and stores it as content-addressed chunks alongside backups.
A discontinuity in the repo's WAL inventory between two
LSNs, typically caused by a slot drop or an asynchronous
failover. The gap auditor detects gaps and emits
wal.gap_detected; the manifest of any backup taken after
a gap explicitly records the gap range; PITR inside the
gap window is refused with a clear error. See
wal pipeline § gap auditor.
The explicit operator command that drops and recreates a replication slot, accepting whatever WAL gap that introduces (which it reports honestly). Runbook: R6 — Slot dropped, gap detected.
"Write Once, Read Many" — the immutability primitive
configured per-repo (worm: true, retention: 7y). Backed
by S3 Object Lock (Compliance mode), Azure immutable blob,
NetApp SnapLock, or POSIX chattr +i. Enforced via the
storage plugin's SetRetention.