Skip to content

pg_hardstorage_wal_push.md

Latest commit

 

History

History
97 lines (75 loc) · 5.8 KB

pg_hardstorage_wal_push.md

File metadata and controls

97 lines (75 loc) · 5.8 KB
Raw
title pg hardstorage wal push
description CLI reference for the pg hardstorage wal push command.
tags
cli
reference

pg_hardstorage wal push

Archive one segment via archive_command (PG-invoked)

Synopsis

Push one PostgreSQL WAL segment file into the repository.

PG invokes this command from archive_command during normal operation:

archive_command = 'pg_hardstorage wal push db1 %p --repo '

The %p substitution is a path on the PG host; we read the file, chunk it through the CAS, and commit a segment manifest atomically. Re-pushes of an already-committed segment are no-ops.

system_identifier — stamped on every segment manifest so cross-cluster repo contamination is detectable — is derived in this order:

  1. --system-identifier if supplied (zero-cost path). This is the unsigned-decimal value pg_control_system() reports (SELECT system_identifier FROM pg_control_system()) — the SAME form rule 2 derives from the segment header. Passing it in any other base would mismatch header-derived pushes and trip a spurious splitbrain.system_identifier_mismatch.
  2. Otherwise read directly from the segment file's first-page XLogLongPageHeader.xlp_sysid — every WAL segment PG writes carries it. No libpq round-trip per call.
  3. Otherwise --pg-connection if supplied (legacy path).

The default canonical archive_command shape — repo only, no extra flags — works because of (2).

Transparent Data Encryption (TDE):

Pass --tde when the source PostgreSQL has TDE enabled (CYBERTEC PGEE, pg_tde, EDB TDE). Under TDE the segment file on disk is ciphertext, so the on-segment header read (precedence rule 2 above) is meaningless — it would either fail outright OR return bogus bytes that happen to look header-shaped. --tde skips rule 2 unconditionally; you MUST supply --system-identifier or --pg-connection.

Exit codes: 0 - segment archived (or already present)

0 - error; PG will retry per archive_timeout

pg_hardstorage wal push <deployment> <segment-path> [flags]

Options

  -h, --help                        help for push
      --kek string                  KEK reference for WAL encryption under a cloud KMS (e.g. aws-kms://...); MUST match the deployment's base-backup KEKRef. Local-KEK encryption is automatic when a keyring kek.bin is present.
      --kms-config stringToString   cloud KMS provider config (region/endpoint/credentials); only consulted when --kek is a cloud scheme — the same values base backups use. (default [])
      --pg-connection string        libpq connection string — used once to fetch the system_identifier
      --repo string                 repository URL (file://, s3://, ...) — must already exist (required)
      --system-identifier string    explicit pg_control system_identifier (skip libpq round-trip)
      --tde                         the source PG has Transparent Data Encryption enabled; skip on-segment header parsing (requires --system-identifier or --pg-connection). See docs/explanation/tde-awareness.md.
      --wal-segsize int             cluster wal_segment_size in megabytes (matches initdb --wal-segsize); a segment file whose length differs from this is refused as truncated/corrupt (default 16)

Options inherited from parent commands

      --airgapped airgapped: strict                                                       refuse outbound endpoints (LLM providers, sinks, OTLP collectors) outside loopback / RFC1918 / explicit airgap.allowlist. Also enabled by PG_HARDSTORAGE_AIRGAPPED=1 or airgapped: strict in the config file.
  -c, --config string                                                                     path to config file (default: XDG/FHS lookup)
      --cpu-profile go tool pprof <path>                                                  write a pprof CPU profile to this path for the duration of the command (go tool pprof <path> to analyse). Off when empty.
      --mem-profile string                                                                write a pprof heap profile to this path at command exit. Off when empty.
      --no-color                                                                          disable ANSI color in text output
      --on-error-llm                                                                      on a structured-error failure, drop into the matching LLM helper skill (auto_on_error trigger). Also enabled by PG_HARDSTORAGE_ON_ERROR_LLM=1.
      --otel-endpoint string                                                              OpenTelemetry OTLP/HTTP endpoint (e.g. http://otel-collector:4318); empty disables tracing
      --otel-stdout                                                                       also export OpenTelemetry traces to stderr (useful for dev)
  -o, --output string                                                                     output format: text|json|ndjson|yaml|template|csv|markdown|html|tap|junit|pdf (default: text on TTY, json off-TTY)
      --profile-port go tool pprof http://127.0.0.1:6060/debug/pprof/profile?seconds=30   if non-zero, expose net/http/pprof on 127.0.0.1:<port> for live profiling of long-running commands (e.g. go tool pprof http://127.0.0.1:6060/debug/pprof/profile?seconds=30). Off when zero.
  -q, --quiet                                                                             suppress non-essential output
      --template string                                                                   Go text/template applied when --output template (or implied if --template is set without --output)

SEE ALSO