Update ghcr.io/streamingfast/firehose-core Docker tag to v1.14.4

[renovate]

This PR contains the following updates:

Package	Update	Change
ghcr.io/streamingfast/firehose-core	minor	v1.12.3 → v1.14.4

---

Release Notes

streamingfast/firehose-core (ghcr.io/streamingfast/firehose-core)

v1.14.4

Compare Source

Removed from docker image

• The 'grpc_health_probe' binary is no longer included in the docker image. You can use the HTTP '/healthz' endpoints instead or use your own GRPC poller.

Added

• merger and relayer now expose an HTTP /healthz endpoint on a dedicated port via the new --merger-http-healthz-addr (default :10013) and --relayer-http-healthz-addr (default :10018) flags. Set the flag to an empty string to disable. The endpoint returns HTTP 200 when the service is ready and 503 otherwise (including during the common-system-shutdown-signal-delay graceful-shutdown window).
• Config file now supports a global: section for setting persistent (global) flags such as shift-ports, log-format, log-to-file, etc. These flags can also still be set under the command-specific section (e.g. start.flags), but global: is more intuitive for flags that apply regardless of command.
• tools compare-blocks: A single block number (e.g. 2713) is now accepted as the range argument, automatically expanding to the 100-block bundle that contains that block (e.g. 2700:2799).
• Substreams Index optimisation: Optimized ClockDistributor to skip blocks earlier and faster when using block filter.
• Substreams: add substreams_tier2_max_concurrent_requests and substreams_tier1_active_requests_hard_limit metrics to prometheus

Fixed

• tools substreams store-size: Fix N/A shown for all stores. For local state stores, the compressed file size is now displayed as a fallback when no uncompressed metadata is available. For GCS state stores, the compressed size is now returned even when files lack datasize metadata, and both .kv and .kv.zst file extensions are now accepted. The "Live (uncompressed)" column is renamed to "Live Size" and now shows "Not found" when no state exists for a module. The computed module hash is shown in the table for easy manual GCS path verification. A warning is shown when no state data is found for any module, hinting at a possible wrong --state-store URL.
• Substreams: Fix server-side bug that would cause Blocks request to fail after a few retries with 'load full store (...) load store stream: opening file for streaming: not found' when depending on a store that is being merged slowly

Changed

• tools compare-blocks: --diff flag is now a string flag instead of a boolean. --diff or --diff=inline prints inline diffs using diffx; --diff=editor opens $DIFF_EDITOR (falls back to diff -u); --diff=<cmd> uses that command as the diff editor (e.g. --diff=vimdiff). The separate --diff-editor flag has been removed.
• tools compare-blocks: diff output uses diffx for richer unified diffs with line numbers, ANSI color, and inline character-level highlighting.

v1.14.3

Compare Source

Fixed

• MergedBlocksWriter.WriteBundle: close the read end of the internal pipe and wait for the producer goroutine to finish before returning. Previously, if the underlying Store.WriteObject returned without consuming the reader (e.g. an S3 destination skipping an existing object when overwrite is disabled), the producer goroutine would block forever on its next pipe write, leaking the goroutine and pinning the bundle's blocks in memory. This affected tools download-from-firehose against S3 destinations that already contained any of the requested bundles.

v1.14.2

Compare Source

• Library dsession bumped to latest version which brings:

  • Bug fixes around session handling at scale.

• Library dstore bumped to latest version v0.2.3 which brings these changes:

  • GCS store: disable gRPC DirectPath when both a project is set and client_protocol=grpc is used, preventing connection issues in that configuration.
  • S3 store: share a single HTTP transport across all store clones for proper HTTP/2 multiplexing, replacing the previous per-clone transport that broke connection sharing.

v1.14.1

Compare Source

Updated

• Library dstore bumped to latest version which brings these changes:
  • GCS store: opt-in gRPC transport via client_protocol=grpc query parameter (e.g. gs://bucket/path?client_protocol=grpc). Defaults to the existing HTTP client; the gRPC client is selected only when this parameter is explicitly set.
  • S3 store: storageClass query parameter is deprecated in favour of storage_class; a warning is logged when the old form is used. storage_class query parameter as the canonical snake_case name for storageClass.
  • S3 store: each store clone now gets its own transport for failure isolation; adds ResponseHeaderTimeout to prevent hung requests and configures HTTP/2 health checks via x/net/http2; default connection pool sizes are reduced.

Fixed

• Substreams: Fix server-side bug that would prevent forkableHub from correctly updating metrics when receiving partial or out-of-order blocks

v1.14.0

Compare Source

Added

• Added --shift-ports global flag that shifts all Firehose service port numbers by a given offset, useful for running multiple instances on the same machine without port conflicts. Both server listen addresses and internal client connection addresses are shifted so wiring stays consistent. Infrastructure ports (Prometheus metrics, pprof, log-level-switcher) are also shifted. Example: fire{chain} start --shift-ports 100 shifts all ports by +100.
• Added '--merger-max-merging-threads' (defaults: 4) so that the merger can merge blocks in parallel (still using way less RAM than previous one-block-preloading method)
• S3 store: configurable HTTP connection pool via DSTORE_S3_MAX_IDLE_CONNS, DSTORE_S3_MAX_IDLE_CONNS_PER_HOST, DSTORE_S3_IDLE_CONN_TIMEOUT env vars

Changed

• Removed parallel preloading of one-block-files to reduce RAM usage when merging big blocks.

[!NOTE]
With this change, HEAD block timestamp is now updated maximum every 5 seconds instead of at every block, by reading the first 500 bytes of the last one-block-file.

Fixed

• S3 store: fixed goroutine leak caused by connection pool exhaustion on single-host S3 stores (e.g. MinIO); HTTP body is now explicitly drained and closed, and the transport is configured with MaxIdleConnsPerHost=100 by default

v1.13.3

Compare Source

• Fix substreams support for requests with 'application/grpc-web*' content-type (old connectweb library)

v1.13.2

Compare Source

• Add 'mindreader stats' info log every 30secs with performance metrics

v1.13.1

Compare Source

Added

• Add firecore tools networks list command to display registered networks from The Graph Networks Registry with their Firehose and Substreams endpoints. Supports --name-only flag for listing only network IDs and --only flag for filtering networks using a regular expression.
• Add substreams-tier2-authenticator flag to specify the authenticator to use for tier2 requests. Can be 'trust://' (default, same as previous behavior) or 'secret://'
• Add substreams-tier1-subrequests-secret-key flag to specify the secret key to use for tier1 subrequests authentication when using 'secret://' authenticator on tier2
• Add reader-node-grpc-secret-key flag to specify the secret key to use for reader node gRPC authentication
• Add ?secret=... parsing to relayer-sources
• Add Prometheus metrics for reader test mode: track blocks compared, success/failure counts, and success/failure percentages for easy monitoring at interval stats.

Changed

• Refactor reader test mode Prometheus metrics to fix incorrect success/failure percentage calculation caused by unaccounted blocks. Renamed blocks_matched_total -> blocks_compared_matched_total and blocks_mismatched_total -> blocks_compared_mismatched_total. The blocks_compared_total metric now counts only blocks that were fully compared (matched + mismatched). Added three new metrics: blocks_seen_total (all attempted blocks), blocks_reorg_total (skipped due to re-org/ID mismatch), and blocks_fetch_failure_total (failed to fetch from production). Invariants: blocks_seen == blocks_reorg + blocks_fetch_failure + blocks_compared and blocks_compared == blocks_compared_matched + blocks_compared_mismatched.

Fixed

• Fix substreams/firehose endpoints detection of supported compression: do not fail on 'algo;q=x.y' syntax
• Fix substreams tier2 jobs behind load balancer: will now retry forever on 'Unavailable: no healthy upstream' errors
• Fix relayer failing to get back to live if reader blocks are unlinkable after a long period, and merger has removed one-blocks: it will now shutdown in that case, so it can be restarted.

v1.13.0

Compare Source

Substreams Performance (RPC V4, New blocks from last partial)

• RPC V4 protocol with BlockScopedDatas batching: Multiple BlockScopedData messages are now batched into a single BlockScopedDatas response, reducing gRPC round-trips and message framing overhead during backfill. Clients automatically fall back V4 → V3 → V2 when connecting to older servers, so no flag changes are required.
• S2 compression is now the default: Replaces gzip as the default compression algorithm, providing ~3-5x faster compression/decompression with comparable ratios. The client automatically negotiates compression with the server.
• VTProtobuf fast serialization: Both client and server now use vtprotobuf for protobuf marshaling/unmarshaling, providing ~2-3x faster serialization with reduced memory allocations.
• Server-side message buffering: Configurable via --substreams-tier1-output-buffer-size flag (default: 100 blocks) or MESSAGE_BUFFER_MAX_DATA_SIZE environment variable (default: 10MB).
• Improved Connect/gRPC protocol selection: Server now efficiently routes requests to the appropriate handler based on content-type, improving performance by ~15% for pure gRPC clients.
• New blocks from last partial: "Last partial blocks" are now accepted interchangeably with new blocks, allowing faster full blocks for requests that do not ask for partial blocks.

Added

• Add firecore tools substreams logs connections <user_id> command to query Cloud Logging and show Substreams connections for an organization. Correlates incoming requests with stats by trace ID and presents a summary table showing active, closed, and error connections with details like network, source IP, module, duration, and blocks processed.

Removed

• Remove alpha partial blocks support in firehose service (only exposed via substreams)

v1.12.8

Compare Source

• Substreams: Improved 'partial blocks': support new pbbstream's "LastPartial" field, fix 'undo' scenarios for stores
• Substreams: Improved 'partial blocks': support new pbbstream's "LastPartial" field, fix 'undo' scenarios for stores
• Reduce RAM usage with partial blocks (relayer, substreams, firehose)
• Prevent panic if transactionTrace.receipt is nil in LogFilter (even if it is not a normal scenario)

v1.12.7

Compare Source

• Bump Golang to build to 1.25

Substreams

• Fix issue where a retry on dstore while writing a fullKV would corrupt the file, making it unreadable. Fix prevents this and also now deletes affected files when they are detected
• Fix bug where so request could get stuck forever (until the clients drops or server restarts).
• Fix issue where transient HTTP/2 stream errors (e.g., INTERNAL_ERROR) from dstore were being treated as fatal errors instead of being retried. These transient network errors are now detected and retried with exponential backoff.

v1.12.6

Compare Source

• Added bucketed prometheus metrics head_block_relative_time_sum to help investigate latency and pipeline performance:
  • "app=firehose_output" and "app=substreams_output" that shows latency between outputing live blocks and their blocktime.
  • "app=relayer" for latency at relayer's input
  • "app=reader-node" for latency at reader's input
• Bump base64 library to use a much faster one in reader
• dstore: bumped google storage lib to v1.59.1 to fix a bug in their multi-range downloader, in case it affects us

Substreams

• Fixed underflow in 'FailedPrecondition desc = request needs to process a total of x blocks' error when running from 'substreams run' with a start-block in the future.

v1.12.5

Compare Source

Substreams fixes

• Fix issue where "live backfiller" would not create segments after reconnecting with a cursor starting from a previous quicksave, causing delays in future reconnection
• Prevent "panic" when log messages are too large: instead, they will be truncated with a 'some logs were truncated' message.
• Raise max individual log message size from 128k to 512k
• Raise max log message size for a full block from 128k to 5MiB
• Reduce log level from Warn to Debug when we fail to get or set the store size (for backends that don't support it)

Substreams Partial blocks (experimental)

• Removed PartialsData message and brought back this data inside the good old BlockScopedData
• added the following fields to BlockScopedData:
  • bool is_partial to indicate if this block is a partial block. The following two fields are only present when is_partial==true
  • optional bool is_last_partial to indicate if this is the last partial of a given block (with correct block hash)
  • optional uint32 partial_index to indicate the index of this partial block within the full block
• renamed partial_blocks_only flag to partial_blocks on substreams Blocks request
• removed include_partial_blocks flag from substreams Blocks request

AWS Store

• Migrated from AWS SDK for Go v1 to v2 (github.com/aws/aws-sdk-go → github.com/aws/aws-sdk-go-v2)

Azure store

• Added support for "workload identity credentials" in Azure. Order of preference is:
  • If AZURE_STORAGE_KEY is set, use shared key credential (previous behavior)
    • Otherwise, use DefaultAzureCredential which supports:
      • Managed Identity (for Azure resources)
      • Service Principal (via AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID)
      • Azure CLI credentials
      • Visual Studio Code credentials

v1.12.4

Compare Source

Partial blocks

Added experimental support for partial blocks (e.g. Flashblocks on Base)

See https://docs.substreams.dev/reference-material/chains-and-endpoints/flashblocks for details about how they work in Substreams.

• SUBSTREAMS_BIGGEST_PARTIAL_BLOCK_INDEX environment variable now specifies the index to use when bundling the "last partial block" from the full block. (default: 10, for Base)
• Added flag --include-partial-blocks on tools firehose-client

Bugfixes

• Substreams: fixed issue where "live backfiller" would not create segments after reconnecting with a cursor starting from a previous quicksave, causing delays in future reconnection

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[renovate]

Edited/Blocked Notification

Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.

You can manually request rebase by checking the rebase/retry box above.

⚠️ Warning: custom changes will be lost.