Skip to content

Commit 2919bf5

Browse files
haiyuan-eng-googlecopybara-github
authored andcommitted
fix(plugins): harden BigQuery agent analytics against fail-open privacy, GCS concurrency, and startup-loss gaps
Close several correctness and privacy gaps in the BigQuery agent analytics plugin: - content_formatter failures fail closed with a [FORMATTER_FAILED] sentinel instead of falling back to the unformatted payload; the exception is logged without the payload and counted as formatter_failed in get_drop_stats(). - A final sanitizer pass runs over the fully assembled attributes tree before json.dumps, covering state deltas, custom tags, labels, nested structures, and temp:-scoped keys. A new _sanitize_json_blob redacts sensitive keys inside JSON-encoded string blobs (e.g. cached credential JSON): it decodes container-shaped strings first, enforces max_content_length before json.loads, and fails closed to [UNPARSEABLE_JSON_BLOB] for anything it cannot parse and verify. - GCS offload paths are call-local: parse() and _parse_content_object() take trace_id/span_id as keyword arguments and build every object path from those values, so concurrent two-part offloads never collide on an object name. - Table readiness becomes a startup requirement with bounded retry: _ensure_schema_exists raises on failure, and _ensure_started records it, keeps _started False, and retries on a later event after exponential backoff (2s to 60s cap). Setup is coalesced across event loops and threads, so a persistent outage costs one setup RPC per backoff window. Rows arriving while setup is unavailable are counted as setup_unavailable. Also: enabled=False is a hard no-op with zero side effects, runtime settings are validated at construction, and plugin-level drop counters survive shutdown. Co-authored-by: Haiyuan Cao <haiyuan@google.com> PiperOrigin-RevId: 949235728
1 parent 9cf295d commit 2919bf5

2 files changed

Lines changed: 2034 additions & 97 deletions

File tree

0 commit comments

Comments
 (0)