Skip to content

feat(gcp_stackdriver_logs sink): Support user-defined insertId#24968

Open
garethpelly wants to merge 9 commits into
vectordotdev:masterfrom
garethpelly:add-stackdriver-insertId-field
Open

feat(gcp_stackdriver_logs sink): Support user-defined insertId#24968
garethpelly wants to merge 9 commits into
vectordotdev:masterfrom
garethpelly:add-stackdriver-insertId-field

Conversation

@garethpelly

@garethpelly garethpelly commented Mar 19, 2026

Copy link
Copy Markdown
Contributor

Summary

This adds support for setting a custom insertId at the root level of the LogEntry, allowing users to define their own unique identifier for each log entry.

In a similar way to how the severity_key field works with the sink now, we extract a custom insertId from log events via an insert_id_key configuration option. The named field is removed from the log event if present.

Vector configuration

sources:
  dummy_logs:
    type: "demo_logs"
    format: "syslog"
    interval: 1

transforms:
  parse_logs:
    type: "remap"
    inputs: ["dummy_logs"]
    source: |
      . = parse_syslog!(string!(.message))
      .my_insert_id = uuid_v4()

sinks:
  my_sink_id:
    type: gcp_stackdriver_logs
    project_id: "my-project"
    log_id: "test_log"
    inputs:
      - parse_logs
    insert_id_key: "my_insert_id"
    resource:
      type: "gce_instance"
      project_id: "my-project"
      instanceId: "Twilight"
      zone: "us-central1-a"

How did you test this PR?

  • Ran unit tests
  • Tested using my Google account and inspected logs in Google Cloud Logging

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@github-actions github-actions Bot added the domain: sinks Anything related to the Vector's sinks label Mar 19, 2026
@garethpelly garethpelly changed the title enhancement(gcp_stackdriver_logs sink): Support user-defined insertId feat(gcp_stackdriver_logs sink): Support user-defined insertId Mar 19, 2026
@garethpelly garethpelly force-pushed the add-stackdriver-insertId-field branch from ad3acb6 to 947e021 Compare March 19, 2026 22:31
@github-actions github-actions Bot added the domain: external docs Anything related to Vector's external, public documentation label Mar 20, 2026
@garethpelly garethpelly force-pushed the add-stackdriver-insertId-field branch from a36e9c2 to ae35824 Compare March 20, 2026 10:19
@github-actions github-actions Bot added domain: ci Anything related to Vector's CI environment domain: external docs Anything related to Vector's external, public documentation and removed domain: external docs Anything related to Vector's external, public documentation labels Mar 20, 2026
@garethpelly garethpelly force-pushed the add-stackdriver-insertId-field branch from 8dd3d4f to ae35824 Compare March 20, 2026 10:56
@github-actions github-actions Bot removed the domain: external docs Anything related to Vector's external, public documentation label Mar 20, 2026
@github-actions github-actions Bot added the domain: external docs Anything related to Vector's external, public documentation label Mar 20, 2026
@garethpelly garethpelly marked this pull request as ready for review March 20, 2026 12:09
@garethpelly garethpelly requested review from a team as code owners March 20, 2026 12:09

@OliviaShoup OliviaShoup left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the PR! 🚀

@garethpelly

Copy link
Copy Markdown
Contributor Author

@vectordotdev/vector please review 🙏

@garethpelly

Copy link
Copy Markdown
Contributor Author

@vectordotdev/vector gentlest of nudges, let me know if there is anything else you need me to do

@pront

pront commented Apr 14, 2026

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@garethpelly

Copy link
Copy Markdown
Contributor Author

@pront Thanks! Is this fit for merging now?

@garethpelly

Copy link
Copy Markdown
Contributor Author

@vectordotdev/vector Can we merge?

@github-actions github-actions Bot added the docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. label May 23, 2026
@pront pront added the sink: gcp_stackdriver_logs Anything `gcp_stackdriver_logs` sink related label May 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. domain: ci Anything related to Vector's CI environment domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks sink: gcp_stackdriver_logs Anything `gcp_stackdriver_logs` sink related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support user-defined insertId in gcp_stackdriver_logs sink

3 participants