[datadog_observability_pipeline] Add ClickHouse destination (OPA-5308)#3810
Draft
Jansen-w wants to merge 2 commits into
Draft
[datadog_observability_pipeline] Add ClickHouse destination (OPA-5308)#3810Jansen-w wants to merge 2 commits into
Jansen-w wants to merge 2 commits into
Conversation
…es TF provider Implements the clickhouse destination component (OPA-5308) with full field surface matching the OPW v1 schema: endpoint_url_key, database, table, format (json_each_row/json_as_object/json_as_string/arrow_stream), skip_unknown_fields, date_time_best_effort, compression (algorithm+level), auth (basic strategy), batch (max_events, timeout_secs), batch_encoding (arrow_stream codec), tls, and buffer. Registers the destination in the main resource model, schema, expand, and flatten switch statements. Adds three acceptance tests covering the full config, minimal config, and arrow_stream format with batch_encoding. NOTE: This PR requires a go.mod bump to a new datadogV2 SDK pseudoversion that includes ObservabilityPipelineClickhouseDestination types (OPA-5308 API spec PR not yet merged). Build will fail until that pseudoversion is available. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
|
Add required auth_strategy field to http_server source blocks in the clickhouse destination acceptance tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
clickhousedestination to the Observability Pipelines Terraform providerendpoint_url_key,database,table,format,skip_unknown_fields,date_time_best_effort,compression,auth,batch,batch_encoding,tls,bufferarrow_streamformat withbatch_encodingSDK Dependency — BLOCKER
This PR will not compile until the following dependency is resolved:
The
datadogV2.ObservabilityPipelineClickhouseDestinationSDK types do not yet exist in thedatadog-api-client-gomodule. The API spec PR (OPA-5308) must be merged first, and thengo.modmust be bumped to the new pseudoversion that includes the generated ClickHouse types.Once the API spec PR lands:
API_CLIENT_VERSION=<new-pseudoversion> make update-go-clientgo build ./...andgo vet ./...to verifyFiles Changed
datadog/fwprovider/observability_pipeline/clickhouse_destination.go— new component filedatadog/fwprovider/resource_datadog_observability_pipeline.go— registration (model, schema, expand, flatten)datadog/tests/resource_datadog_observability_pipeline_test.go— acceptance testsTest plan
go.modto new pseudoversiongo build ./...passesgo vet ./...passesRECORD=false TESTARGS="-run TestAccDatadogObservabilityPipeline_clickhouse" make testaccwith cassettes🤖 Generated with Claude Code