Skip to content

Commit 806619a

Browse files
api-clients-generation-pipeline[bot]ci.datadog-api-spec
api-clients-generation-pipeline[bot]
and
ci.datadog-api-spec
authored
Adding all Observability Pipeline new hec_token related features into the api (#3399)
Co-authored-by: ci.datadog-api-spec <packages@datadoghq.com>
1 parent aa49333 commit 806619a

File tree

23 files changed

+766
-8
lines changed

23 files changed

+766
-8
lines changed

‎.generator/schemas/v2/openapi.yaml‎

Lines changed: 63 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -44467,6 +44467,40 @@ components:
4446744467
type: string
4446844468
x-enum-varnames:
4446944469
- ELASTICSEARCH
44470+
ObservabilityPipelineEnrichmentTableFieldEventLookup:
44471+
description: Looks up a value from a field path in the log event.
44472+
properties:
44473+
event:
44474+
description: The path to the field in the log event to use as the lookup key.
44475+
example: log.user.id
44476+
type: string
44477+
required:
44478+
- event
44479+
type: object
44480+
ObservabilityPipelineEnrichmentTableFieldSecretLookup:
44481+
description: Looks up a value stored as a pipeline secret.
44482+
properties:
44483+
secret:
44484+
description: The name of the secret containing the lookup key value.
44485+
example: MY_LOOKUP_SECRET
44486+
type: string
44487+
required:
44488+
- secret
44489+
type: object
44490+
ObservabilityPipelineEnrichmentTableFieldStringPath:
44491+
description: A plain field path in the log event, used as the lookup key.
44492+
example: log.user.id
44493+
type: string
44494+
ObservabilityPipelineEnrichmentTableFieldVrlLookup:
44495+
description: Evaluates a VRL expression to produce the lookup key.
44496+
properties:
44497+
vrl:
44498+
description: A VRL expression that returns the value to use as the lookup key.
44499+
example: .log.user.id
44500+
type: string
44501+
required:
44502+
- vrl
44503+
type: object
4447044504
ObservabilityPipelineEnrichmentTableFile:
4447144505
description: Defines a static enrichment table loaded from a CSV file.
4447244506
properties:
@@ -44517,6 +44551,16 @@ components:
4451744551
type: string
4451844552
x-enum-varnames:
4451944553
- CSV
44554+
ObservabilityPipelineEnrichmentTableFileKeyItemField:
44555+
description: |-
44556+
Specifies the source of the key value used for enrichment table lookups.
44557+
Can be a plain field path string or an object specifying `event`, `vrl`, or `secret`.
44558+
example: log.user.id
44559+
oneOf:
44560+
- $ref: "#/components/schemas/ObservabilityPipelineEnrichmentTableFieldStringPath"
44561+
- $ref: "#/components/schemas/ObservabilityPipelineEnrichmentTableFieldEventLookup"
44562+
- $ref: "#/components/schemas/ObservabilityPipelineEnrichmentTableFieldVrlLookup"
44563+
- $ref: "#/components/schemas/ObservabilityPipelineEnrichmentTableFieldSecretLookup"
4452044564
ObservabilityPipelineEnrichmentTableFileKeyItems:
4452144565
description: Defines how to map log fields to enrichment table columns during lookups.
4452244566
properties:
@@ -44527,9 +44571,7 @@ components:
4452744571
comparison:
4452844572
$ref: "#/components/schemas/ObservabilityPipelineEnrichmentTableFileKeyItemsComparison"
4452944573
field:
44530-
description: The `items` `field`.
44531-
example: log.user.id
44532-
type: string
44574+
$ref: "#/components/schemas/ObservabilityPipelineEnrichmentTableFileKeyItemField"
4453344575
required:
4453444576
- column
4453544577
- comparison
@@ -47550,6 +47592,8 @@ components:
4755047592
description: Name of the environment variable or secret that holds the Splunk HEC token.
4755147593
example: SPLUNK_HEC_TOKEN
4755247594
type: string
47595+
token_strategy:
47596+
$ref: "#/components/schemas/ObservabilityPipelineSplunkHecDestinationTokenStrategy"
4755347597
type:
4755447598
$ref: "#/components/schemas/ObservabilityPipelineSplunkHecDestinationType"
4755547599
required:
@@ -47568,6 +47612,16 @@ components:
4756847612
x-enum-varnames:
4756947613
- JSON
4757047614
- RAW_MESSAGE
47615+
ObservabilityPipelineSplunkHecDestinationTokenStrategy:
47616+
description: Controls how the Splunk HEC token is supplied. Use `custom` to provide a token with `token_key`, or `from_source` to forward the token received from an upstream Splunk HEC source.
47617+
enum:
47618+
- custom
47619+
- from_source
47620+
example: custom
47621+
type: string
47622+
x-enum-varnames:
47623+
- CUSTOM
47624+
- FROM_SOURCE
4757147625
ObservabilityPipelineSplunkHecDestinationType:
4757247626
default: splunk_hec
4757347627
description: The destination type. Always `splunk_hec`.
@@ -47591,6 +47645,12 @@ components:
4759147645
description: The unique identifier for this component. Used in other parts of the pipeline to reference this component (for example, as the `input` to downstream components).
4759247646
example: splunk-hec-source
4759347647
type: string
47648+
store_hec_token:
47649+
description: |-
47650+
When `true`, the Splunk HEC token from the incoming request is stored in the event metadata.
47651+
This allows downstream components to forward the token to other Splunk HEC destinations.
47652+
example: true
47653+
type: boolean
4759447654
tls:
4759547655
$ref: "#/components/schemas/ObservabilityPipelineTls"
4759647656
type:

‎docs/datadog_api_client.v2.model.rst‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19758,6 +19758,27 @@ datadog\_api\_client.v2.model.observability\_pipeline\_elasticsearch\_destinatio
1975819758
:members:
1975919759
:show-inheritance:
1976019760

19761+
datadog\_api\_client.v2.model.observability\_pipeline\_enrichment\_table\_field\_event\_lookup module
19762+
-----------------------------------------------------------------------------------------------------
19763+
19764+
.. automodule:: datadog_api_client.v2.model.observability_pipeline_enrichment_table_field_event_lookup
19765+
:members:
19766+
:show-inheritance:
19767+
19768+
datadog\_api\_client.v2.model.observability\_pipeline\_enrichment\_table\_field\_secret\_lookup module
19769+
------------------------------------------------------------------------------------------------------
19770+
19771+
.. automodule:: datadog_api_client.v2.model.observability_pipeline_enrichment_table_field_secret_lookup
19772+
:members:
19773+
:show-inheritance:
19774+
19775+
datadog\_api\_client.v2.model.observability\_pipeline\_enrichment\_table\_field\_vrl\_lookup module
19776+
---------------------------------------------------------------------------------------------------
19777+
19778+
.. automodule:: datadog_api_client.v2.model.observability_pipeline_enrichment_table_field_vrl_lookup
19779+
:members:
19780+
:show-inheritance:
19781+
1976119782
datadog\_api\_client.v2.model.observability\_pipeline\_enrichment\_table\_file module
1976219783
-------------------------------------------------------------------------------------
1976319784

@@ -19779,6 +19800,13 @@ datadog\_api\_client.v2.model.observability\_pipeline\_enrichment\_table\_file\_
1977919800
:members:
1978019801
:show-inheritance:
1978119802

19803+
datadog\_api\_client.v2.model.observability\_pipeline\_enrichment\_table\_file\_key\_item\_field module
19804+
-------------------------------------------------------------------------------------------------------
19805+
19806+
.. automodule:: datadog_api_client.v2.model.observability_pipeline_enrichment_table_file_key_item_field
19807+
:members:
19808+
:show-inheritance:
19809+
1978219810
datadog\_api\_client.v2.model.observability\_pipeline\_enrichment\_table\_file\_key\_items module
1978319811
-------------------------------------------------------------------------------------------------
1978419812

@@ -21039,6 +21067,13 @@ datadog\_api\_client.v2.model.observability\_pipeline\_splunk\_hec\_destination\
2103921067
:members:
2104021068
:show-inheritance:
2104121069

21070+
datadog\_api\_client.v2.model.observability\_pipeline\_splunk\_hec\_destination\_token\_strategy module
21071+
-------------------------------------------------------------------------------------------------------
21072+
21073+
.. automodule:: datadog_api_client.v2.model.observability_pipeline_splunk_hec_destination_token_strategy
21074+
:members:
21075+
:show-inheritance:
21076+
2104221077
datadog\_api\_client.v2.model.observability\_pipeline\_splunk\_hec\_destination\_type module
2104321078
--------------------------------------------------------------------------------------------
2104421079

‎examples/v2/observability-pipelines/ValidatePipeline_1267410221.py‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
"""
2+
Validate an observability pipeline with Splunk HEC destination token_strategy returns "OK" response
3+
"""
4+
5+
from datadog_api_client import ApiClient, Configuration
6+
from datadog_api_client.v2.api.observability_pipelines_api import ObservabilityPipelinesApi
7+
from datadog_api_client.v2.model.observability_pipeline_config import ObservabilityPipelineConfig
8+
from datadog_api_client.v2.model.observability_pipeline_config_processor_group import (
9+
ObservabilityPipelineConfigProcessorGroup,
10+
)
11+
from datadog_api_client.v2.model.observability_pipeline_data_attributes import ObservabilityPipelineDataAttributes
12+
from datadog_api_client.v2.model.observability_pipeline_datadog_agent_source import (
13+
ObservabilityPipelineDatadogAgentSource,
14+
)
15+
from datadog_api_client.v2.model.observability_pipeline_datadog_agent_source_type import (
16+
ObservabilityPipelineDatadogAgentSourceType,
17+
)
18+
from datadog_api_client.v2.model.observability_pipeline_filter_processor import ObservabilityPipelineFilterProcessor
19+
from datadog_api_client.v2.model.observability_pipeline_filter_processor_type import (
20+
ObservabilityPipelineFilterProcessorType,
21+
)
22+
from datadog_api_client.v2.model.observability_pipeline_spec import ObservabilityPipelineSpec
23+
from datadog_api_client.v2.model.observability_pipeline_spec_data import ObservabilityPipelineSpecData
24+
from datadog_api_client.v2.model.observability_pipeline_splunk_hec_destination import (
25+
ObservabilityPipelineSplunkHecDestination,
26+
)
27+
from datadog_api_client.v2.model.observability_pipeline_splunk_hec_destination_token_strategy import (
28+
ObservabilityPipelineSplunkHecDestinationTokenStrategy,
29+
)
30+
from datadog_api_client.v2.model.observability_pipeline_splunk_hec_destination_type import (
31+
ObservabilityPipelineSplunkHecDestinationType,
32+
)
33+
34+
body = ObservabilityPipelineSpec(
35+
data=ObservabilityPipelineSpecData(
36+
attributes=ObservabilityPipelineDataAttributes(
37+
config=ObservabilityPipelineConfig(
38+
destinations=[
39+
ObservabilityPipelineSplunkHecDestination(
40+
id="splunk-hec-destination",
41+
inputs=[
42+
"my-processor-group",
43+
],
44+
type=ObservabilityPipelineSplunkHecDestinationType.SPLUNK_HEC,
45+
token_key="SPLUNK_HEC_TOKEN",
46+
token_strategy=ObservabilityPipelineSplunkHecDestinationTokenStrategy.CUSTOM,
47+
),
48+
],
49+
processor_groups=[
50+
ObservabilityPipelineConfigProcessorGroup(
51+
enabled=True,
52+
id="my-processor-group",
53+
include="service:my-service",
54+
inputs=[
55+
"datadog-agent-source",
56+
],
57+
processors=[
58+
ObservabilityPipelineFilterProcessor(
59+
enabled=True,
60+
id="filter-processor",
61+
include="status:error",
62+
type=ObservabilityPipelineFilterProcessorType.FILTER,
63+
),
64+
],
65+
),
66+
],
67+
sources=[
68+
ObservabilityPipelineDatadogAgentSource(
69+
id="datadog-agent-source",
70+
type=ObservabilityPipelineDatadogAgentSourceType.DATADOG_AGENT,
71+
),
72+
],
73+
),
74+
name="Pipeline with Splunk HEC token_strategy",
75+
),
76+
type="pipelines",
77+
),
78+
)
79+
80+
configuration = Configuration()
81+
with ApiClient(configuration) as api_client:
82+
api_instance = ObservabilityPipelinesApi(api_client)
83+
response = api_instance.validate_pipeline(body=body)
84+
85+
print(response)

‎examples/v2/observability-pipelines/ValidatePipeline_2960728933.py‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
"""
2+
Validate an observability pipeline with Splunk HEC source store_hec_token returns "OK" response
3+
"""
4+
5+
from datadog_api_client import ApiClient, Configuration
6+
from datadog_api_client.v2.api.observability_pipelines_api import ObservabilityPipelinesApi
7+
from datadog_api_client.v2.model.observability_pipeline_config import ObservabilityPipelineConfig
8+
from datadog_api_client.v2.model.observability_pipeline_config_processor_group import (
9+
ObservabilityPipelineConfigProcessorGroup,
10+
)
11+
from datadog_api_client.v2.model.observability_pipeline_data_attributes import ObservabilityPipelineDataAttributes
12+
from datadog_api_client.v2.model.observability_pipeline_datadog_logs_destination import (
13+
ObservabilityPipelineDatadogLogsDestination,
14+
)
15+
from datadog_api_client.v2.model.observability_pipeline_datadog_logs_destination_type import (
16+
ObservabilityPipelineDatadogLogsDestinationType,
17+
)
18+
from datadog_api_client.v2.model.observability_pipeline_filter_processor import ObservabilityPipelineFilterProcessor
19+
from datadog_api_client.v2.model.observability_pipeline_filter_processor_type import (
20+
ObservabilityPipelineFilterProcessorType,
21+
)
22+
from datadog_api_client.v2.model.observability_pipeline_spec import ObservabilityPipelineSpec
23+
from datadog_api_client.v2.model.observability_pipeline_spec_data import ObservabilityPipelineSpecData
24+
from datadog_api_client.v2.model.observability_pipeline_splunk_hec_source import ObservabilityPipelineSplunkHecSource
25+
from datadog_api_client.v2.model.observability_pipeline_splunk_hec_source_type import (
26+
ObservabilityPipelineSplunkHecSourceType,
27+
)
28+
29+
body = ObservabilityPipelineSpec(
30+
data=ObservabilityPipelineSpecData(
31+
attributes=ObservabilityPipelineDataAttributes(
32+
config=ObservabilityPipelineConfig(
33+
destinations=[
34+
ObservabilityPipelineDatadogLogsDestination(
35+
id="datadog-logs-destination",
36+
inputs=[
37+
"my-processor-group",
38+
],
39+
type=ObservabilityPipelineDatadogLogsDestinationType.DATADOG_LOGS,
40+
),
41+
],
42+
processor_groups=[
43+
ObservabilityPipelineConfigProcessorGroup(
44+
enabled=True,
45+
id="my-processor-group",
46+
include="service:my-service",
47+
inputs=[
48+
"splunk-hec-source",
49+
],
50+
processors=[
51+
ObservabilityPipelineFilterProcessor(
52+
enabled=True,
53+
id="filter-processor",
54+
include="status:error",
55+
type=ObservabilityPipelineFilterProcessorType.FILTER,
56+
),
57+
],
58+
),
59+
],
60+
sources=[
61+
ObservabilityPipelineSplunkHecSource(
62+
id="splunk-hec-source",
63+
type=ObservabilityPipelineSplunkHecSourceType.SPLUNK_HEC,
64+
store_hec_token=True,
65+
),
66+
],
67+
),
68+
name="Pipeline with Splunk HEC store_hec_token",
69+
),
70+
type="pipelines",
71+
),
72+
)
73+
74+
configuration = Configuration()
75+
with ApiClient(configuration) as api_client:
76+
api_instance = ObservabilityPipelinesApi(api_client)
77+
response = api_instance.validate_pipeline(body=body)
78+
79+
print(response)

0 commit comments

Comments
 (0)