Skip to content

observability_pipelines: Add enabled flag to all processors #2827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
api-clients-generation-pipeline wants to merge 1 commit into from
Closed

observability_pipelines: Add enabled flag to all processors #2827

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .generator/schemas/v2/openapi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28208,6 +28208,10 @@ components:
description: The `add_env_vars` processor adds environment variable values to
log events.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: The unique identifier for this component. Used to reference
this processor in the pipeline.
Expand Down Expand Up @@ -28267,6 +28271,10 @@ components:
ObservabilityPipelineAddFieldsProcessor:
description: The `add_fields` processor adds static key-value fields to logs.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
fields:
description: A list of static fields (key-value pairs) that is added to
each log event processed by this component.
Expand Down Expand Up @@ -28768,6 +28776,10 @@ components:
Remap Language (VRL)](https://vector.dev/docs/reference/vrl/) scripts with
advanced filtering capabilities.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: The unique identifier for this processor.
example: remap-vrl-processor
Expand Down Expand Up @@ -28940,6 +28952,10 @@ components:
properties:
action:
$ref: '#/components/schemas/ObservabilityPipelineDatadogTagsProcessorAction'
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: The unique identifier for this component. Used to reference
this component in other parts of the pipeline (for example, as the `input`
Expand Down Expand Up @@ -29025,6 +29041,10 @@ components:
ObservabilityPipelineDedupeProcessor:
description: The `dedupe` processor removes duplicate fields in log events.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
fields:
description: A list of log field paths to check for duplicates.
example:
Expand Down Expand Up @@ -29266,6 +29286,10 @@ components:
description: The `enrichment_table` processor enriches logs using a static CSV
file or GeoIP database.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
file:
$ref: '#/components/schemas/ObservabilityPipelineEnrichmentTableFile'
geoip:
Expand Down Expand Up @@ -29329,6 +29353,10 @@ components:
on a Datadog search query. Logs that match the `include` query are passed
through; others are discarded.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: The unique identifier for this component. Used to reference
this component in other parts of the pipeline (for example, as the `input`
Expand Down Expand Up @@ -29439,6 +29467,10 @@ components:

'
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: The unique identifier for this component. Used to reference
this component in other parts of the pipeline.
Expand Down Expand Up @@ -30015,6 +30047,10 @@ components:
description: The `ocsf_mapper` processor transforms logs into the OCSF schema
using a predefined mapping configuration.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: The unique identifier for this component. Used to reference
this component in other parts of the pipeline.
Expand Down Expand Up @@ -30149,6 +30185,10 @@ components:
by Datadog.
example: true
type: boolean
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: A unique identifier for this processor.
example: parse-grok-processor
Expand Down Expand Up @@ -30273,6 +30313,10 @@ components:
and flattens it into the event. This is useful when logs contain embedded
JSON as a string.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
field:
description: The name of the log field that contains a JSON string.
example: message
Expand Down Expand Up @@ -30336,6 +30380,10 @@ components:
the filter query continue through the pipeline.
example: false
type: boolean
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: The unique identifier for this component. Used to reference
this component in other parts of the pipeline (for example, as the `input`
Expand Down Expand Up @@ -30468,6 +30516,10 @@ components:
description: The `reduce` processor aggregates and merges logs based on matching
keys and merge strategies.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
group_by:
description: A list of fields used to group log events for merging.
example:
Expand Down Expand Up @@ -30564,6 +30616,10 @@ components:
ObservabilityPipelineRemoveFieldsProcessor:
description: The `remove_fields` processor deletes specified fields from logs.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
fields:
description: A list of field names to be removed from each log event.
example:
Expand Down Expand Up @@ -30611,6 +30667,10 @@ components:
ObservabilityPipelineRenameFieldsProcessor:
description: The `rename_fields` processor changes field names.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
fields:
description: A list of rename rules specifying which fields to rename in
the event, what to rename them to, and whether to preserve the original
Expand Down Expand Up @@ -30750,6 +30810,10 @@ components:
description: The `sample` processor allows probabilistic sampling of logs at
a fixed rate.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: The unique identifier for this component. Used to reference
this component in other parts of the pipeline (for example, as the `input`
Expand Down Expand Up @@ -30801,6 +30865,10 @@ components:
description: The `sensitive_data_scanner` processor detects and optionally redacts
sensitive data in log events.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
id:
description: The unique identifier for this component. Used to reference
this component in other parts of the pipeline (e.g., as input to downstream
Expand Down Expand Up @@ -31806,6 +31874,10 @@ components:
description: The `throttle` processor limits the number of events that pass
through over a given time window.
properties:
enabled:
description: The processor passes through all events if it is set to `false`.
Defaults to `true`.
type: boolean
group_by:
description: Optional list of fields used to group events before the threshold
has been reached.
Expand Down
12 changes: 11 additions & 1 deletion src/datadog_api_client/v2/model/observability_pipeline_add_env_vars_processor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,13 @@
# Copyright 2019-Present Datadog, Inc.
from __future__ import annotations

from typing import List, TYPE_CHECKING
from typing import List, Union, TYPE_CHECKING

from datadog_api_client.model_utils import (
ModelNormal,
cached_property,
unset,
UnsetType,
)


Expand All @@ -31,6 +33,7 @@ def openapi_types(_):
)

return {
"enabled": (bool,),
"id": (str,),
"include": (str,),
"inputs": ([str],),
Expand All @@ -39,6 +42,7 @@ def openapi_types(_):
}

attribute_map = {
"enabled": "enabled",
"id": "id",
"include": "include",
"inputs": "inputs",
Expand All @@ -53,11 +57,15 @@ def __init__(
inputs: List[str],
type: ObservabilityPipelineAddEnvVarsProcessorType,
variables: List[ObservabilityPipelineAddEnvVarsProcessorVariable],
enabled: Union[bool, UnsetType] = unset,
**kwargs,
):
"""
The ``add_env_vars`` processor adds environment variable values to log events.

:param enabled: The processor passes through all events if it is set to ``false``. Defaults to ``true``.
:type enabled: bool, optional

:param id: The unique identifier for this component. Used to reference this processor in the pipeline.
:type id: str

Expand All @@ -73,6 +81,8 @@ def __init__(
:param variables: A list of environment variable mappings to apply to log fields.
:type variables: [ObservabilityPipelineAddEnvVarsProcessorVariable]
"""
if enabled is not unset:
kwargs["enabled"] = enabled
super().__init__(kwargs)

self_.id = id
Expand Down
12 changes: 11 additions & 1 deletion src/datadog_api_client/v2/model/observability_pipeline_add_fields_processor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,13 @@
# Copyright 2019-Present Datadog, Inc.
from __future__ import annotations

from typing import List, TYPE_CHECKING
from typing import List, Union, TYPE_CHECKING

from datadog_api_client.model_utils import (
ModelNormal,
cached_property,
unset,
UnsetType,
)


Expand All @@ -27,6 +29,7 @@ def openapi_types(_):
)

return {
"enabled": (bool,),
"fields": ([ObservabilityPipelineFieldValue],),
"id": (str,),
"include": (str,),
Expand All @@ -35,6 +38,7 @@ def openapi_types(_):
}

attribute_map = {
"enabled": "enabled",
"fields": "fields",
"id": "id",
"include": "include",
Expand All @@ -49,11 +53,15 @@ def __init__(
include: str,
inputs: List[str],
type: ObservabilityPipelineAddFieldsProcessorType,
enabled: Union[bool, UnsetType] = unset,
**kwargs,
):
"""
The ``add_fields`` processor adds static key-value fields to logs.

:param enabled: The processor passes through all events if it is set to ``false``. Defaults to ``true``.
:type enabled: bool, optional

:param fields: A list of static fields (key-value pairs) that is added to each log event processed by this component.
:type fields: [ObservabilityPipelineFieldValue]

Expand All @@ -69,6 +77,8 @@ def __init__(
:param type: The processor type. The value should always be ``add_fields``.
:type type: ObservabilityPipelineAddFieldsProcessorType
"""
if enabled is not unset:
kwargs["enabled"] = enabled
super().__init__(kwargs)

self_.fields = fields
Expand Down
3 changes: 3 additions & 0 deletions src/datadog_api_client/v2/model/observability_pipeline_config_processor_item.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ def __init__(self, **kwargs):
"""
A processor for the pipeline.

:param enabled: The processor passes through all events if it is set to `false`. Defaults to `true`.
:type enabled: bool, optional

:param id: The unique identifier for this component. Used to reference this component in other parts of the pipeline (for example, as the `input` to downstream components).
:type id: str

Expand Down
12 changes: 11 additions & 1 deletion src/datadog_api_client/v2/model/observability_pipeline_custom_processor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,13 @@
# Copyright 2019-Present Datadog, Inc.
from __future__ import annotations

from typing import List, TYPE_CHECKING
from typing import List, Union, TYPE_CHECKING

from datadog_api_client.model_utils import (
ModelNormal,
cached_property,
unset,
UnsetType,
)


Expand Down Expand Up @@ -37,6 +39,7 @@ def openapi_types(_):
)

return {
"enabled": (bool,),
"id": (str,),
"include": (str,),
"inputs": ([str],),
Expand All @@ -45,6 +48,7 @@ def openapi_types(_):
}

attribute_map = {
"enabled": "enabled",
"id": "id",
"include": "include",
"inputs": "inputs",
Expand All @@ -58,11 +62,15 @@ def __init__(
inputs: List[str],
remaps: List[ObservabilityPipelineCustomProcessorRemap],
type: ObservabilityPipelineCustomProcessorType,
enabled: Union[bool, UnsetType] = unset,
**kwargs,
):
"""
The ``custom_processor`` processor transforms events using `Vector Remap Language (VRL) <https://vector.dev/docs/reference/vrl/>`_ scripts with advanced filtering capabilities.

:param enabled: The processor passes through all events if it is set to ``false``. Defaults to ``true``.
:type enabled: bool, optional

:param id: The unique identifier for this processor.
:type id: str

Expand All @@ -78,6 +86,8 @@ def __init__(
:param type: The processor type. The value should always be ``custom_processor``.
:type type: ObservabilityPipelineCustomProcessorType
"""
if enabled is not unset:
kwargs["enabled"] = enabled
super().__init__(kwargs)
include = kwargs.get("include", "*")

Expand Down
Loading
Loading