OBSDA-1383: Make Splunk output sourcetype configurable in CLF #3251
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -1262,6 +1262,7 @@ type SplunkAuthentication struct { | |||||||||||||||||
|
|
||||||||||||||||||
| // Splunk Deliver log data to Splunk’s HTTP Event Collector | ||||||||||||||||||
| // Provides optional extra properties for `type: splunk_hec` ('splunk_hec_logs' after Vector 0.23 | ||||||||||||||||||
| // +kubebuilder:validation:XValidation:rule="!has(self.sourceType) || has(self.payloadKey)",message="sourceType can only be set when payloadKey is defined" | ||||||||||||||||||
| type Splunk struct { | ||||||||||||||||||
| // Authentication sets credentials for authenticating the requests. | ||||||||||||||||||
| // | ||||||||||||||||||
|
|
@@ -1314,7 +1315,7 @@ type Splunk struct { | |||||||||||||||||
| // Source identifies the origin of a log event. | ||||||||||||||||||
| // The Source can be a combination of static and dynamic values consisting of field paths followed by `||` followed by another field path or a static value. | ||||||||||||||||||
| // A dynamic value is encased in single curly brackets `{}` and MUST end with a static fallback value separated with `||`. | ||||||||||||||||||
| // Static values can only contain alphanumeric characters along with dashes, underscores, dots, colons and forward slashes. | ||||||||||||||||||
| // If not specified will be detected according to .log_source and .log_type value. | ||||||||||||||||||
| // Details see in: docs/features/logforwarding/outputs/splunk-forwarding.adoc | ||||||||||||||||||
| // | ||||||||||||||||||
|
|
@@ -1327,10 +1328,39 @@ type Splunk struct { | |||||||||||||||||
| // 3. foo.{.bar.baz||.qux.quux.corge||.grault||"nil"}-waldo.fred{.plugh||"none"} | ||||||||||||||||||
| // | ||||||||||||||||||
| // +kubebuilder:validation:Optional | ||||||||||||||||||
| // +kubebuilder:validation:Pattern:=`^(([a-zA-Z0-9-_.:\/])*(\{(\.[a-zA-Z0-9_]+|\."[^"]+")+((\|\|)(\.[a-zA-Z0-9_]+|\.?"[^"]+")+)*\|\|"[^"]*"\})*)*$` | ||||||||||||||||||
| // +operator-sdk:csv:customresourcedefinitions:type=spec,displayName="Source",xDescriptors={"urn:alm:descriptor:com.tectonic.ui:text"} | ||||||||||||||||||
| Source string `json:"source,omitempty"` | ||||||||||||||||||
|
|
||||||||||||||||||
| // SourceType can be used to specify a pretrained or custom source type in Splunk, but can only be set when PayloadKey is defined. | ||||||||||||||||||
| // | ||||||||||||||||||
| // WARNING: The administrator is responsible for configuring the pipeline so the source type matches the log entry. The collector makes no effort or validation to ensure they match. | ||||||||||||||||||
| // | ||||||||||||||||||
| // If SourceType is not specified, the source type used is `_json`. If using PayloadKey without SourceType, the source type used will be either `_json` or `generic_single_line`, depending on the structure of the final event payload. | ||||||||||||||||||
| // Details in: docs/features/logforwarding/outputs/splunk-forwarding.adoc | ||||||||||||||||||
| // | ||||||||||||||||||
| // The SourceType can be a combination of static and dynamic values consisting of field paths followed by `||` followed by another field path or a static value. | ||||||||||||||||||
| // A dynamic value is encased in single curly brackets `{}` and MUST end with a static fallback value separated with `||`. | ||||||||||||||||||
| // | ||||||||||||||||||
| // Static values can only contain alphanumeric characters along with dashes, underscores, dots, colons and forward slashes. | ||||||||||||||||||
| // | ||||||||||||||||||
| // Examples: | ||||||||||||||||||
| // | ||||||||||||||||||
| // 1. {.kubernetes.labels."splunk/sourcetype"||"generic_single_line"} | ||||||||||||||||||
| // | ||||||||||||||||||
| // 2. log4j | ||||||||||||||||||
| // | ||||||||||||||||||
| // 3. foo-{.bar||"none"} | ||||||||||||||||||
| // | ||||||||||||||||||
| // 4. {.foo||.bar||"missing"} | ||||||||||||||||||
| // | ||||||||||||||||||
| // 5. foo.{.bar.baz||.qux.quux.corge||.grault||"nil"}-waldo.fred{.plugh||"none"} | ||||||||||||||||||
| // | ||||||||||||||||||
| // +kubebuilder:validation:Optional | ||||||||||||||||||
| // +kubebuilder:validation:Pattern:=`^(([a-zA-Z0-9-_.:\/])*(\{(\.[a-zA-Z0-9_]+|\."[^"]+")+((\|\|)(\.[a-zA-Z0-9_]+|\.?"[^"]+")+)*\|\|"[^"]*"\})*)*$` | ||||||||||||||||||
| // +operator-sdk:csv:customresourcedefinitions:type=spec,displayName="SourceType",xDescriptors={"urn:alm:descriptor:com.tectonic.ui:text"} | ||||||||||||||||||
| SourceType string `json:"sourceType,omitempty"` | ||||||||||||||||||
|
Comment on lines
+1359
to
+1362
There was a problem hiding this comment.
The pattern's static character class A user wanting a static colon-separated sourcetype has no clean workaround: 🐛 Proposed fix — add
|
||||||||||||||||||
| // +kubebuilder:validation:Optional | |
| // +kubebuilder:validation:Pattern:=`^(([a-zA-Z0-9-_.\/])*(\{(\.[a-zA-Z0-9_]+|\."[^"]+")+((\|\|)(\.[a-zA-Z0-9_]+|\.?"[^"]+")+)*\|\|"[^"]*"\})*)*$` | |
| // +operator-sdk:csv:customresourcedefinitions:type=spec,displayName="SourceType",xDescriptors={"urn:alm:descriptor:com.tectonic.ui:text"} | |
| SourceType string `json:"sourceType,omitempty"` | |
| // +kubebuilder:validation:Optional | |
| // +kubebuilder:validation:Pattern:=`^(([a-zA-Z0-9-_.:\/])*(\{(\.[a-zA-Z0-9_]+|\."[^"]+")+((\|\|)(\.[a-zA-Z0-9_]+|\.?"[^"]+")+)*\|\|"[^"]*"\})*)*$` | |
| // +operator-sdk:csv:customresourcedefinitions:type=spec,displayName="SourceType",xDescriptors={"urn:alm:descriptor:com.tectonic.ui:text"} | |
| SourceType string `json:"sourceType,omitempty"` |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@api/observability/v1/output_types.go` around lines 1359 - 1362, The
kubebuilder validation pattern on the SourceType field (the struct tag on
SourceType string `json:"sourceType,omitempty"`) omits ':' in the static
character class causing valid colon-separated sourcetypes to be rejected; update
the +kubebuilder:validation:Pattern annotation to include ':' inside the static
class (e.g. add : to `[a-zA-Z0-9-_.\/]`) so values like "my:custom:sourcetype"
validate, and also correct the doc comment/example that shows `"log4j"` (remove
the stray quotes) so examples match the intended valid values.
✅ Addressed in commit 17bcba0
There was a problem hiding this comment.
@vparfonov @Clee2691 This pattern matcher is restricted by VRL. Do we have any idea if VRL would allow us to add a colon for this type? Given our template utilities it may complicate the other fields and we may need to think about how we would accept this suggestion?
There was a problem hiding this comment.
Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!
Uh oh!
There was an error while loading. Please reload this page.