feat(config): Add setting to skip processors before aggregators

[calvinmachado20]

Summary

Addresses open issue: 17006

#14882 has introced the skip_processors_after_aggregators config option. Are there any plans to add a similar skip_processors_before_aggregators config option?

Expected behavior

Processors only run after the aggregators and are skipped before the aggregators.

Actual behavior

Manual workaround with adding, filtering and removing tags is required to achieve the above.

Checklist

• No AI generated code was used in this PR
• AI generated code used in this PR follows the InfluxData Policy on AI-Generated Code Contributions

Related issues

resolves #17006

[R290]

I'd be happy to test this, but the Telegraf bot artifacts are missing due to a flaky test?

[calvinmachado20]

Checking this

[calvinmachado20]

@srebhan the failures appear to be flaky, how do you want to proceed here?

[calvinmachado20]

@R290 the artifacts are generated, feel free to give this feature a run

[R290]

Behavior is expected. Setting both skip_processors_after_aggregators and skip_processors_before_aggregators to true, gives an error.

My test case consists of reading 2 metrics from file, one with field x and the other with field y. They are added together into field z with a Starlark processor. This only works if the metrics are merged beforehand with the merge aggregator. Setting skip_processors_before_aggregators to true results in the intended behavior.

[calvinmachado20]

Thanks for testing this out

[srebhan]

Thanks for your contribution @calvinmachado20! Just one comment/question...

[srebhan]

Thanks @calvinmachado20!

One more thing: I think it's not good to mix config-related things into the agent code. So can we please check for the valid setup (we are not skipping before and after) as well as the conditionals to the config code just like SkipProcessorsAfterAggregators does it!?! I.e. we simply don't add processors to a.Config.Processors if SkipProcessorsBeforeAggregators is true...

This ensures that config related stuff is kept in config-land and does not sneak into the agent code.

[calvinmachado20]

@srebhan thanks for the suggestion, completely agree with you, implemented the same

[calvinmachado20]

@srebhan does this PR look good to go?

[srebhan]

Thanks @calvinmachado20! I think you need to restore the warning!

[srebhan]

Thanks for the update @calvinmachado20! There are three more documentation placement comments from my side...

[calvinmachado20]

@srebhan updated the docs as suggested

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

. DEB	. RPM	. TAR . GZ	. ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

[srebhan]

Nice! Thanks a lot @calvinmachado20!