optimize debezium#960
Merged
Merged
Conversation
Restrict captured columns to those required by consumers and disable schema inclusion in messages to reduce payload size. Added payload hoisting and updated the topic cleanup policy to 'delete'. Additionally, added tests to verify the new configuration settings.
Enable snapshot isolation on the NBS_ODSE database to support Debezium snapshot seeding and set the connector snapshot isolation mode to 'snapshot'. Additionally, cleanup and standardize SQL script casing for better readability and maintainability.
hawley-skylight
approved these changes
Jul 17, 2026
ericnagel
approved these changes
Jul 17, 2026
ericbuckley
marked this pull request as ready for review
July 17, 2026 16:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This pull request optimizes the Debezium configuration for CDC (Change Data Capture) on ODSE/SRTE databases by refining the column inclusion lists and connector settings. These changes reduce the volume of data captured, improving performance and reducing the load on both the source database and downstream consumers. Furthermore, new unit tests have been implemented to ensure that the connector configurations remain compliant with these optimized standards.
Additional Notes
These changes ensure that the CDC pipeline only captures essential data, improving efficiency and reliability.
column.include.listin connector configurations to restrict data capture to required fields.value.converter.schemas.enableand enforce correct payload envelope handling.DebeziumConfigTestto validate connector configurations against defined requirements.Checklist