Skip to content

Fix shutdown and failover handling in RDS source plugin#5624

Merged
oeyh merged 2 commits into
opensearch-project:mainfrom
oeyh:rds/handle-shutdown-and-failover
Apr 18, 2025
Merged

Fix shutdown and failover handling in RDS source plugin#5624
oeyh merged 2 commits into
opensearch-project:mainfrom
oeyh:rds/handle-shutdown-and-failover

Conversation

@oeyh
Copy link
Copy Markdown
Collaborator

@oeyh oeyh commented Apr 18, 2025 •
edited
Loading

Description

  • Fixes source plugin shutdown by adding missed resource cleanup code in a few places
  • Fixes Aurora failover handing by not shutting down stream worker on disconnect. Rather, let the client to restore connection by itself (in MySQL case) or let it throw exception and cause stream partition to be re-acquired (in Postgres case)
  • Adds more logging for information and for troubleshooting

Issues Resolved

N/A

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

oeyh added 2 commits April 16, 2025 15:10
@oeyh
Fix MySQL shutdown sequence
c4e9682 
Signed-off-by: Hai Yan <oeyh@amazon.com>
@oeyh
Fix failover handling
1a65b91 
Signed-off-by: Hai Yan <oeyh@amazon.com>
@oeyh oeyh merged commit 6c70559 into opensearch-project:main Apr 18, 2025
44 of 47 checks passed
Davidding4718 pushed a commit to Davidding4718/data-prepper that referenced this pull request Apr 25, 2025
@oeyh @Galactus22625
Fix shutdown and failover handling in RDS source plugin (opensearch-p…
ed7be7c 
…roject#5624)

* Fix MySQL shutdown sequence

Signed-off-by: Hai Yan <oeyh@amazon.com>

* Fix failover handling

Signed-off-by: Hai Yan <oeyh@amazon.com>

---------

Signed-off-by: Hai Yan <oeyh@amazon.com>
Davidding4718 pushed a commit to Davidding4718/data-prepper that referenced this pull request Apr 25, 2025
@oeyh @Davidding4718
Fix shutdown and failover handling in RDS source plugin (opensearch-p…
96fd03a 
…roject#5624)

* Fix MySQL shutdown sequence

Signed-off-by: Hai Yan <oeyh@amazon.com>

* Fix failover handling

Signed-off-by: Hai Yan <oeyh@amazon.com>

---------

Signed-off-by: Hai Yan <oeyh@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@divbok divbok divbok approved these changes

@chenqi0805 chenqi0805 chenqi0805 approved these changes

@sb2k16 sb2k16 Awaiting requested review from sb2k16 sb2k16 is a code owner

@engechas engechas Awaiting requested review from engechas

@san81 san81 Awaiting requested review from san81 san81 is a code owner

@srikanthjg srikanthjg Awaiting requested review from srikanthjg srikanthjg is a code owner

@graytaylor0 graytaylor0 Awaiting requested review from graytaylor0 graytaylor0 is a code owner

@dinujoh dinujoh Awaiting requested review from dinujoh dinujoh is a code owner

@kkondaka kkondaka Awaiting requested review from kkondaka kkondaka is a code owner

@KarstenSchnitter KarstenSchnitter Awaiting requested review from KarstenSchnitter KarstenSchnitter is a code owner

@dlvenable dlvenable Awaiting requested review from dlvenable dlvenable is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@oeyh @divbok @chenqi0805