[FLINK-36143][ES6][ES7] Intro retry-on-conflict param to resolve Sink ES occurred "version conflict"#109
[FLINK-36143][ES6][ES7] Intro retry-on-conflict param to resolve Sink ES occurred "version conflict"#109leosanqing wants to merge 2 commits into
Conversation
… ES occurred "version conflict"
|
Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html) |
|
@schulzp @alpinegizmo @aljoscha @rmetzger Has any body to review my code? thk ❤️❤️ |
schulzp
left a comment
There was a problem hiding this comment.
Thank you for your patch! I am not an official maintainer. Your changes look reasonable, however, they lack test coverage. One other thing I noticed: The newly introduced constant RETRY_ON_CONFLICT_NUM does not follow the naming convention of other config options in ElasticsearchConnectorOptions: Most of them appear to end on [...]_OPTION but do not have the type-indicating suffix. Renaming it to sink.retries-on-conflict could eliminate the confusion wether this is a boolean flag or a number.
… ES occurred "version conflict"
|
Thanks for your advices,and I fixed this. Has anybody can review my code?
|
|
@rmetzger hi, thx for your triggering. I see there has some error when run IT case. And there also has same error in before PR . Is there some problems in project? The leatest merged pr doesn't have this step, latest un-merged PRs both have this error. |
When I used ES connector Write update datas, it occurred "version conflict", like this " org.elasticsearch.index.engine.VersionConflictEngineException: [project][1140860]: version conflict, current version [5] is different than the one provided [4]
"
When we set the conf failure-handler = ignore, It lead to lost data, and if set the conf failure-handler = fail, job will restore. this cost is too heavy. maybe second try it will success.
so I intro this conf param, this will make job failed cost smaller.