[backend] fix(scenario): deletion take too long (#5208)#5932
Open
gabriel-peze wants to merge 3 commits into
Open
[backend] fix(scenario): deletion take too long (#5208)#5932gabriel-peze wants to merge 3 commits into
gabriel-peze wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aims to reduce scenario (and related entity) deletion time by batching analytics-engine (Elasticsearch/OpenSearch) deletes and by pre-clearing inject dependency rows to avoid slow/erroneous cascade behavior during scenario removal.
Changes:
- Batch
DATA_DELETEindex events per-transaction and flush once after commit (instead of one ES call per deleted entity). - Add a native bulk delete to clear
injects_dependenciesrows for a scenario prior to deleting the scenario. - Add unit tests validating the new EngineListener batching behavior.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| openaev-model/src/main/java/io/openaev/engine/EngineListener.java | Batch delete IDs during a transaction and flush once on afterCommit. |
| openaev-model/src/main/java/io/openaev/database/repository/InjectDependenciesRepository.java | Add native bulk deletion of inject dependency rows by scenario id (tenant-scoped). |
| openaev-api/src/main/java/io/openaev/service/scenario/ScenarioService.java | Call dependency cleanup before deleting a scenario (and wrap delete in a transaction). |
| openaev-api/src/main/java/io/openaev/rest/inject/service/InjectService.java | Add service method delegating to the new bulk dependency delete repository method. |
| openaev-api/src/test/java/io/openaev/engine/EngineListenerTest.java | New unit tests for batching/commit/rollback behavior. |
Comment on lines
+329
to
+332
| @org.springframework.transaction.annotation.Transactional(rollbackFor = Exception.class) | ||
| public void clearInjectDependenciesByScenarioId(String scenarioId) { | ||
| injectDependenciesRepository.deleteAllByScenarioId(scenarioId); | ||
| } |
Comment on lines
501
to
505
| """ | ||
| // For each EsBase attribute of each document | ||
| for (String key : ctx._source.keySet().toArray()) { | ||
| // If it's a "base_XXX_side" (means String id or List of ids), we delete only in the "base_XXX_side" of the EsBase the object id deleted before with the deleteByQuery | ||
| // If it's a "base_XXX_side" (means String id or List of ids), remove all deleted ids from this field. | ||
| if(key.startsWith("base_") && key.endsWith("_side") && ctx._source[key] != null) { |
Comment on lines
436
to
440
| """ | ||
| // For each EsBase attribute of each document | ||
| for (String key : ctx._source.keySet().toArray()) { | ||
| // If it's a "base_XXX_side" (means String id or List of ids), we delete only in the "base_XXX_side" of the EsBase the object id deleted before with the deleteByQuery | ||
| // If it's a "base_XXX_side" (means String id or List of ids), remove all deleted ids from this field. | ||
| if(key.startsWith("base_") && key.endsWith("_side") && ctx._source[key] != null) { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Testing Instructions
Related issues