[Alerting V2][Serverless & 9.5][M2] Fix forced-recovery step in tutorial#7375
Open
nastasha-solomon wants to merge 2 commits into
Open
[Alerting V2][Serverless & 9.5][M2] Fix forced-recovery step in tutorial#7375nastasha-solomon wants to merge 2 commits into
nastasha-solomon wants to merge 2 commits into
Conversation
Contributor
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
Contributor
🔍 Preview links for changed docs |
Contributor
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
In a Slack convo with @jasonrhodes, I learned that deleting the degraded documents entirely (
_delete_by_query) doesn't reliably move the episode toinactive. With no documents left forcheckout, the ES|QL query returns zero rows, which the system treats as "no data" rather than a confirmed non-breaching evaluation (which is a distinct state from genuine recovery). This meant the tutorial's forced-recovery step didn't actually produce the recovery behavior it was meant to demonstrate.To fix this, I replaced the
_delete_by_querycall with an_update_by_querycall that rewrites the degraded documents'latency_msvalues to a healthy level (300ms). This keeps a row forcheckoutin the query results, just under the threshold, so the rule evaluates a genuine recovery condition.I also made some minor improvements to the tutorial overall:
NOTE: kibana#277561 prevents the alert episode from switching to
inactiveat the end of the tutorial. That PR was recently merged, so the final lifecycle change should happen as the tutorial describes once the fix is available in serverless.Preview
Create a rule and observe the alert lifecycle
Generative AI disclosure