ref(seer): Remove should_run_automation from get_issue_summary by trevor-e · Pull Request #112219 · getsentry/sentry

trevor-e · 2026-04-03T19:37:30Z

get_issue_summary was doing too much — generating the summary, caching it,
and optionally triggering automation via a should_run_automation boolean flag.
This made it misleading as a "get" function and hid a significant side effect
behind a default parameter.

This PR moves the run_automation call out of get_issue_summary and into
generate_summary_and_run_automation, the task that actually wants automation.
Callers now explicitly decide whether to run automation after getting a summary.

get_issue_summary now returns a 3-tuple (summary_data, status_code, event)
so callers that need the event for automation have access to it.

Also renames generate_issue_summary_only → generate_summary_and_fixability_score
to better describe what the task does (the Celery task name string is unchanged).

Note: the GroupAiSummaryEndpoint previously passed should_run_automation=True
(the default) but used source=SeerAutomationSource.ISSUE_DETAILS. run_automation
has an early return for that source (if source == ISSUE_DETAILS: return), so
automation was never actually triggered from that endpoint — the flag was a no-op
there. This refactor doesn't change that behavior.

Move the run_automation call out of get_issue_summary and into the generate_summary_and_run_automation task, where the intent is explicit. This removes a hidden side effect from what should be a summary-only function and lets callers decide whether to trigger automation. Also renames generate_issue_summary_only to generate_summary_and_fixability_score to better reflect what it does. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

src/sentry/tasks/seer/autofix.py

trevor-e · 2026-04-03T19:40:08Z

src/sentry/seer/autofix/issue_summary.py

-            )
-
-    return summary_dict, 200
+    return summary_dict, 200, event


Moved the automation responsibility to the calling spot, it was getting too complicated tracing all the various paths and implicit behavior.

We can't remove this here. The old Seer SKU (not per seat orgs) still use this flow.
It is the "old Seer" in this diagram - https://miro.com/app/board/uXjVJqn1-fQ=/?focusWidget=3458764648325594380

I think that should still be preserved.

def kick_off_seer_automation(job: PostProcessJob) -> None: ... if not is_seer_seat_based_tier_enabled(group.organization): generate_summary_and_run_automation.delay(group.id, trigger_path="old_seer_automation") def generate_summary_and_run_automation(group_id: int, **kwargs) -> None: if status_code == 200 and event is not None: run_automation(group, AnonymousUser(), event, SeerAutomationSource.POST_PROCESS)

The generate_summary_and_run_automation() func now explicitly calls run_automation() instead of relying on get_issue_summary() to call it.

src/sentry/seer/autofix/issue_summary.py

tests/sentry/seer/autofix/test_issue_summary.py

trevor-e · 2026-04-03T19:43:29Z

src/sentry/tasks/seer/autofix.py

    retry=Retry(times=3, delay=3, on=(Exception,)),
 )
-def generate_issue_summary_only(group_id: int) -> None:
+def generate_summary_and_fixability_score(group_id: int) -> None:


Can revert the naming if you find it confusing to not match the task name, but the _only suffix was really confusing to read when it clearly does not only generate the summary lol.

This error handler was never triggered in production. Removing it lets the task retry mechanism handle failures instead of silently swallowing them. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

src/sentry/tasks/seer/autofix.py

Two early returns in get_issue_summary (feature flag disabled, AI features hidden) were still returning 2-tuples after the signature changed to 3-tuples. Also fixes two test mocks for _generate_summary with the same issue. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…loss On retry, the summary is cached so get_issue_summary returns event=None, causing the automation guard to skip entirely. Catching the exception prevents a wasted retry and logs the failure instead of silently dropping automation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

src/sentry/seer/autofix/issue_summary.py

tests/sentry/seer/autofix/test_issue_summary.py

get_issue_summary no longer calls run_automation, so the _trigger_autofix_task and get_autofix_state patches and assertions were stale. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Missed caller of get_issue_summary that was still unpacking 2 values instead of 3. Would have caused ValueError at runtime for Slack alert notifications. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

github-actions · 2026-04-03T20:24:26Z

Backend Test Failures

Failures on 88054dd in this run:

tests/sentry/integrations/utils/test_issue_summary_for_alerts.py::FetchIssueSummaryTest::test_fetch_issue_summary_with_hide_ai_features_disabled

— log

[gw0] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
tests/sentry/integrations/utils/test_issue_summary_for_alerts.py:74: in test_fetch_issue_summary_with_hide_ai_features_disabled
    assert result == mock_summary
E   AssertionError: assert None == {'headline': 'Test AI Summary', 'possibleCause': 'Test cause', 'whatsWrong': 'Something went wrong'}

tests/sentry/seer/endpoints/test_group_ai_summary.py::GroupAiSummaryEndpointTest::test_endpoint_with_error_response — log

[gw0] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
tests/sentry/seer/endpoints/test_group_ai_summary.py:61: in test_endpoint_with_error_response
    assert response.status_code == 400
E   assert 500 == 400
E    +  where 500 = <Response status_code=500, "application/json">.status_code

tests/sentry/integrations/slack/test_message_builder.py::BuildGroupAttachmentTest::test_compact_alerts_with_ai_summary — log

[gw1] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
tests/sentry/integrations/slack/test_message_builder.py:1205: in test_compact_alerts_with_ai_summary
    assert found_initial_guess, "Initial Guess context block not found"
E   AssertionError: Initial Guess context block not found
E   assert False

tests/sentry/integrations/slack/test_message_builder.py::BuildGroupAttachmentTest::test_build_group_block_with_ai_summary

— log

[gw1] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
tests/sentry/integrations/slack/test_message_builder.py:957: in test_build_group_block_with_ai_summary
    assert "Identity not found" in blocks["blocks"][0]["text"]["text"]
E   AssertionError: assert 'Identity not found' in ':red_circle: <http://testserver/organizations/baz/issues/84/?referrer=slack|*IntegrationError*>'

tests/sentry/integrations/slack/test_message_builder.py::BuildGroupAttachmentTest::test_build_group_block_with_ai_summary_text_truncation

— log

[gw0] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
tests/sentry/integrations/slack/test_message_builder.py:1044: in test_build_group_block_with_ai_summary_text_truncation
    assert "First line of text..." in title_text
E   AssertionError: assert 'First line of text...' in ':red_circle: <http://testserver/organizations/baz/issues/72/?referrer=slack|*IntegrationError*>'

Several test files were still mocking get_issue_summary with 2-tuple return values instead of 3-tuples: the endpoint error test, Slack message builder tests, and issue_summary_for_alerts tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

JoshFerge · 2026-04-03T20:46:35Z

src/sentry/tasks/seer/autofix.py

+    from django.contrib.auth.models import AnonymousUser
+
+    from sentry.seer.autofix.issue_summary import get_issue_summary, run_automation


do these imports need to be inlined or can they go to top of file?

post_process.py is very picky about imports which is why this is all inlined.

src/sentry/integrations/utils/issue_summary_for_alerts.py

Mihir-Mavalankar

kick_off_seer_automation still needs to support Seer customers on the old plan. I think these changes break that

Mihir-Mavalankar · 2026-04-03T21:12:26Z

src/sentry/seer/autofix/issue_summary.py

-            )
-
-    return summary_dict, 200
+    return summary_dict, 200, event


We can't remove this here. The old Seer SKU (not per seat orgs) still use this flow.
It is the "old Seer" in this diagram - https://miro.com/app/board/uXjVJqn1-fQ=/?focusWidget=3458764648325594380

Mihir-Mavalankar · 2026-04-03T21:14:11Z

src/sentry/tasks/seer/autofix.py

+        try:
+            run_automation(group, AnonymousUser(), event, SeerAutomationSource.POST_PROCESS)
+        except Exception:
+            logger.exception("seer.automation.run_automation_failed", extra={"group_id": group.id})


I think this issue warden caught is correct.

From what I can tell this issue is present in the old code as well. get_issue_summary() checks:

# 1. Check cache first if cached_summary := cache.get(cache_key): return convert_dict_key_case(cached_summary, snake_to_camel_case), 200

which early returns before the automation gets triggered. We can fix if you think this is a bug but trying to keep this PR having no behavior changes.

Although to Warden's point maybe this should be named generate_summary_and_maybe_run_automation()

Mihir-Mavalankar · 2026-04-03T21:16:27Z

Also maybe this PR is worth breaking down into 2 things:

Renames and small non logic changes
Any potential changes to logic of autofix automations.

src/sentry/seer/autofix/issue_summary.py

source was only used to pass to run_automation which was removed from this function. get_issue_summary still accepts source for billing logging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

src/sentry/tasks/seer/autofix.py

src/sentry/integrations/utils/issue_summary_for_alerts.py

…e_summary The refactor to return event from get_issue_summary removed the run_automation call that was previously embedded in _generate_summary. This restores the ALERT automation path by calling run_automation explicitly after a successful summary generation. Co-Authored-By: Claude Opus 4.6 <noreply@example.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 901ed71. Configure here.}

cursor · 2026-04-08T15:23:45Z

src/sentry/integrations/utils/issue_summary_for_alerts.py

+                            logger.exception(
+                                "seer.automation.run_automation_failed",
+                                extra={"group_id": group.id},
+                            )


Alert path runs automation without timeout protection

Medium Severity

The run_automation call, which makes HTTP requests to the Seer API, now runs synchronously on the main thread after the timed issue summary generation. This work is no longer covered by the timeout, allowing slow Seer responses to block the alert notification thread.

^{Reviewed by Cursor Bugbot for commit 901ed71. Configure here.}

github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Apr 3, 2026

vercel bot deployed to Preview April 3, 2026 19:39 View deployment

sentry-warden bot reviewed Apr 3, 2026

View reviewed changes

src/sentry/tasks/seer/autofix.py Outdated Show resolved Hide resolved

trevor-e commented Apr 3, 2026

View reviewed changes

cursor bot reviewed Apr 3, 2026

View reviewed changes

src/sentry/seer/autofix/issue_summary.py Show resolved Hide resolved

tests/sentry/seer/autofix/test_issue_summary.py Show resolved Hide resolved

trevor-e commented Apr 3, 2026

View reviewed changes

ref(seer): Remove dead try/catch around run_automation

aefbfde

This error handler was never triggered in production. Removing it lets the task retry mechanism handle failures instead of silently swallowing them. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vercel bot deployed to Preview April 3, 2026 19:47 View deployment

cursor bot reviewed Apr 3, 2026

View reviewed changes

src/sentry/tasks/seer/autofix.py Outdated Show resolved Hide resolved

trevor-e and others added 2 commits April 3, 2026 15:49

vercel bot deployed to Preview April 3, 2026 19:52 View deployment

vercel bot deployed to Preview April 3, 2026 19:55 View deployment

cursor bot reviewed Apr 3, 2026

View reviewed changes

src/sentry/seer/autofix/issue_summary.py Show resolved Hide resolved

unused

b2998a8

vercel bot deployed to Preview April 3, 2026 19:59 View deployment

cursor bot reviewed Apr 3, 2026

View reviewed changes

tests/sentry/seer/autofix/test_issue_summary.py Show resolved Hide resolved

fix(seer): Remove stale automation assertion from web vitals test

f708a56

get_issue_summary no longer calls run_automation, so the _trigger_autofix_task and get_autofix_state patches and assertions were stale. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vercel bot deployed to Preview April 3, 2026 20:07 View deployment

fix(seer): Fix tuple unpacking in fetch_issue_summary for alerts

a913659

Missed caller of get_issue_summary that was still unpacking 2 values instead of 3. Would have caused ValueError at runtime for Slack alert notifications. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vercel bot deployed to Preview April 3, 2026 20:16 View deployment

vercel bot deployed to Preview April 3, 2026 20:33 View deployment

trevor-e marked this pull request as ready for review April 3, 2026 20:45

trevor-e requested review from a team as code owners April 3, 2026 20:45

JoshFerge reviewed Apr 3, 2026

View reviewed changes

src/sentry/integrations/utils/issue_summary_for_alerts.py Outdated Show resolved Hide resolved

Mihir-Mavalankar reviewed Apr 3, 2026

View reviewed changes

unused

359fcc0

vercel bot deployed to Preview April 3, 2026 21:37 View deployment

cursor bot reviewed Apr 3, 2026

View reviewed changes

src/sentry/seer/autofix/issue_summary.py Show resolved Hide resolved

ref(seer): Remove unused source param from _generate_summary

ab1e9fa

source was only used to pass to run_automation which was removed from this function. get_issue_summary still accepts source for billing logging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vercel bot deployed to Preview April 3, 2026 22:46 View deployment

sentry bot reviewed Apr 3, 2026

View reviewed changes

src/sentry/tasks/seer/autofix.py Show resolved Hide resolved

cursor bot reviewed Apr 3, 2026

View reviewed changes

src/sentry/integrations/utils/issue_summary_for_alerts.py Outdated Show resolved Hide resolved

vercel bot deployed to Preview April 8, 2026 15:16 View deployment

cursor bot reviewed Apr 8, 2026

View reviewed changes

trevor-e closed this Apr 8, 2026

		from django.contrib.auth.models import AnonymousUser

		from sentry.seer.autofix.issue_summary import get_issue_summary, run_automation

Uh oh!

Conversation

trevor-e commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backend Test Failures

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Mihir-Mavalankar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Mihir-Mavalankar commented Apr 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Apr 8, 2026

Choose a reason for hiding this comment

Alert path runs automation without timeout protection

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

trevor-e commented Apr 3, 2026 •

edited

Loading

github-actions bot commented Apr 3, 2026 •

edited

Loading