Skip to content

Commit 9104be8

Browse files
abhizipstackclaude
andauthored
feat: Deploy & Run History follow-ups — polling, metrics, deep-links, DB columns, server-side filters (#64)
* feat: deep-link success toast to Run History + auto-expand latest run The Quick Deploy success toast now includes a clickable "View in Run History →" link that navigates to /project/job/history?task=<id>, preselecting the job. On arrival, the Run History page auto-expands the most recent run (first row) in addition to any FAILURE rows, so the user immediately sees the deploy they just triggered. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: pre-fill create-job form from Quick Deploy 0-candidates CTA When no job covers the current model, clicking "Go to Scheduler" now navigates to /project/job/list?create=1&project=<pid>&model=<name>. The Jobs List reads these params: auto-opens the create drawer, and JobDeploy pre-enables the specified model in Model Configuration with the config panel auto-expanded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: promote trigger + scope to real DB columns on TaskRunHistory Previously stored only in kwargs JSON, making server-side filtering impossible. Now first-class nullable CharField columns with DB indexes, written by trigger_scheduled_run alongside kwargs. - Migration 0002 adds trigger (scheduled/manual) and scope (job/model) columns with defaults matching existing behavior. - celery_tasks.py writes both the columns and kwargs (backward compat). - Frontend getRunTriggerScope prefers top-level row.trigger / row.scope (from serializer) and falls back to kwargs for pre-migration rows. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: server-side Run History filtering by trigger, scope, status Replaces client-side filtering with server-side query params on the task_run_history endpoint. Filter changes now trigger a fresh API call with ?trigger=manual&scope=model&status=FAILURE, so results are accurate across all pages (previously client-side filtering only worked on the visible page). Backend accepts optional trigger, scope, status query params and applies them as Django ORM filters against the new DB columns from the previous migration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: live deploy progress polling on Quick Deploy button After dispatching a deploy, the Quick Deploy button flips to "Deploying…" with a spinner and polls the latest run status every 5s. On terminal state (SUCCESS/FAILURE/REVOKED): - Clears the polling interval - Shows a completion toast with status + deep-link to Run History - Refreshes the explorer (status badges) and recent-runs cache Polling auto-cleans on component unmount. The button returns to its normal state when the run finishes or the component unmounts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: capture runtime metrics from BASE_RESULT into TaskRunHistory After DAG execution (success or failure), trigger_scheduled_run now serializes BASE_RESULT into run.result as JSON with per-model status/end_status and aggregate passed/failed counts. Frontend insights panel renders a metrics bar when result is present: "N models attempted · X passed · Y failed" plus per-model breakdown. Falls back gracefully to scope/models display for older runs without result data. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: toast JSX links not rendering — add renderMarkdown: false The notify service defaults to renderMarkdown: true, which wraps description in ReactMarkdown. When description is JSX (our <a> link), ReactMarkdown stringifies it via JSON.stringify, rendering as raw text instead of a clickable link. Added renderMarkdown: false to both the dispatch toast and the polling-completion toast. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: hide metrics bar when result has no model data Old runs have result as {} or with total=0. Guard with record.result?.total > 0 so the metrics bar only renders when there's actual execution data. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: extract clean model name from class repr in run metrics BASE_RESULT.node_name stores str(cls) which renders as <class 'project.models.mdoela.Mdoela'>. Extract the module name (second-to-last dotted segment) so metrics show "mdoela" instead of the full class repr. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use class name for metrics instead of module name A model file can define multiple classes (e.g. SourceMdoela + Mdoela) in the same module. Using [-2] (module name) made them indistinguishable. Switch to [-1] (class name) so the metrics display shows "SourceMdoela (OK), Mdoela (OK)" instead of "mdoela (OK), mdoela (OK)". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: filter out Source classes from run metrics No-code models generate a *Source class (e.g. MdoelaSource) for DAG dependency resolution alongside the user's actual model class. Both execute as DAG nodes and appear in BASE_RESULT, but users only care about their own models. Filter out classes ending with "Source" from the metrics serialization so the count and per-model list reflect user-created models only. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Source classes start with Source, not end with it SourceMdoela, DevPaymentsSource — the sample projects use both conventions. The generated no-code models use the prefix pattern (SourceX). Changed endswith to startswith to match. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR #64 review — pagination filters, stale global, closure bugs - Pass active filters through handlePagination and handleRefresh (P1) - Snapshot-then-clear BASE_RESULT to prevent stale metrics across worker reuse (P1) - Fix handleRefresh stale closure deps (P2) - Forward project URL param from goToScheduler to JobDeploy (P2) - Prefer DB columns over kwargs for trigger/scope in list_recent_runs_for_model (P2) - Sanitize taskId with encodeURIComponent in toast deep-links (CodeQL) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: prettier formatting for toast deep-link JSX Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address Greptile review — BASE_RESULT ordering + pagination deps P1: _mark_failure was called after _clear_base_result(), so failure metrics were always empty. Swapped order: capture metrics first via _mark_failure, then clear the global. P1: getRunHistoryList had currentPage/pageSize in useCallback deps, causing infinite re-creation on pagination. Removed — they're passed as explicit arguments, not captured from closure. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use filterQueries.job instead of envInfo.id in pagination envInfo.id updates only after getRunHistoryList completes, creating a race window where pagination could fetch data for the previously selected job if the user switches jobs and changes page before the new data arrives. filterQueries.job updates immediately on job selection, so pagination always targets the correct job. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: eliminate double-fetch race and stale deep-link expand in run history - Remove getRunHistoryList and pageSize from filter effect deps to prevent double-fetch when handleJobChange sets filterQuery.job - Let the filter effect be the sole fetch trigger for job/filter changes - Use a ref to track deep-link consumption so auto-expand only fires once on initial arrival, not on every data refresh Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve eslint/prettier errors in run history Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: make trigger/scope nullable so pre-migration rows use kwargs fallback Existing rows get NULL instead of "scheduled"/"job" defaults, allowing the kwargs-based fallback to correctly identify manual/model-scope runs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: preserve prefillModel when project-change effect resets model configs The async getProjectModels callback was clearing modelConfigs after the prefillModel effect had already set the pre-checked model. Now preserves the prefilled model during reset. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 24d477c commit 9104be8

9 files changed

Lines changed: 399 additions & 56 deletions

File tree

backend/backend/core/scheduler/celery_tasks.py

Lines changed: 80 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,19 @@ def _send_notification(user_task: UserTaskDetails, run: TaskRunHistory, success:
147147
_send_slack_notification(user_task, run, success)
148148

149149

150+
# ---------------------------------------------------------------------------
151+
# BASE_RESULT cleanup helper
152+
# ---------------------------------------------------------------------------
153+
154+
def _clear_base_result():
155+
"""Clear the module-level BASE_RESULT global to prevent stale data across worker reuse."""
156+
try:
157+
from visitran.events.printer import BASE_RESULT
158+
BASE_RESULT.clear()
159+
except Exception:
160+
pass
161+
162+
150163
# ---------------------------------------------------------------------------
151164
# Job chaining helper
152165
# ---------------------------------------------------------------------------
@@ -264,6 +277,8 @@ def trigger_scheduled_run(
264277
start_time=timezone.now(),
265278
user_task_detail=user_task,
266279
kwargs=run_kwargs,
280+
trigger=trigger,
281+
scope=scope,
267282
)
268283

269284
# ── Mark task as running ──────────────────────────────────────────
@@ -321,11 +336,47 @@ def trigger_scheduled_run(
321336
else:
322337
app_context.execute_visitran_run_command(environment_id=environment_id)
323338

339+
# ── Capture execution metrics from BASE_RESULT ──────────────
340+
try:
341+
from visitran.events.printer import BASE_RESULT
342+
343+
# Snapshot and immediately clear the global to prevent stale
344+
# data leaking into a subsequent run on the same worker process.
345+
results_snapshot = list(BASE_RESULT)
346+
BASE_RESULT.clear()
347+
348+
def _clean_name(raw):
349+
if "'" in raw:
350+
return raw.split("'")[1].split(".")[-1]
351+
return raw
352+
353+
user_results = [
354+
r for r in results_snapshot
355+
if not _clean_name(r.node_name).startswith("Source")
356+
]
357+
run.result = {
358+
"models": [
359+
{
360+
"name": _clean_name(r.node_name),
361+
"status": r.status,
362+
"end_status": r.end_status,
363+
"sequence": r.sequence_num,
364+
}
365+
for r in user_results
366+
],
367+
"total": len(user_results),
368+
"passed": sum(1 for r in user_results if r.end_status == "OK"),
369+
"failed": sum(1 for r in user_results if r.end_status == "FAIL"),
370+
}
371+
except Exception:
372+
_clear_base_result()
373+
logger.debug("Could not capture BASE_RESULT metrics", exc_info=True)
374+
324375
# ── Mark success ──────────────────────────────────────────────
325376
success = True
326377
run.status = "SUCCESS"
327378
run.end_time = timezone.now()
328-
run.save(update_fields=["status", "end_time"])
379+
run.save(update_fields=["status", "end_time", "result"])
329380

330381
user_task.status = TaskStatus.SUCCESS
331382
user_task.task_completion_time = run.end_time
@@ -336,11 +387,13 @@ def trigger_scheduled_run(
336387
error_msg = str(exc) if str(exc) else f"Job exceeded timeout of {timeout}s"
337388
logger.warning("Job %s timed out: %s", user_task.task_name, error_msg)
338389
_mark_failure(run, user_task, error_msg)
390+
_clear_base_result()
339391

340392
except Exception as exc:
341393
error_msg = str(exc)
342394
logger.exception("Job %s failed: %s", user_task.task_name, error_msg)
343395
_mark_failure(run, user_task, error_msg)
396+
_clear_base_result()
344397

345398
# ── Retry logic ───────────────────────────────────────────────────
346399
if not success and user_task.max_retries > 0 and retry_num < user_task.max_retries:
@@ -380,10 +433,35 @@ def trigger_scheduled_run(
380433

381434
def _mark_failure(run: TaskRunHistory, user_task: UserTaskDetails, error_msg: str):
382435
"""Helper to mark a run and its parent task as failed."""
436+
try:
437+
from visitran.events.printer import BASE_RESULT
438+
439+
def _clean(raw):
440+
return raw.split("'")[1].split(".")[-1] if "'" in raw else raw
441+
442+
user_results = [
443+
r for r in BASE_RESULT if not _clean(r.node_name).startswith("Source")
444+
]
445+
run.result = {
446+
"models": [
447+
{
448+
"name": _clean(r.node_name),
449+
"status": r.status,
450+
"end_status": r.end_status,
451+
"sequence": r.sequence_num,
452+
}
453+
for r in user_results
454+
],
455+
"total": len(user_results),
456+
"passed": sum(1 for r in user_results if r.end_status == "OK"),
457+
"failed": sum(1 for r in user_results if r.end_status == "FAIL"),
458+
}
459+
except Exception:
460+
pass
383461
run.status = "FAILURE"
384462
run.end_time = timezone.now()
385463
run.error_message = error_msg
386-
run.save(update_fields=["status", "end_time", "error_message"])
464+
run.save(update_fields=["status", "end_time", "error_message", "result"])
387465

388466
user_task.status = TaskStatus.FAILED
389467
user_task.task_completion_time = run.end_time
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
from django.db import migrations, models
2+
3+
4+
class Migration(migrations.Migration):
5+
6+
dependencies = [
7+
("job_scheduler", "0001_initial"),
8+
]
9+
10+
operations = [
11+
migrations.AddField(
12+
model_name="taskrunhistory",
13+
name="trigger",
14+
field=models.CharField(
15+
choices=[("scheduled", "Scheduled"), ("manual", "Manual")],
16+
default="scheduled",
17+
null=True,
18+
blank=True,
19+
help_text="How the run was initiated: cron/interval schedule or manual dispatch.",
20+
max_length=20,
21+
),
22+
),
23+
migrations.AddField(
24+
model_name="taskrunhistory",
25+
name="scope",
26+
field=models.CharField(
27+
choices=[("job", "Full job"), ("model", "Single model")],
28+
default="job",
29+
null=True,
30+
blank=True,
31+
help_text="Whether the run executed all job models or a single model.",
32+
max_length=20,
33+
),
34+
),
35+
migrations.AddIndex(
36+
model_name="taskrunhistory",
37+
index=models.Index(
38+
fields=["trigger"], name="job_schedul_trigger_idx"
39+
),
40+
),
41+
migrations.AddIndex(
42+
model_name="taskrunhistory",
43+
index=models.Index(
44+
fields=["scope"], name="job_schedul_scope_idx"
45+
),
46+
),
47+
]

backend/backend/core/scheduler/models.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,22 @@ class TaskRunHistory(DefaultOrganizationMixin, BaseModel):
132132
kwargs = models.JSONField(blank=True, null=True)
133133
result = models.JSONField(blank=True, null=True)
134134
error_message = models.TextField(blank=True, null=True)
135+
trigger = models.CharField(
136+
max_length=20,
137+
choices=[("scheduled", "Scheduled"), ("manual", "Manual")],
138+
default="scheduled",
139+
null=True,
140+
blank=True,
141+
help_text="How the run was initiated: cron/interval schedule or manual dispatch.",
142+
)
143+
scope = models.CharField(
144+
max_length=20,
145+
choices=[("job", "Full job"), ("model", "Single model")],
146+
default="job",
147+
null=True,
148+
blank=True,
149+
help_text="Whether the run executed all job models or a single model.",
150+
)
135151

136152
user_task_detail = models.ForeignKey(
137153
UserTaskDetails,
@@ -154,6 +170,8 @@ class Meta:
154170
models.Index(
155171
fields=["user_task_detail"], name="job_schedul_user_ta_5cd43a_idx"
156172
),
173+
models.Index(fields=["trigger"], name="job_schedul_trigger_idx"),
174+
models.Index(fields=["scope"], name="job_schedul_scope_idx"),
157175
]
158176

159177
def __str__(self):

backend/backend/core/scheduler/views.py

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -595,7 +595,19 @@ def task_run_history(request, project_id, user_task_id):
595595
if _is_valid_project_id(project_id):
596596
query["project__project_uuid"] = project_id
597597
task = UserTaskDetails.objects.get(**query)
598-
runs = TaskRunHistory.objects.filter(user_task_detail=task).order_by("-start_time")
598+
runs = TaskRunHistory.objects.filter(user_task_detail=task)
599+
600+
trigger_filter = request.GET.get("trigger")
601+
scope_filter = request.GET.get("scope")
602+
status_filter = request.GET.get("status")
603+
if trigger_filter:
604+
runs = runs.filter(trigger=trigger_filter)
605+
if scope_filter:
606+
runs = runs.filter(scope=scope_filter)
607+
if status_filter:
608+
runs = runs.filter(status=status_filter)
609+
610+
runs = runs.order_by("-start_time")
599611
total = runs.count()
600612

601613
offset = (page - 1) * limit
@@ -759,13 +771,13 @@ def list_recent_runs_for_model(request, project_id, model_name):
759771
env = task.environment
760772
kwargs = run.kwargs or {}
761773
models_override = kwargs.get("models_override") or []
762-
# Back-compat: rows written before the trigger/scope split only
763-
# carried kwargs.source=="quick_deploy" as their manual-model marker.
774+
# Prefer first-class DB columns; fall back to kwargs for rows
775+
# written before the trigger/scope migration.
764776
legacy_source = kwargs.get("source")
765-
trigger = kwargs.get("trigger") or (
777+
trigger = run.trigger or kwargs.get("trigger") or (
766778
"manual" if legacy_source == "quick_deploy" else "scheduled"
767779
)
768-
scope = kwargs.get("scope") or (
780+
scope = run.scope or kwargs.get("scope") or (
769781
"model" if models_override or legacy_source == "quick_deploy" else "job"
770782
)
771783
data.append({

frontend/src/ide/editor/no-code-model/no-code-model.jsx

Lines changed: 89 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -259,6 +259,7 @@ function NoCodeModel({ nodeData }) {
259259
runTaskForModel,
260260
runTask,
261261
listRecentRunsForModel,
262+
getLatestRunStatus,
262263
} = useJobService();
263264

264265
const [quickDeployModal, setQuickDeployModal] = useState({
@@ -273,8 +274,10 @@ function NoCodeModel({ nodeData }) {
273274
const [recentRunsState, setRecentRunsState] = useState({
274275
loading: false,
275276
runs: [],
276-
fetchedFor: null, // model name the current runs are for
277+
fetchedFor: null,
277278
});
279+
const [deployPolling, setDeployPolling] = useState(null);
280+
const pollingRef = useRef(null);
278281

279282
const modelName =
280283
nodeData?.node?.title ||
@@ -1921,16 +1924,29 @@ function NoCodeModel({ nodeData }) {
19211924
);
19221925
const envName = selected?.environment_name || "the selected environment";
19231926
const jobName = selected?.task_name || "";
1927+
const taskId = encodeURIComponent(quickDeployModal.selectedTaskId);
19241928
notify({
19251929
type: "success",
19261930
message: "Deploy Triggered",
1927-
description:
1928-
selectedScope === "job"
1929-
? `Job "${jobName}" is running on "${envName}" (all enabled models). Check Run History for progress.`
1930-
: `"${currentModelName}" is running on "${envName}" via job "${jobName}". Check Run History for progress.`,
1931+
renderMarkdown: false,
1932+
description: (
1933+
<span>
1934+
{selectedScope === "job"
1935+
? `Job "${jobName}" is running on "${envName}" (all enabled models). `
1936+
: `"${currentModelName}" is running on "${envName}" via job "${jobName}". `}
1937+
<a
1938+
href={`/project/job/history?task=${taskId}`}
1939+
onClick={(e) => {
1940+
e.preventDefault();
1941+
navigate(`/project/job/history?task=${taskId}`);
1942+
}}
1943+
>
1944+
View in Run History →
1945+
</a>
1946+
</span>
1947+
),
19311948
});
1932-
setRefreshModels(true);
1933-
setRecentRunsState((prev) => ({ ...prev, fetchedFor: null }));
1949+
startDeployPolling(quickDeployModal.selectedTaskId);
19341950
setQuickDeployModal((prev) => ({
19351951
...prev,
19361952
open: false,
@@ -1942,9 +1958,71 @@ function NoCodeModel({ nodeData }) {
19421958
}
19431959
};
19441960

1961+
const startDeployPolling = (taskId) => {
1962+
if (pollingRef.current) clearInterval(pollingRef.current);
1963+
setDeployPolling({ taskId, status: "STARTED" });
1964+
pollingRef.current = setInterval(async () => {
1965+
try {
1966+
const run = await getLatestRunStatus(projectId, taskId);
1967+
if (!run) return;
1968+
const terminal = ["SUCCESS", "FAILURE", "REVOKED"].includes(run.status);
1969+
if (terminal) {
1970+
clearInterval(pollingRef.current);
1971+
pollingRef.current = null;
1972+
setDeployPolling(null);
1973+
setRefreshModels(true);
1974+
setRecentRunsState((prev) => ({ ...prev, fetchedFor: null }));
1975+
notify({
1976+
type: run.status === "SUCCESS" ? "success" : "error",
1977+
message:
1978+
run.status === "SUCCESS" ? "Deploy Completed" : "Deploy Failed",
1979+
renderMarkdown: false,
1980+
description: (
1981+
<span>
1982+
{run.status === "SUCCESS"
1983+
? "Model deployed successfully."
1984+
: run.error_message || "Check Run History for details."}{" "}
1985+
<a
1986+
href={`/project/job/history?task=${encodeURIComponent(
1987+
taskId
1988+
)}`}
1989+
onClick={(e) => {
1990+
e.preventDefault();
1991+
navigate(
1992+
`/project/job/history?task=${encodeURIComponent(taskId)}`
1993+
);
1994+
}}
1995+
>
1996+
View in Run History →
1997+
</a>
1998+
</span>
1999+
),
2000+
});
2001+
} else {
2002+
setDeployPolling((prev) =>
2003+
prev ? { ...prev, status: run.status } : null
2004+
);
2005+
}
2006+
} catch {
2007+
// Silently retry on next interval
2008+
}
2009+
}, 5000);
2010+
};
2011+
2012+
useEffect(() => {
2013+
return () => {
2014+
if (pollingRef.current) clearInterval(pollingRef.current);
2015+
};
2016+
}, []);
2017+
19452018
const goToScheduler = () => {
19462019
setQuickDeployModal((prev) => ({ ...prev, open: false }));
1947-
navigate("/project/job/list");
2020+
const params = new URLSearchParams();
2021+
params.set("create", "1");
2022+
if (projectId) params.set("project", projectId);
2023+
const modelTitle = nodeData?.node?.title;
2024+
if (modelTitle) params.set("model", modelTitle);
2025+
navigate(`/project/job/list?${params.toString()}`);
19482026
};
19492027

19502028
const runTransformation = (spec) => {
@@ -2945,9 +3023,10 @@ function NoCodeModel({ nodeData }) {
29453023
!can_write ||
29463024
!nodeData?.node?.title
29473025
}
2948-
icon={<PlayCircleOutlined />}
3026+
loading={!!deployPolling}
3027+
icon={!deployPolling ? <PlayCircleOutlined /> : undefined}
29493028
>
2950-
Quick Deploy
3029+
{deployPolling ? "Deploying…" : "Quick Deploy"}
29513030
</Button>
29523031
<Dropdown
29533032
trigger={["click"]}

0 commit comments

Comments
 (0)