Skip to content

Commit 57aeafe

Browse files
pescnclaude
pescn
and
claude
authored
feat(playground): add Chat UI and comparison testing (#74)
* style: apply code formatter Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(playground): add database schema, migrations, and queries Add 5 new tables for the playground feature: - playground_conversations: chat conversations with model/params - playground_messages: individual messages in conversations - playground_test_cases: reusable test case templates - playground_test_runs: test run executions across models - playground_test_results: per-model results with metrics Includes CRUD query functions and Drizzle migration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(playground): add backend admin API endpoints Add REST endpoints for playground resources under /api/admin/playground: - Conversations: CRUD + message management - Test cases: CRUD with message templates - Test runs: create/list/delete with per-model results - Test results: update status, response, and metrics Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(playground): add frontend chat and comparison testing UI Add a new "Playground" section with two tabs: Chat tab: - Three-pane layout: conversation list, chat area, parameter sidebar - Model and API key selection via dropdowns - Real-time streaming responses via SSE - Conversation persistence with auto-generated titles - Configurable parameters: system prompt, temperature, top_p, etc. Compare tab: - Create reusable test cases with message templates - Run test cases against multiple models simultaneously - Side-by-side comparison of responses with metrics (tokens, TTFT, duration) Also adds shadcn/ui components (scroll-area, slider, textarea), i18n keys for both en-US and zh-CN, and sidebar navigation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(playground): address code review feedback Backend: - Extract shared roleSchema for consistent role validation - Return 404 when test result not found (was returning 200 null) - Return 500 status on test run creation failure (was returning 200) - Add pagination limit upper bound (max 200) Frontend: - Fix stopSequences input UX: normalize on blur instead of every keystroke - Abort previous stream before starting new one in useChatStreaming - Use functional updater in handleSend to avoid stale closure - Add try/catch error handling to handleSend, handleSaveAsTestCase, handleCreate, handleDelete, handleUpdate - Use instant scroll during streaming, smooth scroll otherwise - Forward all supported params in comparison test runs - Fix test-case-editor to include params in save payload - Replace hard-coded English labels with i18n keys - Fix NavContent anti-pattern (function call instead of component) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(playground): address follow-up code review feedback - Add null checks for conversation and test case creation (500 on failure) - Add try/catch for conversation creation in chat handleSend - Add try/catch for assistant message persistence in onDone - Move sendMessage side effect out of setMessages state updater - Add toast for test run creation failure - Add nested try/catch for PUT in comparison catch block - Add missing `t` and `messages` to useCallback dependencies Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(playground): add FK cascade/set-null actions and atomic test run creation Add proper ON DELETE actions to playground FK constraints: - CASCADE for parent-child relationships (messages, test runs, results) - SET NULL for nullable external references (api keys, completions) Wrap test run + result creation in a database transaction for atomicity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(playground): add message insert null check, TTFT translation, and outer catch - Add null check with 500 status for insertPlaygroundMessage POST handler - Fix TTFT zh-CN translation to match existing terminology (首 Token 返回时间) - Add catch block to handleRunComparison outer try for user error feedback Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(playground): validate testCaseId, fail transaction on insert error, fix running PUT scope - Validate testCaseId exists before creating test run (404 if not found) - Throw error in transaction when result insert fails instead of silently skipping - Move running status PUT inside inner try block so failures are properly caught Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 454e721 commit 57aeafe

84 files changed

Lines changed: 7467 additions & 1654 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎backend/drizzle/0017_superb_sentinels.sql‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
CREATE TYPE "public"."playground_message_role" AS ENUM('system', 'user', 'assistant');--> statement-breakpoint
2+
CREATE TYPE "public"."playground_test_result_status" AS ENUM('pending', 'running', 'completed', 'failed');--> statement-breakpoint
3+
CREATE TABLE "playground_conversations" (
4+
"id" integer PRIMARY KEY GENERATED ALWAYS AS IDENTITY (sequence name "playground_conversations_id_seq" INCREMENT BY 1 MINVALUE 1 MAXVALUE 2147483647 START WITH 1 CACHE 1),
5+
"title" varchar(255) NOT NULL,
6+
"model" varchar(63) NOT NULL,
7+
"api_key_id" integer,
8+
"params" jsonb,
9+
"created_at" timestamp DEFAULT now() NOT NULL,
10+
"updated_at" timestamp DEFAULT now() NOT NULL,
11+
"deleted" boolean DEFAULT false NOT NULL
12+
);
13+
--> statement-breakpoint
14+
CREATE TABLE "playground_messages" (
15+
"id" integer PRIMARY KEY GENERATED ALWAYS AS IDENTITY (sequence name "playground_messages_id_seq" INCREMENT BY 1 MINVALUE 1 MAXVALUE 2147483647 START WITH 1 CACHE 1),
16+
"conversation_id" integer NOT NULL,
17+
"role" "playground_message_role" NOT NULL,
18+
"content" varchar NOT NULL,
19+
"completion_id" integer,
20+
"created_at" timestamp DEFAULT now() NOT NULL
21+
);
22+
--> statement-breakpoint
23+
CREATE TABLE "playground_test_cases" (
24+
"id" integer PRIMARY KEY GENERATED ALWAYS AS IDENTITY (sequence name "playground_test_cases_id_seq" INCREMENT BY 1 MINVALUE 1 MAXVALUE 2147483647 START WITH 1 CACHE 1),
25+
"title" varchar(255) NOT NULL,
26+
"description" varchar,
27+
"messages" jsonb NOT NULL,
28+
"params" jsonb,
29+
"created_at" timestamp DEFAULT now() NOT NULL,
30+
"updated_at" timestamp DEFAULT now() NOT NULL,
31+
"deleted" boolean DEFAULT false NOT NULL
32+
);
33+
--> statement-breakpoint
34+
CREATE TABLE "playground_test_results" (
35+
"id" integer PRIMARY KEY GENERATED ALWAYS AS IDENTITY (sequence name "playground_test_results_id_seq" INCREMENT BY 1 MINVALUE 1 MAXVALUE 2147483647 START WITH 1 CACHE 1),
36+
"test_run_id" integer NOT NULL,
37+
"model" varchar(63) NOT NULL,
38+
"status" "playground_test_result_status" DEFAULT 'pending' NOT NULL,
39+
"response" varchar,
40+
"prompt_tokens" integer,
41+
"completion_tokens" integer,
42+
"ttft" integer,
43+
"duration" integer,
44+
"error_message" varchar,
45+
"completion_id" integer,
46+
"created_at" timestamp DEFAULT now() NOT NULL,
47+
"updated_at" timestamp DEFAULT now() NOT NULL
48+
);
49+
--> statement-breakpoint
50+
CREATE TABLE "playground_test_runs" (
51+
"id" integer PRIMARY KEY GENERATED ALWAYS AS IDENTITY (sequence name "playground_test_runs_id_seq" INCREMENT BY 1 MINVALUE 1 MAXVALUE 2147483647 START WITH 1 CACHE 1),
52+
"test_case_id" integer NOT NULL,
53+
"api_key_id" integer,
54+
"models" jsonb NOT NULL,
55+
"created_at" timestamp DEFAULT now() NOT NULL,
56+
"deleted" boolean DEFAULT false NOT NULL
57+
);
58+
--> statement-breakpoint
59+
ALTER TABLE "playground_conversations" ADD CONSTRAINT "playground_conversations_api_key_id_api_keys_id_fk" FOREIGN KEY ("api_key_id") REFERENCES "public"."api_keys"("id") ON DELETE no action ON UPDATE no action;--> statement-breakpoint
60+
ALTER TABLE "playground_messages" ADD CONSTRAINT "playground_messages_conversation_id_playground_conversations_id_fk" FOREIGN KEY ("conversation_id") REFERENCES "public"."playground_conversations"("id") ON DELETE no action ON UPDATE no action;--> statement-breakpoint
61+
ALTER TABLE "playground_messages" ADD CONSTRAINT "playground_messages_completion_id_completions_id_fk" FOREIGN KEY ("completion_id") REFERENCES "public"."completions"("id") ON DELETE no action ON UPDATE no action;--> statement-breakpoint
62+
ALTER TABLE "playground_test_results" ADD CONSTRAINT "playground_test_results_test_run_id_playground_test_runs_id_fk" FOREIGN KEY ("test_run_id") REFERENCES "public"."playground_test_runs"("id") ON DELETE no action ON UPDATE no action;--> statement-breakpoint
63+
ALTER TABLE "playground_test_results" ADD CONSTRAINT "playground_test_results_completion_id_completions_id_fk" FOREIGN KEY ("completion_id") REFERENCES "public"."completions"("id") ON DELETE no action ON UPDATE no action;--> statement-breakpoint
64+
ALTER TABLE "playground_test_runs" ADD CONSTRAINT "playground_test_runs_test_case_id_playground_test_cases_id_fk" FOREIGN KEY ("test_case_id") REFERENCES "public"."playground_test_cases"("id") ON DELETE no action ON UPDATE no action;--> statement-breakpoint
65+
ALTER TABLE "playground_test_runs" ADD CONSTRAINT "playground_test_runs_api_key_id_api_keys_id_fk" FOREIGN KEY ("api_key_id") REFERENCES "public"."api_keys"("id") ON DELETE no action ON UPDATE no action;

‎backend/drizzle/0018_messy_spectrum.sql‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
ALTER TABLE "playground_conversations" DROP CONSTRAINT "playground_conversations_api_key_id_api_keys_id_fk";
2+
--> statement-breakpoint
3+
ALTER TABLE "playground_messages" DROP CONSTRAINT "playground_messages_conversation_id_playground_conversations_id_fk";
4+
--> statement-breakpoint
5+
ALTER TABLE "playground_messages" DROP CONSTRAINT "playground_messages_completion_id_completions_id_fk";
6+
--> statement-breakpoint
7+
ALTER TABLE "playground_test_results" DROP CONSTRAINT "playground_test_results_test_run_id_playground_test_runs_id_fk";
8+
--> statement-breakpoint
9+
ALTER TABLE "playground_test_results" DROP CONSTRAINT "playground_test_results_completion_id_completions_id_fk";
10+
--> statement-breakpoint
11+
ALTER TABLE "playground_test_runs" DROP CONSTRAINT "playground_test_runs_test_case_id_playground_test_cases_id_fk";
12+
--> statement-breakpoint
13+
ALTER TABLE "playground_test_runs" DROP CONSTRAINT "playground_test_runs_api_key_id_api_keys_id_fk";
14+
--> statement-breakpoint
15+
ALTER TABLE "playground_conversations" ADD CONSTRAINT "playground_conversations_api_key_id_api_keys_id_fk" FOREIGN KEY ("api_key_id") REFERENCES "public"."api_keys"("id") ON DELETE set null ON UPDATE no action;--> statement-breakpoint
16+
ALTER TABLE "playground_messages" ADD CONSTRAINT "playground_messages_conversation_id_playground_conversations_id_fk" FOREIGN KEY ("conversation_id") REFERENCES "public"."playground_conversations"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
17+
ALTER TABLE "playground_messages" ADD CONSTRAINT "playground_messages_completion_id_completions_id_fk" FOREIGN KEY ("completion_id") REFERENCES "public"."completions"("id") ON DELETE set null ON UPDATE no action;--> statement-breakpoint
18+
ALTER TABLE "playground_test_results" ADD CONSTRAINT "playground_test_results_test_run_id_playground_test_runs_id_fk" FOREIGN KEY ("test_run_id") REFERENCES "public"."playground_test_runs"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
19+
ALTER TABLE "playground_test_results" ADD CONSTRAINT "playground_test_results_completion_id_completions_id_fk" FOREIGN KEY ("completion_id") REFERENCES "public"."completions"("id") ON DELETE set null ON UPDATE no action;--> statement-breakpoint
20+
ALTER TABLE "playground_test_runs" ADD CONSTRAINT "playground_test_runs_test_case_id_playground_test_cases_id_fk" FOREIGN KEY ("test_case_id") REFERENCES "public"."playground_test_cases"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
21+
ALTER TABLE "playground_test_runs" ADD CONSTRAINT "playground_test_runs_api_key_id_api_keys_id_fk" FOREIGN KEY ("api_key_id") REFERENCES "public"."api_keys"("id") ON DELETE set null ON UPDATE no action;

‎backend/drizzle/meta/0008_snapshot.json‎

Lines changed: 28 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -92,9 +92,7 @@
9292
"api_keys_key_unique": {
9393
"name": "api_keys_key_unique",
9494
"nullsNotDistinct": false,
95-
"columns": [
96-
"key"
97-
]
95+
"columns": ["key"]
9896
}
9997
},
10098
"policies": {},
@@ -224,25 +222,17 @@
224222
"name": "completions_api_key_id_api_keys_id_fk",
225223
"tableFrom": "completions",
226224
"tableTo": "api_keys",
227-
"columnsFrom": [
228-
"api_key_id"
229-
],
230-
"columnsTo": [
231-
"id"
232-
],
225+
"columnsFrom": ["api_key_id"],
226+
"columnsTo": ["id"],
233227
"onDelete": "no action",
234228
"onUpdate": "no action"
235229
},
236230
"completions_upstream_id_upstreams_id_fk": {
237231
"name": "completions_upstream_id_upstreams_id_fk",
238232
"tableFrom": "completions",
239233
"tableTo": "upstreams",
240-
"columnsFrom": [
241-
"upstream_id"
242-
],
243-
"columnsTo": [
244-
"id"
245-
],
234+
"columnsFrom": ["upstream_id"],
235+
"columnsTo": ["id"],
246236
"onDelete": "no action",
247237
"onUpdate": "no action"
248238
}
@@ -252,9 +242,7 @@
252242
"completions_id_unique": {
253243
"name": "completions_id_unique",
254244
"nullsNotDistinct": false,
255-
"columns": [
256-
"id"
257-
]
245+
"columns": ["id"]
258246
}
259247
},
260248
"policies": {},
@@ -366,25 +354,17 @@
366354
"name": "embeddings_api_key_id_api_keys_id_fk",
367355
"tableFrom": "embeddings",
368356
"tableTo": "api_keys",
369-
"columnsFrom": [
370-
"api_key_id"
371-
],
372-
"columnsTo": [
373-
"id"
374-
],
357+
"columnsFrom": ["api_key_id"],
358+
"columnsTo": ["id"],
375359
"onDelete": "no action",
376360
"onUpdate": "no action"
377361
},
378362
"embeddings_model_id_models_id_fk": {
379363
"name": "embeddings_model_id_models_id_fk",
380364
"tableFrom": "embeddings",
381365
"tableTo": "models",
382-
"columnsFrom": [
383-
"model_id"
384-
],
385-
"columnsTo": [
386-
"id"
387-
],
366+
"columnsFrom": ["model_id"],
367+
"columnsTo": ["id"],
388368
"onDelete": "no action",
389369
"onUpdate": "no action"
390370
}
@@ -394,9 +374,7 @@
394374
"embeddings_id_unique": {
395375
"name": "embeddings_id_unique",
396376
"nullsNotDistinct": false,
397-
"columns": [
398-
"id"
399-
]
377+
"columns": ["id"]
400378
}
401379
},
402380
"policies": {},
@@ -509,12 +487,8 @@
509487
"name": "models_provider_id_providers_id_fk",
510488
"tableFrom": "models",
511489
"tableTo": "providers",
512-
"columnsFrom": [
513-
"provider_id"
514-
],
515-
"columnsTo": [
516-
"id"
517-
],
490+
"columnsFrom": ["provider_id"],
491+
"columnsTo": ["id"],
518492
"onDelete": "no action",
519493
"onUpdate": "no action"
520494
}
@@ -524,10 +498,7 @@
524498
"models_provider_system_name_unique": {
525499
"name": "models_provider_system_name_unique",
526500
"nullsNotDistinct": false,
527-
"columns": [
528-
"provider_id",
529-
"system_name"
530-
]
501+
"columns": ["provider_id", "system_name"]
531502
}
532503
},
533504
"policies": {},
@@ -622,9 +593,7 @@
622593
"providers_name_unique": {
623594
"name": "providers_name_unique",
624595
"nullsNotDistinct": false,
625-
"columns": [
626-
"name"
627-
]
596+
"columns": ["name"]
628597
}
629598
},
630599
"policies": {},
@@ -679,9 +648,7 @@
679648
"settings_key_unique": {
680649
"name": "settings_key_unique",
681650
"nullsNotDistinct": false,
682-
"columns": [
683-
"key"
684-
]
651+
"columns": ["key"]
685652
}
686653
},
687654
"policies": {},
@@ -773,38 +740,26 @@
773740
"name": "srv_logs_related_api_key_id_api_keys_id_fk",
774741
"tableFrom": "srv_logs",
775742
"tableTo": "api_keys",
776-
"columnsFrom": [
777-
"related_api_key_id"
778-
],
779-
"columnsTo": [
780-
"id"
781-
],
743+
"columnsFrom": ["related_api_key_id"],
744+
"columnsTo": ["id"],
782745
"onDelete": "no action",
783746
"onUpdate": "no action"
784747
},
785748
"srv_logs_related_upstream_id_upstreams_id_fk": {
786749
"name": "srv_logs_related_upstream_id_upstreams_id_fk",
787750
"tableFrom": "srv_logs",
788751
"tableTo": "upstreams",
789-
"columnsFrom": [
790-
"related_upstream_id"
791-
],
792-
"columnsTo": [
793-
"id"
794-
],
752+
"columnsFrom": ["related_upstream_id"],
753+
"columnsTo": ["id"],
795754
"onDelete": "no action",
796755
"onUpdate": "no action"
797756
},
798757
"srv_logs_related_completion_id_completions_id_fk": {
799758
"name": "srv_logs_related_completion_id_completions_id_fk",
800759
"tableFrom": "srv_logs",
801760
"tableTo": "completions",
802-
"columnsFrom": [
803-
"related_completion_id"
804-
],
805-
"columnsTo": [
806-
"id"
807-
],
761+
"columnsFrom": ["related_completion_id"],
762+
"columnsTo": ["id"],
808763
"onDelete": "no action",
809764
"onUpdate": "no action"
810765
}
@@ -814,9 +769,7 @@
814769
"srv_logs_id_unique": {
815770
"name": "srv_logs_id_unique",
816771
"nullsNotDistinct": false,
817-
"columns": [
818-
"id"
819-
]
772+
"columns": ["id"]
820773
}
821774
},
822775
"policies": {},
@@ -922,40 +875,22 @@
922875
"public.completions_status": {
923876
"name": "completions_status",
924877
"schema": "public",
925-
"values": [
926-
"pending",
927-
"completed",
928-
"failed"
929-
]
878+
"values": ["pending", "completed", "failed"]
930879
},
931880
"public.model_type": {
932881
"name": "model_type",
933882
"schema": "public",
934-
"values": [
935-
"chat",
936-
"embedding"
937-
]
883+
"values": ["chat", "embedding"]
938884
},
939885
"public.provider_type": {
940886
"name": "provider_type",
941887
"schema": "public",
942-
"values": [
943-
"openai",
944-
"openai-responses",
945-
"anthropic",
946-
"azure",
947-
"ollama"
948-
]
888+
"values": ["openai", "openai-responses", "anthropic", "azure", "ollama"]
949889
},
950890
"public.srv_logs_level": {
951891
"name": "srv_logs_level",
952892
"schema": "public",
953-
"values": [
954-
"unspecific",
955-
"info",
956-
"warn",
957-
"error"
958-
]
893+
"values": ["unspecific", "info", "warn", "error"]
959894
}
960895
},
961896
"schemas": {},
@@ -968,4 +903,4 @@
968903
"schemas": {},
969904
"tables": {}
970905
}
971-
}
906+
}

0 commit comments

Comments
 (0)