Skip to content

Commit fa0370e

Browse files
os-zhuangclaude
andauthored
refactor(ai): skills-only AI surface — real Actions instead of 10 fictional tools, retire the two agents (#512)
* refactor(ai): skills-only AI surface — real Actions instead of 10 fictional tools, retire the two agents (objectstack#3820, ADR-0109) HotCRM was the corpus that motivated objectstack#3820: 6 skills declaring 16 tool references, of which 10 named tools that exist nowhere. The copilot's instructions therefore claimed abilities it did not have — the runtime silently drops an unresolved tool, so the model improvised or failed when asked to use one. Every instance passed `objectstack validate` and `lint` cleanly. The fix is mostly deletion, because the references were a category error: "analyze the pipeline", "generate email copy", "score this lead" are REASONING, not tools. What actually needed a tool was the handful of steps that change state — and those already exist as HotCRM Actions, reachable through the `action_<name>` tools the runtime materialises from them (ADR-0109's default path: no tool records to author). - case_triage: `triage_case` → the priority rubric moves into instructions; `action_escalate_case` / `action_close_case` do the work. Also repoints the hand-off that named `response_drafting`, a skill that never existed (the audit's "hand-off to a nonexistent skill"), at the real `email_drafting`. - email_drafting: 4 fictional copy tools → the model writes the copy; `action_send_email` sends it, so recipient resolution, permissions and audit match the UI path. - lead_qualification: `analyze_lead` / `suggest_next_action` → BANT reasoning in instructions over `get_record`; `action_convert_lead` / `action_schedule_followup` act, and conversion requires confirmation. - revenue_forecasting: 3 fictional analytics tools → `aggregate_data` / `query_records` / `visualize_data` with the actual forecasting method spelled out in instructions. - live_data, customer_360: already correct, untouched but for prose. The two agents (`sales_copilot`, `service_copilot`) are DELETED. ADR-0063 §2 closed `*.agent.ts` to third parties and cloud#904 made the runtime refuse non-platform agent records, so they were unreachable metadata — carrying them is the ADR-0078 lie. The capability lives in the skills, which attach to the platform assistant by surface affinity. Verified with the merged reference-integrity rule (objectstack#3885 `validate-ai-tool-references`): 22 tool references, 0 findings — was 16 references with 10 dead. typecheck, `objectstack validate`, and the 67-test suite pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BHjroNkLkajskKbJaidko4 * fix(ai): only reference Actions that actually materialise as tools (ADR-0011) Follow-up to the skills rewrite in this PR. The first pass routed state changes through `action_escalate_case` / `action_close_case` / `action_send_email` — but the runtime materialises `action_<name>` ONLY for an Action that opts in (`ai.exposed` + `ai.description`) AND has a headless path. All three are `type: 'modal'`: they collect a reason / resolution / final copy from a person, so they are UI-only and no tool is ever generated. Referencing them would have re-created the bug this PR fixes, one layer down. - `convert_lead` and `schedule_followup` are flow-typed with targets, so they only needed the opt-in: both now carry `ai.exposed` + an LLM-facing description. `convert_lead` keeps its `confirmText`, which the runtime reads as approval-required — an agent invocation lands in the HITL queue instead of converting a lead unattended, which is what we want for an irreversible outcome. - `case_triage` and `email_drafting` no longer claim to act. They read, reason, and hand a ready-to-paste `reason` / `resolution` / draft to the user, pointing at the Escalate / Close / Send Email button. The instructions say plainly that the assistant cannot press it — a modal Action staying human-driven is the right shape for these, not a gap. Re-verified with the corrected rule (objectstack#3894, which now models AI exposure): 19 tool references, 0 findings. Both directions hold — the two opted-in flow Actions resolve, the three modal ones would not. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BHjroNkLkajskKbJaidko4 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent a257433 commit fa0370e

12 files changed

Lines changed: 160 additions & 144 deletions

objectstack.config.ts

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ import * as dashboards from './src/dashboards/index.js';
99
import * as datasets from './src/datasets/index.js';
1010
import * as reports from './src/reports/index.js';
1111
import { allFlows } from './src/flows/index.js';
12-
import { allAgents } from './src/agents/index.js';
1312
import { allSkills } from './src/skills/index.js';
1413
import * as profiles from './src/profiles/index.js';
1514
import * as apps from './src/apps/index.js';
@@ -79,7 +78,6 @@ export default defineStack({
7978
datasets: Object.values(datasets),
8079
reports: Object.values(reports),
8180
flows: allFlows,
82-
agents: allAgents,
8381
skills: allSkills,
8482
permissions: Object.values(profiles),
8583
apps: Object.values(apps),

src/actions/lead.actions.ts

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,17 @@ export const ConvertLeadAction: Action = {
2626
confirmText: 'Are you sure you want to convert this lead?',
2727
successMessage: 'Lead converted successfully!',
2828
refreshAfter: true,
29+
// ADR-0011 — opt this Action in to AI, which materialises the
30+
// `action_convert_lead` tool the `lead_qualification` skill references.
31+
// Flow-typed with a target, so it has a headless path. `confirmText` makes
32+
// the runtime treat it as approval-requiring, so an agent invocation lands
33+
// in the HITL queue rather than converting a lead unattended — which is the
34+
// behaviour we want for an irreversible outcome.
35+
ai: {
36+
exposed: true,
37+
description:
38+
'Converts a qualified lead into an account, contact and opportunity. Irreversible — requires human approval before it runs.',
39+
},
2940
};
3041

3142
/**
@@ -56,6 +67,13 @@ export const ScheduleFollowUpAction: Action = {
5667
visible: P`record.is_converted == false && record.status != "unqualified" && record.status != "converted"`,
5768
successMessage: 'Follow-up scheduled.',
5869
refreshAfter: true,
70+
// ADR-0011 — materialises `action_schedule_followup` for the
71+
// `lead_qualification` skill. Additive and reversible, so no approval gate.
72+
ai: {
73+
exposed: true,
74+
description:
75+
'Schedules the next follow-up task on a lead, assigned to its owner, so the next touch lands on the rep list.',
76+
},
5977
};
6078

6179
/**

src/agents/index.ts

Lines changed: 0 additions & 24 deletions
This file was deleted.

src/agents/sales-copilot.agent.ts

Lines changed: 0 additions & 41 deletions
This file was deleted.

src/agents/service-copilot.agent.ts

Lines changed: 0 additions & 29 deletions
This file was deleted.

src/docs/crm_admin.md

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -86,8 +86,13 @@ revisits:
8686
> on each record's detail page — they are intentionally **not** duplicated here.
8787
> This guide documents only what those screens can't tell you.
8888
89-
## AI copilots
90-
91-
Two assistants ship with HotCRM: the **Sales Copilot** and the **Service
92-
Copilot**. They operate over live CRM data and the actions exposed to them; the
93-
end user simply asks the assistant — no agent selection required.
89+
## AI skills
90+
91+
HotCRM ships **skills**, not assistants of its own. The platform provides the
92+
assistant (ObjectStack ADR-0063: the runtime owns exactly two agents and the
93+
surface you are in binds one); an app extends it by authoring skills that
94+
attach by surface affinity. HotCRM's six — live data, lead qualification, case
95+
triage, email drafting, revenue forecasting, customer 360 — operate over live
96+
CRM data and reach anything that *changes* state through the same Actions the
97+
UI buttons use, so permissions and audit are identical. The end user simply
98+
asks — no agent selection required.

src/objects/knowledge_article.object.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ import { F, P, cel } from '@objectstack/spec';
77
* Knowledge Article Object
88
*
99
* Reusable, search-indexed answers that back the Support Knowledge Base
10-
* and ground the Service Copilot's "Suggest a resolution" skill.
10+
* and ground the assistant's case-resolution skills.
1111
*
1212
* Lifecycle: draft → in_review → published → archived.
1313
* Audience: public (customer portal visible) | internal (agent-only).

src/skills/case-triage.skill.ts

Lines changed: 37 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,48 @@
22

33
import { defineSkill } from '@objectstack/spec';
44

5+
/**
6+
* Case Triage — prioritise a support case and route it to the right action.
7+
*
8+
* ADR-0109: this skill declares NO tool records. Triage itself is
9+
* reasoning, not a tool — the agent reads the case with the platform's
10+
* data tools and applies the rubric below.
11+
*
12+
* It deliberately calls NOTHING that mutates. Escalate and Close are
13+
* `type: 'modal'` Actions: they collect a reason / resolution from a
14+
* person, so they have no headless path and the runtime never
15+
* materialises tools for them (ADR-0011). That is the right shape here —
16+
* the agent supplies the judgement, the human supplies the words and the
17+
* decision — so the skill recommends the button instead of pretending to
18+
* press it.
19+
*/
520
export const CaseTriageSkill = defineSkill({
621
name: 'case_triage',
722
label: 'Case Triage',
8-
description: 'Triages a support case, assigns priority, and recommends the next action.',
23+
description: 'Triages a support case, assigns a priority with its justification, and points at the escalate/close action.',
924

10-
instructions: `When the user asks to triage, prioritise, or
11-
classify a case:
12-
1. Call triage_case with the current record.
13-
2. If priority resolves to "critical", immediately recommend
14-
escalation and draft an internal notification message.
15-
3. Otherwise, propose a customer-facing response by handing off to
16-
the response_drafting skill (do NOT inline-draft here).`,
25+
instructions: `When the user asks to triage, prioritise, or classify a case:
1726
18-
tools: ['triage_case'],
27+
1. Read the case first — call \`describe_object\` for \`crm_case\` (the
28+
schema is alive; admins add fields), then \`get_record\` for the case
29+
at hand. Never triage from memory of a previous turn.
30+
2. Assign a priority yourself from what you read. There is no triage
31+
tool and you do not need one — weigh, in order: customer tier and
32+
contract value, whether the customer is blocked with no workaround,
33+
how long the case has been open against its SLA, and the sentiment
34+
of the latest customer message.
35+
3. State the priority and the ONE reason that drove it. Cite the case
36+
ID and the field values you used.
37+
4. If the priority is critical, say so and point the user at
38+
**Escalate Case** on the record header — offer a ready-to-paste
39+
\`reason\` built from step 3. You cannot escalate yourself, and should
40+
not imply otherwise.
41+
5. If the case is resolved in substance, point at **Close Case** and
42+
offer a \`resolution\` summary the user can paste.
43+
6. For the customer-facing reply, hand off to the \`email_drafting\`
44+
skill rather than drafting it here.`,
45+
46+
tools: ['describe_object', 'get_record'],
1947

2048
triggerPhrases: [
2149
'triage this case',

src/skills/email-drafting.skill.ts

Lines changed: 28 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,40 @@
22

33
import { defineSkill } from '@objectstack/spec';
44

5+
/**
6+
* Email Drafting — write the copy, then send it through the real Action.
7+
*
8+
* ADR-0109: writing a subject line and personalising a body is what the
9+
* model does; it is not a tool call. Sending is HotCRM's `send_email`
10+
* Action — `type: 'modal'`, so it collects the final copy from a person
11+
* and has no headless path (ADR-0011). The agent therefore drafts and
12+
* hands over; a human presses Send. For outbound email that review step
13+
* is a feature, not a limitation.
14+
*/
515
export const EmailDraftingSkill = defineSkill({
616
name: 'email_drafting',
717
label: 'Email Drafting',
8-
description: 'Drafts personalised outbound emails and optimises subject lines for open rate.',
18+
description: 'Drafts personalised outbound emails grounded in live contact data, ready for a human to review and send.',
919

1020
instructions: `When the user asks to draft, write, or optimise an email:
11-
1. Use generate_email_copy with the recipient and a brief intent.
12-
2. Always run optimize_subject_line on the proposed subject before
13-
returning the draft.
14-
3. Personalise the body using personalize_content when the recipient
15-
has known firmographics or recent activity.
16-
4. Return the draft as { subject, body, alternatives } so the UI can
17-
present A/B variants.`,
1821
19-
tools: ['generate_email_copy', 'optimize_subject_line', 'personalize_content', 'generate_email'],
22+
1. Ground it in real data before writing a word: \`get_record\` the
23+
contact (and the related account or opportunity when the request
24+
references one) so names, titles and recent activity are accurate.
25+
Use \`query_records\` when you need the recent history.
26+
2. Write the copy yourself — subject and body. You do not have, and do
27+
not need, a copy-generation tool. Aim for: a subject under 60
28+
characters naming the concrete value, a first line referencing
29+
something specific to THIS recipient rather than a generic
30+
pleasantry, one clear ask, and no more than 150 words.
31+
3. Offer two subject-line variants, say which you recommend and why,
32+
so the user can A/B them.
33+
4. Show the draft and stop. You cannot send — say so plainly rather
34+
than implying a send is queued.
35+
5. Point the user at **Send Email** on the contact record, where the
36+
subject and body you drafted can be pasted, reviewed and sent.`,
37+
38+
tools: ['get_record', 'query_records'],
2039

2140
triggerPhrases: [
2241
'draft an email',

src/skills/lead-qualification.skill.ts

Lines changed: 33 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -3,27 +3,48 @@
33
import { defineSkill } from '@objectstack/spec';
44

55
/**
6-
* Lead Qualification Skill — qualifies inbound leads using BANT
7-
* (Budget, Authority, Need, Timeline) and emits a 0–100 score.
6+
* Lead Qualification — score an inbound lead with BANT, then act on it.
7+
*
8+
* ADR-0109: scoring is judgement, not a tool call. The lead is read
9+
* with the platform's data tools; the two outcomes that change state
10+
* (`convert_lead`, `schedule_followup`) are HotCRM's own Actions,
11+
* reached through their materialised `action_<name>` tools.
812
*
913
* Auto-activates when the user is viewing a lead record so the LLM
10-
* naturally surfaces the "Qualify Lead" capability without the user
11-
* having to pick an agent.
14+
* surfaces the capability without the user having to ask for it.
1215
*/
1316
export const LeadQualificationSkill = defineSkill({
1417
name: 'lead_qualification',
1518
label: 'Lead Qualification',
1619
description: 'Qualifies inbound leads using BANT criteria and assigns a 0–100 score.',
1720

18-
instructions: `When the user asks to qualify, score, or analyze a lead:
19-
1. Fetch the current lead with analyze_lead.
20-
2. Apply BANT (Budget, Authority, Need, Timeline) reasoning.
21-
3. Return a numeric score (0–100), a one-line justification per BANT
22-
dimension, and the recommended next action.
23-
4. If the score >= 70, propose calling suggest_next_action to draft
24-
the follow-up.`,
21+
instructions: `When the user asks to qualify, score, or analyse a lead:
22+
23+
1. Read the lead — \`describe_object\` for \`crm_lead\` first (fields
24+
change), then \`get_record\`. Use \`query_records\` to pull the
25+
related activity history when the lead has any.
26+
2. Score it yourself, 0–100, with BANT. There is no scoring tool:
27+
Budget (is spend confirmed or inferable from company size?),
28+
Authority (is the contact a decision maker?), Need (is there a
29+
stated problem you solve?), Timeline (is there a date?). Weight
30+
them evenly unless the user says otherwise.
31+
3. Report the score, one line of justification per BANT dimension
32+
naming the field or activity you read, and the single recommended
33+
next step. Cite the lead ID.
34+
4. Score >= 70 and the user agrees → call \`action_convert_lead\`.
35+
5. Otherwise, or when the user wants to nurture, call
36+
\`action_schedule_followup\` with the date you recommend. Say why
37+
that date.
38+
6. Never convert without explicit confirmation — conversion is not
39+
reversible from this surface.`,
2540

26-
tools: ['analyze_lead', 'suggest_next_action'],
41+
tools: [
42+
'describe_object',
43+
'get_record',
44+
'query_records',
45+
'action_convert_lead',
46+
'action_schedule_followup',
47+
],
2748

2849
triggerPhrases: [
2950
'qualify this lead',

0 commit comments

Comments
 (0)