Skip to content

Align custom agent instructions contract #159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
leo-paz merged 1 commit into from
Jul 1, 2026
Merged

Align custom agent instructions contract #159

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/custom-agent-unified-instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@outlit/cli": patch
---

Remove stale custom-agent criteria flags now that guidance belongs in unified instructions.
4 changes: 3 additions & 1 deletion docs/ai-integrations/platform-actions.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ For example, a CLI user or agent can create a draft template and then explicitly

```bash
outlit agents create --template churn --json
outlit agents create --type custom --display-name "Renewal risk" --instructions "Find risk" --surface-criteria "Surface risky customers" --json
outlit agents create --type custom --display-name "Renewal risk" --instructions "Find risky customers and skip already resolved issues." --json
outlit signals options --json
outlit signals create --file ./signal.json --json
outlit destinations options --json
Expand All @@ -66,6 +66,8 @@ Template creation creates supported template resources in `draft` mode. Draft cr

Automation create and update actions are agent-centered. Callers provide `agentId`; update bodies also provide `name`, `enabled`, and `triggerType` explicitly. Outlit maps the agent ID to the hosted-agent processor internally and does not require callers to construct raw processor JSON.

For custom agents, put all surfacing and skipping guidance in `instructions`. Older split fields such as `surfaceCriteria` and `skipCriteria` are no longer accepted by the platform action create/update contract.

## Safety Model

Platform actions are designed to make configuration inspectable before they make it broadly mutable.
Expand Down
2 changes: 1 addition & 1 deletion docs/cli/commands.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -583,7 +583,7 @@ outlit agents runs list 10000000-0000-4000-8000-000000000004 --json
outlit agents runs get 10000000-0000-4000-8000-000000000004 run_123 --json
outlit agents runs start 10000000-0000-4000-8000-000000000004 --client-request-id smoke-123 --json
outlit agents create --template churn --json
outlit agents create --type custom --display-name "Renewal risk" --instructions "Find risk" --surface-criteria "Surface risky customers" --json
outlit agents create --type custom --display-name "Renewal risk" --instructions "Find risky customers and skip already resolved issues." --json
outlit agents update 10000000-0000-4000-8000-000000000004 --display-name "Renewal risk" --json
outlit agents update 10000000-0000-4000-8000-000000000004 --instructions "Prioritize recent escalations" --json
outlit agents update 10000000-0000-4000-8000-000000000004 --action-keys send_slack_notification --json
Expand Down
34 changes: 6 additions & 28 deletions docs/openapi.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -274,8 +274,7 @@
"value": {
"type": "custom",
"displayName": "Renewal risk",
"instructions": "Find risk.",
"surfaceCriteria": "Surface risky customers.",
"instructions": "Find risky customers and skip already resolved issues.",
"actionKeys": ["send_slack_notification"]
}
}
Expand Down Expand Up @@ -7881,7 +7880,7 @@
},
"CreateAgentCustomRequest": {
"type": "object",
"required": ["type", "displayName", "instructions", "surfaceCriteria"],
"required": ["type", "displayName", "instructions"],
"properties": {
"type": {
"type": "string",
Expand All @@ -7895,17 +7894,7 @@
"instructions": {
"type": "string",
"minLength": 1,
"maxLength": 10000
},
"surfaceCriteria": {
"type": "string",
"minLength": 1,
"maxLength": 10000
},
"skipCriteria": {
"type": "string",
"minLength": 1,
"maxLength": 10000
"maxLength": 32000
},
"maxItemsToSurface": {
"type": "integer",
Expand Down Expand Up @@ -10506,22 +10495,11 @@
},
"CustomAgentSettingsSummary": {
"type": "object",
"required": [
"instructions",
"surfaceCriteria",
"skipCriteria",
"maxItemsToSurface",
"outputMode"
],
"required": ["instructions", "maxItemsToSurface", "outputMode"],
"properties": {
"instructions": {
"type": "string"
},
"surfaceCriteria": {
"type": "string"
},
"skipCriteria": {
"type": ["string", "null"]
"type": "string",
"maxLength": 32000
},
"maxItemsToSurface": {
"type": "integer",
Expand Down
19 changes: 4 additions & 15 deletions packages/cli/src/commands/agents/create.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,9 @@ import {
} from "../../lib/platform-input"

function hasCustomAgentFlags(args: Record<string, unknown>): boolean {
return [
"display-name",
"instructions",
"surface-criteria",
"skip-criteria",
"max-items-to-surface",
"action-keys",
].some((key) => args[key] !== undefined)
return ["display-name", "instructions", "max-items-to-surface", "action-keys"].some(
(key) => args[key] !== undefined,
)
}

export default defineCommand({
Expand All @@ -31,7 +26,7 @@ export default defineCommand({
"",
"Examples:",
" outlit agents create --template churn --json",
" outlit agents create --type custom --display-name 'Renewal risk' --instructions 'Find risk' --surface-criteria 'Surface risky customers' --json",
" outlit agents create --type custom --display-name 'Renewal risk' --instructions 'Find risky renewals and skip already resolved issues.' --json",
"",
AGENT_JSON_HINT,
].join("\n"),
Expand All @@ -49,8 +44,6 @@ export default defineCommand({
},
"display-name": { type: "string", description: "Custom agent display name" },
instructions: { type: "string", description: "Custom agent instructions" },
"surface-criteria": { type: "string", description: "Criteria for surfacing items" },
"skip-criteria": { type: "string", description: "Optional criteria for skipping items" },
"max-items-to-surface": {
type: "string",
description: "Maximum items the custom agent should surface per run",
Expand Down Expand Up @@ -106,10 +99,6 @@ export default defineCommand({
type: "custom",
displayName: requiredTrimmedString(args["display-name"], "--display-name", json),
instructions: requiredTrimmedString(args.instructions, "--instructions", json),
surfaceCriteria: requiredTrimmedString(args["surface-criteria"], "--surface-criteria", json),
...(optionalTrimmedString(args["skip-criteria"])
? { skipCriteria: optionalTrimmedString(args["skip-criteria"]) }
: {}),
maxItemsToSurface: parseIntegerFlag(
args["max-items-to-surface"],
10,
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/src/commands/agents/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ export default defineCommand({
" outlit agents runs list 10000000-0000-4000-8000-000000000004 --json",
" outlit agents runs start 10000000-0000-4000-8000-000000000004 --json",
" outlit agents create --template churn --json",
" outlit agents create --type custom --display-name 'Renewal risk' --instructions 'Find risk' --surface-criteria 'Surface risky customers' --json",
" outlit agents create --type custom --display-name 'Renewal risk' --instructions 'Find risky renewals and skip already resolved issues.' --json",
" outlit agents update 10000000-0000-4000-8000-000000000004 --display-name 'Renewal risk' --json",
" outlit agents update 10000000-0000-4000-8000-000000000004 --instructions 'New instructions' --json",
" outlit agents update 10000000-0000-4000-8000-000000000004 --action-keys send_slack_notification --json",
Expand Down
2 changes: 0 additions & 2 deletions packages/cli/src/commands/completions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,8 +263,6 @@ const COMMANDS: readonly CmdDef[] = [
{ name: "--type", desc: "Agent type to create" },
{ name: "--display-name", desc: "Agent display name" },
{ name: "--instructions", desc: "Agent instructions" },
{ name: "--surface-criteria", desc: "Criteria for surfacing items" },
{ name: "--skip-criteria", desc: "Optional criteria for skipping items" },
{ name: "--max-items-to-surface", desc: "Maximum items surfaced per run" },
{ name: "--action-keys", desc: "Comma-separated action keys" },
],
Expand Down
6 changes: 2 additions & 4 deletions packages/cli/tests/commands/agents/create.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,7 @@ describe("agents create", () => {
args: {
type: "custom",
"display-name": "Renewal risk",
instructions: "Find renewal risk.",
"surface-criteria": "Surface risky renewals.",
instructions: "Find risky renewals and skip already resolved issues.",
"action-keys": "send_slack_notification, create_task",
json: true,
},
Expand All @@ -83,8 +82,7 @@ describe("agents create", () => {
expect(mockCallTool).toHaveBeenCalledWith("outlit_agent_create", {
type: "custom",
displayName: "Renewal risk",
instructions: "Find renewal risk.",
surfaceCriteria: "Surface risky renewals.",
instructions: "Find risky renewals and skip already resolved issues.",
maxItemsToSurface: 10,
actionKeys: ["send_slack_notification", "create_task"],
})
Expand Down
6 changes: 2 additions & 4 deletions packages/cli/tests/commands/platform-lifecycle.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,8 +226,7 @@ describe("platform lifecycle commands", () => {
args: {
type: "custom",
"display-name": "Renewal risk",
instructions: "Find renewal risk.",
"surface-criteria": "Surface risky renewals.",
instructions: "Find risky renewals and skip already resolved issues.",
"action-keys": "send_slack_notification",
json: true,
},
Expand Down Expand Up @@ -262,8 +261,7 @@ describe("platform lifecycle commands", () => {
expect(mockCallTool).toHaveBeenNthCalledWith(2, "outlit_agent_create", {
type: "custom",
displayName: "Renewal risk",
instructions: "Find renewal risk.",
surfaceCriteria: "Surface risky renewals.",
instructions: "Find risky renewals and skip already resolved issues.",
maxItemsToSurface: 10,
actionKeys: ["send_slack_notification"],
})
Expand Down
6 changes: 2 additions & 4 deletions packages/cli/tests/lib/client.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -648,8 +648,7 @@ describe("client.callTool()", () => {
await client.callTool("outlit_agent_create", {
type: "custom",
displayName: "Renewal risk",
instructions: "Find risk",
surfaceCriteria: "Surface risky customers",
instructions: "Find risky customers and skip already resolved issues.",
maxItemsToSurface: 10,
actionKeys: ["send_slack_notification"],
})
Expand Down Expand Up @@ -758,8 +757,7 @@ describe("client.callTool()", () => {
JSON.stringify({
type: "custom",
displayName: "Renewal risk",
instructions: "Find risk",
surfaceCriteria: "Surface risky customers",
instructions: "Find risky customers and skip already resolved issues.",
maxItemsToSurface: 10,
actionKeys: ["send_slack_notification"],
}),
Expand Down
33 changes: 18 additions & 15 deletions tests/docs/openapi-spec.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,27 +66,31 @@ describe("docs OpenAPI spec", () => {
"/api/agent-actions",
"/api/agent-templates",
"/api/agents",
"/api/agents/{agentId}/runs",
"/api/agents/{agentId}/runs/{runId}",
"/api/agents/{id}",
"/api/agents/{id}/disable",
"/api/agents/{id}/enable",
"/api/agents/{id}/rename",
"/api/agents/{id}/runs",
"/api/agents/{id}/runs/{runId}",
"/api/automations",
"/api/automations/options",
"/api/automations/{automationId}/runs",
"/api/automations/{automationId}/runs/{runId}",
"/api/automations/{id}",
"/api/automations/{id}/archive",
"/api/automations/{id}/disable",
"/api/automations/{id}/enable",
"/api/automations/{id}/runs",
"/api/automations/{id}/runs/{runId}",
"/api/destinations",
"/api/destinations/options",
"/api/destinations/{id}",
"/api/destinations/{id}/archive",
"/api/destinations/{id}/disable",
"/api/destinations/{id}/enable",
"/api/i/v1/{publicKey}/events",
"/api/identity/merge-suggestions",
"/api/identity/merge-suggestions/{id}",
"/api/identity/merge-suggestions/{id}/queue",
"/api/identity/merge-suggestions/{id}/reject",
"/api/integrations",
"/api/integrations/capabilities",
"/api/integrations/connect",
Expand Down Expand Up @@ -137,17 +141,19 @@ describe("docs OpenAPI spec", () => {
"GET /api/agent-actions",
"GET /api/agent-templates",
"GET /api/agents",
"GET /api/agents/{agentId}/runs",
"GET /api/agents/{agentId}/runs/{runId}",
"GET /api/agents/{id}",
"GET /api/agents/{id}/runs",
"GET /api/agents/{id}/runs/{runId}",
"GET /api/automations",
"GET /api/automations/options",
"GET /api/automations/{automationId}/runs",
"GET /api/automations/{automationId}/runs/{runId}",
"GET /api/automations/{id}",
"GET /api/automations/{id}/runs",
"GET /api/automations/{id}/runs/{runId}",
"GET /api/destinations",
"GET /api/destinations/options",
"GET /api/destinations/{id}",
"GET /api/identity/merge-suggestions",
"GET /api/identity/merge-suggestions/{id}",
"GET /api/integrations",
"GET /api/integrations/capabilities",
"GET /api/integrations/connect/status",
Expand All @@ -165,10 +171,10 @@ describe("docs OpenAPI spec", () => {
"PATCH /api/settings/report",
"PATCH /api/signals/{id}",
"POST /api/agents",
"POST /api/agents/{agentId}/runs",
"POST /api/agents/{id}/disable",
"POST /api/agents/{id}/enable",
"POST /api/agents/{id}/rename",
"POST /api/agents/{id}/runs",
"POST /api/automations",
"POST /api/automations/{id}/archive",
"POST /api/automations/{id}/disable",
Expand All @@ -177,6 +183,8 @@ describe("docs OpenAPI spec", () => {
"POST /api/destinations/{id}/archive",
"POST /api/destinations/{id}/disable",
"POST /api/destinations/{id}/enable",
"POST /api/identity/merge-suggestions/{id}/queue",
"POST /api/identity/merge-suggestions/{id}/reject",
"POST /api/integrations/connect",
"POST /api/integrations/disconnect",
"POST /api/integrations/setup-step",
Expand Down Expand Up @@ -427,15 +435,10 @@ describe("docs OpenAPI spec", () => {
},
additionalProperties: false,
})
expect(schemas.UpdateDestinationRequest?.oneOf?.[1]?.properties?.isDefault).toMatchObject({
expect(schemas.UpdateDestinationRequest?.properties?.isDefault).toMatchObject({
type: "boolean",
const: true,
})
expect(schemas.UpdateDestinationRequest?.oneOf?.[0]?.properties?.url).toMatchObject({
type: "string",
format: "uri",
pattern: expect.stringContaining("localhost"),
})
expect(schemas.AuthoredSignalDefinition).toMatchObject({
type: "object",
required: ["schemaVersion", "subjectType", "detection"],
Expand Down
Loading