Skip to content

Commit 7d3df10

Browse files
Copilothotlong
Copilot
and
hotlong
committed
feat(ai): add unified cost tracking schemas to AI protocols
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
1 parent 5013b3a commit 7d3df10

20 files changed

+391
-15
lines changed

‎content/docs/references/ai/conversation.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ const result = ConversationAnalyticsSchema.parse(data);
7777
| **toolCallId** | `string` | optional | Tool call ID this message responds to |
7878
| **name** | `string` | optional | Name of the function/user |
7979
| **tokens** | `object` | optional | Token usage for this message |
80+
| **cost** | `number` | optional | Cost for this message in USD |
8081
| **pinned** | `boolean` | optional | Prevent removal during pruning |
8182
| **importance** | `number` | optional | Importance score for pruning |
8283
| **embedding** | `number[]` | optional | Vector embedding for semantic search |
@@ -97,6 +98,8 @@ const result = ConversationAnalyticsSchema.parse(data);
9798
| **tokenBudget** | `object` | ✅ | |
9899
| **messages** | `object[]` | optional | |
99100
| **tokens** | `object` | optional | |
101+
| **totalTokens** | `object` | optional | Total tokens across all messages |
102+
| **totalCost** | `number` | optional | Total cost for this session in USD |
100103
| **status** | `Enum<'active' \| 'paused' \| 'completed' \| 'archived'>` | optional | |
101104
| **createdAt** | `string` | ✅ | ISO 8601 timestamp |
102105
| **updatedAt** | `string` | ✅ | ISO 8601 timestamp |

‎content/docs/references/ai/cost.mdx‎

Lines changed: 32 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,15 +12,32 @@ description: Cost protocol schemas
1212
## TypeScript Usage
1313

1414
```typescript
15-
import { BillingPeriodSchema, BudgetLimitSchema, BudgetStatusSchema, BudgetTypeSchema, CostAlertSchema, CostAlertTypeSchema, CostAnalyticsSchema, CostBreakdownDimensionSchema, CostBreakdownEntrySchema, CostEntrySchema, CostMetricTypeSchema, CostOptimizationRecommendationSchema, CostQueryFiltersSchema, CostReportSchema } from '@objectstack/spec/ai';
16-
import type { BillingPeriod, BudgetLimit, BudgetStatus, BudgetType, CostAlert, CostAlertType, CostAnalytics, CostBreakdownDimension, CostBreakdownEntry, CostEntry, CostMetricType, CostOptimizationRecommendation, CostQueryFilters, CostReport } from '@objectstack/spec/ai';
15+
import { AIOperationCostSchema, BillingPeriodSchema, BudgetLimitSchema, BudgetStatusSchema, BudgetTypeSchema, CostAlertSchema, CostAlertTypeSchema, CostAnalyticsSchema, CostBreakdownDimensionSchema, CostBreakdownEntrySchema, CostEntrySchema, CostMetricTypeSchema, CostOptimizationRecommendationSchema, CostQueryFiltersSchema, CostReportSchema, TokenUsageSchema } from '@objectstack/spec/ai';
16+
import type { AIOperationCost, BillingPeriod, BudgetLimit, BudgetStatus, BudgetType, CostAlert, CostAlertType, CostAnalytics, CostBreakdownDimension, CostBreakdownEntry, CostEntry, CostMetricType, CostOptimizationRecommendation, CostQueryFilters, CostReport, TokenUsage } from '@objectstack/spec/ai';
1717

1818
// Validate data
19-
const result = BillingPeriodSchema.parse(data);
19+
const result = AIOperationCostSchema.parse(data);
2020
```
2121

2222
---
2323

24+
## AIOperationCost
25+
26+
### Properties
27+
28+
| Property | Type | Required | Description |
29+
| :--- | :--- | :--- | :--- |
30+
| **operationId** | `string` | ✅ | |
31+
| **operationType** | `Enum<'conversation' \| 'orchestration' \| 'prediction' \| 'rag' \| 'nlq'>` | ✅ | |
32+
| **agentName** | `string` | optional | Agent that performed the operation |
33+
| **modelId** | `string` | ✅ | |
34+
| **tokens** | `object` | ✅ | |
35+
| **cost** | `number` | ✅ | Cost in USD |
36+
| **timestamp** | `string` | ✅ | |
37+
| **metadata** | `Record<string, any>` | optional | |
38+
39+
---
40+
2441
## BillingPeriod
2542

2643
### Allowed Values
@@ -316,3 +333,15 @@ const result = BillingPeriodSchema.parse(data);
316333
| **format** | `Enum<'summary' \| 'detailed' \| 'executive'>` | optional | |
317334
| **currency** | `string` | optional | |
318335

336+
---
337+
338+
## TokenUsage
339+
340+
### Properties
341+
342+
| Property | Type | Required | Description |
343+
| :--- | :--- | :--- | :--- |
344+
| **prompt** | `integer` | ✅ | Input tokens |
345+
| **completion** | `integer` | ✅ | Output tokens |
346+
| **total** | `integer` | ✅ | Total tokens |
347+

‎content/docs/references/ai/nlq.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,8 @@ const result = EntitySchema.parse(data);
152152
| **totalCount** | `integer` | optional | |
153153
| **executionTime** | `number` | optional | Execution time in milliseconds |
154154
| **needsClarification** | `boolean` | ✅ | Whether query needs clarification |
155+
| **tokens** | `object` | optional | Token usage for this query |
156+
| **cost** | `number` | optional | Cost for this query in USD |
155157
| **suggestions** | `string[]` | optional | Query refinement suggestions |
156158

157159
---

‎content/docs/references/ai/orchestration.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,8 @@ const result = AIOrchestrationSchema.parse(data);
6969
| **tasksSucceeded** | `integer` | ✅ | Number of tasks succeeded |
7070
| **tasksFailed** | `integer` | ✅ | Number of tasks failed |
7171
| **taskResults** | `object[]` | optional | |
72+
| **tokens** | `object` | optional | Total token usage for this execution |
73+
| **cost** | `number` | optional | Total cost for this execution in USD |
7274
| **error** | `string` | optional | |
7375
| **startedAt** | `string` | ✅ | ISO timestamp |
7476
| **completedAt** | `string` | optional | ISO timestamp |

‎content/docs/references/ai/predictive.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,8 @@ const result = EvaluationMetricsSchema.parse(data);
131131
| **confidence** | `number` | optional | Confidence score (0-1) |
132132
| **probabilities** | `Record<string, number>` | optional | Class probabilities (for classification) |
133133
| **explanation** | `object` | optional | |
134+
| **tokens** | `object` | optional | Token usage for this prediction (if AI-powered) |
135+
| **cost** | `number` | optional | Cost for this prediction in USD |
134136
| **metadata** | `object` | optional | |
135137

136138
---

‎content/docs/references/ai/rag-pipeline.mdx‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,9 @@ const result = ChunkingStrategySchema.parse(data);
157157
| **query** | `string` | ✅ | |
158158
| **results** | `object[]` | ✅ | |
159159
| **context** | `string` | ✅ | Assembled context for LLM |
160-
| **tokensUsed** | `integer` | optional | |
160+
| **tokens** | `object` | optional | Token usage for this query |
161+
| **cost** | `number` | optional | Cost for this query in USD |
162+
| **tokensUsed** | `integer` | optional | Deprecated: use tokens.total instead |
161163
| **retrievalTime** | `number` | optional | Retrieval time in milliseconds |
162164

163165
---

‎packages/spec/json-schema/ai/AIOperationCost.json‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
{
2+
"$ref": "#/definitions/AIOperationCost",
3+
"definitions": {
4+
"AIOperationCost": {
5+
"type": "object",
6+
"properties": {
7+
"operationId": {
8+
"type": "string"
9+
},
10+
"operationType": {
11+
"type": "string",
12+
"enum": [
13+
"conversation",
14+
"orchestration",
15+
"prediction",
16+
"rag",
17+
"nlq"
18+
]
19+
},
20+
"agentName": {
21+
"type": "string",
22+
"description": "Agent that performed the operation"
23+
},
24+
"modelId": {
25+
"type": "string"
26+
},
27+
"tokens": {
28+
"type": "object",
29+
"properties": {
30+
"prompt": {
31+
"type": "integer",
32+
"minimum": 0,
33+
"description": "Input tokens"
34+
},
35+
"completion": {
36+
"type": "integer",
37+
"minimum": 0,
38+
"description": "Output tokens"
39+
},
40+
"total": {
41+
"type": "integer",
42+
"minimum": 0,
43+
"description": "Total tokens"
44+
}
45+
},
46+
"required": [
47+
"prompt",
48+
"completion",
49+
"total"
50+
],
51+
"additionalProperties": false
52+
},
53+
"cost": {
54+
"type": "number",
55+
"minimum": 0,
56+
"description": "Cost in USD"
57+
},
58+
"timestamp": {
59+
"type": "string",
60+
"format": "date-time"
61+
},
62+
"metadata": {
63+
"type": "object",
64+
"additionalProperties": {}
65+
}
66+
},
67+
"required": [
68+
"operationId",
69+
"operationType",
70+
"modelId",
71+
"tokens",
72+
"cost",
73+
"timestamp"
74+
],
75+
"additionalProperties": false
76+
}
77+
},
78+
"$schema": "http://json-schema.org/draft-07/schema#"
79+
}

‎packages/spec/json-schema/ai/AIOrchestrationExecutionResult.json‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,38 @@
7676
"additionalProperties": false
7777
}
7878
},
79+
"tokens": {
80+
"type": "object",
81+
"properties": {
82+
"prompt": {
83+
"type": "integer",
84+
"minimum": 0,
85+
"description": "Input tokens"
86+
},
87+
"completion": {
88+
"type": "integer",
89+
"minimum": 0,
90+
"description": "Output tokens"
91+
},
92+
"total": {
93+
"type": "integer",
94+
"minimum": 0,
95+
"description": "Total tokens"
96+
}
97+
},
98+
"required": [
99+
"prompt",
100+
"completion",
101+
"total"
102+
],
103+
"additionalProperties": false,
104+
"description": "Total token usage for this execution"
105+
},
106+
"cost": {
107+
"type": "number",
108+
"minimum": 0,
109+
"description": "Total cost for this execution in USD"
110+
},
79111
"error": {
80112
"type": "string"
81113
},

‎packages/spec/json-schema/ai/ConversationMessage.json‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -157,22 +157,32 @@
157157
"prompt": {
158158
"type": "integer",
159159
"minimum": 0,
160-
"description": "Tokens in prompt"
160+
"description": "Input tokens"
161161
},
162162
"completion": {
163163
"type": "integer",
164164
"minimum": 0,
165-
"description": "Tokens in completion"
165+
"description": "Output tokens"
166166
},
167167
"total": {
168168
"type": "integer",
169169
"minimum": 0,
170170
"description": "Total tokens"
171171
}
172172
},
173+
"required": [
174+
"prompt",
175+
"completion",
176+
"total"
177+
],
173178
"additionalProperties": false,
174179
"description": "Token usage for this message"
175180
},
181+
"cost": {
182+
"type": "number",
183+
"minimum": 0,
184+
"description": "Cost for this message in USD"
185+
},
176186
"pinned": {
177187
"type": "boolean",
178188
"default": false,

‎packages/spec/json-schema/ai/ConversationSession.json‎

Lines changed: 44 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -302,22 +302,32 @@
302302
"prompt": {
303303
"type": "integer",
304304
"minimum": 0,
305-
"description": "Tokens in prompt"
305+
"description": "Input tokens"
306306
},
307307
"completion": {
308308
"type": "integer",
309309
"minimum": 0,
310-
"description": "Tokens in completion"
310+
"description": "Output tokens"
311311
},
312312
"total": {
313313
"type": "integer",
314314
"minimum": 0,
315315
"description": "Total tokens"
316316
}
317317
},
318+
"required": [
319+
"prompt",
320+
"completion",
321+
"total"
322+
],
318323
"additionalProperties": false,
319324
"description": "Token usage for this message"
320325
},
326+
"cost": {
327+
"type": "number",
328+
"minimum": 0,
329+
"description": "Cost for this message in USD"
330+
},
321331
"pinned": {
322332
"type": "boolean",
323333
"default": false,
@@ -411,6 +421,38 @@
411421
],
412422
"additionalProperties": false
413423
},
424+
"totalTokens": {
425+
"type": "object",
426+
"properties": {
427+
"prompt": {
428+
"type": "integer",
429+
"minimum": 0,
430+
"description": "Input tokens"
431+
},
432+
"completion": {
433+
"type": "integer",
434+
"minimum": 0,
435+
"description": "Output tokens"
436+
},
437+
"total": {
438+
"type": "integer",
439+
"minimum": 0,
440+
"description": "Total tokens"
441+
}
442+
},
443+
"required": [
444+
"prompt",
445+
"completion",
446+
"total"
447+
],
448+
"additionalProperties": false,
449+
"description": "Total tokens across all messages"
450+
},
451+
"totalCost": {
452+
"type": "number",
453+
"minimum": 0,
454+
"description": "Total cost for this session in USD"
455+
},
414456
"status": {
415457
"type": "string",
416458
"enum": [

0 commit comments

Comments
 (0)