Skip to content

Commit d808ca8

Browse files
andrewklatzkeandrewklatzke
authored
feat: implements _template methods for fetching non-interpolated config (#1774)
**Requirements** - [x] I have added test coverage for new or changed functionality - [x] I have followed the repository's [pull request submission guidelines](../blob/main/CONTRIBUTING.md#submitting-pull-requests) - [x] I have validated my changes against all supported platform versions **Related issues** https://launchdarkly.atlassian.net/jira/software/c/projects/AIC/boards/2045?selectedIssue=AIC-2843 **Describe the solution you've provided** Implements three methods: - `completionConfigTemplate` - `agentConfigTemplate` - `judgeConfigTemplate` To mirror the base names. Each of these retrieves the config without running the interpolation process. **Describe alternatives you've considered** Alternatively users can use the base `.variation` method to retrieve the config, but then things like typing are foisted onto the consumer rather than the SDK itself. We're opting to add a method so that users still receive the niceties (tracker, types). **Additional context** Add any other context about the pull request here. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Additive public API on the server-ai SDK; existing interpolated config paths are unchanged and coverage is added for the new methods. > > **Overview** > Adds **`completionConfigTemplate`**, **`agentConfigTemplate`**, and **`judgeConfigTemplate`** on `LDAIClient` / `LDAIClientImpl` so callers can load typed AI configs (including `createTracker`) while keeping Mustache placeholders like `{{name}}` and `{{ldctx.key}}` verbatim—useful for previews, auditing, and storing templates for later rendering. > > Evaluation is extended with an **`interpolate`** flag on `_evaluate`; template methods call it with `interpolate: false` and emit separate **`$ld:ai:usage:*-config-template`** tracking events. Existing `completionConfig` / `agentConfig` / `judgeConfig` behavior is unchanged. > > Tests cover placeholder preservation, tracking, disabled defaults, and no `ldctx` substitution for completion and agent templates. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 1c93e6f. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent c74dae7 commit d808ca8

3 files changed

Lines changed: 331 additions & 0 deletions

File tree

‎packages/sdk/server-ai/__tests__/LDAIClientImpl.test.ts‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1100,3 +1100,211 @@ describe('tools map support', () => {
11001100
expect(result.tools).toBeUndefined();
11011101
});
11021102
});
1103+
1104+
describe('template methods', () => {
1105+
it('completionConfigTemplate preserves Mustache placeholders in messages', async () => {
1106+
const client = new LDAIClientImpl(mockLdClient);
1107+
const key = 'test-flag';
1108+
const defaultValue: LDAICompletionConfigDefault = { enabled: false };
1109+
1110+
const mockVariation = {
1111+
model: { name: 'gpt-4' },
1112+
provider: { name: 'openai' },
1113+
messages: [
1114+
{ role: 'system', content: 'Hello {{name}}' },
1115+
{ role: 'user', content: 'Score: {{score}}' },
1116+
],
1117+
_ldMeta: { variationKey: 'v1', enabled: true, mode: 'completion' },
1118+
};
1119+
mockLdClient.variation.mockResolvedValue(mockVariation);
1120+
1121+
const result = await client.completionConfigTemplate(key, testContext, defaultValue);
1122+
1123+
expect(result.messages).toEqual([
1124+
{ role: 'system', content: 'Hello {{name}}' },
1125+
{ role: 'user', content: 'Score: {{score}}' },
1126+
]);
1127+
expect(result.enabled).toBe(true);
1128+
expect(result.createTracker).toBeDefined();
1129+
});
1130+
1131+
it('completionConfigTemplate emits the correct usage tracking event', async () => {
1132+
const client = new LDAIClientImpl(mockLdClient);
1133+
const key = 'test-flag';
1134+
1135+
const mockVariation = {
1136+
messages: [{ role: 'system', content: 'Hello {{name}}' }],
1137+
_ldMeta: { variationKey: 'v1', enabled: true, mode: 'completion' },
1138+
};
1139+
mockLdClient.variation.mockResolvedValue(mockVariation);
1140+
1141+
await client.completionConfigTemplate(key, testContext);
1142+
1143+
expect(mockLdClient.track).toHaveBeenCalledWith(
1144+
'$ld:ai:usage:completion-config-template',
1145+
testContext,
1146+
key,
1147+
1,
1148+
);
1149+
});
1150+
1151+
it('completionConfigTemplate uses disabled default when no defaultValue is provided', async () => {
1152+
const client = new LDAIClientImpl(mockLdClient);
1153+
const key = 'test-flag';
1154+
1155+
const mockVariation = {
1156+
_ldMeta: { variationKey: 'v1', enabled: false, mode: 'completion' },
1157+
};
1158+
mockLdClient.variation.mockResolvedValue(mockVariation);
1159+
1160+
const result = await client.completionConfigTemplate(key, testContext);
1161+
1162+
expect(result.enabled).toBe(false);
1163+
});
1164+
1165+
it('agentConfigTemplate preserves Mustache placeholders in instructions', async () => {
1166+
const client = new LDAIClientImpl(mockLdClient);
1167+
const key = 'test-agent';
1168+
const defaultValue: LDAIAgentConfigDefault = { enabled: false };
1169+
1170+
const mockVariation = {
1171+
model: { name: 'gpt-4' },
1172+
provider: { name: 'openai' },
1173+
instructions: 'You are a research assistant specializing in {{topic}}.',
1174+
_ldMeta: { variationKey: 'v1', enabled: true, mode: 'agent' },
1175+
};
1176+
mockLdClient.variation.mockResolvedValue(mockVariation);
1177+
1178+
const result = await client.agentConfigTemplate(key, testContext, defaultValue);
1179+
1180+
expect(result.instructions).toBe('You are a research assistant specializing in {{topic}}.');
1181+
expect(result.enabled).toBe(true);
1182+
expect(result.createTracker).toBeDefined();
1183+
});
1184+
1185+
it('agentConfigTemplate emits the correct usage tracking event', async () => {
1186+
const client = new LDAIClientImpl(mockLdClient);
1187+
const key = 'test-agent';
1188+
1189+
const mockVariation = {
1190+
instructions: 'You are a {{role}}.',
1191+
_ldMeta: { variationKey: 'v1', enabled: true, mode: 'agent' },
1192+
};
1193+
mockLdClient.variation.mockResolvedValue(mockVariation);
1194+
1195+
await client.agentConfigTemplate(key, testContext);
1196+
1197+
expect(mockLdClient.track).toHaveBeenCalledWith(
1198+
'$ld:ai:usage:agent-config-template',
1199+
testContext,
1200+
key,
1201+
1,
1202+
);
1203+
});
1204+
1205+
it('agentConfigTemplate uses disabled default when no defaultValue is provided', async () => {
1206+
const client = new LDAIClientImpl(mockLdClient);
1207+
const key = 'test-agent';
1208+
1209+
const mockVariation = {
1210+
_ldMeta: { variationKey: 'v1', enabled: false, mode: 'agent' },
1211+
};
1212+
mockLdClient.variation.mockResolvedValue(mockVariation);
1213+
1214+
const result = await client.agentConfigTemplate(key, testContext);
1215+
1216+
expect(result.enabled).toBe(false);
1217+
});
1218+
1219+
it('judgeConfigTemplate preserves Mustache placeholders in messages', async () => {
1220+
const client = new LDAIClientImpl(mockLdClient);
1221+
const key = 'test-judge';
1222+
const defaultValue: LDAIJudgeConfigDefault = { enabled: false };
1223+
1224+
const mockVariation = {
1225+
model: { name: 'gpt-4' },
1226+
provider: { name: 'openai' },
1227+
evaluationMetricKey: 'relevance',
1228+
messages: [
1229+
{ role: 'system', content: 'You are a judge evaluating {{criteria}}.' },
1230+
{ role: 'user', content: 'Score: {{score}}' },
1231+
],
1232+
_ldMeta: { variationKey: 'v1', enabled: true, mode: 'judge' },
1233+
};
1234+
mockLdClient.variation.mockResolvedValue(mockVariation);
1235+
1236+
const result = await client.judgeConfigTemplate(key, testContext, defaultValue);
1237+
1238+
expect(result.messages).toEqual([
1239+
{ role: 'system', content: 'You are a judge evaluating {{criteria}}.' },
1240+
{ role: 'user', content: 'Score: {{score}}' },
1241+
]);
1242+
expect(result.enabled).toBe(true);
1243+
expect(result.createTracker).toBeDefined();
1244+
});
1245+
1246+
it('judgeConfigTemplate emits the correct usage tracking event', async () => {
1247+
const client = new LDAIClientImpl(mockLdClient);
1248+
const key = 'test-judge';
1249+
1250+
const mockVariation = {
1251+
evaluationMetricKey: 'relevance',
1252+
messages: [{ role: 'system', content: 'You are a {{role}}.' }],
1253+
_ldMeta: { variationKey: 'v1', enabled: true, mode: 'judge' },
1254+
};
1255+
mockLdClient.variation.mockResolvedValue(mockVariation);
1256+
1257+
await client.judgeConfigTemplate(key, testContext);
1258+
1259+
expect(mockLdClient.track).toHaveBeenCalledWith(
1260+
'$ld:ai:usage:judge-config-template',
1261+
testContext,
1262+
key,
1263+
1,
1264+
);
1265+
});
1266+
1267+
it('judgeConfigTemplate uses disabled default when no defaultValue is provided', async () => {
1268+
const client = new LDAIClientImpl(mockLdClient);
1269+
const key = 'test-judge';
1270+
1271+
const mockVariation = {
1272+
_ldMeta: { variationKey: 'v1', enabled: false, mode: 'judge' },
1273+
};
1274+
mockLdClient.variation.mockResolvedValue(mockVariation);
1275+
1276+
const result = await client.judgeConfigTemplate(key, testContext);
1277+
1278+
expect(result.enabled).toBe(false);
1279+
});
1280+
1281+
it('completionConfigTemplate does not apply ldctx interpolation', async () => {
1282+
const client = new LDAIClientImpl(mockLdClient);
1283+
const key = 'test-flag';
1284+
1285+
const mockVariation = {
1286+
messages: [{ role: 'system', content: 'User: {{ldctx.key}}' }],
1287+
_ldMeta: { variationKey: 'v1', enabled: true, mode: 'completion' },
1288+
};
1289+
mockLdClient.variation.mockResolvedValue(mockVariation);
1290+
1291+
const result = await client.completionConfigTemplate(key, testContext);
1292+
1293+
expect(result.messages?.[0].content).toBe('User: {{ldctx.key}}');
1294+
});
1295+
1296+
it('agentConfigTemplate does not apply ldctx interpolation', async () => {
1297+
const client = new LDAIClientImpl(mockLdClient);
1298+
const key = 'test-agent';
1299+
1300+
const mockVariation = {
1301+
instructions: 'Context key: {{ldctx.key}}',
1302+
_ldMeta: { variationKey: 'v1', enabled: true, mode: 'agent' },
1303+
};
1304+
mockLdClient.variation.mockResolvedValue(mockVariation);
1305+
1306+
const result = await client.agentConfigTemplate(key, testContext);
1307+
1308+
expect(result.instructions).toBe('Context key: {{ldctx.key}}');
1309+
});
1310+
});

‎packages/sdk/server-ai/src/LDAIClientImpl.ts‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,11 +37,14 @@ import { aiSdkLanguage, aiSdkName, aiSdkVersion } from './sdkInfo';
3737
*/
3838
const TRACK_SDK_INFO = '$ld:ai:sdk:info';
3939
const TRACK_USAGE_COMPLETION_CONFIG = '$ld:ai:usage:completion-config';
40+
const TRACK_USAGE_COMPLETION_CONFIG_TEMPLATE = '$ld:ai:usage:completion-config-template';
4041
const TRACK_USAGE_CREATE_CHAT = '$ld:ai:usage:create-chat';
4142
const TRACK_USAGE_CREATE_AGENT = '$ld:ai:usage:create-agent';
4243
const TRACK_USAGE_JUDGE_CONFIG = '$ld:ai:usage:judge-config';
44+
const TRACK_USAGE_JUDGE_CONFIG_TEMPLATE = '$ld:ai:usage:judge-config-template';
4345
const TRACK_USAGE_CREATE_JUDGE = '$ld:ai:usage:create-judge';
4446
const TRACK_USAGE_AGENT_CONFIG = '$ld:ai:usage:agent-config';
47+
const TRACK_USAGE_AGENT_CONFIG_TEMPLATE = '$ld:ai:usage:agent-config-template';
4548
const TRACK_USAGE_AGENT_CONFIGS = '$ld:ai:usage:agent-configs';
4649
const TRACK_USAGE_AGENT_GRAPH = '$ld:ai:usage:agent-graph';
4750

@@ -84,6 +87,7 @@ export class LDAIClientImpl implements LDAIClient {
8487
variables?: Record<string, unknown>,
8588
graphKey?: string,
8689
defaultAiProvider?: SupportedAIProvider,
90+
interpolate: boolean = true,
8791
): Promise<LDAIConfigKind> {
8892
const ldFlagValue = LDAIConfigUtils.toFlagValue(defaultValue, mode);
8993

@@ -127,6 +131,10 @@ export class LDAIClientImpl implements LDAIClient {
127131

128132
const config = LDAIConfigUtils.fromFlagValue(key, value, trackerFactory, evaluator);
129133

134+
if (!interpolate) {
135+
return config;
136+
}
137+
130138
// Apply variable interpolation (always needed for ldctx)
131139
return this._applyInterpolation(config, context, variables);
132140
}
@@ -313,6 +321,62 @@ export class LDAIClientImpl implements LDAIClient {
313321
);
314322
}
315323

324+
async completionConfigTemplate(
325+
key: string,
326+
context: LDContext,
327+
defaultValue?: LDAICompletionConfigDefault,
328+
defaultAiProvider?: SupportedAIProvider,
329+
): Promise<LDAICompletionConfig> {
330+
this._ldClient.track(TRACK_USAGE_COMPLETION_CONFIG_TEMPLATE, context, key, 1);
331+
return (await this._evaluate(
332+
key,
333+
context,
334+
defaultValue ?? disabledAIConfig,
335+
'completion',
336+
undefined,
337+
undefined,
338+
defaultAiProvider,
339+
false,
340+
)) as LDAICompletionConfig;
341+
}
342+
343+
async agentConfigTemplate(
344+
key: string,
345+
context: LDContext,
346+
defaultValue?: LDAIAgentConfigDefault,
347+
defaultAiProvider?: SupportedAIProvider,
348+
): Promise<LDAIAgentConfig> {
349+
this._ldClient.track(TRACK_USAGE_AGENT_CONFIG_TEMPLATE, context, key, 1);
350+
return (await this._evaluate(
351+
key,
352+
context,
353+
defaultValue ?? disabledAIConfig,
354+
'agent',
355+
undefined,
356+
undefined,
357+
defaultAiProvider,
358+
false,
359+
)) as LDAIAgentConfig;
360+
}
361+
362+
async judgeConfigTemplate(
363+
key: string,
364+
context: LDContext,
365+
defaultValue?: LDAIJudgeConfigDefault,
366+
): Promise<LDAIJudgeConfig> {
367+
this._ldClient.track(TRACK_USAGE_JUDGE_CONFIG_TEMPLATE, context, key, 1);
368+
return (await this._evaluate(
369+
key,
370+
context,
371+
defaultValue ?? disabledAIConfig,
372+
'judge',
373+
undefined,
374+
undefined,
375+
undefined,
376+
false,
377+
)) as LDAIJudgeConfig;
378+
}
379+
316380
async agentConfigs<const T extends readonly LDAIAgentRequestConfig[]>(
317381
agentConfigs: T,
318382
context: LDContext,

‎packages/sdk/server-ai/src/api/LDAIClient.ts‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,65 @@ export interface LDAIClient {
6767
defaultAiProvider?: SupportedAIProvider,
6868
): Promise<LDAICompletionConfig>;
6969

70+
/**
71+
* Retrieves a completion AI Config with Mustache placeholders left intact (no interpolation).
72+
* Useful for displaying prompt previews or storing templates for later rendering.
73+
*
74+
* @param key The key of the AI Config.
75+
* @param context The LaunchDarkly context object.
76+
* @param defaultValue Optional fallback when the configuration is not available from LaunchDarkly.
77+
* When omitted or null, a disabled default is used.
78+
*
79+
* @returns An {@link LDAICompletionConfig} identical to {@link completionConfig} except that
80+
* `messages[].content` strings are stored verbatim from the flag variation — Mustache
81+
* placeholders such as `{{variable}}` are preserved.
82+
*/
83+
completionConfigTemplate(
84+
key: string,
85+
context: LDContext,
86+
defaultValue?: LDAICompletionConfigDefault,
87+
defaultAiProvider?: SupportedAIProvider,
88+
): Promise<LDAICompletionConfig>;
89+
90+
/**
91+
* Retrieves an agent AI Config with Mustache placeholders left intact (no interpolation).
92+
* Useful for auditing instruction templates or building UI previews.
93+
*
94+
* @param key The key of the AI Config agent.
95+
* @param context The LaunchDarkly context object.
96+
* @param defaultValue Optional fallback when the configuration is not available from LaunchDarkly.
97+
* When omitted or null, a disabled default is used.
98+
*
99+
* @returns An {@link LDAIAgentConfig} identical to {@link agentConfig} except that
100+
* the `instructions` string is stored verbatim from the flag variation — Mustache
101+
* placeholders such as `{{topic}}` are preserved.
102+
*/
103+
agentConfigTemplate(
104+
key: string,
105+
context: LDContext,
106+
defaultValue?: LDAIAgentConfigDefault,
107+
defaultAiProvider?: SupportedAIProvider,
108+
): Promise<LDAIAgentConfig>;
109+
110+
/**
111+
* Retrieves a judge AI Config with Mustache placeholders left intact (no interpolation).
112+
* Useful for auditing judge prompt templates.
113+
*
114+
* @param key The key of the Judge AI Config.
115+
* @param context The LaunchDarkly context object.
116+
* @param defaultValue Optional fallback when the configuration is not available from LaunchDarkly.
117+
* When omitted or null, a disabled default is used.
118+
*
119+
* @returns An {@link LDAIJudgeConfig} identical to {@link judgeConfig} except that
120+
* `messages[].content` strings are stored verbatim from the flag variation — Mustache
121+
* placeholders are preserved.
122+
*/
123+
judgeConfigTemplate(
124+
key: string,
125+
context: LDContext,
126+
defaultValue?: LDAIJudgeConfigDefault,
127+
): Promise<LDAIJudgeConfig>;
128+
70129
/**
71130
* Retrieves and processes a single AI Config agent based on the provided key, LaunchDarkly context,
72131
* and variables. This includes the model configuration and the customized instructions.

0 commit comments

Comments
 (0)