Skip to content
Open
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
9859d08
feat: implements optimize method in SDK, code moved
andrewklatzke Mar 26, 2026
1712e4f
feat: implementation of agent optimization + tests
andrewklatzke Mar 31, 2026
ea596a7
feat: implement ability to use completions or agents for judge calls
andrewklatzke Mar 31, 2026
2fd55e2
feat: all logs -> debug
andrewklatzke Mar 31, 2026
8481690
fix: lints + structured output tool rename
andrewklatzke Mar 31, 2026
f8e5509
fix: lint + missed variable rename
andrewklatzke Mar 31, 2026
c032aaf
fix: sort imports
andrewklatzke Mar 31, 2026
aee6aa7
fix: lint
andrewklatzke Mar 31, 2026
59c7ac7
chore: break up long lines, add spaces where necessary
andrewklatzke Mar 31, 2026
59f03f2
chore: break up another long line
andrewklatzke Mar 31, 2026
e2ff561
chore: fix on_turn path
andrewklatzke Mar 31, 2026
af2dd03
chore: move prompts to own file, better debug info
andrewklatzke Mar 31, 2026
ea43575
chore: update tests, fix cursor feedback
andrewklatzke Mar 31, 2026
2fecd54
feat: implements LD API client, optimize_from_config path
andrewklatzke Apr 1, 2026
d3e1f96
feat: partially implement optimize_from_config
andrewklatzke Apr 1, 2026
44c8c59
feat: ground truth optimization path
andrewklatzke Apr 3, 2026
8f9f1e2
feat: prevent overfitting via prompt changes and post-processing
andrewklatzke Apr 7, 2026
a17fd6e
chore: remove some dead code
andrewklatzke Apr 7, 2026
67fdbf1
chore: remove provided_tool_handlers code
andrewklatzke Apr 7, 2026
3042984
fix: adjust iteration logic so validation doesn't consume them
andrewklatzke Apr 8, 2026
288336e
feat: implement latency & token tracking for optimizations
andrewklatzke Apr 8, 2026
5d76276
feat: add optimization for duration
andrewklatzke Apr 8, 2026
4cb8859
feat: add auto-commit option
andrewklatzke Apr 9, 2026
ba369a2
chore: add tests
andrewklatzke Apr 9, 2026
149aa76
chore: various fixes, improvements for optimization package
andrewklatzke Apr 15, 2026
31c8385
feat: add shared dataclass for calls so they can be handled by same h…
andrewklatzke Apr 15, 2026
55674ae
chore: improve call config, context so they're passable as a single t…
andrewklatzke Apr 16, 2026
8f3468f
fix: success path + add test, cursor feedback
andrewklatzke Apr 16, 2026
7074cfa
feat: dx improvements for optimization package
andrewklatzke Apr 16, 2026
a386a27
chore: update types for lint
andrewklatzke Apr 17, 2026
8d1a868
lint, cursor feedback
andrewklatzke Apr 17, 2026
937542a
chore: additional lint
andrewklatzke Apr 17, 2026
8b3c69f
chore: rename package, imports, and address security review
andrewklatzke Apr 22, 2026
f7ce6d3
merge + conflicts
andrewklatzke Apr 22, 2026
7468372
chore: cursor feedback, remove unnecessary log of response
andrewklatzke Apr 22, 2026
a387f83
chore: fix readme inconsistency
andrewklatzke Apr 22, 2026
312161f
chore: add retry logic and structural validation for variations
andrewklatzke Apr 22, 2026
542c135
chore: remove coolnames, implement generate_slug() cleanup unused files
andrewklatzke Apr 22, 2026
3bce893
chore: removes cleanup unused tool function, remove additionalProperties
andrewklatzke Apr 22, 2026
e8c6692
feat: add token limit handling
andrewklatzke Apr 23, 2026
572a2aa
fix: better handling of params and custom params for optimization
andrewklatzke May 4, 2026
0ed243e
feat: adds ability to use inverted judges
andrewklatzke May 5, 2026
a8f14de
feat: use judge_passed for all calcs
andrewklatzke May 5, 2026
23baeb4
feat: adds ability to use inverted judges (#168)
andrewklatzke May 6, 2026
94de596
feat: adds ability to optimize for cost
andrewklatzke May 6, 2026
dc82818
fix: remove unnecessary token path
andrewklatzke May 6, 2026
92f51fa
fix: better handling of params and custom params for optimization (#163)
andrewklatzke May 7, 2026
365fa94
feat: adds reporting for cost and latency optimization failures
andrewklatzke May 7, 2026
d267832
feat: adds reporting for cost and latency optimization failures (#180)
andrewklatzke May 12, 2026
9bedf9e
fix: don't only evaluate final input in GT results
andrewklatzke May 13, 2026
53f455f
fix: don't only evaluate final input in GT results
andrewklatzke May 13, 2026
66bc1f0
fix: only strip known provider prefixes
andrewklatzke May 13, 2026
f2f0894
fix: address cursor feedback
andrewklatzke May 14, 2026
9c1d8d7
chore: adjust package name
andrewklatzke May 18, 2026
4fc1ecf
fix: pull model configs if available in options path
andrewklatzke May 18, 2026
32dc4d0
chore: (optimization) add provenance file + register with release please
andrewklatzke May 18, 2026
3b4baa3
feat: adds ability to optimize for cost (#172)
andrewklatzke May 18, 2026
d15c6bc
chore: (optimization) add provenance file + register with release ple…
andrewklatzke May 18, 2026
99dd7b8
Merge branch 'main' into aklatzke/AIC-2263/sdk-dx-improvements
andrewklatzke May 20, 2026
849d41f
chore: lint
andrewklatzke May 20, 2026
c51e235
Merge branch 'aklatzke/AIC-2263/sdk-dx-improvements' of github.com:la…
andrewklatzke May 20, 2026
f481ed3
chore: isort
andrewklatzke May 20, 2026
27b248e
chore: line lengths
andrewklatzke May 20, 2026
4545203
chore: fix agent init typing
andrewklatzke May 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/optimization/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ classifiers = [
]
dependencies = [
"launchdarkly-server-sdk-ai>=0.16.0",
"coolname>=2.0.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this? Why such an old version? Is having cool names for variation keys worth the risk of this dependency? Would the 'cool' names even be useful/meaningful, vs a default date-based name?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed in review meeting; will just hand roll this as its pretty simple to implement

]

[project.urls]
Expand Down
35 changes: 33 additions & 2 deletions packages/optimization/src/ldai_optimization/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,42 @@
This package will provide helpers to run selected tools against the LaunchDarkly API from SDK-based workflows.
"""

from ldai_optimization.client import ApiAgentOptimizationClient
from ldai.tracker import TokenUsage

from ldai_optimization.client import OptimizationClient
from ldai_optimization.dataclasses import (
AIJudgeCallConfig,
GroundTruthOptimizationOptions,
GroundTruthSample,
LLMCallConfig,
LLMCallContext,
OptimizationContext,
OptimizationFromConfigOptions,
OptimizationJudge,
OptimizationJudgeContext,
OptimizationOptions,
OptimizationResponse,
ToolDefinition,
)
from ldai_optimization.ld_api_client import LDApiError

__version__ = "0.0.0"

__all__ = [
'__version__',
'ApiAgentOptimizationClient',
'AIJudgeCallConfig',
'GroundTruthOptimizationOptions',
'GroundTruthSample',
'LDApiError',
'LLMCallConfig',
'LLMCallContext',
'OptimizationClient',
'OptimizationContext',
'OptimizationFromConfigOptions',
'OptimizationJudge',
'OptimizationJudgeContext',
'OptimizationOptions',
'OptimizationResponse',
'TokenUsage',
'ToolDefinition',
]
Loading
Loading