Skip to content

Commit e4efdad

Browse files
authored
Added Synthetic Tool Function Call Tool (#85)
* added synthetic tool * refactor synthetic tool files * added synthetic tool doc * modified pyproject.toml
1 parent 2e3ab89 commit e4efdad

5 files changed

Lines changed: 992 additions & 1 deletion

File tree

docs/docs/user-guide/tools/function-call.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,6 +296,56 @@ This tool automatically discovers available functions from connected MCP servers
296296

297297
By integrating `AgentCallingTool` into your event-driven workflow, you can build sophisticated multi-agent systems where each agent can be invoked seamlessly via structured function calls. This approach maintains a clear separation between the LLM's orchestration and the agents' invoke details.
298298

299+
## Synthetic Tool
300+
301+
`SyntheticTool` extends the `FunctionCallTool` that enables LLMs to generate synthetic or modeled data by leveraging another LLM as a data generator. Unlike traditional function call tools that execute predefined logic, `SyntheticTool` uses an LLM to produce plausible, schema-compliant, outputs based on input specifications—perfect for testing, prototyping, or generating realistic mock data.
302+
303+
**Fields**:
304+
305+
| Field | Description |
306+
|------------------------|--------------------------------------------------------------------------------------------------------------------|
307+
| `name` | Descriptive identifier, defaults to `"SyntheticTool"`. |
308+
| `type` | Tool type indicator, defaults to `"SyntheticTool"`. |
309+
| `tool_name` | Name used for function registration and LLM tool calls. |
310+
| `description` | Explanation of what synthetic data this tool generates. |
311+
| `input_model` | Pydantic `BaseModel` class or JSON schema dict defining expected input structure. |
312+
| `output_model` | Pydantic `BaseModel` class or JSON schema dict defining generated output structure. |
313+
| `model` | OpenAI model to use for data generation (e.g., `"gpt-4o-mini"`). |
314+
| `openai_api_key` | API key for OpenAI authentication. |
315+
| `oi_span_type` | OpenInference semantic attribute (`TOOL`), enabling observability and traceability. |
316+
317+
**Methods**:
318+
319+
| Method | Description |
320+
|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
321+
| `get_function_specs` | Returns the function specification (name, description, input schema) for the synthetic. tool |
322+
| `invoke` | Processes tool calls, generates synthetic data via LLM, returns schema-compliant JSON responses. |
323+
| `ensure_strict_schema` | Static method that recursively adds `additionalProperties: false` to JSON schemas for OpenAI strict mode compatibility. |
324+
| `to_dict` | Serializes all relevant fields, including agent metadata and the assigned callable, for debugging or persistence. |
325+
326+
**Workflow Example**:
327+
328+
1. **Schema Definition**: Define input and output schemas using either Pydantic models (type-safe Python) or JSON Schema dicts (flexible).
329+
2. **Function Registration**: The tool automatically generates the `FunctionSpec`, enabling LLMs to discover and call the tool.
330+
3. **Tool Invocation**: When an LLM invokes the tool with arguments:
331+
- Arguments are validated against `input_model` schema
332+
- A prompt is constructed with input/output schema specifications
333+
- LLM generates synthetic data conforming to `output_model`
334+
4. **Structured Output**:
335+
- **Pydantic Mode**: Uses OpenAI's `beta.chat.completions.parse()` with type safety
336+
- **JSON Schema Mode**: Uses `chat.completions.create()` with `strict: True` for schema validation
337+
5. **Response Handling**: Generated data is returned as a `Message` object linked to the original `tool_call_id`.
338+
339+
**Usage and Customization**:
340+
341+
- **Flexible Schema Definition**: By supporting both Pydantic models and JSON schemas, you can choose type-safe Python development or dynamic schema-based configuration without changing the rest of your workflow.
342+
- **Runtime Model Selection**: Easily swap between OpenAI models (e.g., `gpt-5-mini` for cost, `gpt-5` for quality) to balance generation quality and API costs without modifying tool logic.
343+
- **Schema-Driven Generation**: Input and output schemas guide the LLM's data generation, ensuring consistent, validated outputs that conform to your exact specifications.
344+
- **Composable Data Pipelines**: Chain multiple `SyntheticTool` instances where one tool's output becomes another's input, creating sophisticated data generation workflows.
345+
346+
With SyntheticTool, you can rapidly prototype data-driven workflows without building actual data sources, while maintaining full schema compliance and
347+
type safety through Pydantic or JSON Schema validation.
348+
299349
## Best Practices
300350

301351
### Function Design
Lines changed: 340 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,340 @@
1+
import inspect
2+
import json
3+
from typing import Any
4+
from typing import AsyncGenerator
5+
from typing import Dict
6+
from typing import List
7+
8+
from openai import OpenAIError
9+
from openinference.semconv.trace import OpenInferenceSpanKindValues
10+
from pydantic import BaseModel
11+
from pydantic import field_validator
12+
13+
from grafi.common.decorators.record_decorators import record_tool_invoke
14+
from grafi.common.models.function_spec import FunctionSpec
15+
from grafi.common.models.function_spec import ParametersSchema
16+
from grafi.common.models.invoke_context import InvokeContext
17+
from grafi.common.models.message import Message
18+
from grafi.common.models.message import Messages
19+
from grafi.tools.function_calls.function_call_tool import FunctionCallTool
20+
from grafi.tools.function_calls.function_call_tool import FunctionCallToolBuilder
21+
22+
23+
try:
24+
from openai import AsyncOpenAI
25+
except ImportError:
26+
raise ImportError(
27+
"`openai` not installed. Please install using `pip install openai`"
28+
)
29+
30+
31+
class SyntheticTool(FunctionCallTool):
32+
name: str = "SyntheticTool"
33+
type: str = "SyntheticTool"
34+
tool_name: str = ""
35+
description: str = ""
36+
input_model: Any = ""
37+
output_model: Any = ""
38+
model: str = ""
39+
openai_api_key: str = ""
40+
oi_span_type: OpenInferenceSpanKindValues = OpenInferenceSpanKindValues.TOOL
41+
42+
@field_validator("input_model", "output_model")
43+
@classmethod
44+
def validate_pydantic_model_or_schema(cls, v: Any, info) -> Any:
45+
"""
46+
Validate that input_model and output_model are either:
47+
- A Pydantic BaseModel class (not instance) - for type-safe Python usage
48+
- A JSON schema dict - for flexible schema definition
49+
- An empty string (for optional models)
50+
51+
Both Pydantic models and JSON schemas are fully supported for LLM invocation
52+
with strict validation enabled.
53+
54+
Args:
55+
v: The value to validate
56+
info: Pydantic validation info containing field name
57+
58+
Returns:
59+
The validated value
60+
61+
Raises:
62+
ValueError: If the value is not a valid type (e.g., int, str, instances)
63+
"""
64+
if v == "":
65+
return v
66+
67+
if isinstance(v, dict):
68+
return v
69+
70+
if inspect.isclass(v) and issubclass(v, BaseModel):
71+
return v
72+
73+
field_name = info.field_name
74+
raise ValueError(
75+
f"{field_name} must be a Pydantic BaseModel class, "
76+
f"a dict schema, or an empty string. "
77+
f"Got: {type(v).__name__}"
78+
)
79+
80+
def model_post_init(self, _context: Any) -> None:
81+
if self.input_model:
82+
# Handle both dict schemas and Pydantic models
83+
if isinstance(self.input_model, dict):
84+
input_schema = self.input_model
85+
else:
86+
input_schema = self.input_model.model_json_schema()
87+
88+
self.function_specs.append(
89+
FunctionSpec(
90+
name=self.tool_name,
91+
description=self.description,
92+
parameters=ParametersSchema(**input_schema),
93+
)
94+
)
95+
96+
@property
97+
def input_schema(self) -> Dict[str, Any]:
98+
"""Get input schema from Pydantic model."""
99+
if self.input_model:
100+
if isinstance(self.input_model, dict):
101+
return self.input_model
102+
return self.input_model.model_json_schema()
103+
return {}
104+
105+
@property
106+
def output_schema(self) -> Dict[str, Any]:
107+
"""Get output schema from Pydantic model."""
108+
if self.output_model:
109+
if isinstance(self.output_model, dict):
110+
return self.output_model
111+
return self.output_model.model_json_schema()
112+
return {}
113+
114+
@classmethod
115+
def builder(cls) -> "SyntheticToolBuilder":
116+
"""
117+
Return a builder for SyntheticTool.
118+
This method allows for the construction of an SyntheticTool instance with specified parameters.
119+
"""
120+
return SyntheticToolBuilder(cls)
121+
122+
@record_tool_invoke
123+
async def invoke(
124+
self,
125+
invoke_context: InvokeContext,
126+
input_data: Messages,
127+
) -> AsyncGenerator[Messages, None]:
128+
"""
129+
Invokes the synthetic tool by processing incoming tool calls and generating
130+
LLM-based responses for each matching invocation.
131+
132+
Args:
133+
invoke_context (InvokeContext): The context for this invocation.
134+
input_data (Messages): A list of incoming messages that may contain tool calls.
135+
136+
Yields:
137+
AsyncGenerator[Messages, None]: A stream of messages representing the
138+
responses from the LLM for each valid tool call.
139+
140+
Raises:
141+
ValueError: If no tool_calls are found in the input data.
142+
"""
143+
input_msg = input_data[0]
144+
if input_msg.tool_calls is None:
145+
raise ValueError("No tool_calls found for SyntheticTool invocation.")
146+
147+
messages: List[Message] = []
148+
149+
for tool_call in input_msg.tool_calls:
150+
if tool_call.function.name != self.tool_name:
151+
continue
152+
153+
args = json.loads(tool_call.function.arguments)
154+
prompt = self._make_prompt(args)
155+
response = await self._call_llm(prompt)
156+
messages.extend(
157+
self.to_messages(response=response, tool_call_id=tool_call.id)
158+
)
159+
160+
yield messages
161+
162+
def _make_prompt(self, user_input: Dict[str, Any]) -> str:
163+
"""Builds the synthetic execution prompt."""
164+
return f"""
165+
You are a synthetic tool named "{self.tool_name}".
166+
Description: {self.description}
167+
168+
INPUT SCHEMA:
169+
{json.dumps(self.input_schema, indent=2)}
170+
171+
OUTPUT SCHEMA:
172+
{json.dumps(self.output_schema, indent=2)}
173+
174+
USER INPUT:
175+
{json.dumps(user_input, indent=2)}
176+
177+
Return ONLY a JSON object that strictly conforms to the OUTPUT schema.
178+
"""
179+
180+
@staticmethod
181+
def ensure_strict_schema(schema: Dict[str, Any]) -> Dict[str, Any]:
182+
"""
183+
Recursively ensure schema is compatible with OpenAI strict mode.
184+
185+
Adds 'additionalProperties': false to all objects, which is required
186+
for OpenAI's structured outputs strict mode.
187+
188+
Args:
189+
schema: JSON schema dict
190+
191+
Returns:
192+
Modified schema with strict mode requirements
193+
"""
194+
schema = schema.copy()
195+
196+
if schema.get("type") == "object":
197+
schema["additionalProperties"] = False
198+
if "properties" in schema:
199+
schema["properties"] = {
200+
k: SyntheticTool.ensure_strict_schema(v)
201+
for k, v in schema["properties"].items()
202+
}
203+
elif schema.get("type") == "array":
204+
if "items" in schema:
205+
schema["items"] = SyntheticTool.ensure_strict_schema(schema["items"])
206+
207+
return schema
208+
209+
async def _call_llm(self, prompt: str) -> str:
210+
"""
211+
Calls OpenAI with structured output.
212+
Supports both Pydantic models and JSON schemas.
213+
"""
214+
try:
215+
if not self.output_model:
216+
raise ValueError("output_model must be set to call LLM")
217+
218+
client = AsyncOpenAI(api_key=self.openai_api_key)
219+
220+
# If output model is json (dict)
221+
if isinstance(self.output_model, dict):
222+
# Ensure schema is compatible with strict mode
223+
strict_schema = self.ensure_strict_schema(self.output_model)
224+
225+
response_format = {
226+
"type": "json_schema",
227+
"json_schema": {
228+
"name": f"{self.tool_name}_output",
229+
"schema": strict_schema,
230+
"strict": True,
231+
},
232+
}
233+
234+
# Use standard chat completion (not parse)
235+
completion = await client.chat.completions.create(
236+
model=self.model,
237+
messages=[{"role": "user", "content": prompt}],
238+
response_format=response_format,
239+
)
240+
241+
content = completion.choices[0].message.content
242+
if not content:
243+
return json.dumps({"error": "Empty response"})
244+
245+
return content
246+
247+
# If output model is pydantic model
248+
else:
249+
# Use Pydantic mode with parse
250+
completion = await client.beta.chat.completions.parse(
251+
model=self.model,
252+
messages=[{"role": "user", "content": prompt}],
253+
response_format=self.output_model,
254+
)
255+
256+
parsed_response = completion.choices[0].message.parsed
257+
258+
if not parsed_response:
259+
return json.dumps({"error": "Empty response"})
260+
261+
# Return as JSON string
262+
return parsed_response.model_dump_json()
263+
264+
except OpenAIError as exc:
265+
return json.dumps({"error": f"OpenAI API error: {str(exc)}"})
266+
267+
except Exception as e:
268+
return json.dumps({"error": f"LLM call failed: {str(e)}"})
269+
270+
def to_dict(self) -> Dict[str, Any]:
271+
"""
272+
Convert the tool instance to a dictionary representation.
273+
274+
Returns:
275+
Dict[str, Any]: A dictionary representation of the tool.
276+
"""
277+
return {
278+
**super().to_dict(),
279+
"tool_name": self.tool_name,
280+
"description": self.description,
281+
"input_schema": self.input_schema,
282+
"output_schema": self.output_schema,
283+
"model": self.model,
284+
}
285+
286+
@classmethod
287+
async def from_dict(cls, data: Dict[str, Any]) -> "SyntheticTool":
288+
"""
289+
Create a SyntheticTool instance from a dictionary representation.
290+
291+
Args:
292+
data (dict[str, Any]): A dictionary representation of the SyntheticTool.
293+
294+
Returns:
295+
SyntheticTool: A SyntheticTool instance created from the dictionary.
296+
297+
Note:
298+
The client needs to be recreated with an API key from environment
299+
or other secure source as API keys are masked in serialization.
300+
"""
301+
return (
302+
cls.builder()
303+
.tool_name(data.get("tool_name", "synthetic_tool"))
304+
.description(data.get("description", ""))
305+
.input_model(data.get("input_schema", {}))
306+
.output_model(data.get("output_schema", {}))
307+
.model(data.get("model", "gpt-5-mini"))
308+
.openai_api_key(data.get("openai_api_key", ""))
309+
.oi_span_type(OpenInferenceSpanKindValues(data.get("oi_span_type", "TOOL")))
310+
.build()
311+
)
312+
313+
314+
class SyntheticToolBuilder(FunctionCallToolBuilder[SyntheticTool]):
315+
"""Builder for SyntheticTool instances."""
316+
317+
def tool_name(self, name: str) -> "SyntheticToolBuilder":
318+
self.kwargs["tool_name"] = name
319+
self.kwargs["name"] = name
320+
return self
321+
322+
def description(self, desc: str) -> "SyntheticToolBuilder":
323+
self.kwargs["description"] = desc
324+
return self
325+
326+
def input_model(self, model: type[BaseModel]) -> "SyntheticToolBuilder":
327+
self.kwargs["input_model"] = model
328+
return self
329+
330+
def output_model(self, model: type[BaseModel]) -> "SyntheticToolBuilder":
331+
self.kwargs["output_model"] = model
332+
return self
333+
334+
def model(self, model: str) -> "SyntheticToolBuilder":
335+
self.kwargs["model"] = model
336+
return self
337+
338+
def openai_api_key(self, openai_api_key: str) -> "SyntheticToolBuilder":
339+
self.kwargs["openai_api_key"] = openai_api_key
340+
return self

0 commit comments

Comments
 (0)