Skip to content

feat(util-genai): refactor and make API smaller and more user-friendly#4391

Open
lmolkova wants to merge 15 commits intoopen-telemetry:mainfrom
lmolkova:genai-utils-api-cleanup
Open

feat(util-genai): refactor and make API smaller and more user-friendly#4391
lmolkova wants to merge 15 commits intoopen-telemetry:mainfrom
lmolkova:genai-utils-api-cleanup

Conversation

@lmolkova
Copy link
Copy Markdown
Member

@lmolkova lmolkova commented Apr 3, 2026

Replaces the handler.start_llm(invocation) / handler.stop_llm(invocation) pattern with factory methods that construct and start spans in one call, and moves lifecycle ownership onto the invocation object. Existing code continues to work — LLMInvocation, stop_llm, and fail_llm are kept with @deprecated markers.

Before

inv = LLMInvocation(provider="openai", request_model="gpt-4o")
# inv is in invalid state now
handler.start_llm(inv)
try:
    inv.output_messages = [...]
    # need to carry both - handler and inv to here
    handler.stop_llm(inv)
except Exception as exc:
   # why not just pass exc ?
    handler.fail_llm(inv, Error(type=type(exc), message=str(exc)))

After

inv = handler.start_inference("openai", "gpt-4o")
# valid state right away
try:
    inv.output_messages = [...]
    # no need to pass handler here if happens in a different method
    inv.stop()
except Exception as exc:
    # much shorter
    inv.fail(exc); 
    raise
with handler.inference("openai", "gpt-4o") as inv:
    inv.output_messages = [...]

@lmolkova lmolkova force-pushed the genai-utils-api-cleanup branch from 0fc33e7 to b8e7e4e Compare April 3, 2026 05:46
@xrmx xrmx added the gen-ai Related to generative AI label Apr 3, 2026
Copy link
Copy Markdown
Member

@MikeGoldsmith MikeGoldsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I've left one small suggestion for separating invocation types file, but not blocking.

I see you have a few TODOs you'd like to probably resolve before merging too.

@lmolkova
Copy link
Copy Markdown
Member Author

lmolkova commented Apr 3, 2026

@MikeGoldsmith and @DylanRussell thanks for the review!

The todos are the existing ones - it's easy to a address them but I didn't want to include any new features into this pure refactoring PR.

@lmolkova lmolkova force-pushed the genai-utils-api-cleanup branch from 392ae45 to 39ffa9d Compare April 4, 2026 17:36
@lmolkova
Copy link
Copy Markdown
Member Author

lmolkova commented Apr 7, 2026

/cc @Cirilla-zmh

Copy link
Copy Markdown
Member

@MikeGoldsmith MikeGoldsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still looks good but has some conflicts that need resolving first.

@MikeGoldsmith MikeGoldsmith moved this to Approved PRs that need fixes in Python PR digest Apr 8, 2026
@lmolkova lmolkova force-pushed the genai-utils-api-cleanup branch from 4d4e31d to 320bfd0 Compare April 9, 2026 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gen-ai Related to generative AI

Projects

Status: Approved PRs that need fixes

Development

Successfully merging this pull request may close these issues.

6 participants