Skip to content

Pass session (with app_id) to model plugin instances (parity with tool plugins) #311

@ryuta-kobayashi-ug

Description

@ryuta-kobayashi-ug

This is the SDK-side change for langgenius/dify#35772.

Problem

When the plugin daemon dispatches a model invocation (LLM, Embedding, Rerank, etc.), it creates a Session that already contains app_id — the daemon uses a shared InvokePluginRequest struct for both tool and model dispatch, so app_id arrives in the Session correctly.

However, plugin_executor.py does not forward this session to model plugin instances. It passes session to tool plugins but not to model plugins:

# Tool invocation — session IS passed
def invoke_tool(self, session: Session, request):
    tool = tool_cls(
        runtime=ToolRuntime(...),
        session=session,  # ← passed
    )
    yield from tool.invoke(request.tool_parameters)

# Model invocation — session is NOT passed
def invoke_llm(self, session: Session, data):
    model_instance = self.registration.get_model_instance(...)
    return model_instance.invoke(
        data.model,
        data.credentials,
        data.prompt_messages,
        data.model_parameters,
        data.tools,
        data.stop,
        data.stream,
        data.user_id,
        # ← session is not forwarded
    )

This means model plugins cannot access session.app_id, which is needed for provider-side cost attribution (e.g., Bedrock requestMetadata, OpenAI metadata, Vertex AI labels).

Proposed change

  1. In plugin_executor.py, pass session to model plugin instances for all model invocation methods (invoke_llm, invoke_text_embedding, invoke_rerank, invoke_tts, invoke_speech_to_text, invoke_moderation, etc.)
  2. Add an optional session parameter to model base class _invoke() methods (LargeLanguageModel, TextEmbeddingModel, RerankModel, etc.) so plugin authors can access session.app_id when available

This is backward compatible — session would be optional, and existing model plugins that don't use it require no changes.

Parent issue

langgenius/dify#35772

We have a PR ready and can submit it upon agreement on the design direction.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions