Skip to content

feat(gemini): add multimodal embedding support#944

Merged
sixlive merged 2 commits into
prism-php:mainfrom
cyppe:feature/gemini-embedding-2-multimodal-support
Mar 12, 2026
Merged

feat(gemini): add multimodal embedding support#944
sixlive merged 2 commits into
prism-php:mainfrom
cyppe:feature/gemini-embedding-2-multimodal-support

Conversation

@cyppe

@cyppe cyppe commented Mar 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds first-class multimodal embeddings support for Gemini, including the new gemini-embedding-2-preview workflow shape.

What changed

  • adds an Embeddings\Content value object so embeddings requests can represent real content entries instead of only flat text/image lists
  • adds fromAudio(), fromAudios(), fromVideo(), fromVideos(), fromDocument(), fromDocuments() to the embeddings pending request API
  • adds fromContent() and fromContents() so callers can explicitly choose between:
    • one aggregated embedding from multiple parts in a single content entry
    • multiple embeddings in one batch request
  • updates the Gemini embeddings handler to:
    • support image, audio, video, and document parts
    • route single content entries to embedContent
    • route multiple content entries to batchEmbedContents
    • forward Gemini embedding provider options such as title, taskType, and outputDimensionality
  • updates the embeddings docs and Gemini provider docs with multimodal examples
  • adds request-level and provider-level test coverage for the new behavior

Why

Prism already supports image inputs at the generic Embeddings API level, but Gemini's provider implementation still serialized only text parts. That meant models such as gemini-embedding-2-preview could not actually use Prism's multimodal embeddings surface even though the public API suggested they could.

This change aligns the Gemini provider with the current Embeddings API and exposes the input structuring needed by Gemini's multimodal embedding model.

Example

Aggregated multimodal embedding

use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::embeddings()
    ->using(Provider::Gemini, 'gemini-embedding-2-preview')
    ->fromContent([
        'An image of a dog',
        Image::fromLocalPath('/path/to/dog.png'),
    ])
    ->asEmbeddings();

Multiple embeddings in one request

use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::embeddings()
    ->using(Provider::Gemini, 'gemini-embedding-2-preview')
    ->fromContents([
        ['The dog is cute'],
        [Image::fromLocalPath('/path/to/dog.png')],
    ])
    ->asEmbeddings();

References

Test plan

Ran in DDEV:

  • php vendor/bin/pest tests/Embeddings tests/Providers/Gemini
  • php vendor/bin/pest tests/Embeddings/ImageEmbeddingsTest.php tests/Embeddings/MultimodalEmbeddingsTest.php tests/Providers/Gemini/EmbeddingsTest.php tests/Providers/Gemini/MultimodalEmbeddingsTest.php
  • php vendor/bin/pint --test
  • php vendor/bin/rector process --dry-run
  • php vendor/bin/phpstan analyse --verbose --no-ansi

@sixlive
sixlive merged commit 1da1057 into prism-php:main Mar 12, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants