Skip to content

Add Apple Foundation Models support (Apple FM)#2150

Open
jimscard wants to merge 27 commits into
danielmiessler:mainfrom
jimscard:add-fm
Open

Add Apple Foundation Models support (Apple FM)#2150
jimscard wants to merge 27 commits into
danielmiessler:mainfrom
jimscard:add-fm

Conversation

@jimscard

@jimscard jimscard commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Resolves #2154

Add Apple Foundation Models support (Apple FM)

Summary

This PR adds support for Apple Foundation Models (OpenAI-compatible) as a provider in Fabric. It includes robust handling for servers that return SSE (text/event-stream) for non-stream responses by parsing SSE "data:" chunks and concatenating content. A secondary fallback resends a single concatenated user message when SSE parsing yields empty output.

Key changes

  • Register Apple Foundation Models provider (default base URL: http://localhost:1976/v1). API key optional for local servers.
  • Implement direct HTTP fallback for Chat Completions when the SDK returns no choices or errors.
  • parseSSEAndConcat: robust SSE parser using bufio.Reader to avoid scanner limits and extract both JSON delta/message content and raw text data.
  • Secondary fallback: resend single concatenated user message when SSE parse yields empty output for some server behaviors.
  • Unit tests for SSE parsing.
  • README updated with server guidance (fm serve example), tmux snippet, and host-binding warning.

Lint & cleanup (follow-up)

  • Replaced inefficient string concatenation in a loop with strings.Builder for the concatenated-message fallback.
  • Simplified HasPrefix+TrimPrefix to strings.CutPrefix in SSE parsing.
  • These cleanups address CI/linter warnings and were committed in 4ce138c (fix(openai): optimize SSE parsing).

Files touched (high level)

  • internal/plugins/ai/openai_compatible/providers_config.go
  • internal/plugins/ai/openai/chat_completions.go
  • internal/plugins/ai/openai/chat_completions_test.go
  • README.md

Testing performed locally

  • Ran full test suite: go test ./... (all packages report OK locally)
  • Built and ran a sample YouTube summary against a local Apple FM server and validated output.
  • Verified non-stream completions against an Apple FM server returning text/event-stream; SSE parsing and concatenated-message fallback yield plain text.

How to test

  1. Start Apple FM server (requires macOS Golden Gate 27.0 beta 1+):

    # Start in background via tmux (recommended)
    tmux has-session -t fm_server || tmux new -d -s fm_server fm serve --host 0.0.0.0 --port 1976
  2. Verify vendor and models:

    ./fabric --listvendors
    ./fabric --listmodels --vendor "Apple Foundation Models"
  3. Run a non-stream sample to exercise SSE fallback:

    ./fabric -y https://youtu.be/piKWmRBMgRU -p "youtube_summary" --vendor "Apple Foundation Models" --model pcc

Notes for reviewers

  • Please review parseSSEAndConcat for correctness and edge cases.
  • Consider Git LFS for future large artifacts if the project wishes to store binaries in-repo.

Recent commits on add-fm (most recent first):

$(git log add-fm -n 12 --pretty=format:"- %h %s (%an)")

jimscard and others added 26 commits April 10, 2026 16:27
…API key optional)\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…scovery\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tructions\n\nDocument fm serve usage and macOS requirement

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com
…allback for OpenAI-compatible providers

Concatenate SSE 'data:' JSON delta content into a single response when the SDK cannot parse non-stream SSE responses (Apple Foundation Models compatibility).\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…allback; build direct HTTP fallback from messages/opts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…enation of JSON delta SSE messages and plain text SSE data lines

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…E responses)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…roubleshooting SSE empty response

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… parsing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…est.json for comparison

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…message fallback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…d docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…: Copilot <223556219+Copilot@users.noreply.github.com>
…tPrefix)\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for Apple Foundation Models

1 participant