Skip to content

docs: swift quickstart improvements as guided by evals flywheel to improve Agentic experience#1386

Merged
sanchitmehtagit merged 2 commits into
mainfrom
eval-flywheel/swift-quickstart-improvements
Jun 15, 2026
Merged

docs: swift quickstart improvements as guided by evals flywheel to improve Agentic experience#1386
sanchitmehtagit merged 2 commits into
mainfrom
eval-flywheel/swift-quickstart-improvements

Conversation

@sanchitmehtagit

@sanchitmehtagit sanchitmehtagit commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Description

Improves the iOS/Swift quickstart based on eval-flywheel findings. When an LLM
agent follows the current quickstart, it tends to hardcode clientId/domain
directly in Swift source (following the "Advanced Integration / Programmatic
Configuration" section) instead of using Auth0.plist, and is not explicitly
steered toward CredentialsManager for token storage.

This PR adds two guidance callouts to nudge both human and AI readers toward the
secure, idiomatic defaults — without removing the programmatic-configuration
escape hatch:

  • <Info> (Step 5 – Authentication Service): use CredentialsManager for
    token storage; do not store tokens in memory, UserDefaults, or localStorage.
  • <Warning> (Advanced Integration): never hardcode clientId/domain in
    Swift source; prefer Auth0.plist (the quickstart default).

References

  • Eval framework: auth0-evals swift_quickstart eval
  • Related: staging preview for eval testing
    (docs-staging-eval-flywheel-swift-quickstart-v2.auth0-mintlify.app)

Testing

Measured with the swift_quickstart eval (agent + MCP, claude-opus-4-8):

  • Current production docs: Grade B (~57% graders). Agent hardcoded
    credentials in source → Security dimension 0/F, plus L1 webAuth() and L5
    Auth0.plist graders failing.
  • With these docs (via staging content): Grade A (~86% graders). Agent used
    Auth0.plist and stopped hardcoding → security + plist graders pass.

Single-run signal, not a statistical average — directional evidence that the
guidance changes agent behavior in the intended direction.

Checklist

  • I've read and followed CONTRIBUTING.md.
  • I've tested the site build for this change locally.
  • I've made appropriate docs updates for any code or config changes.
  • I've coordinated with the Product Docs and/or Docs Management team about non-trivial changes.

@sanchitmehtagit sanchitmehtagit requested a review from a team as a code owner June 12, 2026 20:47
@github-actions

Copy link
Copy Markdown
Contributor

Summary

Status Count
🔍 Total 2
✅ Successful 0
⏳ Timeouts 0
🔀 Redirected 0
👻 Excluded 2
❓ Unknown 0
🚫 Errors 0
⛔ Unsupported 0
Full Github Actions output

@sanchitmehtagit sanchitmehtagit changed the title Eval flywheel/swift quickstart improvements docs: swift quickstart improvements as guided by evals flywheel to improve Agentic experience Jun 15, 2026
@sanchitmehtagit sanchitmehtagit merged commit 76bf2d8 into main Jun 15, 2026
9 of 10 checks passed
@sanchitmehtagit sanchitmehtagit deleted the eval-flywheel/swift-quickstart-improvements branch June 15, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants