Skip to content

Add blob store support#3606

Merged
michaeljguarino merged 16 commits into
masterfrom
blob-uploads
May 26, 2026
Merged

Add blob store support#3606
michaeljguarino merged 16 commits into
masterfrom
blob-uploads

Conversation

@michaeljguarino

@michaeljguarino michaeljguarino commented May 23, 2026

Copy link
Copy Markdown
Member

There's a few usecases for this:

  • workbench memory - tarball of ai generated files
  • agent run screenrecordings
  • agent run session storage
  • ...etc

Also implemented:

  • workbench job search (vector store required)
  • lua tool calling
  • pg based vector store

Test Plan

Test environment: https://console.your-env.onplural.sh/

Checklist

  • I have added a meaningful title and summary to convey the impact of this PR to a user.
  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have deployed the agent to a test environment and verified that it works as expected (required only when changing agent code).

Plural Flow: console

@michaeljguarino michaeljguarino requested a review from a team May 23, 2026 20:01
@michaeljguarino michaeljguarino added the enhancement New feature or request label May 23, 2026
@greptile-apps

greptile-apps Bot commented May 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR introduces blob/object store support for the console, wiring up S3 and Azure Blob Storage via a Waffle integration to store agent run session recordings, screen recordings, patch files, and workbench memory tarballs.

  • New AgentRunUpload schema, migration, GraphQL mutation (createAgentRunUpload), and cluster-scoped auth; a validate_run/3 helper cleans up duplicated auth logic across update_agent_run, agent_run_uploads, and create_agent_message.
  • Helm chart values and secrets extended for both S3 and Azure object store configuration; runtime config blocks conditionally enable waffle storage when the relevant env vars are present.

Confidence Score: 3/5

The file-type validation bug in Console.Uploads will cause every real upload to be rejected on the first attempt, making the core feature non-functional as shipped.

Two defects affect the main use-cases: the validate/1 whitelist only allows images and blocks all expected file types, and the S3 endpoint env var is threaded through Helm but never applied in the runtime config, silently breaking custom-endpoint deployments. The unofficial fork dependency is an additional concern.

lib/console/uploads.ex (invalid file-type whitelist) and rel/runtime.exs (missing endpoint wiring) need fixes before this feature works end-to-end.

Security Review

  • Supply-chain risk (mix.exs): waffle is sourced from a personal GitHub fork (jopedroliveira/waffle, tag v1.1.9-azure.3) rather than the official elixir-waffle/waffle package. A compromised fork owner could push malicious code under the same tag.
  • No other security issues identified. ACL is correctly set to :private, and the upload mutation is gated behind ClusterAuthenticated middleware.

Important Files Changed

Filename Overview
lib/console/uploads.ex New Waffle upload definition; validate/1 only allows image extensions, blocking all session, screen-recording, and patch uploads
rel/runtime.exs Adds S3 and Azure waffle config blocks; CONSOLE_S3_ENDPOINT is never consumed here despite being wired in Helm secrets
mix.exs Adds waffle, waffle_ecto, and ex_aws_s3 deps; waffle points to a personal GitHub fork rather than the official package
lib/console/schema/agent_run_upload.ex New schema for agent run uploads with session, screen_recording, and patch Waffle attachment fields; looks correct
lib/console/deployments/agents.ex Adds agent_run_uploads/3 with object-store guard; refactors repeated auth logic into validate_run/3 helper
priv/repo/migrations/20260523191654_add_s3_upload_support.exs Creates agent_run_uploads table and adds memory column to workbenches; migration looks correct
charts/console/templates/secrets.yaml Adds object store env vars to Helm secrets; CONSOLE_S3_ENDPOINT is present but unused in runtime config
lib/console/graphql/deployments/agent.ex Adds create_agent_run_upload mutation with ClusterAuthenticated middleware and upload_url resolver helper

Reviews (1): Last reviewed commit: "Add blob store support" | Re-trigger Greptile

Comment thread lib/console/uploads.ex
Comment thread rel/runtime.exs
Comment thread mix.exs
@michaeljguarino michaeljguarino force-pushed the blob-uploads branch 2 times, most recently from 460f677 to bfed81d Compare May 23, 2026 20:31
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

3 similar comments
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino michaeljguarino force-pushed the blob-uploads branch 3 times, most recently from 0b4f303 to 8ebc981 Compare May 24, 2026 15:05
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino michaeljguarino force-pushed the blob-uploads branch 3 times, most recently from 0fcf15f to 7a56ac2 Compare May 24, 2026 17:11
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino michaeljguarino force-pushed the blob-uploads branch 3 times, most recently from b210f04 to 40ad430 Compare May 24, 2026 20:39
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino michaeljguarino force-pushed the blob-uploads branch 2 times, most recently from 79a54b0 to 1cce296 Compare May 24, 2026 21:09
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino michaeljguarino force-pushed the blob-uploads branch 3 times, most recently from 11c936e to dbe2f9a Compare May 24, 2026 23:10
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

1 similar comment
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

1 similar comment
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@michaeljguarino michaeljguarino merged commit 90b7982 into master May 26, 2026
79 checks passed
@michaeljguarino michaeljguarino deleted the blob-uploads branch May 26, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants