Skip to content

Sync upstream v0.62.3 (merge conflicts)#24

Closed
trevorgowing wants to merge 794 commits into
masterfrom
sync/upstream-2026-06-25
Closed

Sync upstream v0.62.3 (merge conflicts)#24
trevorgowing wants to merge 794 commits into
masterfrom
sync/upstream-2026-06-25

Conversation

@trevorgowing

Copy link
Copy Markdown
Member

Upstream Sync — v0.62.3

Merge conflicts were detected when merging upstream release v0.62.3. Manual resolution is required before this PR can be merged into master.

Files with conflicts

.clj-kondo/config.edn
.github/workflows/backend.yml
.github/workflows/docs-generate-base.yml
.github/workflows/embedding-sdk-component-tests.yml
.github/workflows/pre-release.yml
.github/workflows/release-embedding-sdk.yml
.github/workflows/run-tests.yml
.loki/reference/chrome_laptop_Design_System_Colors_Default.png
bin/build/src/build/uberjar.clj
bun.lock
docs/ai/file-based-development.md
docs/ai/mcp.md
docs/ai/overview.md
docs/ai/settings.md
docs/ai/start.md
docs/configuring-metabase/config-template.md
docs/configuring-metabase/environment-variables.md
docs/embedding/guest-embedding.md
docs/usage-and-performance-tools/usage-analytics-reference.md
e2e/support/helpers/e2e-caching-helpers.ts
e2e/test/scenarios/admin/performance/caching.cy.spec.ts
e2e/test/scenarios/dashboard/dashboard-questions.cy.spec.js
e2e/test/scenarios/metabot/usage-auditing.cy.spec.ts
e2e/test/scenarios/permissions/impersonated.cy.spec.js
enterprise/backend/src/metabase_enterprise/data_complexity_score/api.clj
enterprise/backend/src/metabase_enterprise/data_complexity_score/cli.clj
enterprise/backend/src/metabase_enterprise/data_complexity_score/complexity.clj
enterprise/backend/src/metabase_enterprise/data_complexity_score/models/data_complexity_score.clj
enterprise/backend/src/metabase_enterprise/data_complexity_score/task/complexity_score.clj
enterprise/backend/src/metabase_enterprise/gsheets/settings.clj
enterprise/backend/src/metabase_enterprise/metabot/task/ai_usage_trimmer.clj
enterprise/backend/src/metabase_enterprise/remote_sync/api.clj
enterprise/backend/src/metabase_enterprise/remote_sync/impl.clj
enterprise/backend/src/metabase_enterprise/serialization/api.clj
enterprise/backend/src/metabase_enterprise/serialization/schema.clj
enterprise/backend/src/metabase_enterprise/serialization/v2/models.clj
enterprise/backend/test/metabase_enterprise/data_complexity_score/api_test.clj
enterprise/backend/test/metabase_enterprise/data_complexity_score/cli_test.clj
enterprise/backend/test/metabase_enterprise/data_complexity_score/complexity_test.clj
enterprise/backend/test/metabase_enterprise/data_studio/permissions/published_tables_test.clj
enterprise/backend/test/metabase_enterprise/impersonation/driver_test.clj
enterprise/backend/test/metabase_enterprise/remote_sync/api_test.clj
enterprise/backend/test/metabase_enterprise/serialization/api_test.clj
enterprise/backend/test/metabase_enterprise/serialization/models/entity_id_test.clj
enterprise/backend/test/metabase_enterprise/sso/integrations/jwt_test.clj
enterprise/backend/test/metabase_enterprise/sso/settings_test.clj
enterprise/frontend/src/embedding-sdk-package/package.template.json
enterprise/frontend/src/metabase-enterprise/ai-controls/routes.tsx
enterprise/frontend/src/metabase-enterprise/audit_app/metabot-analytics/components/ConversationDetailPage/ConversationDetailPage.tsx
enterprise/frontend/src/metabase-enterprise/audit_app/metabot-analytics/components/ConversationFilters/ConversationFilters.tsx
enterprise/frontend/src/metabase-enterprise/audit_app/metabot-analytics/components/MetabotAnalyticsUpsellPage/MetabotAnalyticsUpsellPage.tsx
enterprise/frontend/src/metabase-enterprise/caching/components/StrategyEditorForDatabases.unit.spec.tsx
enterprise/frontend/src/metabase-enterprise/metabot/components/MetabotAdmin/MetabaseAIProviderSetup.tsx
frontend/src/metabase-types/api/metabot.ts
frontend/src/metabase/admin/ai/AISettingsPage.tsx
frontend/src/metabase/admin/ai/MetabotNavPane.tsx
frontend/src/metabase/admin/ai/MetabotSetup.tsx
frontend/src/metabase/admin/ai/MetabotSetup.unit.spec.tsx
frontend/src/metabase/admin/performance/components/StrategyEditorForDatabases.unit.spec.tsx
frontend/src/metabase/admin/permissions/selectors/data-permissions/permission-editor.unit.spec.tsx
frontend/src/metabase/admin/routes.tsx
frontend/src/metabase/admin/settings/components/SettingsPages/GeneralSettingsPage.tsx
frontend/src/metabase/admin/settings/components/SettingsPages/GeneralSettingsPage.unit.spec.tsx
frontend/src/metabase/api/metabot.ts
frontend/src/metabase/browse/tables/TableBrowser/TableBrowser.tsx
frontend/src/metabase/dashboard/components/DashCard/Dashcard.unit.spec.tsx
frontend/src/metabase/dashboard/components/DashboardHeader/buttons/ExportAsPdfButton.unit.spec.tsx
frontend/src/metabase/dashboard/selectors.ts
frontend/src/metabase/entities/tables.js
frontend/src/metabase/metabot/components/AIProviderConfigurationModal.tsx
frontend/src/metabase/metabot/components/FixSqlQueryButton/FixSqlQueryButton.unit.spec.tsx
frontend/src/metabase/metabot/components/MetabotAdmin/MetabotNavPane.unit.spec.tsx
frontend/src/metabase/metabot/components/MetabotChat/MetabotAgentDataSourcePills.tsx
frontend/src/metabase/metabot/components/MetabotChat/MetabotAgentDataSourcePills.unit.spec.tsx
frontend/src/metabase/metabot/components/MetabotChat/MetabotChat.tsx
frontend/src/metabase/metabot/components/MetabotChat/MetabotChatMessage.tsx
frontend/src/metabase/metabot/components/MetabotChat/MetabotChatMessage.unit.spec.tsx
frontend/src/metabase/metabot/components/MetabotQueryBuilder/MetabotQueryBuilder.tsx
frontend/src/metabase/metabot/hooks/use-user-metabot-permissions.unit.spec.tsx
frontend/src/metabase/metabot/tests/utils.tsx
frontend/src/metabase/redux/downloads.unit.spec.ts
frontend/src/metabase/services.unit.spec.ts
frontend/src/metabase/transforms/components/LockedTransformsBanner/LockedTransformsBanner.tsx
frontend/src/metabase/transforms/components/LockedTransformsHoverCard/LockedTransformsHoverCard.tsx
frontend/src/metabase/transforms/pages/TransformListPage/TransformListPage.tsx
frontend/src/metabase/ui/colors/constants/themes/dark.ts
frontend/src/metabase/ui/colors/constants/themes/light.ts
frontend/src/metabase/ui/components/icons/Icon/icons/index.ts
frontend/src/metabase/visualizations/click-actions/modes/EmbeddingSdkMode.ts
frontend/src/metabase/visualizations/visualizations/CartesianChart/events.unit.spec.ts
modules/drivers/bigquery-cloud-sdk/test/metabase/driver/bigquery_cloud_sdk_test.clj
modules/drivers/druid/src/metabase/driver/druid/query_processor.clj
modules/drivers/mongo/src/metabase/driver/mongo/query_processor.clj
modules/drivers/redshift/src/metabase/driver/redshift.clj
modules/drivers/sqlserver/src/metabase/driver/sqlserver.clj
modules/drivers/sqlserver/test/metabase/driver/sqlserver_test.clj
release/bun.lock
release/package.json
resources/metabot/prompts/system/embedding-next.selmer
resources/metabot/prompts/system/internal.selmer
resources/metabot/prompts/system/natural-language-querying-only.selmer
resources/metabot/prompts/tools/nlq_search.md
resources/metabot/prompts/tools/search.md
resources/metabot/prompts/tools/sql_search.md
resources/metabot/prompts/tools/transform_search.md
src/metabase/agent_api/api.clj
src/metabase/agent_lib/eval/args.clj
src/metabase/agent_lib/mbql_integration/field_resolution.clj
src/metabase/agent_lib/repair/normalize.clj
src/metabase/agent_lib/repair/normalize/forms.clj
src/metabase/agent_lib/repair/normalize/top_level.clj
src/metabase/analytics/llm_token_usage.clj
src/metabase/analytics/prometheus.clj
src/metabase/analyze/fingerprint/fingerprinters.clj
src/metabase/api/macros/defendpoint/tools_manifest.clj
src/metabase/cmd/copy.clj
src/metabase/driver/common/parameters/parse.clj
src/metabase/driver/postgres.clj
src/metabase/driver/sql/query_processor.clj
src/metabase/driver/sql_jdbc/execute.clj
src/metabase/legacy_mbql/jvm_util.clj
src/metabase/legacy_mbql/util.cljc
src/metabase/lib/metadata/result_metadata.cljc
src/metabase/lib/parameters/parse.cljc
src/metabase/mcp/README.md
src/metabase/mcp/api.clj
src/metabase/mcp/resources.clj
src/metabase/mcp/scope.clj
src/metabase/mcp/tools.clj
src/metabase/metabot/feedback.clj
src/metabase/metabot/persistence.clj
src/metabase/metabot/self/claude.clj
src/metabase/metabot/tools/resources.clj
src/metabase/metabot/tools/search.clj
src/metabase/models/resolution.clj
src/metabase/models/serialization.clj
src/metabase/parameters/chain_filter.clj
src/metabase/permissions/models/data_permissions.clj
src/metabase/permissions_rest/data_permissions/graph.clj
src/metabase/plugins/impl.clj
src/metabase/query_processor/card.clj
src/metabase/query_processor/execute.clj
src/metabase/query_processor/middleware/optimize_temporal_filters.clj
src/metabase/query_processor/middleware/wrap_value_literals.clj
src/metabase/sql_parsing/core.clj
src/metabase/sql_parsing/pool.clj
src/metabase/task/impl.clj
src/metabase/transforms/core.clj
src/metabase/transforms/feature_gating.clj
src/metabase/transforms/jobs.clj
src/metabase/transforms/models/job_run.clj
src/metabase/transforms/models/transform.clj
src/metabase/transforms/settings.clj
src/metabase/transforms_rest/api/transform.clj
test/metabase/agent_lib/eval_test.clj
test/metabase/agent_lib/repair/normalize/forms_test.clj
test/metabase/agent_lib/repair_test.clj
test/metabase/agent_lib/runtime/fields_test.clj
test/metabase/analytics/llm_token_usage_test.clj
test/metabase/api/macros/defendpoint/tools_manifest_test.clj
test/metabase/app_db/connection_pool_setup_test.clj
test/metabase/driver/mysql_test.clj
test/metabase/driver/postgres_test.clj
test/metabase/driver/sql/query_processor_test.clj
test/metabase/lib/drill_thru/underlying_records_test.cljc
test/metabase/llm/context_test.clj
test/metabase/mcp/api_test.clj
test/metabase/mcp/resources_test.clj
test/metabase/mcp/tools_test.clj
test/metabase/metabot/agent/user_context_test.clj
test/metabase/metabot/api_test.clj
test/metabase/metabot/context_test.clj
test/metabase/metabot/persistence_test.clj
test/metabase/metabot/self/claude_test.clj
test/metabase/metabot/self_test.clj
test/metabase/metabot/tools/search_test.clj
test/metabase/permissions_rest/data_permissions/graph/bulk_update_test.clj
test/metabase/premium_features/token_check_test.clj
test/metabase/search/appdb/index_test.clj
test/metabase/slackbot/api_test.clj
test/metabase/slackbot/channel_test.clj
test/metabase/slackbot/streaming_test.clj
test/metabase/sql_tools/core_test.clj
test/metabase/transforms/jobs_test.clj
test/metabase/transforms/models/transform_test.clj
test/metabase/util/match_test.cljc

How to resolve

git fetch origin
git checkout sync/upstream-2026-06-25
# Resolve the <<<<<<< markers in each conflicted file, then:
git add -A
git commit --amend --no-edit
git push --force-with-lease origin sync/upstream-2026-06-25

Tip: The most likely files to conflict are the Teal branding files:
frontend/src/metabase/ui/colors/constants/themes/light.ts
frontend/src/metabase/ui/colors/constants/themes/dark.ts
src/metabase/appearance/settings.clj

iethree and others added 30 commits May 27, 2026 22:43
only auto-add issues to open milestones
* Update translation sources 2026-05-26

* Update translations

---------

Co-authored-by: github-actions <github-actions@github.com>
* Migrate question entity-framework hooks off the Questions entity

* Migrate read-only Question consumers off the Questions entity

* Migrate dashboard card actions off the Questions entity

* Migrate query-builder card actions off the Questions entity

* Remove the questions entity

* Do not drop dashcard sizes in api call

* Handle errors in card query

* Add error/loading state to action creator modal

* Avoid refetching when forbidden

* Avoid stale cards in card mutations
* test: cleanup jest warnings from query_builder folder

* fix question download widget test

* proper fixes
…4406)

* Distinguish empty Metabot prompt regeneration from errors

When an admin clicks "Regenerate suggested prompts" today, three different
outcomes look the same: the LLM crashed, the metabot's library is empty
(no models/metrics to summarise), or the LLM ran but produced no
questions. All three previously returned `204 No Content` and the UI had
no way to tell them apart.

This commit makes `generate-sample-prompts` return a structured outcome
the API endpoint surfaces as JSON:

  {:status :generated  :prompt-count N}             - happy path
  {:status :empty :reason :no-library-content}      - nothing to summarise
  {:status :empty :reason :ai-produced-no-prompts}  - LLM returned 0 questions
  {:status :empty :reason :managed-free-limit-reached}
                                                    - managed AI cap reached

The empty-library and managed-free-limit cases skip the LLM call entirely
so we don't burn tokens generating nothing. The endpoint now returns
`200 OK` with the structured body in place of `204 No Content`; the
response schema is declared so docs and TypeScript signature generation
pick it up.

Errors (LLM 5xx, network, etc.) still surface as exceptions and turn
into 4xx/5xx responses — those remain distinct from the `:empty` cases.

Frontend follow-up needed: the admin panel's regenerate-button toast
should branch on `:status` and `:reason` to render a friendly empty-state
message instead of a silent success. Not in this PR (backend-only).

* Make regenerate-prompts response a discriminated union

Collapse the previous `{:status :empty :reason :x}` shape into a single-tag
union `{:status :x}` so callers can switch on one field, and rename
`:prompt-count` to `:prompt_count` to match wire conventions.

* Handle each regenerate-prompts status in admin UI

Surface the new no-library-content, ai-produced-no-prompts, and
managed-free-limit-reached statuses with their own toast copy instead of
collapsing them into a generic error.

* Tighten regenerate-prompts response and tests

- Drop :managed-free-limit-reached from the discriminated union; the 402
  fires before generate-sample-prompts is ever called, so the case is
  unreachable from this endpoint.
- Match exhaustively in the admin UI via ts-pattern so future status
  additions are a compile error.
- Add admin-UI unit tests for the no-library-content,
  ai-produced-no-prompts, and happy-path branches.
- In the backend test, assert that :prompt_count in the response equals
  the actual MetabotPrompt row count (not just the mock's promise).
- Realize the per-source prompt seqs into vectors so we don't re-run
  ->prompt between counting and inserting.
- "summarise" → "summarize".

* Close TOCTOU race on managed-AI lock in regenerate-prompts

`generate-sample-prompts` now calls `check-metabase-managed-free-limit!`
itself instead of soft-returning `{:status :managed-free-limit-reached}`.
That makes the response union genuinely closed: there's no longer a window
where the endpoint's pre-check passes, the function-level check then sees
the lock flipped, and a status not in the Malli `:multi` schema slips
through (which would have surfaced as a 500 from defendpoint validation).

`update-metabot` still wants best-effort behavior — a settings PUT
shouldn't 402 just because prompt regeneration was raced by the lock —
so wrap its `generate-sample-prompts` call in a try/catch that swallows
the locked-account exception and re-throws everything else.

Add a test for that race-tolerance path, asserting both that the locked
402 is swallowed and that a non-locked ex-info still propagates.

* Tighten happy-path regenerate test anchor

The previous `waitFor(negative-assertion)` succeeded immediately because
no toast renders synchronously on click — by the time the mutation
settled, the assertion had already passed, so a regression where the
happy-path branch wrongly fell through to an empty-state toast could
slip past.

Anchor on the button's idle text instead: "Regenerate suggested prompts"
only re-renders after `isRegenerating` flips back to false, which is the
same render where any mistaken toast would have appeared. Now the
negative assertions run only after the mutation has actually resolved.

* docstrings: trim design-rationale from generate-sample-prompts + endpoint

Both docstrings carried multi-paragraph design rationale ("single discriminator
so the frontend can switch on one field", "closed union so the regenerate
endpoint can declare an exact response schema", "no TOCTOU race between
checks"). All of that is implementer-context for someone who'd refactor the
design, not caller-contract material the IDE-hovering caller needs.

Slim each docstring to the caller contract: what's returned, when 402 fires,
and what callers who want best-effort behaviour should do about the
managed-free-limit race. The schema declaration above the endpoint already
shows the exact response shape.

* Tighten regenerate-prompts response schema

Two narrower invariants in the discriminated union:

- `{:closed true}` on each variant so the OpenAPI / TS output is an
  exact union instead of permitting `additionalProperties`.
- `:prompt_count` is `pos-int?`, not `nat-int?`. The `:generated`
  variant is only returned when total > 0 — the zero case branches
  to `:ai-produced-no-prompts`. Encoding that in the schema removes
  an impossible state from the contract.

Inline comment points the next reader at the function that decides
which branch is returned.

* Pre-guard managed-AI lock in startup prompt task

After 4831cc9 made `generate-sample-prompts` always call
`check-metabase-managed-free-limit!`, locked-instance startups would
fail the call and surface as `log/errorf` from the generic catch in
`maybe-generate-suggested-prompts!`. That's noise — the locked state
isn't a failure, just a "not now."

Add a `metabot.usage/managed-free-limit-reached?` branch to the cond so
the task logs a clean info-level skip when the cap is hit, mirroring
the existing "Metabot is disabled" skip. The catch still backstops real
exceptions.

The existing locked-provider test already pins the outcome (no prompts
persisted, generator never called); the new path just changes how the
skip is logged.

* Wrap PUT regenerate in a transaction so a failed regen keeps prompts

* Test that PUT prompt regen rolls back the delete on failure

* Sync OpenAPI specs with regenerate endpoint docstring

* Plainer regenerate-prompts docstring in API docs

* Regenerate OpenAPI specs to include regenerate-prompts response union

The earlier sync commit (9ab0fc9) updated only the endpoint
description, leaving `responses.2XX` as the generic placeholder. Copilot
review on metabase#74406 flagged that generated clients still wouldn't see the
discriminated union this PR introduces.

Run `bun run generate-openapi` and `clojure -M:ee:doc api-documentation`
so both `resources/openapi/openapi.json` and `docs/api.json` carry the
full `oneOf` schema for the 2XX response (`generated` /
`no-library-content` / `ai-produced-no-prompts`). The regen also flushes
unrelated component drift (workspace schemas, etc.) that the nightly
auto-update workflow had not yet caught up to.

* Drop unrelated EE endpoint drift from docs/api.json

* Drop stray blank lines flagged by cljfmt

The earlier merge commit (`8684854bc85`) was made with `--no-verify`
because husky's pre-commit cljfmt was getting SIGKILL'd mid-merge.
CI caught two stray trailing blank lines inside `let`/`with-redefs`
bodies the merge introduced — apply `bin/mage cljfmt-files` to them.

* Use mt/with-dynamic-fn-redefs for the new function-mocking sites

Copilot review on metabase#74406 flagged four new mocks that use
`clojure.core/with-redefs` to redefine defn-style vars (the project
clj-kondo hook prefers `metabase.test/with-dynamic-fn-redefs` because
it's thread-safe).

Switch the four sites I added in this PR:
- metabot-put-skips/recovers tests for managed-AI lock
- metabot-prompt-regenerate-empty-states-test (no-library-content and
  ai-produced-no-prompts branches)

Leaving the two pre-existing `with-redefs` sites in this file (the
ones not flagged by the reviewer) untouched — those weren't introduced
by this PR.

* nits

---------

Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Even when sandboxing is not used, 1 read was being generated for each
group X table whose permissions were impacted by the update. In a sample
query from a user, that was 28 groups X 382 tables =  10,696 reads.

This change includes a unit test that synthesizes this same scale and
makes the same request. It did reproduce the same 10,700+ requests the
user was seeing initially, but with these changes to how sandboxes are
checked (one query per database) it now only makes 87 queries. That's
still a lot but not unreasonable for a complex request.
* setup changes

* fix

* e2e fixes

* fix

* fix

* fix
…etabase#74913)

GHY-3701 force let-user-control-scheduling false for non-stub config entries

build-workspace-config emits :details only — not the schedule cron
columns. infer-db-schedules asserts cache_field_values_schedule and
metadata_sync_schedule are populated whenever :let-user-control-scheduling
is true, so a source DB that opted in would blow up the round-trip on
import. Override the key to false in database-entry so the receiving
side takes the auto-schedule branch. Stubs are unaffected — they emit
:details {} and already fall through to the auto-schedule branch.
deps: bump qs ^6.14.2 -> 6.15.2

Fixes Snyk medium-severity advisory SNYK-JS-QS-16721866
(CVE-2026-8723, NULL pointer dereference in stringify with
arrayFormat=comma + encodeValuesOnly when arrays contain
null/undefined).
* check pg grant statements rights

* tests

* fix

* redshift

* fix

* fix

* fix

* fix
…ase#74743)

* do not return mcp apps tools for clients without ui capability

* use a stronger shape for mcp ui capability detection

* tidy up session.clj session helpers

* add extended session id validation

* mcp ui tools should declare required client extensions

* remove rollback for varchar length extension

* update argument order

* parse uuid with parse-uuid utility

* log when capability payload is blank

* log when capability payload cannot be decoded

* warn if the mcp session id is too long

* document preserving slash with json/decode+kw

* use with-dynamic-fn-redefs for unit test

* remove unused supported-extensions option

* check scope matches first

* re-add non-null constraint after varchar(254) change

* disable mcp apps ui support if capability payload version mismatches

* allow known/older version for capability payload

* fix cljfmt lint
* Test connection button for Remote Sync

* remove branch and mode from connection test
* deps: use safe version of js-cookie

* pin
…etabase#74758)

* Reduce and converge Metabot system prompts on objective-based style

Converge the embedding and internal system prompts onto the lean,
objective-based style of the NLQ prompt, and collapse the duplicated
verification/grounding guidance into a single source of truth.

- New shared includes (grounding, data-sources, mbql-shape, communication,
  discovery) lifted from the NLQ prompt; nlq/embedding/internal now compose
  them. Retire critical-constraints, verification-triggers, samples-vs-results,
  disclosure-protocol, and anti-patterns (the verification rule was restated
  ~6x; now once).
- internal.selmer 447->107 lines, embedding-next.selmer 419->79; permission
  gating kept but expressed through stable <!-- gate:X -->/<!-- denial:X -->
  anchors so wording can change without breaking tests.
- Verification reframed from "verify EVERY literal, no exceptions" to
  "verify literals whose format you can't predict" to stop wasteful
  sample reads; narration/clarification contradictions resolved.
- Discovery reframed navigate-first: read_resource is a first-class peer of
  search. Fold the four near-identical search prompt files into the discovery
  include; move search parameter guidance into Malli field descriptions and
  enrich the tool docstrings.
- De-duplicate the SQL read-only contract: canonical in create_sql_query.md,
  pointer in edit/replace, one-liner in internal.
- prompt_gating_test asserts on the new anchors instead of prose.

* Address PR review on Metabot prompt converge

- Include grounding/data-sources/discovery in the SQL profile so it gets
  the same discovery and verification guidance as the other profiles
  after the per-tool search prompts were retired.
- Drop the literal tool-name reference in grounding.selmer so the snippet
  is not coupled to specific tool names; discovery.selmer keeps the
  names since it is specifically about distinguishing them.
- Move shared snippets under shared/prompt_snippets/ to mark the
  directory as a contract: files placed there must be safe to embed in
  the cacheable prefix.
- Add prompt-cache tests asserting no Selmer directives appear in
  shared/prompt_snippets/ and that the cacheable prefix of every system
  template is stable across per-request values (current_time, user
  info, viewing context, recent views).

* Tune SQL profile to address regressions found in benchmark runs

- Add an SQL-specific "Searching effectively" section to the SQL
  profile so the agent gets concrete multi-concept and
  semantic/keyword search examples that used to live in the deleted
  sql_search.md.
- Replace the previous "Handle Uncertainty" framing (which described
  ask_for_sql_clarification as "your only way to send text") with an
  explicit tool-routing table. SQL bodies go through
  create_sql_query/edit_sql_query/replace_sql_query;
  ask_for_sql_clarification is reserved for short clarifying
  questions about structural ambiguity only.
- Rewrite ask_for_sql_clarification.md from the ground up: scope to
  short clarifying questions, explicit denylist for SQL bodies and for
  questions discovery can answer, and a one-sentence constraint on
  the question argument.
- Reinforce the routing in the profile's Remember section so the
  cache-prefix tail nails the rule.
- Bump SQL profile max-iterations from 10 to 20 so the agent has
  budget for the longer discovery patterns the new prompt encourages.

* Silence reflection warning in prompt_cache_test

Add (set! *warn-on-reflection* true) so the type hint on ^java.io.File
in snippet-files is checked by the compiler instead of producing a
clj-kondo warning.
Loading the `:workspace` config-file section (at boot or at runtime via `POST /api/ee/advanced-config/`) only required the `:config-text-file` feature. They should both require `:workspaces`.
* Revert "Use existing commit for cljfmt (metabase#74885)"

This reverts commit d4d8c53.

* Revert "Fix cljfmt formatting  (metabase#74892)"

This reverts commit ef186e4.
* BOT-1524 Save data sources used in a metabot conversation

* Handle implicit joins

* Simplify query-tables-and-cards with lib/all-referenced-entity-ids

* Cleanup native-tables parsing, remove uneeded macaw-tables

* Use lib more. banish mbql destructuring and raw mbql in tests

* Remove MetabotUsedTable from module config model-exports since nothing external uses it

* banish deprecated mt/mbql-query in tests

* Clean up docstrings in used_tables.clj

* Calm down with "MBQL 5" references

* Improve description for metabot_used_table

* Push seq down into when

* whitespace cleanups

* Use with-temp rather than t2/insert! and t2/delete! in used table tests

* Small formatting cleanup in persistence tests

* Use t2/with-transaction :rollback-only true rather than mt/with-model-cleanup in persistence tests

* Cleanup finalize-records-used-tables-test

* Formatting cleanup in ->transform-sql-parts and ->transform-python-parts

* More persistent test cleanups

* Remove private mp defn for metadata provider

* Rename provider to mp

* Formatting cleanups in used tables tests

* Pass in metadata-provider to extract-used-tables defaulting to lib-be/application-database-metadata-provider

And update the tests to take advantage of not need real cards in the app db and mark tests ^:parallel

* Use mt/with-dynamic-fn-redefs and mark tests ^:parallel

* Remove explicit ^:sequential

* Improve testing description strings

* Improve combined tool calls used tables test

* Only visit cards shared between tool calls once in extract-used-tables

* Fix trailing whitespace

* Fix lint warning about with-redefs

* Fix race in app-mention-triggers-response-test

* Add prometheus metrics for used table extraction duration and failures

* Move used tables extraction to background thread

* review feedback: remove macaw mentions

* review feedback: composed transducer rather than lazy seq

* review feedback: add some debug logs with more context

* review feedback: add bounded queue and increase threads to 4 for the used tables extraction executor

* Add 15s timeout via u/with-timeout for used tables extraction

* Add separate worker and waiter executors to ensure work remains bounded

Don't use u/with-timeout which runs the extraction on the shared future thread pool. Instead, create a separate worker
pool to ensure worker threads remain bounded even if they are stuck in cpu-bound state where they ignore thread
cancellations.

* Rename extract-used-tables-with-timing to extract-used-tables-with-timing!

* Allow idle threads to be reclaimed after 5 minutes

* Add test for extraction timeout and improve test for queue saturation
* Move Explore button into the chart card (UXW-4237)

* Address review: drop dead mock, move zIndex to CSS module

* Drop now-redundant Group wrapper around single Menu

* CSS cleanup: drop dead border-radius, move position:relative into class

* Fix metric-page e2e: explore link is now on the chart card, not the header

* Bump Explore button overlay spacing to xl (32px)

* Drop module mocks in MetricAbout spec; use real card metadata + endpoint stubs

* Use size=xs on Explore button for closer-to-design height
…etabase#74375)

* Check for VolumeOperationAllowedLocalPaths connecting to databricks.

* Move additional options check to validate-db-details! method.

* Review suggestions: check a few other cases.
…abase#74926)

LLMs sometimes place a clause's options at the END of a fixed-arity
tuple clause instead of position 1, e.g.

  ["time-interval" {} <expr> -1 "month" {"include-current" true}]

`time-interval` is a strict 5-tuple, so the extra element fails Malli
with a generic tuple-shape error and the construct_notebook_query tool
rejects the query. The trailing map is unambiguously misplaced options
(no fixed-arity tuple clause legitimately has a map-typed last arg).

Add a generic repair pass `merge-trailing-options*` that detects this
across every fixed-arity tuple clause registered with
`metabase.lib.schema.mbql-clause` — the expected-element-count table is
derived once at namespace load from `tag-registry`, so new fixed-arity
clauses get coverage automatically. Variadic `:catn`/`:repeat` clauses
(`and`, `or`, `case`, `coalesce`, `=`, `in`, …) are excluded — their
arity isn't fixed, so a trailing map there could be a legitimate arg.

When the pattern matches, merge the trailing map into position-1 options
(trailing keys win on conflict, since the trailing map is where the LLM
deliberately wrote content) and drop the trailing element. Idempotent by
construction.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
github-automation-metabase and others added 28 commits June 18, 2026 22:24
…TINCT (metabase#76077)" (metabase#76115)

Fix OOM in table-sync permission setup by SELECTing DISTINCT (metabase#76077) (metabase#76100)

When the sync-and-analyze job creates new tables, `set-default-table-permissions!`
calls `load-perm-context`, which loaded *every* table-level `data_permissions`
row for the database to figure out, per (group, perm-type, schema), the set of
distinct permission values already in use. The result set grew with the table
count, so on databases with very many tables it materialized millions of rows
(2.5M+ `org.postgresql.core.Tuple` / `toucan2.instance.Instance` in the heap
dump that prompted this) and OOM'd the JVM on a scheduler thread.

`schema-vals-idx` only ever needs the *distinct* (group_id, perm_type,
schema_name, perm_value) tuples — it reduces the rows into a set keyed by
(group, perm-type, schema), so duplicate rows were already being collapsed in
memory. Pushing that DISTINCT into SQL makes the loaded set bounded by
groups × perm-types × schemas × values instead of growing with table count.

Rows loaded by the introspection query (measured; one perm-type, one schema):

    groups  tables   before (groups×tables)   after (DISTINCT)
      5       10              100                    20
      5      100            1,000                    20
      5    1,000           10,000                    20   <- flat across tables
     10      100            2,000                    40
     50      100           10,000                   200   <- linear in groups

Before, the result scaled with the table count (the thing that blew up); after,
it is flat across table count and grows only linearly with the number of groups.
The produced `schema-vals-idx` is byte-identical because it accumulates into a
set — verified in a regression test.

Not scoping the query to the synced tables' schemas (a further reduction):
a `schema_name IN (...)` filter would add NULL/casing correctness risk, and the
DISTINCT collapse alone removes the geometric blowup that caused the OOM.

Co-authored-by: dpsutton <dan@dpsutton.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…abase#76127)

* BigQuery - `sample-table` with adaptive page size (metabase#76068)

* ci: re-trigger CI

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Limit schemas in workspaces (metabase#75405)

limit schemas in workspaces

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
…se#76171)

Fix memory leak with notifications and GraalVM (metabase#76145)

* Fix memory leak with table notifications

* fix

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
…ation selector" (metabase#76179)

Fix sensible visualization options in MCP Apps visualization selector (metabase#75876)

* fix visualization selector showing illegal visualizations

* reset the stored default display when loading a new query

* refactor: cleanup logic to capture default display

* update unit test name to be clearer

* add unit test to captureDefaultDisplay

* update prompt in MCP resource to not mention display types or viz settings

* do not accept stale query results after query changes

Co-authored-by: Phoomparin Mano <poom@metabase.com>
merge

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Make SQL guardrail skills always-on (metabase#76176)

* Make SQL guardrail skills always-on

The on-demand skills system demoted read_resource and the SQL
create/edit/replace guidance from always-present system-prompt content to
catalog-only skills the agent had to load itself. With the guardrails gone
from the prompt, the SQL profile would loop read_resource calls (no
"drill, don't re-search"/"honor truncation" rules) and fire redundant
edit_sql_query calls instead of one replace_sql_query (no edit-vs-replace
guidance).

Mark read-resource, create-sql-query, edit-sql-query, and replace-sql-query
as always-on so their bodies are inlined into the prompt prefix again, while
keeping the bulky rarely-used skills on-demand. Harden load_skill
registration to fire when the manifest has any catalog or always-on skills,
so the dialect-preload invariant no longer depends on read-resource sitting
in the catalog.

* Make ask-for-sql-clarification always-on to restore SQL termination

The :sql profile is :required-tool-call?, which maps to Anthropic
tool_choice {:type "any"} — the model is forced to emit a tool call on
every turn and can never stop on free text. The agent loop only terminates
on max-iterations or a tool result carrying :final-response?. Among the SQL
profile's tools, ask_for_sql_clarification is the only one that sets it, so
it is the model's de facto termination signal.

The skills commit (ec842e1) moved the clarification guidance from the
always-inlined tool instructions to a catalog-only on-demand skill. Its
content was unchanged, but it was no longer present in the prompt, so the
model lost awareness of its terminal/handoff role and instead spun on
read_resource until hitting max-iterations (observed: a single edit_sql_query
followed by 19 redundant read_resource calls).

Mark the skill always-on so its guidance is inlined again, restoring the
pre-commit behavior. Together with the other SQL guardrail skills, the
:sql profile's catalog is now empty (all guidance inline, as before);
load_skill stays registered via the manifest's always-on entries so dialect
preload keeps working.

* Exclude always-on skills from the loadable/advertised skill set

Always-on skill bodies are already inlined in the system prompt, so the
model must neither see them in the catalog nor be able to load_skill them.
The catalog already removed them, but record-loadable-skill-ids! still
recorded every visible skill, leaving always-on ids loadable — so the agent
could burn an iteration re-fetching a skill it already has inline.

Record only the on-demand (non-always-on) skills as loadable, and build the
catalog from that same set.

* Make always-on a per-profile decision instead of a skill-global flag

A skill's always-on/on-demand status was a boolean on the skill itself, so it
applied to every profile exposing the matching tool. That is wrong: the SQL
profile wants its SQL skills inlined on every turn, but the internal profile
(which shares read_resource/create/edit/replace_sql_query) wants them
on-demand to keep its prompt prefix lean.

Move the decision to the profile: a profile lists :always-on-skills (skill
ids) whose bodies are inlined; every other relevant skill stays on-demand in
the catalog. build-skill-manifest now partitions on the profile's set rather
than the skill's flag. The skill-level :always-on frontmatter is removed, and
register-profile! validates that :always-on-skills resolve to real skills.

The :sql profile inlines read-resource + the create/edit/replace/clarification
SQL skills; no other profile opts in, restoring their pre-skills on-demand
behavior. Adds tests for the per-profile split and for always-on bodies
actually appearing in the rendered system prompt.

* Don't render the load_skill catalog header when the catalog is empty

Selmer treats an empty vector as truthy, so a profile whose skills are all
always-on (empty catalog) still rendered the "# Available skills … load the
skill(s) you need with load_skill" header with nothing listed — nudging the
model into pointless load_skill calls. Pass the catalog through not-empty so
an empty catalog is nil (falsy) and the header is omitted.

* Let a profile mark tools terminal; end :sql turn after a successful write

Under :required-tool-call? the model is forced to emit a tool call every
turn and can only stop on max-iterations or a :final-response? result. After
a clean edit_sql_query the SQL profile had no terminal tool to reach for, so
it spun on read_resource to max-iterations (e.g. 1 edit + 19 redundant reads).

Add a per-profile :terminal-tools set: a *successful* call to one of these
tools ends the turn. Success is detected by the presence of :structured-output
(the query/chart tools attach it only on success, returning just an error
:output otherwise), so a failed edit still lets the model self-correct.
Terminality is per-profile — the same write tools stay non-terminal in
profiles like :internal that need multi-step flows.

The :sql profile marks create/edit/replace_sql_query terminal. register-profile!
validates that :terminal-tools refer to tools the profile exposes.

* Replace tool-level :final-response? with per-profile :terminal-tools

Terminality was expressed two ways: the new per-profile :terminal-tools and
the older tool-level :final-response? flag (set by ask_for_sql_clarification
and the document_construct_*_chart tools). Two mechanisms for one concept —
and, like always-on on skills, the tool-level flag couldn't vary by profile.

Consolidate on :terminal-tools. Both tools already pair their terminal result
with :structured-output (failures return only :output), so success-gated
:terminal-tools is behaviour-preserving:
- :sql adds ask_for_sql_clarification to its terminal set.
- :document-generate-content marks document_construct_model_chart and
  document_construct_sql_chart terminal (schema collection stays preparatory).

Remove :final-response? from the tools and drop has-final-response? from the
loop. No frontend/persistence consumer read the flag.

Co-authored-by: Mihael Konjević <konjevic@gmail.com>
…6204)

Limit max sync / scan / fingerprint fields (metabase#76189)

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Do not fingerprint MongoDB binary blobs (metabase#76214)

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Fix test flake related to search analytics (metabase#75576)
…d" (metabase#76229)

Admin notifications group by notification id not card id (metabase#75928)

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
…g from results (metabase#76233)

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Fix OOM with json/decode (metabase#76221)

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
…base#76256)

Fix OOM with sync metadata for postgres/redshift (metabase#76218)

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
)

Preserve nulls in day-of-week calculation (metabase#76180)

Closes QUE2-687

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Fix OOM with BigQuery and Redshift sync (metabase#76276)

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Include dashId in downloaded diagnostics (metabase#75803)

* Include dashId in downloaded diagnostics

* e2e test

---------

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>

* Fixing backport

---------

Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
metabase#76274)

* Fix appdb reindex misdirecting writes to the live index (metabase#76199)

* Fixing backport

---------

Co-authored-by: Nathan Voxland <nathan@metabase.com>
Improve dataset creation for Bigquery (metabase#76031)

* Stop deleting datasets for now in bigquery.

* Don't assume the database existing means all tables exist.
Update generated docs (release-x.62.x)

Co-authored-by: github-actions <github-actions@github.com>
…etabase#76335)

Validate sqlserver additional options when connecting (metabase#76293)

validate sqlserver optios when connecting

Co-authored-by: Riley Thompson <riley@metabase.com>
metabase#76267)

Dont override lib temporal types if driver type is text (metabase#75663)

* dont override lib temporal types if driver type is text

* make test driver agnostic

* fix tests

* add sqlite test

* rename unreliable-driver-type? to prefer-lib-type?



---------

Co-authored-by: Riley Thompson <riley@metabase.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…tion" (metabase#76339)

UXW-4443: Re-encrypt additional columns during key rotation (metabase#75536)

* UXW-4443: Re-encrypt additional columns during key rotation

There were 6 columns that we weren't correctly rotating during key
rotation. This is a mechanical change to add them to the list of things
we rotate. The "Right Way" to fix this would probably be closer to
UXW-4444 (a refactor that mechanically prevents this from happening
again) but this triage makes sense for now.

[UXW-4443: Re-encrypt additional encrypted columns during key rotation](https://linear.app/metabase/issue/UXW-4443/re-encrypt-additional-encrypted-columns-during-key-rotation)

* fix test

Co-authored-by: John Swanson <john.swanson@metabase.com>
Further pdf export improvements (metabase#76016)

* use cssText

* include font props on <text>, <tspan>, and <textPath> nodes

* re-apply SVG transform after cssText copy in html2canvas patch

Chrome's getComputedStyle().cssText is empty, so the cssText copy wiped the Leaflet grid SVG's transform and offset its serialized bounds, re-breaking the metabase#68751 PDF-export grid fix. Re-apply transform/transform-origin/transform-box after the cssText copy; Firefox keeps the bulk-copy perf win. Re-enable the GridMapPdfExport loki test with its baseline reference.

* resolve vars in svg props

* fix hidden row chart axis labels

* gauge value positioning

* fix gauge chart labels

* re-skip grid map loki test

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
…6369)

Notification temp storage across dashboard (metabase#76355)

Co-authored-by: dpsutton <dan@dpsutton.com>
…metabase#76378)

docs: Add note about public links and database routing (metabase#76152)

Add note about public links and database routing

Clarified that public links do not function with database routing.

Co-authored-by: Alex Yarosh <alexandra@metabase.com>
@trevorgowing trevorgowing deleted the sync/upstream-2026-06-25 branch June 28, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.