Skip to content

fix(core): preserve locale casing in API locale filter (#1551)#2174

Merged
ascorbic merged 11 commits into
mainfrom
fix/1551-locale-casing
Jul 22, 2026
Merged

fix(core): preserve locale casing in API locale filter (#1551)#2174
ascorbic merged 11 commits into
mainfrom
fix/1551-locale-casing

Conversation

@ascorbic

@ascorbic ascorbic commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Fixes content and search APIs returning zero results when filtering by a locale with an uppercase region or script subtag, such as zh-TW, pt-BR, or zh-Hant.

The localeCode validator lowercased explicit locale values while configuration, stored content, and public queries preserve BCP-47 casing. SQLite and D1 compare these values case-sensitively. This preserves validated input casing and canonicalizes existing content rows to configured locale casing.

This replaces #1572 because its organization-owned fork rejected maintainer pushes while resolving merge conflicts.

Closes #1551

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (or targeted tests for my change)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (not applicable: no admin UI strings)
  • I have added a changeset (if this PR changes a published package)
  • New features link to an approved Discussion: https://github.com/emdash-cms/emdash/discussions/... (not applicable: bug fix)

AI-generated code disclosure

  • This PR includes AI-generated code: Claude Opus 4.8 and GPT-5.6 Sol

Screenshots / test output

Verified after merging current main:

  • pnpm lint:quick
  • pnpm --filter emdash exec vitest run tests/unit/api/schemas.test.ts tests/unit/database/migrations/054_canonicalize_locale_casing.test.ts (48 tests)
  • pnpm --filter emdash exec vitest run tests/integration/database/migrations.test.ts (19 tests)
  • pnpm build
  • pnpm typecheck

Try this PR

Open a fresh playground →

A full working EmDash site, deployed from this branch. Each visit gets its own session-scoped sandbox: no login needed and no shared state. Try the admin, edit content, hit the public site.

Tracks fix/1551-locale-casing. Updated automatically when the playground redeploys.

marcusbellamyshaw-cell and others added 4 commits July 12, 2026 09:44
The `localeCode` validator lowercased the `?locale=` filter and create-body
locale, but config `locales`/`defaultLocale`, the stored `locale` column, and
the public query path all keep the raw BCP-47 casing. As a result the content
and search APIs returned zero rows for any locale with an uppercase region or
script subtag (e.g. zh-TW, pt-BR, zh-Hant).

Drop the `.toLowerCase()` transform so the value is preserved verbatim;
validation stays case-insensitive. This also matches the sibling
`localeFilterQuery` (taxonomies/menus), which never lowercased.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…#1551)

The shared `localeFilterQuery` (used by taxonomy and menu endpoints) still
used a plain `z.string()` while `contentListQuery` and the search schemas
adopted the stricter, casing-preserving `localeCode`. Reusing `localeCode`
here tightens BCP-47 validation and keeps locale handling consistent across
the API: a malformed `?locale=` value is now rejected instead of silently
matching zero rows. Addresses non-blocking review feedback on #1572.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses review feedback on #1572: removing localeCode's lowercasing
transform fixed new locale-filtered queries against the canonical
casing, but pre-existing rows already stored under the lowercased form
(e.g. zh-tw) were still missed by an exact locale = 'zh-TW' filter.

Adds migration 044, which canonicalizes the locale column on every
ec_* table against getI18nConfig().locales, case-insensitively. No-op
when i18n isn't configured. Not reversible (down is a no-op) -- the
original incorrect casing isn't recoverable and re-lowercasing would
reintroduce the bug.

Updates the "partially applied" migration-runner test to include the
new trailing migration.
Copilot AI review requested due to automatic review settings July 21, 2026 13:24
@changeset-bot

changeset-bot Bot commented Jul 21, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 730e414

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
Name Type
emdash Patch
@emdash-cms/cloudflare Patch
@emdash-cms/sandbox-workerd Patch
@emdash-cms/plugin-mcp-smoke Patch
@emdash-cms/fixture-perf-site Patch
@emdash-cms/perf-demo-site Patch
@emdash-cms/cache-demo-site Patch
@emdash-cms/do-demo-site Patch
@emdash-cms/do-solo-demo-site Patch
@emdash-cms/admin Patch
@emdash-cms/auth Patch
@emdash-cms/blocks Patch
@emdash-cms/gutenberg-to-portable-text Patch
@emdash-cms/x402 Patch
create-emdash Patch
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 21, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-playground 730e414 Jul 22 2026, 06:00 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 21, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-demo-cache 730e414 Jul 22 2026, 06:01 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 21, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-demo-do 730e414 Jul 22 2026, 05:59 AM

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
docs 6a07613 Jul 21 2026, 01:27 PM

@pkg-pr-new

pkg-pr-new Bot commented Jul 21, 2026

Copy link
Copy Markdown

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@2174

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@2174

@emdash-cms/auth-atproto

npm i https://pkg.pr.new/@emdash-cms/auth-atproto@2174

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@2174

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@2174

@emdash-cms/contentful-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/contentful-to-portable-text@2174

emdash

npm i https://pkg.pr.new/emdash@2174

create-emdash

npm i https://pkg.pr.new/create-emdash@2174

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@2174

@emdash-cms/plugin-cli

npm i https://pkg.pr.new/@emdash-cms/plugin-cli@2174

@emdash-cms/plugin-types

npm i https://pkg.pr.new/@emdash-cms/plugin-types@2174

@emdash-cms/registry-client

npm i https://pkg.pr.new/@emdash-cms/registry-client@2174

@emdash-cms/registry-lexicons

npm i https://pkg.pr.new/@emdash-cms/registry-lexicons@2174

@emdash-cms/registry-verification

npm i https://pkg.pr.new/@emdash-cms/registry-verification@2174

@emdash-cms/sandbox-workerd

npm i https://pkg.pr.new/@emdash-cms/sandbox-workerd@2174

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@2174

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@2174

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@2174

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@2174

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@2174

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@2174

@emdash-cms/plugin-field-kit

npm i https://pkg.pr.new/@emdash-cms/plugin-field-kit@2174

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@2174

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@2174

commit: 730e414

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes locale-filtered content/search endpoints returning zero rows when the locale includes uppercase subtags (e.g. zh-TW, pt-BR, zh-Hant) by preserving the validated locale string’s original casing and backfilling previously-lowercased stored locales.

Changes:

  • Remove lowercasing transform from the localeCode schema and apply localeCode validation to the shared localeFilterQuery.
  • Add migration 054_canonicalize_locale_casing and register it in the migration runner.
  • Add unit/integration regression tests and a changeset entry for the fix.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
packages/core/src/api/schemas/common.ts Stops lowercasing locale values and tightens ?locale= validation via localeCode.
packages/core/src/database/migrations/054_canonicalize_locale_casing.ts Backfills existing ec_* rows whose locale casing doesn’t match configured locale casing.
packages/core/src/database/migrations/runner.ts Registers migration 054 in the static migration map.
packages/core/tests/unit/api/schemas.test.ts Adds schema-level regression coverage ensuring locale casing is preserved and malformed values reject.
packages/core/tests/unit/database/migrations/054_canonicalize_locale_casing.test.ts Adds unit coverage for the backfill migration behavior.
packages/core/tests/integration/database/migrations.test.ts Includes migration 054 in integration migration coverage.
.changeset/localecode-preserve-casing.md Publishes release notes for the bug fix and migration backfill.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +31 to +35
await sql`
UPDATE ${table}
SET locale = ${locale}
WHERE lower(locale) = lower(${locale}) AND locale != ${locale}
`.execute(db);
import { listTablesLike } from "../dialect-helpers.js";

/**
* Backfill migration for #1572.
Comment thread packages/core/src/api/schemas/common.ts Outdated
"emdash": patch
---

Fixes the content and search APIs returning zero results when filtering by a locale with an uppercase region or script subtag (e.g. `?locale=zh-TW`, `pt-BR`, `zh-Hant`). The `localeCode` validator lowercased the value, but config locales, the stored `locale` column, and the public query path all preserve the original casing, so the filter matched nothing. Validation stays case-insensitive; the value is now preserved verbatim. The taxonomy and menu endpoints' shared `?locale=` filter now applies the same BCP-47 validation, so a malformed locale is rejected with a clear error instead of silently matching nothing. A migration backfills existing content rows that were previously stored under the lowercased casing so upgraded sites match immediately, not just new content. Closes #1551.

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right fix for #1551. Removing the unconditional lowercasing in localeCode and relying on the casing that the Astro config already preserves is the correct direction. The migration is needed, and the tests cover the core backfill path.

I checked the changed schemas, the new migration, the runner registration, the integration migration test, and the unit schema/migration tests. I also traced call-sites of localeCode, localeFilterQuery, getI18nConfig, and the denormalized locale columns in content_taxonomies.

Headline findings:

  1. The migration only updates ec_* tables, not the denormalized content_taxonomies.locale column. Migration 051 denormalizes content status/locale/etc. onto content_taxonomies so taxonomy-filtered listings can seek by pivot index, then re-check the real row. If ec_post.locale is canonicalized to zh-TW but the pivot still says zh-tw, the loader’s ct.locale = ${locale} predicate will filter those rows out before the re-check ever runs. The migration must keep the pivot in sync, and the unit test should assert that.

  2. AGENTS.md comment discipline is violated in the new code and tests. Issue/PR references (#1551, #1572) and PR-describing prose appear in JSDoc blocks and test names. Those references belong in commit messages / PR descriptions, not in code comments.

  3. Unverified edge-case risk: because SQLite’s (slug, locale) unique index is case-sensitive, a site that already created a duplicate row under the canonical casing (e.g. because the canonical query returned nothing) will hit a UNIQUE constraint failed when the migration tries to canonicalize the older row. I didn’t find logic to detect/merge such duplicates. Worth a sanity-check, but I couldn’t confirm it would occur without the project’s runtime data.

  4. Future-write casing: explicit locale values in contentCreateBody are now stored verbatim. If a write ever uses a non-canonical casing (e.g. zh-tw while zh-TW is configured), canonical queries will again return zero rows. The admin picker likely prevents this; flagging as a cautious note rather than a blocker.

The PR is close, but the stale content_taxonomies.locale issue should be fixed before merge.

Comment on lines +28 to +38
for (const tableName of tableNames) {
const table = sql.ref(tableName);
for (const locale of locales) {
await sql`
UPDATE ${table}
SET locale = ${locale}
WHERE lower(locale) = lower(${locale}) AND locale != ${locale}
`.execute(db);
}
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[needs fixing] The migration canonicalizes locale on every ec_* table, but it leaves the denormalized content_taxonomies.locale column stale. Migration 051 copies locale from ec_* onto content_taxonomies so the loader can index-seek taxonomy+locale filters (loader.ts uses ct.locale = ${locale} before re-checking ec_*). After this migration, the pivot will still say zh-tw while the content row says zh-TW, so taxonomy-filtered listings will miss rows.

Update content_taxonomies in the same per-table loop, using the collection slug (ec_<slug>collection = <slug>):

Suggested change
for (const tableName of tableNames) {
const table = sql.ref(tableName);
for (const locale of locales) {
await sql`
UPDATE ${table}
SET locale = ${locale}
WHERE lower(locale) = lower(${locale}) AND locale != ${locale}
`.execute(db);
}
}
}
for (const tableName of tableNames) {
const table = sql.ref(tableName);
const slug = tableName.slice("ec_".length);
for (const locale of locales) {
await sql`
UPDATE ${table}
SET locale = ${locale}
WHERE lower(locale) = lower(${locale}) AND locale != ${locale}
`.execute(db);
await sql`
UPDATE content_taxonomies
SET locale = ${locale}
WHERE collection = ${slug}
AND lower(locale) = lower(${locale})
AND locale != ${locale}
`.execute(db);
}
}

Add a unit test that inserts a content_taxonomies row for the lowercased locale and asserts it is also rewritten to the canonical casing.

import { listTablesLike } from "../dialect-helpers.js";

/**
* Backfill migration for #1572.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[needs fixing] AGENTS.md: comments must not reference issues, PRs, or review threads, and should not repeat the PR description. The large JSDoc here is a change narrative. Trim it to a concise, future-facing explanation.

Suggested change
* Backfill migration for #1572.
/**
* Canonicalize locale casing on existing content rows.
*
* The old `localeCode` schema lowercased explicit values, so a site with a
* configured locale like `zh-TW` could have rows stored as `zh-tw`. Since
* config, columns, and query params now preserve casing, update any row whose
* locale case-insensitively matches a configured locale to that locale's
* canonical form.
*/

Comment thread packages/core/src/api/schemas/common.ts Outdated
* BCP 47 locale code — language with optional script/region subtags (e.g. en, en-US, pt-BR, es-419, zh-Hant).
* Validation is case-insensitive, but the value is preserved verbatim: config `locales`/`defaultLocale`, the
* stored `locale` column, and the public query path all keep the raw casing, so lowercasing the `?locale=`
* filter here made it match zero rows for locales with uppercase subtags (#1551).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[needs fixing] AGENTS.md: remove the issue reference from the comment. The rationale (preserve raw casing because the stored column and query use it) is worth keeping; the (#1551) marker is not.

Suggested change
* filter here made it match zero rows for locales with uppercase subtags (#1551).
* stored `locale` column, and the public query path all keep the raw casing, so lowercasing the `?locale=`
* filter here made it match zero rows for locales with uppercase subtags.

import { setupTestDatabaseWithCollections, teardownTestDatabase } from "../../../utils/test-db.js";

/**
* Regression coverage for #1572.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[needs fixing] AGENTS.md: remove the issue reference and PR-describing prose from the comment. The describe name already identifies the migration; the block comment can be deleted or replaced with a one-line purpose.

Suggested change
* Regression coverage for #1572.
/** Migration 054 rewrites content rows to the configured locale casing. */

* canonical casing, but pre-existing rows would still be missed by an exact
* `locale = 'zh-TW'` filter without a data backfill.
*/
describe("migration 054: canonicalize locale casing (#1572)", () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[needs fixing] AGENTS.md: test names should not reference issue/PR numbers.

Suggested change
describe("migration 054: canonicalize locale casing (#1572)", () => {
describe("migration 054: canonicalize locale casing", () => {

});
});

describe("localeCode validator (#1551)", () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[needs fixing] AGENTS.md: remove the issue reference from the test describe name.

Suggested change
describe("localeCode validator (#1551)", () => {
describe("localeCode validator", () => {

@github-actions github-actions Bot added review/awaiting-author Reviewed; waiting on the author to respond and removed review/needs-review No maintainer or bot review yet labels Jul 21, 2026
@github-actions github-actions Bot added review/needs-rereview Author pushed changes since the last review and removed review/awaiting-author Reviewed; waiting on the author to respond labels Jul 21, 2026
@ascorbic ascorbic added the bot:review Trigger an emdashbot code review on this PR label Jul 21, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right fix for #1551. Removing the unconditional lowercasing in localeCode and relying on the BCP-47 casing already preserved by Astro config is the correct direction, and the migration back-fills existing ec_* rows and the denormalized content_taxonomies pivot.

What changed since the last review:

  • Fixed: the migration now keeps content_taxonomies.locale in sync with ec_*. The up() function updates content tables and then updates pivot rows whose authoritative content row has already been canonicalized, exactly what the loader's ct.locale = ${locale} seek needs.
  • Fixed: the migration guards against the case-sensitive (slug, locale) unique index on ec_* by skipping rows that would collide with an existing canonical row.
  • Fixed: issue/PR number references in comments are gone.

Still open: AGENTS.md comment discipline is still violated by PR-describing prose in the new JSDoc, test block comment, migration down(), and migration test. These narrate the bug that was fixed ("made it match zero rows", "pre-fix row", "the bug this fixes") rather than giving a durable "why" to future readers.

Approach check: the change fits EmDash's architecture. localeCode preserving casing works because getI18nConfig() already carries the config's casing, the write path defaults to defaultLocale, and TaxonomyRepository stamps the content row's locale onto new pivot rows. I didn't find another codepath that lowercases locales.

One cautious note: localeFilterQuery now requires BCP-47 format with preserved casing for list endpoints, but createMenuBody, createTaxonomyDefBody, createTermBody, and createRelationBody still accept z.string().min(1) for locale. A write using a non-canonical casing could be stored and then missed by a canonical list query. The admin UI likely forces the canonical value via a picker, so this is a consistency gap rather than a user-visible bug right now.

Headline conclusion: the functional fix is solid and the pivot backfill is now correct. Clean up the remaining comment narrative and consider widening localeCode to the other create bodies, then this can land.


Findings

  • [needs fixing] packages/core/src/api/schemas/common.ts:56-61

    The JSDoc explains the non-obvious "why" in its first two sentences, but the final clause narrates the PR bug ("so lowercasing the ?locale= filter here made it match zero rows..."). AGENTS.md says comments must not be PR descriptions or summaries of the change.

    /**
     * BCP 47 locale code — language with optional script/region subtags (e.g. en, en-US, pt-BR, es-419, zh-Hant).
     * Validation is case-insensitive, but the value is preserved verbatim because the site config,
     * stored `locale` columns, and public query path all keep the raw BCP-47 casing.
     */
    
  • [needs fixing] packages/core/tests/unit/api/schemas.test.ts:128-131

    The block comment restates what the tests assert and then narrates the bug that was fixed ("lowercasing the ?locale= filter ... made it miss every row stored under an uppercase subtag"). Test comments should explain non-obvious invariants, not PR history.

    describe("localeCode validator", () => {
    	// The site config, stored locale columns, and public query path keep raw BCP-47 casing.
    	// The schema must preserve that casing instead of normalizing it.
    
  • [needs fixing] packages/core/src/database/migrations/054_canonicalize_locale_casing.ts:46-48

    The down() comment is useful in stating the migration is irreversible, but "reintroduce the bug this fixes" references the PR rather than the durable invariant. Keep the irreversibility reason without the bug narrative.

    export async function down(_db: Kysely<unknown>): Promise<void> {
    	// Not reversible: the original casing is not recoverable.
    }
    
  • [needs fixing] packages/core/tests/unit/database/migrations/054_canonicalize_locale_casing.test.ts:32

    "pre-fix" is PR-describing narrative. The comment should describe the test setup, not which code version it simulates.

    		// Simulate a row saved under the lowercased locale.
    
  • [suggestion] packages/core/src/api/schemas/menus.ts:27

    createMenuBody.locale still accepts any non-empty string, while localeFilterQuery now requires a BCP-47 code with preserved casing. A menu created with a non-canonical casing (e.g. zh-tw when the configured locale is zh-TW) will be stored verbatim and then missed by GET /_emdash/api/menus?locale=zh-TW. The same applies to createTaxonomyDefBody.locale, createTermBody.locale, and createRelationBody.locale. Consider importing localeCode from common.js and using it for these body schemas as well, so writes cannot introduce a casing that the fixed list filters will miss.

@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Jul 21, 2026
@github-actions github-actions Bot added review/awaiting-author Reviewed; waiting on the author to respond and removed review/needs-rereview Author pushed changes since the last review labels Jul 21, 2026
@ascorbic ascorbic added the bot:review Trigger an emdashbot code review on this PR label Jul 21, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a re-review of the locale-casing fix for #1551. The approach remains correct: preserving BCP-47 casing in localeCode and canonicalizing writes to the configured casing is the right way to fix the SQLite/D1 case-sensitivity mismatch.

The first issue from the prior review is resolved. All the write bodies flagged previously (createMenuBody, createTaxonomyDefBody, createTermBody, createRelationBody, bylineCreateBody, bylineTranslationCreateBody) now use localeCode, and the corresponding handlers plus the admin bylines route call resolveConfiguredLocale before storing or filtering. That gap is closed.

Two issues remain:

  1. Taxonomy read endpoints do not canonicalize ?locale=. handleTaxonomyList, handleTermList, handleTermGet, handleTermUpdate, and handleTermDelete pass the raw query-string locale directly to the repository. Because localeCode now preserves caller casing, a request like ?locale=zh-tw will query for exact 'zh-tw' rows and miss terms/defs stored with canonical 'zh-TW' by the fixed write handlers. This is the same class of casing mismatch the PR is meant to eliminate, just moved to the taxonomy path. Content, menu, relation, and byline endpoints already canonicalize; taxonomy reads should too.

  2. The runtime repair is still narrower than the changeset claims. repairLocaleCasing only rewrites ec_% content tables and content_taxonomies, but the changeset says it fixes “locale-filtered content, search, taxonomy, and menu results” and that “existing content rows saved with lowercased locale values are repaired automatically.” Existing rows in _emdash_menus, _emdash_taxonomy_defs, taxonomies, _emdash_relations, _emdash_bylines, and _emdash_menu_items are not repaired, so the new locale-preserving filters still miss historical lowercased values there.

Both are fixable extensions of the same pattern the PR already uses for content.


Findings

  • [needs fixing] packages/core/src/api/handlers/taxonomies.ts:179

    Taxonomy definition list filters by options.locale without canonicalizing it to the configured casing. Since localeCode now preserves caller casing, a request like GET /_emdash/api/taxonomies?locale=zh-tw queries _emdash_taxonomy_defs.locale = 'zh-tw' and misses definitions stored with canonical zh-TW by handleTaxonomyCreate.

    handleContentList, handleMenuList, and handleRelationList already canonicalize the locale filter via resolveConfiguredLocale; taxonomy reads should do the same:

    		const locale = options.locale ? resolveConfiguredLocale(options.locale) : undefined;
    		if (locale !== undefined) query = query.where("locale", "=", locale);
    
  • [needs fixing] packages/core/src/api/handlers/taxonomies.ts:395

    Term read paths also pass the raw options.locale to the repository: handleTermList (line 395), handleTermGet (line 641), handleTermUpdate (line 749), and handleTermDelete (line 838). With localeCode preserving caller casing and handleTermCreate storing canonical zh-TW, a ?locale=zh-tw lookup returns no results. Canonicalize options.locale before passing it to repo.findByName/repo.findBySlug:

    		const locale = options.locale ? resolveConfiguredLocale(options.locale) : undefined;
    		const terms = await repo.findByName(taxonomyName, { locale });
    

    The same fix applies to the repo.findBySlug(taxonomyName, termSlug, locale) calls at lines 641, 749, and 838.

  • [needs fixing] packages/core/src/i18n/repair-locale-casing.ts:12

    repairLocaleCasing only discovers ec_% content tables and then updates content_taxonomies, so it does not repair existing locale values in _emdash_menus, _emdash_taxonomy_defs, taxonomies, _emdash_relations, _emdash_bylines, or _emdash_menu_items.

    The changeset says the fix covers “locale-filtered content, search, taxonomy, and menu results” and that “existing content rows saved with lowercased locale values are repaired automatically.” Historical lowercased rows in those tables remain missed by the new locale-preserving list filters. Either extend repairLocaleCasing to those tables (with the same slug-uniqueness collision handling used for content), or narrow the changeset to the tables the repair actually rewrites:

    Fixes locale-filtered content and search results for locales with uppercase region or script subtags. Existing content rows saved with lowercased locale values are repaired automatically.
    

@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Jul 21, 2026
@github-actions github-actions Bot added review/awaiting-author Reviewed; waiting on the author to respond and removed review/needs-rereview Author pushed changes since the last review labels Jul 21, 2026
@github-actions github-actions Bot added review/needs-rereview Author pushed changes since the last review and removed review/awaiting-author Reviewed; waiting on the author to respond labels Jul 21, 2026
@ascorbic ascorbic added the bot:review Trigger an emdashbot code review on this PR label Jul 21, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right fix for #1551: preserve BCP-47 casing in localeCode, resolve every caller-supplied locale to the exact casing configured for the site, canonicalize writes, and repair existing content rows at runtime. Both issues from the prior review are addressed: taxonomy read endpoints (handleTaxonomyList, handleTermList, handleTermGet, handleTermUpdate, handleTermDelete) now call resolveConfiguredLocale, and the changeset was narrowed to match the tables repairLocaleCasing actually rewrites (ec_% content tables and content_taxonomies pivots). The implementation fits EmDash's handler/repository split and follows SQL-safety conventions.

I checked the diff, the changed handler/schema/seed/runtime files, and call sites for the new resolveConfiguredLocale. The only remaining gap is the public search/suggest path. searchQuery and searchSuggestQuery were updated to use localeCode (so casing is preserved), but packages/core/src/search/query.ts still passes options.locale straight into the FTS SQL. A GET /_emdash/api/search?q=hello&locale=zh-tw will therefore look for rows whose locale is exactly 'zh-tw', miss the canonical 'zh-TW' rows stored by the fixed write path, and return empty results. Content, menu, relation, byline, and taxonomy handlers all canonicalize before querying; search should do the same.

This is a fixable extension of the same pattern already applied everywhere else. Once searchSingleCollection and getSuggestions resolve the locale, the changeset's claim that "search results" are fixed will be accurate.


Findings

  • [needs fixing] packages/core/src/search/query.ts:253

    searchSingleCollection filters c.locale with the raw options.locale value. Because searchQuery now preserves caller casing, a request like ?locale=zh-tw queries for exact 'zh-tw' and misses entries indexed under canonical 'zh-TW' by the fixed write path. Content, menu, relation, byline, and taxonomy handlers all call resolveConfiguredLocale before filtering; search should too.

    Add the import and canonicalize the locale before building the FTS SQL:

    import { resolveConfiguredLocale } from "../i18n/config.js";
    
    	const locale = options.locale ? resolveConfiguredLocale(options.locale) : undefined;
    

    The same change is needed for getSuggestions around line 400 (const locale = options.locale;). A regression test should cover searching/suggesting with a lowercase input locale against content stored with canonical casing (e.g. zh-TW).

@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Jul 21, 2026
@github-actions github-actions Bot added review/awaiting-author Reviewed; waiting on the author to respond and removed review/needs-rereview Author pushed changes since the last review labels Jul 21, 2026
@github-actions github-actions Bot added review/needs-rereview Author pushed changes since the last review and removed review/awaiting-author Reviewed; waiting on the author to respond labels Jul 21, 2026
@ascorbic ascorbic added the bot:review Trigger an emdashbot code review on this PR label Jul 21, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right fix for #1551 and the follow-up addresses the remaining gap from the prior review.

What changed since last review:

  • packages/core/src/search/query.ts now canonicalizes the locale in both searchSingleCollection and getSuggestions via resolveConfiguredLocale, so public search/suggest requests with ?locale=zh-tw correctly match zh-TW rows stored by the fixed write path.
  • Regression tests in tests/integration/search/suggest.test.ts cover search and suggest with lowercase input against canonical-cased content.

What I checked:

  • The diff, all changed handler/schema/seed/runtime files, and the new repairLocaleCasing implementation.
  • SQL safety in repairLocaleCasing: table identifiers come from listTablesLike and are passed through sql.ref(); configured locale values are parameterized; no raw interpolation of untrusted input.
  • Call sites for resolveConfiguredLocale across content, menus, relations, taxonomies, bylines, and now search/suggest — every locale-filtered read/write path in the diff canonicalizes.
  • The runtime repair gating: it only runs when at least one configured locale has a hyphen or uppercase letter, or a previous repair key exists and the version changed, then records the new version in options.
  • Tests added for canonical create/list/get, seed validation, runtime repair, and case-variant collision handling.

Headline conclusion: the prior finding is resolved, the change is consistent with EmDash's handler/repository split, and the patch appears ready to merge.

@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Jul 21, 2026
@github-actions github-actions Bot added review/approved Approved; no new commits since overlap and removed review/needs-rereview Author pushed changes since the last review labels Jul 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Overlapping PRs

This PR modifies files that are also changed by other open PRs:

This may cause merge conflicts or duplicated work. A maintainer will coordinate.

@github-actions

Copy link
Copy Markdown
Contributor

Visual regression: 1 screen changed

These admin screens render differently on this PR. Confirm every change below is intended before accepting.

Screen Image Compare
dashboard-rtl expected · actual

Maintainers: if every change above is intended, comment /accept-baselines
to commit the regenerated Linux baselines to this PR.

Measured head: 730e4143de47fdf1c37bba04b364a9b60f4e3a13. /accept-baselines accepts the snapshots for this commit.

github-actions Bot pushed a commit that referenced this pull request Jul 22, 2026
@ascorbic

Copy link
Copy Markdown
Collaborator Author

/accept-baselines

@github-actions

Copy link
Copy Markdown
Contributor

ℹ️ The committed baselines already match the candidates; nothing to commit.

@ascorbic
ascorbic merged commit c568876 into main Jul 22, 2026
46 of 47 checks passed
@ascorbic
ascorbic deleted the fix/1551-locale-casing branch July 22, 2026 09:10
@emdashbot emdashbot Bot mentioned this pull request Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

localeCode schema lowercases the ?locale= filter, returning zero rows for locales with uppercase subtags

3 participants