Skip to content

test(drivers): a conformance run that discovers zero drivers is a failure, not an OK (#4646) - #4648

Merged
os-zhuang merged 1 commit into
mainfrom
claude/turso-driver-evaluation-beoewn
Aug 2, 2026
Merged

test(drivers): a conformance run that discovers zero drivers is a failure, not an OK (#4646)#4648
os-zhuang merged 1 commit into
mainfrom
claude/turso-driver-evaluation-beoewn

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Closes #4646.

问题

scripts/check-driver-conformance.mjs 从磁盘发现 driver 包,根目录写死在 DRIVERS_DIRlistDir 吞掉 ENOENT 返回 [],而三条不变量全部遍历「发现到的集合」:

于是一个失效的 DRIVERS_DIR 会打印 OK — 0 covered cell(s) 并 exit 0。

先把范围说准:CI 从来没有这个暴露。 lint.yml:304 跑的是 pnpm check:driver-conformance,即 --self-test && audit,self-test 里有一条 driver 发现断言会先失败。假绿只出现在脚本 header 第 21 行文档化的 node scripts/check-driver-conformance.mjs 裸调用上。

为什么不是「self-test 已经管了,就这样吧」

那条守卫本身有两个问题:

  1. 它读 drivers.length >= 3 && drivers.includes('driver-sql') —— 一个硬编码的 driver 名和数量,出现在唯一一份声明「driver 从磁盘发现、绝不列表」(:145)的脚本里。下次加 driver 或挪包时两处都要手改,而那正是守卫该发挥作用的时刻。
  2. 失败信息 discovers driver packages from disk 既不提 DRIVERS_DIR 也不说路径已失效,踩中的人得自己去找。

改动

audit() 增加第四条不变量 DISCOVERED,信息里带上实际搜索的目录:

x DISCOVERED: no driver package found under packages/drivers/. Either these
  packages moved and DRIVERS_DIR is stale, or they are gone. Every other
  invariant iterates the discovered set, so a zero-driver run reports OK having
  checked nothing — it fails here instead.

检查体抽成 discoveredErrors(drivers),好让 self-test 驱动这条不变量本身而不是它的替身;self-test 里对 driver 名和数量的断言随之删除。

为什么不给 case-set 轴加同样的检查

它不会这样烂掉,实测过。 CASE_SETS 是脚本里声明的预期,所以 spec/src/data 消失时 CLASSIFIED 的反向对账会逐条报错:

x CLASSIFIED: CASE_SETS names FILTER_LOGIC_CASES, which filter-logic-conformance.ts no longer exports.
... (5 条)

driver 轴是纯磁盘发现、没有任何声明预期可对账 —— 这个不对称正是「零」只在一条轴上可达的原因,也正是 DISCOVERED 补上的那一块。给 case-set 轴再加一层是冗余。

验证

场景 修复前 修复后
正常 audit OK,20 cells OK,20 cells(不变)
正常 --self-test OK OK
DRIVERS_DIR 失效 → audit exit 0,"OK — 0 covered cell(s)" exit 1,DISCOVERED 指名路径
DRIVERS_DIR 失效 → --self-test exit 1 exit 1
CASE_SETS_DIR 失效 → audit exit 1(CLASSIFIED × 5) 不变

pnpm check:driver-conformance(self-test + audit)全绿,eslint 干净。失效场景通过复制脚本改常量的方式复现,未改动仓库内容。

来由

来自 turso driver 归属评估(#4645):那次重组要把 packages/plugins/driver-* 挪到 packages/drivers/*,正好会动 DRIVERS_DIR 和 self-test 里那两个硬编码值。但这个缺陷独立于重组成立,所以单独出 PR,作为 #4645 的前置项。

分支名不走 claude/issue-<n>-<slug> 约定,因为本会话被指派了固定分支 claude/turso-driver-evaluation-beoewn


Generated by Claude Code

…lure, not an OK (#4646)

check-driver-conformance discovers driver packages from disk under a hardcoded
DRIVERS_DIR. listDir swallows ENOENT and returns [], and all three invariants
iterate the discovered set — CONSUMED over `drivers`, RECONCILED over LEDGER
(empty since #4405, the intended steady state), CLASSIFIED not over drivers at
all. A stale DRIVERS_DIR therefore printed `OK — 0 covered cell(s)` and exit 0.

CI never had this exposure: lint.yml runs `pnpm check:driver-conformance`, which
is `--self-test && audit`, and the self-test carried a discovery assertion. The
false green was on the bare `node scripts/check-driver-conformance.mjs` the
script's own header documents as a usage.

Leaving the guard there was wrong twice over. It read
`drivers.length >= 3 && drivers.includes('driver-sql')` — a hardcoded name and
count inside the one script whose stated rule is that drivers come from disk and
are never listed, so both wanted hand-editing on the next driver added or
package moved, which is when the guard earns its keep. And its failure text
named neither DRIVERS_DIR nor the stale path.

DISCOVERED is now a fourth invariant in audit(), and its message names the
directory searched. The self-test drives the invariant in both directions
instead of standing in for it, and asserts nothing about which drivers exist.

The case-set axis cannot rot this way and is left alone: CASE_SETS is a declared
expectation, so a vanished spec/src/data fails CLASSIFIED's reverse direction
with one error per case-set. The driver axis is disk-discovery with nothing
declared to reconcile against — that asymmetry is why zero was reachable on one
axis and not the other, and it is what DISCOVERED supplies.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VTj56JLhVk585TA8Ez3yGK
@vercel

vercel Bot commented Aug 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 2, 2026 1:53pm

Request Review

@github-actions github-actions Bot added size/s documentation Improvements or additions to documentation tooling and removed size/s labels Aug 2, 2026
@os-zhuang
os-zhuang marked this pull request as ready for review August 2, 2026 14:30
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 2, 2026
Merged via the queue into main with commit fad5240 Aug 2, 2026
18 checks passed
@os-zhuang
os-zhuang deleted the claude/turso-driver-evaluation-beoewn branch August 2, 2026 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

check-driver-conformance 的 audit 路径对「发现零个 driver」报 OK — 零发现该是失败,且唯一的守卫硬编码了 driver-sql

2 participants