chore: add migration tracking scripts and docs

Author: ggazzo

docs/api-endpoint-migration.md

@@ -358,22 +358,29 @@ integrations: {
 
 ### Handling nullable types
 
-When a field can be `null`, combine `nullable: true` with `$ref`:
+Ajv does **not** allow `nullable: true` without `type`. Since `$ref` schemas don't have a `type` at the reference site, you cannot use `{ nullable: true, $ref: '...' }`. Instead, use `oneOf` with `{ type: 'null' }`:
 
 ```typescript
-// Nullable $ref
-report: { nullable: true, $ref: '#/components/schemas/IModerationReport' },
+// Nullable $ref — use oneOf with null
+report: {
+  oneOf: [
+    { $ref: '#/components/schemas/IModerationReport' },
+    { type: 'null' },
+  ],
+},
 
-// Nullable union
+// Nullable union — add null as another oneOf branch
 integration: {
-  nullable: true,
   oneOf: [
     { $ref: '#/components/schemas/IIncomingIntegration' },
     { $ref: '#/components/schemas/IOutgoingIntegration' },
+    { type: 'null' },
   ],
 },
 ```
 
+> **Note**: `nullable: true` works fine with `type`, e.g., `{ type: 'string', nullable: true }`. The restriction only applies when combining `nullable` with `$ref` (no `type` present).
+
 ### Handling intersection types with `allOf`
 
 When a response intersects a type with additional properties (e.g., `VideoConferenceInstructions & { providerName: string }`), use `allOf`:
@@ -440,6 +447,44 @@ This happens because `oneOf` requires **exactly one** match, but the value is bo
 
 **Long-term fix**: Revise the core-typings to narrow `ts` to `string` (which is what MongoDB aggregation pipelines and `JSON.stringify` actually return), or adjust the AJV/typia schema generation to handle `Date | string` unions correctly (e.g., using `anyOf` instead of `oneOf`, or collapsing `Date` into `string`).
 
+### Known Pitfall: Mapped utility types (`Nullable<>`, `Partial<>`, etc.)
+
+Typia does **not** resolve custom mapped types when generating JSON schemas. For example, the following pattern:
+
+```typescript
+type Nullable<T, K extends keyof T> = { [P in K]: T[P] | null } & Omit<T, K>;
+
+export interface IVideoConferenceUser
+  extends Nullable<Pick<Required<IUser>, '_id' | 'username' | 'name' | 'avatarETag'>, 'avatarETag'> {
+  ts: Date;
+}
+```
+
+Produces `avatarETag: { type: 'string' }` **without** `nullable: true`, even though the resolved type is `string | null`. This causes response validation to reject `null` values when `coerceTypes` is `false`.
+
+**Fix**: Declare nullable fields explicitly instead of relying on mapped types:
+
+```typescript
+// BEFORE — typia loses the | null
+extends Nullable<Pick<Required<IUser>, '_id' | 'username' | 'name' | 'avatarETag'>, 'avatarETag'>
+
+// AFTER — typia correctly generates nullable: true
+extends Pick<Required<IUser>, '_id' | 'username' | 'name'> {
+  avatarETag: string | null;
+}
+```
+
+After changing the type, rebuild core-typings (`yarn workspace @rocket.chat/core-typings run build`) to regenerate the schema.
+
+**How to detect**: If a `$ref` schema rejects `null` values at runtime but the TypeScript type allows `null`, check if the type uses a mapped utility type. Inspect the generated schema:
+
+```bash
+node -e "const { schemas } = require('./packages/core-typings/dist/Ajv.js'); \
+  console.log(JSON.stringify(schemas.components.schemas.YourType, null, 2))"
+```
+
+Look for fields that should have `nullable: true` but don't.
+
 ### Adding a new type to typia
 
 If you need a `$ref` for a type that is not yet registered:
@@ -524,6 +569,36 @@ Source: `apps/meteor/app/api/server/v1/invites.ts`
 4. **Augment `Endpoints`**: Use `declare module '@rocket.chat/rest-typings'` to merge the extracted types into the global `Endpoints` interface. This is what makes `useEndpoint('listInvites')` and similar SDK calls type-safe.
 5. **Import `ExtractRoutesFromAPI`** from `'../ApiClass'` (relative to the endpoint file in `v1/`).
 
+### Keep types in `rest-typings` (do NOT remove them)
+
+The `declare module` augmentation via `ExtractRoutesFromAPI` only works within the `apps/meteor` compilation unit. External packages (`ddp-client`, `rest-client`, etc.) compile independently and **do not see** the augmented types — they only see the types exported from `@rocket.chat/rest-typings`.
+
+**When migrating an endpoint, keep its type definition in `rest-typings` unchanged.** The augmentation adds response schema types on top of the existing definition. Removing the type from `rest-typings` will break external package consumers.
+
+This duplication is temporary — see `docs/api-definitions-package-plan.md` for the planned consolidation.
+
+### Use `as const` on options variables
+
+When endpoint options are stored in a separate variable (required for sharing between action factories), add `as const` so that `authRequired: true` is inferred as the literal `true`, not `boolean`. This matters because `TypedThis` uses a conditional type:
+
+```typescript
+userId: TOptions['authRequired'] extends true ? string : string | undefined;
+```
+
+Without `as const`, `authRequired` is `boolean`, and `userId` becomes `string | undefined` — forcing unnecessary guards in the action body.
+
+```typescript
+// Correct — userId is string, bodyParams is typed
+const myEndpointProps = {
+	authRequired: true,
+	body: isMyProps,
+	response: { ... },
+} as const;
+
+// Inline options don't need as const — TypeScript infers literals from the generic context
+API.v1.post('myEndpoint', { authRequired: true, body: isMyProps, response: { ... } }, async function action() { ... });
+```
+
 ### What augmentation enables
 
 Once the `Endpoints` interface is augmented, the entire stack benefits:
@@ -598,15 +673,59 @@ When migrating an endpoint, search for its tests and update:
 
 ## Tracking Migration Progress
 
+Two scripts help track migration progress and identify type-safety issues.
+
+### `scripts/list-unmigrated-api-endpoints.mjs`
+
+Lists all endpoints that still use the legacy `addRoute` pattern and need migration.
+
 ```bash
-# Summary by file
+# Human-readable report grouped by file
 node scripts/list-unmigrated-api-endpoints.mjs
 
-# Full list with line numbers (JSON)
+# Machine-readable JSON with file paths and line numbers
 node scripts/list-unmigrated-api-endpoints.mjs --json
 ```
 
-The script scans for `API.v1.addRoute` and `API.default.addRoute` calls in `apps/meteor/app/api/`.
+The script scans `apps/meteor/app/api/` for `API.v1.addRoute(...)` and `API.default.addRoute(...)` calls, extracting the route path, HTTP methods, file, and line number. Endpoints using this pattern lack compile-time type checking on request params and response shapes.
+
+### `scripts/analyze-weak-types.mjs`
+
+Analyzes `packages/rest-typings/` for "weak" types — generic types that provide little or no type safety in endpoint definitions.
+
+```bash
+# Full report grouped by endpoint
+node scripts/analyze-weak-types.mjs
+
+# JSON output for tooling/CI
+node scripts/analyze-weak-types.mjs --json
+
+# Only check AJV schema definitions (type: 'object' with no properties, etc.)
+node scripts/analyze-weak-types.mjs --schema-only
+
+# Only check TypeScript type definitions (any, Record<string, any>, etc.)
+node scripts/analyze-weak-types.mjs --ts-only
+```
+
+Weak types detected:
+
+| Pattern | Level | Risk |
+|---|---|---|
+| `any`, `unknown` | TypeScript | No type checking at all |
+| `object` (bare) | TypeScript | Accepts any non-primitive |
+| `Record<string, any>`, `Record<string, unknown>` | TypeScript | Untyped key-value bag |
+| `any[]`, `Array<any>` | TypeScript | Untyped array |
+| `Partial<IMessage>`, `Partial<IRoom>`, `Partial<IUser>` | TypeScript | Overly broad — accepts any subset of a large interface |
+| `{ type: 'object' }` without `properties` | AJV Schema | No runtime validation of object shape |
+| `{ type: 'array' }` without `items` | AJV Schema | No runtime validation of array elements |
+
+### Recommended workflow
+
+1. **Pick a domain** to migrate (e.g., channels, users, teams).
+2. **Run `list-unmigrated-api-endpoints.mjs`** to see which endpoints still use `addRoute`.
+3. **Run `analyze-weak-types.mjs`** to check if the existing `rest-typings` for that domain have weak types.
+4. **Migrate the endpoint**: move from `addRoute` to the typed pattern, and strengthen any weak types in `rest-typings` at the same time.
+5. **Re-run both scripts** to verify the endpoint no longer appears in either report.
 
 ## Reference Files
 
@@ -625,3 +744,96 @@ The script scans for `API.v1.addRoute` and `API.default.addRoute` calls in `apps
 | Request validators (examples)    | `packages/rest-typings/src/v1/moderation/`       |
 | Router implementation            | `packages/http-router/src/Router.ts`             |
 | Unmigrated endpoints script      | `scripts/list-unmigrated-api-endpoints.mjs`      |
+| Weak types analysis script       | `scripts/analyze-weak-types.mjs`                 |
+
+## Post-Migration Cleanup Checklist
+
+After migrating endpoints, run through these improvements to maximize quality. These are not blocking but should be addressed before the migration is considered complete.
+
+### 1. Extract shared schemas
+
+Avoid duplicating the same schema inline across endpoints. Common patterns to extract:
+
+```typescript
+// Void success — reuse across all endpoints returning only { success: true }
+const voidSuccessResponse = ajv.compile<void>({
+  type: 'object',
+  properties: { success: { type: 'boolean', enum: [true] } },
+  required: ['success'],
+  additionalProperties: false,
+});
+
+// Paginated response — reuse for list endpoints
+const paginatedUsersResponse = ajv.compile<{ users: IUser[]; count: number; offset: number; total: number }>({
+  type: 'object',
+  properties: {
+    users: { type: 'array' },
+    count: { type: 'number' },
+    offset: { type: 'number' },
+    total: { type: 'number' },
+    success: { type: 'boolean', enum: [true] },
+  },
+  required: ['users', 'count', 'offset', 'total', 'success'],
+  additionalProperties: false,
+});
+
+// User identifier body — reuse for endpoints accepting userId/username/user
+const userIdentifierBodySchema = ajv.compile<{ userId?: string; username?: string; user?: string }>({
+  type: 'object',
+  properties: {
+    userId: { type: 'string' },
+    username: { type: 'string' },
+    user: { type: 'string' },
+  },
+  additionalProperties: false,
+});
+```
+
+**Important:** Declare shared `const` schemas **before** their first usage to avoid temporal dead zone (TDZ) errors. JavaScript `const` is not hoisted.
+
+### 2. Strengthen relaxed object schemas
+
+After initial migration, review schemas using `{ type: 'object' }` without `properties`. These pass any object shape through without validation.
+
+**Acceptable cases** (document with a comment):
+- Dynamic/schemaless objects (e.g., user preferences, custom fields)
+- Complex types not yet in typia (e.g., `IExportOperation`)
+
+**Should be fixed:**
+- Types that have a `$ref` available (e.g., `IUser` → `{ $ref: '#/components/schemas/IUser' }`)
+- Array items without schema (e.g., `{ type: 'array' }` → `{ type: 'array', items: { $ref: '...' } }`)
+
+Run `node scripts/analyze-weak-types.mjs --schema-only` to find all instances.
+
+### 3. Add `authRequired: false` explicitly
+
+Endpoints that are intentionally public should declare `authRequired: false` explicitly rather than relying on the default. This makes the intent clear and self-documenting.
+
+### 4. Add OpenAPI tags
+
+Endpoints without `tags` in their options are excluded from the generated OpenAPI spec. Add appropriate tags:
+
+```typescript
+{
+  authRequired: true,
+  tags: ['Users'],
+  // ...
+}
+```
+
+### 5. Create a changeset
+
+If the migration changes error types (e.g., `'invalid-params'` → `'error-invalid-params'`), this is a breaking change for API consumers matching on `errorType`. Create a changeset:
+
+```bash
+yarn changeset
+```
+
+Use `minor` bump for the affected packages and document the error type change.
+
+### 6. Add missing test coverage
+
+For each migrated endpoint, verify:
+- **Validator tests** exist for the body/query schema (valid, invalid, extra properties)
+- **E2E tests** cover success (200), unauthorized (401), invalid params (400), and forbidden (403) cases
+- **Error type assertions** use `'error-invalid-params'` (not the old `'invalid-params'`)