Skip to content

Commit 6dd575d

Browse files
authored
Update typed FieldMask in sdk-core (#101)
## 🥞 Stacked PR Use this [link](https://github.com/databricks/sdk-js/pull/101/files) to review incremental changes. - [**stack/field-mask-sdk-core**](#101) [[Files changed](https://github.com/databricks/sdk-js/pull/101/files)] - [stack/generator-field-mask](#102) [[Files changed](https://github.com/databricks/sdk-js/pull/102/files/9662ed8687549fc04fb0dd2588bb8d3c5e59853b..b49f0f673265f605667d95501e6beaea9630647d)] - [stack/field-mask-convertors](#100) [[Files changed](https://github.com/databricks/sdk-js/pull/100/files/b49f0f673265f605667d95501e6beaea9630647d..7b206bc649bc57d0915f922997954ce0323ff8ef)] --------- ## Summary Update the `FieldMask<T>` in `@databricks/sdk-core/wkt`. `FieldMask.build` validates paths against a per-message schema, translates them to wire format, and stores the result privately. `toString()` serializes to the comma-separated wire string the server expects. ## Why Databricks accepts field-mask paths in proto snake_case, but generated TS interfaces use camelCase. The existing `FieldMask` didn't translate or validate — typos went silently to the server. This is the runtime foundation the generator needs; the follow-up PR emits the per-message schemas and factories that call `FieldMask.build`. ### Why validation is at runtime, not compile time A compile-time `FieldPaths<T>` conditional type (computing the string-literal union of valid paths for each generated interface) was explored and rejected. TypeScript's conditional-type evaluation has a hard recursion depth limit — any proto message that references itself or forms a cycle through another message (common in Databricks' schemas: `Node { child: Node }`, `A → B → A`) triggers `TS2589: Type instantiation is excessively deep and possibly infinite`. Verified by probing against the real sdk-js `tsconfig`. Because we can't express a complete set of valid paths at the type level for the schemas we actually ship, `FieldMask.build` walks the schema at runtime and throws on a mismatch instead. The schema itself is a plain runtime object with lazy `() => FieldMaskSchema` child references, which keeps recursive and mutually-recursive protos describable as finite graphs. ## What changed ### Interface changes - **`FieldMask<T = unknown>`** — private constructor, `@internal static build<T>(paths, schema)` as the sole entry point, `toString(): string` returning the joined wire paths. Phantom `T` distinguishes masks across messages at compile time. - **`FieldMaskSchema`, `FieldMaskSchemaField`** — new exports. The shape the generator emits per message; `children?: () => FieldMaskSchema` keeps recursive/mutual-recursive protos cycle-safe. - **Removed**: `FieldPaths<T>`, `FieldMask.of(...)`, `FieldMask.append(...)`. See the "Why validation is at runtime" note above for the rationale. ### Behavioral changes `FieldMask.build` throws on an unknown path segment or descent past a scalar leaf. Before, any string was accepted. ### Internal changes `walkFieldMaskPath` and `normalize` are module-private in `packages/core/src/wkt/fieldmask.ts`. ## How is this tested? Table-driven tests (`it.each`) in `packages/core/tests/wkt/fieldmask.test.ts`: - **Valid paths translate and serialize** (8 cases): flat, nested, deep, multi-path sort, dedup, parent subsumption, empty input, self-cycle at arbitrary depth. - **Invalid paths throw** (3 cases): unknown top-level, unknown nested, descent past scalar.
1 parent abc2d31 commit 6dd575d

3 files changed

Lines changed: 158 additions & 168 deletions

File tree

packages/core/src/wkt/fieldmask.ts

Lines changed: 70 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,7 @@
1-
// True if any property of T is callable, indicating a class instance (e.g.
2-
// Temporal.Instant, Date) rather than a plain data interface.
3-
type HasMethods<T> = true extends {
4-
[K in keyof T]-?: NonNullable<T[K]> extends (...args: never[]) => unknown
5-
? true
6-
: never;
7-
}[keyof T]
8-
? true
9-
: false;
10-
11-
/**
12-
* Utility type that derives all valid dot-separated field paths from a
13-
* TypeScript interface. Provides compile-time path validation for
14-
* {@link FieldMask}.
15-
*
16-
* Recursion stops at arrays, index signatures (Record/Map), and class
17-
* instances with methods (e.g. Temporal.Instant, Date). These are treated
18-
* as leaf nodes.
19-
*
20-
* @example
21-
* ```ts
22-
* interface Cluster {
23-
* name: string;
24-
* config: {numWorkers: number; scaling: {min: number}};
25-
* }
26-
* // "name" | "config" | "config.numWorkers" | "config.scaling" | "config.scaling.min"
27-
* type ClusterPaths = FieldPaths<Cluster>;
28-
* ```
29-
*/
30-
export type FieldPaths<T, Prefix extends string = ''> = {
31-
[K in keyof T & string]: NonNullable<T[K]> extends unknown[]
32-
? `${Prefix}${K}` // Array field — leaf, do not recurse.
33-
: NonNullable<T[K]> extends Record<string, unknown>
34-
? string extends keyof NonNullable<T[K]>
35-
? `${Prefix}${K}` // Index signature (Record/Map) — leaf, do not recurse.
36-
: HasMethods<NonNullable<T[K]>> extends true
37-
? `${Prefix}${K}` // Class instance with methods — leaf, do not recurse.
38-
: `${Prefix}${K}` | FieldPaths<NonNullable<T[K]>, `${Prefix}${K}.`>
39-
: `${Prefix}${K}`;
40-
}[keyof T & string];
41-
42-
// Remove duplicates and paths subsumed by a parent (e.g. "config" subsumes
43-
// "config.numWorkers").
44-
function normalize<P extends string>(paths: P[]): P[] {
1+
// Remove duplicates and paths subsumed by a parent (e.g. "config" subsumes "config.numWorkers").
2+
function normalize(paths: string[]): string[] {
453
const unique = [...new Set(paths)].sort();
46-
const result: P[] = [];
4+
const result: string[] = [];
475
for (const path of unique) {
486
const isSubsumed = result.some(existing => path.startsWith(existing + '.'));
497
if (!isSubsumed) {
@@ -54,33 +12,78 @@ function normalize<P extends string>(paths: P[]): P[] {
5412
}
5513

5614
/**
57-
* A type-safe field mask implementing google.protobuf.FieldMask semantics.
58-
* Provides compile-time path validation via {@link FieldPaths}. Paths are
59-
* always normalized: duplicates and paths subsumed by a parent are removed.
60-
*
61-
* @example
62-
* ```ts
63-
* const mask = FieldMask.of<FieldPaths<Cluster>>(
64-
* 'displayName',
65-
* 'config.numWorkers'
66-
* );
67-
* ```
15+
* One field entry in a {@link FieldMaskSchema}: its wire-format name and, for message-typed fields, a lazy reference to the nested message's schema. Array, map, enum, and scalar fields omit `children`.
6816
*/
69-
export class FieldMask<TPath extends string = string> {
70-
/** The list of field paths in this mask. */
71-
readonly paths: TPath[];
17+
export interface FieldMaskSchemaField {
18+
readonly wire: string;
19+
readonly children?: () => FieldMaskSchema;
20+
}
7221

73-
private constructor(paths: TPath[]) {
74-
this.paths = normalize(paths);
22+
/**
23+
* Structural description of one message's FieldMask-reachable fields. Maps each typescript field name to its wire-format name and, for message-typed fields, a lazy `() => FieldMaskSchema` reference that lets recursive and mutually-recursive messages describe themselves.
24+
*/
25+
export type FieldMaskSchema = Readonly<Record<string, FieldMaskSchemaField>>;
26+
27+
// Walk a dot-separated typescript field name path against a schema, returning the equivalent wire-format path. Returns `undefined` when any segment fails: a name that isn't a field of the current message, or a non-terminal segment that doesn't reference another message.
28+
function walkFieldMaskPath(
29+
schema: FieldMaskSchema,
30+
path: string
31+
): string | undefined {
32+
const segments = path.split('.');
33+
const wireSegments: string[] = [];
34+
let current: FieldMaskSchema = schema;
35+
for (let i = 0; i < segments.length; i++) {
36+
const seg = segments[i];
37+
// Existence check before lookup: `current[seg]` is typed as FieldMaskSchemaField without noUncheckedIndexedAccess, so an undefined check downstream would be flagged "unnecessary".
38+
if (!(seg in current)) return undefined;
39+
const field = current[seg];
40+
wireSegments.push(field.wire);
41+
if (i < segments.length - 1) {
42+
if (field.children === undefined) return undefined;
43+
current = field.children();
44+
}
7545
}
46+
return wireSegments.join('.');
47+
}
48+
49+
/**
50+
* A field mask implementing google.protobuf.FieldMask semantics.
51+
*/
52+
export class FieldMask<T = unknown> {
53+
// Phantom marker: keeps `FieldMask<Alert>` and `FieldMask<Query>` compile-time distinct under TypeScript's otherwise-structural typing. Never set at runtime.
54+
declare private readonly _tag: T;
7655

77-
/** Create a field mask from one or more paths. */
78-
static of<P extends string>(...paths: P[]): FieldMask<P> {
79-
return new FieldMask(paths);
56+
// Stored post-translation, normalized wire-format paths.
57+
private readonly paths: string[];
58+
59+
private constructor(paths: string[]) {
60+
this.paths = paths;
61+
}
62+
63+
/**
64+
* Build a FieldMask from typescript field name paths against the target message's schema. Validates every path by walking each segment through the schema and throws Error when any segment fails.
65+
*
66+
* Reserved for generated per-message factories; user code should call the factory (e.g. `alertFieldMask(...)`), which supplies the schema before delegating here.
67+
*
68+
* @internal
69+
*/
70+
static build<T>(paths: string[], schema: FieldMaskSchema): FieldMask<T> {
71+
const normalized = normalize(paths);
72+
const wire: string[] = [];
73+
for (const p of normalized) {
74+
const w = walkFieldMaskPath(schema, p);
75+
if (w === undefined) {
76+
throw new Error(`Unknown field path "${p}"`);
77+
}
78+
wire.push(w);
79+
}
80+
return new FieldMask<T>(wire);
8081
}
8182

82-
/** Return a new mask with additional paths appended. */
83-
append(...paths: TPath[]): FieldMask<TPath> {
84-
return new FieldMask([...this.paths, ...paths]);
83+
/**
84+
* Serialize the mask to the wire-format string.
85+
*/
86+
toString(): string {
87+
return this.paths.join(',');
8588
}
8689
}

packages/core/src/wkt/index.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
export {FieldMask} from './fieldmask';
2-
export type {FieldPaths} from './fieldmask';
2+
export type {FieldMaskSchema, FieldMaskSchemaField} from './fieldmask';
33
export type {JsonValue, JsonObject} from './value';
Lines changed: 87 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -1,133 +1,120 @@
11
import {describe, it, expect} from 'vitest';
22
import {FieldMask} from '../../src/wkt';
3-
import type {FieldPaths} from '../../src/wkt';
3+
import type {FieldMaskSchema} from '../../src/wkt';
44

5-
// Simulates a class instance with methods (e.g. Temporal.Instant).
6-
interface ClassLike {
7-
epochMilliseconds: number;
8-
toString(): string;
9-
}
5+
// Alert-like non-cyclic schema with nested Condition + Operand, used to drive FieldMask through its `@internal` build entry point.
6+
const operandSchema: FieldMaskSchema = {
7+
column: {wire: 'column'},
8+
value: {wire: 'value'},
9+
};
10+
const conditionSchema: FieldMaskSchema = {
11+
op: {wire: 'op'},
12+
operand: {wire: 'operand', children: () => operandSchema},
13+
};
14+
const alertSchema: FieldMaskSchema = {
15+
displayName: {wire: 'display_name'},
16+
condition: {wire: 'condition', children: () => conditionSchema},
17+
};
1018

11-
// Test interface for FieldPaths derivation.
12-
interface Cluster {
13-
name: string;
14-
displayName: string;
15-
state: string;
16-
config: {
17-
numWorkers: number;
18-
scaling: {
19-
minReplicas: number;
20-
maxReplicas: number;
21-
};
22-
};
23-
tags: string[];
24-
labels: Record<string, string>;
25-
createTime?: ClassLike;
26-
}
19+
// Self-referential schema used to verify cycle-safe construction.
20+
const nodeSchema: FieldMaskSchema = {
21+
label: {wire: 'label'},
22+
child: {wire: 'child', children: () => nodeSchema},
23+
};
2724

28-
// Verify FieldPaths derives correct paths at compile time.
29-
type ClusterPaths = FieldPaths<Cluster>;
30-
const _checkPaths: ClusterPaths[] = [
31-
'name',
32-
'displayName',
33-
'state',
34-
'config',
35-
'config.numWorkers',
36-
'config.scaling',
37-
'config.scaling.minReplicas',
38-
'config.scaling.maxReplicas',
39-
'tags',
40-
'labels',
41-
'createTime', // Leaf — ClassLike has methods, so no recursion.
42-
];
43-
// Suppress unused variable warning.
44-
void _checkPaths;
45-
46-
// Verify that ClassLike properties are NOT included as paths.
47-
// If FieldPaths recursed into ClassLike, "createTime.epochMilliseconds" would
48-
// be a valid path. This assignment must fail at compile time.
49-
// @ts-expect-error - ClassLike is a leaf; its properties are not valid paths.
50-
const _badPath: ClusterPaths = 'createTime.epochMilliseconds';
51-
void _badPath;
52-
53-
describe('FieldMask', () => {
54-
describe('of', () => {
55-
const testCases: {name: string; input: string[]; want: string[]}[] = [
56-
{name: 'single path', input: ['name'], want: ['name']},
25+
describe('FieldMask.build', () => {
26+
describe('valid paths translate and serialize', () => {
27+
const cases: {
28+
name: string;
29+
schema: FieldMaskSchema;
30+
input: string[];
31+
want: string;
32+
}[] = [
5733
{
58-
name: 'multiple paths',
59-
input: ['displayName', 'name'],
60-
want: ['displayName', 'name'],
34+
name: 'flat path',
35+
schema: alertSchema,
36+
input: ['displayName'],
37+
want: 'display_name',
6138
},
6239
{
63-
name: 'deduplicates paths',
64-
input: ['name', 'name', 'state'],
65-
want: ['name', 'state'],
40+
name: 'nested path',
41+
schema: alertSchema,
42+
input: ['condition.op'],
43+
want: 'condition.op',
6644
},
6745
{
68-
name: 'removes paths subsumed by a parent',
69-
input: ['config.numWorkers', 'config', 'name'],
70-
want: ['config', 'name'],
46+
name: 'deeply nested path',
47+
schema: alertSchema,
48+
input: ['condition.operand.column'],
49+
want: 'condition.operand.column',
7150
},
7251
{
73-
name: 'removes deeply subsumed paths',
74-
input: ['config.scaling.minReplicas', 'config.scaling', 'config'],
75-
want: ['config'],
52+
name: 'multiple paths joined in sorted order',
53+
schema: alertSchema,
54+
input: ['displayName', 'condition.op'],
55+
want: 'condition.op,display_name',
7656
},
7757
{
78-
name: 'does not subsume paths sharing a prefix without a dot boundary',
79-
input: ['foo', 'foobar'],
80-
want: ['foo', 'foobar'],
58+
name: 'duplicates collapse before translation',
59+
schema: alertSchema,
60+
input: ['displayName', 'displayName'],
61+
want: 'display_name',
62+
},
63+
{
64+
name: 'children subsumed by a parent are dropped',
65+
schema: alertSchema,
66+
input: ['condition.op', 'condition'],
67+
want: 'condition',
68+
},
69+
{
70+
name: 'empty input serializes to empty string',
71+
schema: alertSchema,
72+
input: [],
73+
want: '',
74+
},
75+
{
76+
name: 'arbitrary depth through a self-cycle',
77+
schema: nodeSchema,
78+
input: ['child.child.child.label'],
79+
want: 'child.child.child.label',
8180
},
82-
{name: 'empty mask', input: [], want: []},
8381
];
8482

85-
it.each(testCases)('$name', ({input, want}) => {
86-
const mask = FieldMask.of(...input);
87-
expect(mask.paths).toStrictEqual(want);
83+
it.each(cases)('$name', ({schema, input, want}) => {
84+
const mask = FieldMask.build<unknown>(input, schema);
85+
expect(mask.toString()).toBe(want);
8886
});
8987
});
9088

91-
describe('append', () => {
92-
const testCases: {
89+
describe('invalid paths throw', () => {
90+
const cases: {
9391
name: string;
94-
initial: string[];
95-
append: string[];
96-
want: string[];
92+
schema: FieldMaskSchema;
93+
input: string[];
94+
msg: string;
9795
}[] = [
9896
{
99-
name: 'adds new paths',
100-
initial: ['name'],
101-
append: ['state'],
102-
want: ['name', 'state'],
97+
name: 'unknown top-level field',
98+
schema: alertSchema,
99+
input: ['bogus'],
100+
msg: 'Unknown field path "bogus"',
103101
},
104102
{
105-
name: 'deduplicates when appending existing paths',
106-
initial: ['name', 'state'],
107-
append: ['name'],
108-
want: ['name', 'state'],
103+
name: 'unknown nested field',
104+
schema: alertSchema,
105+
input: ['condition.bogus'],
106+
msg: 'Unknown field path "condition.bogus"',
109107
},
110108
{
111-
name: 'subsumes child when parent is appended',
112-
initial: ['config.numWorkers'],
113-
append: ['config'],
114-
want: ['config'],
109+
name: 'descent past a scalar leaf',
110+
schema: alertSchema,
111+
input: ['displayName.nope'],
112+
msg: 'Unknown field path "displayName.nope"',
115113
},
116114
];
117115

118-
it.each(testCases)('$name', ({initial, append, want}) => {
119-
const mask = FieldMask.of(...initial).append(...append);
120-
expect(mask.paths).toStrictEqual(want);
121-
});
122-
});
123-
124-
describe('type safety with FieldPaths', () => {
125-
it('works with derived FieldPaths type', () => {
126-
const mask = FieldMask.of<FieldPaths<Cluster>>(
127-
'displayName',
128-
'config.numWorkers'
129-
);
130-
expect(mask.paths).toStrictEqual(['config.numWorkers', 'displayName']);
116+
it.each(cases)('$name', ({schema, input, msg}) => {
117+
expect(() => FieldMask.build<unknown>(input, schema)).toThrowError(msg);
131118
});
132119
});
133120
});

0 commit comments

Comments
 (0)