Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# Changelog

## [0.9.0](https://github.com/bbopen/tywrap/compare/v0.8.0...v0.9.0) (2026-07-11)

Typed value-RPC, one job done well. 0.9.0 removes the stateful instance API whose handles silently broke under pooling, collapses the transport stack it no longer needs, and makes the generated types tell the truth: a type you see in a generated wrapper is now either backed by a declaration and a codec, or it is `unknown` — and what comes back over the wire is validated against it at runtime. Generated wrappers must be regenerated: the IR schema is now `0.4.0` on both sides, and a version mismatch fails generation with a clear message.

### Breaking

- **The stateful instance API is gone ([#264](https://github.com/bbopen/tywrap/issues/264), [#265](https://github.com/bbopen/tywrap/issues/265), [#266](https://github.com/bbopen/tywrap/issues/266)).** A handle lived in one pool worker while calls routed to any worker, so instance methods returned wrong results under `maxProcesses > 1`. The server no longer accepts `instantiate`/`call_method`/`dispose_instance` (structured unknown-method error), the client and wire protocol are call-only (`call | meta`), and generated classes expose only static and classmethod members routed through ordinary calls. Classes that lose members carry a migration note pointing at value-returning module functions.
- **Generated types degrade honestly ([#267](https://github.com/bbopen/tywrap/issues/267)).** A return type tywrap cannot resolve to a local declaration or a codec-backed value is emitted as `unknown` instead of a bare, undeclared TypeScript name — and every degrade feeds the generation report, so `--fail-on-warn` builds fail on them by design. Two long-catalogued type lies are fixed with the same stroke: Python `bytes` maps to `Uint8Array` (what the decoder actually delivers), and `set[T]` maps to `T[]` (sets arrive as arrays; sending a `Set` was already a loud error). Iterator-protocol returns (`Generator`, `Iterator`, `AsyncGenerator`, `AsyncIterator`) degrade to `unknown` for the same reason: the bridge rejects iterator objects loudly at serialization, so those annotations could never carry a value — while `Iterable[T]`/`Sequence[T]` keep mapping to types an actually-decoded list satisfies.
- **Returns are validated at runtime ([#268](https://github.com/bbopen/tywrap/issues/268)).** Every generated module-function and static-method call checks the decoded result against its declared type before the promise resolves; a Python function annotated `-> int` that returns a string now throws `BridgeValidationError` (with the declared type, the received shape, and the call site) instead of surfacing as a mistyped `Promise<number>`. Columnar values are checked by provenance — marker, dims, dtype — so DataFrame/ndarray returns validate by shape and the Arrow fast path stays walk-free. `unknown`, `void`, and `Any` returns validate nothing, by design.
- **The IR is a pinned, versioned contract ([#269](https://github.com/bbopen/tywrap/issues/269)).** Generation writes `<module>.contract.json` next to the generated code — byte-stable across machines and Python processes — so `generate --check` flags drift; `contractInput` regenerates from the pinned file without spawning Python; and `IR_VERSION` is `0.4.0` on both sides, checked at generation time with an actionable mismatch error.
- **Call results are never cached, and requests get exactly one attempt.** The heuristic `enableCache` result cache is removed (a "pure function" guess that could serve stale results), along with the request retry machinery (retries could reuse an in-flight request id and cross-wire responses). Timeouts and errors surface to the caller unchanged — once.
- **The codec rejects what JSON would mangle.** `Map` and `Set` values in requests fail loudly with a conversion hint instead of serializing as `{}`; `bytesHandling: 'passthrough'` is removed; non-finite numbers keep failing as before.
- **Pool and transport defaults tightened.** `maxConcurrentPerProcess` defaults to `1` (the Python bridge is a serial loop; concurrency comes from more workers, and pipelining is explicit opt-in). Pool limits are validated at `NodeBridge` construction (synchronous throw) rather than first use. The `TYWRAP_TRANSPORT_CHUNKING`/`TYWRAP_TRANSPORT_FRAME_PROTOCOL`/`TYWRAP_TRANSPORT_MAX_FRAME_BYTES` negotiation env vars are gone: the npm package ships its own Python bridge, so version skew is impossible and `tywrap-frame/1` framing is simply always on for the subprocess transport.

### Features

- **Security model documented.** `SECURITY.md` states the bridge trust model — the two bounds are the `TYWRAP_ALLOWED_MODULES` allowlist and the on-by-default private-attribute block (`TYWRAP_ALLOW_PRIVATE_ATTRS` opts out) — plus the server-author contract for network exposure and private vulnerability reporting. ([#262](https://github.com/bbopen/tywrap/issues/262), [#263](https://github.com/bbopen/tywrap/issues/263))
- **Docs describe what the types actually cover** — annotation-derived typing with `unknown` fallbacks, not "full type safety"; Deno marked experimental and untested in CI. ([#261](https://github.com/bbopen/tywrap/issues/261))

### Internal

- The subprocess transport stack collapsed from six layers to a merged pool/transport with atomic threshold restarts: a restarting worker can never receive dispatches, cancelled requests never count toward restart thresholds, and disposal fences out racing restarts so no orphan Python process survives.
- Python-side codec unified: `safe_codec.py` merged into `tywrap_bridge_core.py`; plain values no longer cold-import the scientific stack (first-call serialize cost 1.6 s → microseconds); duplicate first frames surface the duplicate-sequence protocol error instead of silently resetting reassembly.
- The test suite was rebuilt around discrimination: an adversarial cross-library menagerie (`test/menagerie/`) catalogues every round-trip behavior as honest/loud/known, ~2,600 lines of vacuous or tautological tests were removed, and the resurrected adversarial suite caught real cross-PR interactions before they shipped.
- `tywrap-ir` is published as `0.3.0` (IR schema `0.4.0`).

## [0.8.0](https://github.com/bbopen/tywrap/compare/v0.7.0...v0.8.0) (2026-06-01)

The large-payload transport release — the second half of the scientific data plane. A result that exceeds a single JSONL line no longer has to fit on one line: the subprocess bridge splits it into frames and reassembles it byte-for-byte. The wire protocol is unchanged (`tywrap/1`); framing is a separate, additive `tywrap-frame/1` protocol negotiated at startup, so a 0.7.x bridge and a 0.8.0 client still talk, and an oversize payload sent to a bridge that cannot chunk fails loudly instead of silently truncating.
Expand Down
29 changes: 24 additions & 5 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,26 @@ The detailed technical appendix for the scientific data plane lives in

## Recently Shipped

### v0.9.0: typed value-RPC

`v0.9.0` narrowed tywrap to one job done well: typed value-RPC between
TypeScript and Python (roadmap #260–#270). The stateful instance API is gone
(#264–#266) — a handle lived in one pool worker while calls routed to any
worker, so instance methods silently broke under pooling; generated classes now
expose only static and classmethod members routed through ordinary calls. The
generated types stopped lying (#267): a return tywrap cannot back with a
declaration or a codec is `unknown`, `bytes` maps to `Uint8Array`, `set[T]` to
`T[]`. Decoded returns are validated against the declared type at runtime, with
Comment on lines +18 to +20

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Fix the malformed type-mapping sentence.

“a return tywrap cannot back with a declaration or a codec” is unclear and should be rewritten to match the precise wording in the changelog.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ROADMAP.md` around lines 18 - 20, Rewrite the malformed type-mapping sentence
in ROADMAP.md to use precise changelog wording, clearly describing the
relationship between return type wrappers, declarations, and codecs. Preserve
the surrounding mappings for unknown, Uint8Array, and arrays unchanged.

`BridgeValidationError` carrying the declared type, received shape, and call
site (#268). The IR is a pinned, versioned contract (#269): generation writes a
byte-stable `<module>.contract.json`, `generate --check` catches drift,
`contractInput` regenerates without spawning Python, and `IR_VERSION 0.4.0` is
enforced on both sides. Result caching and request retries are removed, the
subprocess transport stack collapsed from six layers to a merged pool/transport
with atomic restarts, framing negotiation is gone (always-on), and `SECURITY.md`
documents the bridge trust model (#262, #263). Docs claim what the tool does,
not more (#261).

### v0.8.0: large-payload transport

`v0.8.0` is the second half of the scientific data plane (#237). When a request
Expand Down Expand Up @@ -93,19 +113,18 @@ and bridge live.

## Now

The scientific data plane (#237) is complete as of `v0.8.0`. The next release
theme is not yet locked; candidates are drawn from **Later** below.
The typed value-RPC contract pass (#260) is complete as of `v0.9.0`, and the
scientific data plane (#237) as of `v0.8.0`. The next release theme is not yet
locked; candidates are drawn from **Later** below.

See [docs/codec-roadmap.md](./docs/codec-roadmap.md) for the deeper technical
appendix behind the data-plane work.

## Later

These items are intentionally not part of the scientific data plane (`0.7.0`/`0.8.0`):
These items are intentionally deferred:

- GPU-native transport such as DLPack or Arrow CUDA
- HTTP server lifecycle management owned by Tywrap
- app-level HMR beyond Tywrap wrapper regeneration and bridge reload
- unsafe default model-serialization paths such as implicit pickle or joblib
- broader runtime surface removals that should wait for a later major-version
contract pass
8 changes: 4 additions & 4 deletions docs/public/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2763,7 +2763,7 @@ so a payload can exceed the JSONL line ceiling. The capability is statically
[Transport framing](https://bbopen.github.io/tywrap/transport-framing.md) for the wire format.

`supportsStreaming` (incremental results for a single request) is `false` on
every backend; it is not implemented in 0.8.0.
every backend; it is not implemented as of 0.9.0.

### `maxFrameBytes`

Expand Down Expand Up @@ -2817,7 +2817,7 @@ through a `PooledTransport` lease reassembles correctly (see

## Scope: subprocess only

`tywrap-frame/1` is **subprocess-only** for 0.8.0. The subprocess transport is
`tywrap-frame/1` is **subprocess-only** as of 0.9.0. The subprocess transport is
the only backend with a real frame ceiling — the JSONL line-length limit. The
other backends stay single-frame and keep `supportsChunking: false`:

Expand All @@ -2827,7 +2827,7 @@ other backends stay single-frame and keep `supportsChunking: false`:
- **Pyodide** is in-memory (`maxFrameBytes: Number.POSITIVE_INFINITY`,
JSON-only); there is no wire to fragment.

`supportsStreaming` stays `false` on every backend in 0.8.0.
`supportsStreaming` stays `false` on every backend as of 0.9.0.

## Layering

Expand Down Expand Up @@ -2906,7 +2906,7 @@ construction. The codepoint-boundary rule is a sender obligation: a frame's
`data` MUST NOT split a multi-byte UTF-8 sequence, so the receiver can decode
each frame as valid UTF-8 and concatenate without re-aligning bytes across frame
edges. `utf8-base64` remains a defined alternative in the wire schema for future
use (e.g. a non-text payload), but tywrap does not emit it in 0.8.0.
use (e.g. a non-text payload), but tywrap does not emit it as of 0.9.0.

> Implementation note for later workstreams: `total` and the per-frame split
> points are computed over the message's UTF-8 **byte** length against
Expand Down
2 changes: 1 addition & 1 deletion docs/transport-capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ so a payload can exceed the JSONL line ceiling. The capability is statically
[Transport framing](./transport-framing.md) for the wire format.

`supportsStreaming` (incremental results for a single request) is `false` on
every backend; it is not implemented in 0.8.0.
every backend; it is not implemented as of 0.9.0.

### `maxFrameBytes`

Expand Down
6 changes: 3 additions & 3 deletions docs/transport-framing.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ through a `PooledTransport` lease reassembles correctly (see

## Scope: subprocess only

`tywrap-frame/1` is **subprocess-only** for 0.8.0. The subprocess transport is
`tywrap-frame/1` is **subprocess-only** as of 0.9.0. The subprocess transport is
the only backend with a real frame ceiling — the JSONL line-length limit. The
other backends stay single-frame and keep `supportsChunking: false`:

Expand All @@ -27,7 +27,7 @@ other backends stay single-frame and keep `supportsChunking: false`:
- **Pyodide** is in-memory (`maxFrameBytes: Number.POSITIVE_INFINITY`,
JSON-only); there is no wire to fragment.

`supportsStreaming` stays `false` on every backend in 0.8.0.
`supportsStreaming` stays `false` on every backend as of 0.9.0.

## Layering

Expand Down Expand Up @@ -106,7 +106,7 @@ construction. The codepoint-boundary rule is a sender obligation: a frame's
`data` MUST NOT split a multi-byte UTF-8 sequence, so the receiver can decode
each frame as valid UTF-8 and concatenate without re-aligning bytes across frame
edges. `utf8-base64` remains a defined alternative in the wire schema for future
use (e.g. a non-text payload), but tywrap does not emit it in 0.8.0.
use (e.g. a non-text payload), but tywrap does not emit it as of 0.9.0.

> Implementation note for later workstreams: `total` and the per-frame split
> points are computed over the message's UTF-8 **byte** length against
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "tywrap",
"version": "0.8.0",
"version": "0.9.0",
"description": "Generate TypeScript bindings for Python libraries with precise types where tywrap can resolve them — Node.js, experimental Deno, Bun, and browsers via Pyodide.",
"type": "module",
"main": "dist/index.js",
Expand Down Expand Up @@ -34,6 +34,7 @@
"files": [
"dist",
"runtime",
"!runtime/__pycache__",
"src",
"README.md",
"SECURITY.md"
Expand Down
30 changes: 18 additions & 12 deletions src/core/generator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,18 @@
'Promise',
'Record',
]);
// Iterator-protocol objects can never cross the bridge: the Python side
// rejects them loudly at serialization, so a return typed as one of these
// could never carry a value. Returns degrade to `unknown` (and count as a
// degrade for --fail-on-warn). Iterable/Sequence are NOT here — a list
// satisfies those annotations and decodes to an array, which structurally
// satisfies the emitted TS type.
private readonly nonDecodableReturnGenerics = new Set([
'Generator',
'AsyncGenerator',
'Iterator',
'AsyncIterator',
]);
private readonly reservedTsIdentifiers = new Set([
'default',
'delete',
Expand Down Expand Up @@ -363,7 +375,7 @@
): string {
const base = `typeof ${valueExpr} === 'object' && ${valueExpr} !== null && !globalThis.Array.isArray(${valueExpr}) && (Object.getPrototypeOf(${valueExpr}) === Object.prototype || Object.getPrototypeOf(${valueExpr}) === null)`;

const keyCheck = (() => {

Check warning on line 378 in src/core/generator.ts

View workflow job for this annotation

GitHub Actions / lint

Missing return type on function
if (options.requiredKwOnlyNames.length > 0) {
return options.requiredKwOnlyNames
.map(k => `Object.prototype.hasOwnProperty.call(${valueExpr}, ${JSON.stringify(k)})`)
Expand Down Expand Up @@ -466,18 +478,9 @@
if (leaf === 'NDArray' || leaf === 'ndarray') {
return { kind: 'marker', marker: 'ndarray' };
}
if (
[
'list',
'List',
'Sequence',
'Iterable',
'Iterator',
'Generator',
'set',
'frozenset',
].includes(leaf)
) {
// Iterator/Generator are deliberately absent: those returns emit
// `unknown` (see nonDecodableReturnGenerics) and validate nothing.
if (['list', 'List', 'Sequence', 'Iterable', 'set', 'frozenset'].includes(leaf)) {
return {
kind: 'array',
element: schema(current.typeArgs[0] ?? { kind: 'custom', name: 'Any' }),
Expand Down Expand Up @@ -583,7 +586,7 @@
const tsTypeForValue = (p: (typeof filteredParams)[number]): string =>
this.typeToTsFromPython(p.type, genericContext, 'value');

const kwargsType = (() => {

Check warning on line 589 in src/core/generator.ts

View workflow job for this annotation

GitHub Actions / lint

Missing return type on function
if (!needsKwargsParam) {
return '';
}
Expand Down Expand Up @@ -664,7 +667,7 @@
for (let i = requiredPosCount; i <= positionalParams.length; i++) {
const head = positionalParams.slice(0, i).map(p => renderPositionalParam(p, true));
const rest: string[] = [];
const v = (() => {

Check warning on line 670 in src/core/generator.ts

View workflow job for this annotation

GitHub Actions / lint

Missing return type on function
if (!varArgsParam) {
return null;
}
Expand Down Expand Up @@ -902,7 +905,7 @@
return `${pname}${opt}: ${methodTsValueType(p)}`;
};

const kwargsType = (() => {

Check warning on line 908 in src/core/generator.ts

View workflow job for this annotation

GitHub Actions / lint

Missing return type on function
if (!needsKwargsParam) {
return '';
}
Expand Down Expand Up @@ -944,7 +947,7 @@
const overloads: string[] = [];
if (needsKwargsParam && requiredKwOnlyNames.length > 0) {
const firstOptionalIndex = positionalParams.findIndex(p => p.optional);
const requiredPosCount =

Check warning on line 950 in src/core/generator.ts

View workflow job for this annotation

GitHub Actions / lint

'requiredPosCount' is already declared in the upper scope on line 895 column 15
firstOptionalIndex >= 0 ? firstOptionalIndex : positionalParams.length;
for (let i = requiredPosCount; i <= positionalParams.length; i++) {
const head = positionalParams.slice(0, i).map(p => renderPositionalParam(p, true));
Expand Down Expand Up @@ -1207,6 +1210,9 @@
if (!/^[A-Za-z_$][A-Za-z0-9_$]*$/.test(g.name)) {
return this.degradeType(g);
}
if (mappingContext === 'return' && this.nonDecodableReturnGenerics.has(g.name)) {
return this.degradeType(g);
}
if (ctx && !this.builtinGenericNames.has(g.name) && !this.isLocalTypeIdentity(g, ctx)) {
return this.degradeType(g);
}
Expand Down
8 changes: 5 additions & 3 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
/**
* tywrap - TypeScript wrapper for Python libraries with full type safety
* tywrap - TypeScript bindings for Python libraries
*
* @description Build-time code generation system that makes Python libraries
* feel native in TypeScript with zero runtime overhead
* @description Build-time code generation that produces typed wrappers for
* Python modules: precise types where annotations resolve, `unknown` where
* they cannot, and runtime validation of decoded returns against the
* declared types.
*
* This is the package root. It intentionally exposes only the stable,
* consumer-facing surface. Runtime plumbing (codec, transport, bridge
Expand Down
2 changes: 1 addition & 1 deletion src/version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@
* Regenerate with: node scripts/generate-version.mjs
*/

export const VERSION: string = "0.8.0";
export const VERSION: string = "0.9.0";
80 changes: 80 additions & 0 deletions test/generator.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,86 @@ describe('CodeGenerator', () => {
expect(degraded).toEqual(['other.Remote']);
});

it('degrades iterator-protocol returns to unknown with a no-op validator', () => {
// A generator/iterator object can never cross the bridge (serialization
// rejects it loudly), so Generator<...> as a return type could never
// carry a value.
const degraded: string[] = [];
const honestGenerator = new CodeGenerator(undefined, {
onTypeDegrade: typeName => degraded.push(typeName),
});
const returnType = {
kind: 'generic',
name: 'Generator',
module: 'typing',
typeArgs: [
{ kind: 'primitive', name: 'int' },
{ kind: 'primitive', name: 'None' },
{ kind: 'primitive', name: 'None' },
],
};
const code = honestGenerator.generateModuleDefinition({
name: 'local_module',
functions: [
{
name: 'counting',
signature: { parameters: [], returnType, isAsync: false, isGenerator: true },
decorators: [],
isAsync: false,
isGenerator: true,
returnType,
parameters: [],
},
],
classes: [],
typeAliases: [],
imports: [],
exports: [],
});

expect(code.typescript).toContain('Promise<unknown>');
expect(code.typescript).not.toContain('Generator<');
expect(code.typescript).toContain('{"kind":"any"}');
expect(degraded).toEqual(['typing.Generator']);
});

it('keeps Iterable returns as arrays — a decoded list satisfies them honestly', () => {
const degraded: string[] = [];
const honestGenerator = new CodeGenerator(undefined, {
onTypeDegrade: typeName => degraded.push(typeName),
});
const returnType = {
kind: 'generic',
name: 'Iterable',
module: 'typing',
typeArgs: [{ kind: 'primitive', name: 'int' }],
};
const code = honestGenerator.generateModuleDefinition({
name: 'local_module',
functions: [
{
name: 'values',
signature: { parameters: [], returnType, isAsync: false, isGenerator: false },
decorators: [],
isAsync: false,
isGenerator: false,
returnType,
parameters: [],
},
],
classes: [],
typeAliases: [],
imports: [],
exports: [],
});

expect(code.typescript).toContain('Promise<Iterable<number>>');
expect(code.typescript).toContain(
'{"kind":"array","element":{"kind":"primitive","type":"number"}}'
);
expect(degraded).toEqual([]);
});

it('degrades qualified generic heads and module-colliding leaves', () => {
const degraded: string[] = [];
const honestGenerator = new CodeGenerator(undefined, {
Expand Down
Loading
Loading