feat: Add finishInto() for zero-copy finish into existing buffers#2196
feat: Add finishInto() for zero-copy finish into existing buffers#2196jiang1997 wants to merge 2 commits intoprotobufjs:masterfrom
Conversation
|
Do you plan to use |
To be honest, my initial motivation was simply to solve an issue in my own project (serializing directly into Wasm memory to avoid extra copies). I haven't really explored using it to speed up the library internally yet. |
|
One approach to integrate it into protobuf.js itself might be to use |
205e22e to
2f41276
Compare
Yeah, totally agree. I've already made the changes — finish() now delegates to finishTo(), all runtime checks have been removed, and the documentation has been updated to note that callers are responsible for ensuring sufficient buffer space. |
| * @returns {Uint8Array} The provided buffer | ||
| */ | ||
| Writer.prototype.finishTo = function finishTo(buf, offset) { | ||
| offset |= 0; |
There was a problem hiding this comment.
This turns offsets >= 2 GiB negative. Perhaps:
| offset |= 0; | |
| offset >>>= 0; |
There was a problem hiding this comment.
Good catch, learned something new! Updated to offset >>>= 0.
There was a problem hiding this comment.
The offset >>>= 0 operation reduces the value to a 32-bit number. Node lets TypedArray indices go beyond 2³², so a large offset might end up near the start of the buffer.
I’d suggest making offset required, not optional, to avoid any unexpected behavior.
What are your thoughts on this, @dcodeIO ?
| * @param {Uint8Array} buf Target buffer | ||
| * @param {number} [offset=0] Offset to start writing at | ||
| * @returns {Uint8Array} The provided buffer |
There was a problem hiding this comment.
Since this returns the provided buffer, I think we can make the JSDoc generic here:
| * @param {Uint8Array} buf Target buffer | |
| * @param {number} [offset=0] Offset to start writing at | |
| * @returns {Uint8Array} The provided buffer | |
| * @param {T} buf Target buffer | |
| * @param {number} [offset=0] Offset to start writing at | |
| * @returns {T} The provided buffer | |
| * @template T extends Uint8Array |
The definitions generator should then generate finishTo<T extends Uint8Array>(buf: T, offset?: number): T, preserving Buffer and other Uint8Array subclasses.
2f41276 to
55f27e4
Compare
|
Thinking about naming a bit here, I guess a good name could be |
Introduce finishInto(), which allows writing directly into an existing buffer (e.g., WebAssembly memory), thereby avoiding extra copying.