Skip to content

Commit 5fd2bae

Browse files
author
ssjia
committed
Update on "[ET-VK] Fix use-after-free in PrepackNode when TensorRefs are shared"
When a model has shared/tied weights (e.g. tied embeddings in transformers), the serialization deduplicates them into a single TensorRef that multiple PrepackNodes reference. Previously, `PrepackNode::create_staging_buffer()` called `tref->free_buffer()` unconditionally after copying weight data to a GPU staging buffer. This meant the first PrepackNode to execute would free the underlying host memory, and subsequent PrepackNodes sharing the same TensorRef would read from a dangling pointer — producing garbage/NaN values in prepacked weight and bias tensors on the GPU. The fix adds a `prepack_use_count` field to `TensorRef` that tracks how many PrepackNodes still need to read from it. Each PrepackNode increments the count in its constructor and decrements it after copying data. The buffer is only freed when the count reaches zero. This preserves the original eager-free behavior for non-shared weights (freeing immediately after the single consumer copies) while correctly deferring the free for shared weights until the last consumer is done — avoiding both the use-after-free and unnecessary peak memory increase. Differential Revision: [D101009402](https://our.internmc.facebook.com/intern/diff/D101009402/) [ghstack-poisoned]
1 parent 586cce1 commit 5fd2bae

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

  • backends/vulkan/runtime/graph/containers

backends/vulkan/runtime/graph/containers/Constant.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ struct TensorRef final {
3333
// this reaches 0, the buffer can be safely freed. This prevents
3434
// use-after-free when multiple PrepackNodes reference the same TensorRef
3535
// (e.g. shared/tied weights).
36-
uint32_t prepack_use_count{0};
36+
int32_t prepack_use_count{0};
3737

3838
explicit TensorRef(
3939
const std::vector<int64_t>& t_sizes,

0 commit comments

Comments
 (0)