Skip to content

Cannot create managed on-host memory with the alignment specified in buffer_requirements #5373

@xuke537

Description

@xuke537

Hi,

I am implementing an NPU dispatch shared object for our NPU. Our NPU has special alignment requirements (aligned with the burst size) for input/output buffers allocated in host memory. I have set the alignment to 256 in buffer_requirements, but it always allocates aligned memory according to LITERT_HOST_MEMORY_BUFFER_ALIGNMENT 64.

Call STACK:

LiteRtTensorBufferT::CreateManagedWithAlignment(LiteRtEnvironment env, LiteRtTensorBufferType buffer_type, const LiteRtRankedTensorType & tensor_type, size_t buffer_size, size_t alignment) (\workspaces\liteRT\LiteRT\litert\runtime\tensor_buffer.cc:676)
LiteRtTensorBufferT::CreateManaged(LiteRtEnvironment env, LiteRtTensorBufferType buffer_type, const LiteRtRankedTensorType & tensor_type, size_t buffer_size) (\workspaces\liteRT\LiteRT\litert\runtime\tensor_buffer.cc:666)
LiteRtCreateManagedTensorBuffer(LiteRtEnvironment env, LiteRtTensorBufferType buffer_type, const LiteRtRankedTensorType * tensor_type, size_t buffer_size, LiteRtTensorBuffer * tensor_buffer) (\workspaces\liteRT\LiteRT\litert\c\litert_tensor_buffer.cc:404)
litert::TensorBuffer::CreateManaged(const litert::Environment & env, litert::TensorBufferType buffer_type, const litert::RankedTensorType & tensor_type, size_t buffer_size) (\workspaces\liteRT\LiteRT\litert\cc\litert_tensor_buffer.cc:51)
litert::CompiledModel::CreateBufferImpl(const litert::Environment & env, const litert::TensorBufferRequirements & buffer_requirements, const litert::RankedTensorType & tensor_type) (\workspaces\liteRT\LiteRT\litert\cc\litert_compiled_model.cc:80)
litert::CompiledModel::CreateInputOutputBuffer(const litert::CompiledModel * const this, size_t signature_index, absl::lts_20250814::string_view tensor_name, bool is_input) (\workspaces\liteRT\LiteRT\litert\cc\litert_compiled_model.cc:108)
litert::CompiledModel::CreateInputOutputBuffers(const litert::CompiledModel * const this, size_t signature_index, bool is_input) (\workspaces\liteRT\LiteRT\litert\cc\litert_compiled_model.cc:136)
litert::CompiledModel::CreateInputBuffers(const litert::CompiledModel * const this, size_t signature_index) (\workspaces\liteRT\LiteRT\litert\cc\litert_compiled_model.h:366)
litert::(anonymous namespace)::RunModel() (\workspaces\liteRT\LiteRT\litert\tools\run_model.cc:319)
main(int argc, char ** argv) (\workspaces\liteRT\LiteRT\litert\tools\run_model.cc:397)

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions