Skip to content

Bug: cattrs hook registration in deserialize.py causing significant latency in multithreaded environments #1670

@k7ragav

Description

@k7ragav

Summary

A change introduced in SDK 25.14.0, specifically in the python/looker_sdk/rtl/serialize.py file, has introduced a major performance bottleneck. The SDK now re-registers a cattrs hook factory on every single API response deserialization, leading to slowness in multithreaded applications. We would also like to know why was this change introduced as there is no proper documentation also behind this change.

The Problem
In the versions >25.14.0, the deserialize.py includes this line:
converter.register_structure_hook_factory(is_sequence, list_structure_factory)

Due to this the multithreading in Python does not work as fast as it is supposed to.

Impact
When running high-concurrency scripts (e.g., managing 5,000+ user attributes across multiple threads), the time taken is nearly an hour. Without this hook registration, in higher versions, the time take is less than 10 minutes.

Steps to Reproduce

  • Use Python SDK 25.16.0 or 26.2.0.
  • Execute a high volume (1,000+) of metadata calls (e.g., user_attribute_user_values) using a ThreadPoolExecutor.
  • Observe high CPU usage and note the speed.
  • Remove the converter.register_structure_hook_factory(is_sequence, list_structure_factory) line from deserialize.py and then observe the CPU and the speed taken

Proposed Fix

  • Maybe instead of inside in deserialize(), it could be placed more globally on the module level so individual API calls are not affected/

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions