Skip to content

Lazy mode silently drops buckaroo_state_change → WS client hangs forever #793

Description

@paddymul

Summary

_handle_buckaroo_state_change (buckaroo/server/websocket_handler.py:73) returns silently when session.mode != "buckaroo". Sessions loaded via POST /load with mode=lazy therefore receive zero response to a buckaroo_state_change WS message — no error frame, no rebroadcast, nothing. The client (AG-Grid, our test driver, any third-party caller) sits waiting on an empty WS read until it times out.

Behaviour matrix (against the Boston restaurant data, full 883K rows):

mode search_pizza (state_change) response
buckaroo (pandas) 1.1 s initial_state rebroadcast with filtered_rows=22366 ✓
lazy (polars) hangs / 20s timeout no frame returned ✗
buckaroo xorq (/load_expr) 11 s initial_state rebroadcast with filtered_rows=22366 ✓

Reproduction

import asyncio, json, tornado.websocket, tornado.httpclient
async def main():
    client = tornado.httpclient.AsyncHTTPClient()
    await client.fetch("http://127.0.0.1:8700/load", method="POST",
        body=json.dumps({"session":"x", "path":"/path/to/file.parquet",
            "mode":"lazy", "no_browser":True}),
        headers={"Content-Type":"application/json"})
    ws = await tornado.websocket.websocket_connect("ws://127.0.0.1:8700/ws/x")
    await ws.read_message()  # initial state arrives fine
    ws.write_message(json.dumps({"type":"buckaroo_state_change",
        "new_state":{"quick_command_args":{"search":["PIZZA"]},
            "post_processing":"","cleaning_method":"","df_display":"main",
            "show_commands":False,"sampled":False,"search_string":"PIZZA"}}))
    # Hangs here. No frame ever arrives.
    frame = await asyncio.wait_for(ws.read_message(), timeout=20.0)
asyncio.run(main())

Why it matters

Either lazy mode supports filter / cleaning / post-processing state changes (in which case the handler needs implementation), or it explicitly doesn't (in which case the WS must return a structured error so the client can render "filter not supported in lazy mode" instead of spinning forever). The current silent-drop is the worst of both: third-party callers can't tell if their request is in flight, lost, or unsupported.

Suggested fix

Two flavors, depending on product intent:

A. Wire state_change into lazy mode. session.ldf is a pl.LazyFrame; quick-command filters can be applied via ldf.filter(...) similar to how handle_infinite_request_lazy already accepts a filtered ldf. Recompute the row count, rebroadcast initial_state. Symmetric with the buckaroo/xorq paths.

B. Reject explicitly. If lazy mode is intentionally read-only:

if session.mode != "buckaroo":
    self.write_message(json.dumps({"type": "error",
        "error_code": "state_change_unsupported_mode",
        "message": f"buckaroo_state_change is not supported in mode={session.mode!r}; "
                   "the lazy reader does not currently apply filters / cleaning / post-processing."}))
    return

A is more useful product-wise; B is a 5-line stop-the-bleed fix. Recommend B first, file A as the follow-up if filter support in lazy mode isn't already on the roadmap.

Test plan

  • Failing-test commit: a Tornado WS test mirroring the repro above, expecting either a non-empty frame within 2s (any frame — error or rebroadcast).
  • Fix commit: A or B above. Existing tests/unit/server/test_load_expr.py and test_rows_first_spike.py continue to pass (they don't touch the lazy path).

Found while stress-testing the smorgasbord branch across pandas/lazy-polars/xorq backends.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions