Skip to content

test: parametrize to_arrow empty stream tests and update docstrings - #17919

Draft
chalmerlowe wants to merge 1 commit into
mainfrom
bugfix/352600521-empty-table-materialization
Draft

test: parametrize to_arrow empty stream tests and update docstrings#17919
chalmerlowe wants to merge 1 commit into
mainfrom
bugfix/352600521-empty-table-materialization

Conversation

@chalmerlowe

Copy link
Copy Markdown
Contributor

Problem

When reading from an empty BigQuery table using the Read API, the stream may return no messages. Previously, this could lead to an AttributeError when trying to parse the schema from a None parser. While a fallback was added to return an empty table, this fallback uses an empty schema (zero columns), which can cause downstream consumers (like Vertex Ray) to fail if they expect specific columns to be present.

Solution

  1. Updated Documentation: Added explanatory comments and updated docstrings in reader.py to clarify the flow between ReadRowsStream.to_arrow() and ReadRowsIterable.to_arrow(), and to inform users about the fallback behavior.
  2. Added Parametrized Tests: Added comprehensive tests to test_reader_v1_arrow.py to verify the behavior of to_arrow() on empty streams, both with and without a read_session provided.

Notes to Reviewers

  • This PR does not change functionality but adds testing and documentation around existing behavior to prevent future regression and aid debugging.
  • To guarantee the correct schema (non-empty) for empty streams, callers should provide the read_session to ReadRowsStream.to_arrow().

Fixes internal bug #352600521

@chalmerlowe chalmerlowe self-assigned this Jul 27, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds documentation and inline comments to clarify the behavior of to_arrow() on empty streams in ReadRowsIterable and ReadRowsStream. It also introduces a new unit test, test_to_arrow_empty_stream, to verify empty stream handling. Feedback on the unit test highlights a potential TypeError when testing ReadRowsIterable because it does not accept a read_session argument, and suggests a code modification to conditionally handle this case.

Comment on lines +222 to +225
read_session = _generate_arrow_read_session(arrow_schema) if use_session else None
expected_schema = arrow_schema if use_session else pyarrow.schema([])

table = reader.to_arrow(read_session)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The class_under_test fixture parametrizes both ReadRowsStream and ReadRowsIterable. Since ReadRowsIterable.to_arrow() does not accept any arguments (unlike ReadRowsStream.to_arrow()), calling reader.to_arrow(read_session) will raise a TypeError when testing ReadRowsIterable (even if read_session is None). We should return early for the session-based test for ReadRowsIterable and call to_arrow() without arguments when use_session is False. Avoid using pytest.skip() here as it would mark the entire test as skipped; use a conditional return instead.

Suggested change
read_session = _generate_arrow_read_session(arrow_schema) if use_session else None
expected_schema = arrow_schema if use_session else pyarrow.schema([])
table = reader.to_arrow(read_session)
if use_session and class_under_test.__name__ == "ReadRowsIterable":
return
read_session = _generate_arrow_read_session(arrow_schema) if use_session else None
expected_schema = arrow_schema if use_session else pyarrow.schema([])
kwargs = {"read_session": read_session} if use_session else {}
table = reader.to_arrow(**kwargs)
References
  1. Use pytest.skip() to skip an entire test. To skip only a portion of a test (e.g., a final verification step), use a conditional return instead, as pytest.skip() would mark the entire test as skipped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant