Describe the bug
CSVDocumentSplitter.run() raises RecursionError on some CSV inputs when both row_split_threshold and column_split_threshold are set. It does not fail on every multi-block CSV, only certain layouts where a nested split is needed on a later row block.
Error message
RecursionError: maximum recursion depth exceeded in comparison
Traceback (most recent call last):
File "repro_csv.py", line 26, in <module>
result = splitter.run([doc])
File "haystack/components/preprocessors/csv_document_splitter.py", line 133, in run
split_dfs = self._recursive_split(
File "haystack/components/preprocessors/csv_document_splitter.py", line 270, in _recursive_split
self._recursive_split(
File "haystack/components/preprocessors/csv_document_splitter.py", line 270, in _recursive_split
self._recursive_split(
[Previous line repeated 178 more times]
File "haystack/components/preprocessors/csv_document_splitter.py", line 262, in _recursive_split
final_tables.extend(self._split_dataframe(df=table, split_threshold=column_split_threshold, axis="column"))
File "haystack/components/preprocessors/csv_document_splitter.py", line 219, in _split_dataframe
split_indices = self._find_split_indices(df=df, split_threshold=split_threshold, axis=axis)
File "haystack/components/preprocessors/csv_document_splitter.py", line 181, in _find_split_indices
empty_elements = df.columns[df.isnull().all(axis=0)].tolist()
...
RecursionError: maximum recursion depth exceeded in comparison
Expected behavior
The splitter returns a list of sub-table Documents (or at least does not crash) for any well-formed CSV input, regardless of how many row/column blocks it contains.
Additional context
Only reproduces with both row_split_threshold and column_split_threshold set together, on a CSV with more than one row block where a later block also needs a column split. A single-threshold splitter (row-only or column-only) does not hit this on the same input.
To Reproduce
Input CSV:
A,B,C,D,E,F
1,2,3,4,5,6
,,,,,
P,Q,,,X,Y
1,2,,,7,8
,,,,M,N
,,,,9,10
R,S,,,,
3,4,,,,
Script:
import sys
from haystack import Document
from haystack.components.preprocessors.csv_document_splitter import CSVDocumentSplitter
csv_content = (
"A,B,C,D,E,F\n"
"1,2,3,4,5,6\n"
",,,,,\n"
"P,Q,,,X,Y\n"
"1,2,,,7,8\n"
",,,,M,N\n"
",,,,9,10\n"
"R,S,,,,\n"
"3,4,,,,\n"
)
sys.setrecursionlimit(200) # lowered only to make the crash fast to observe; default limit (1000) reproduces too
splitter = CSVDocumentSplitter(row_split_threshold=1, column_split_threshold=1)
doc = Document(content=csv_content, id="test_id")
result = splitter.run([doc])
print(f"got {len(result['documents'])} sub-tables")Running this raises the RecursionError above instead of returning sub-table documents.
FAQ Check
System:
- OS: Debian 13 (trixie), container
python:3.11-slim, verified on Linux x86_64
- Haystack version: 3.1.0rc0, editable install at commit
b53e614d1c8f826fe348b3b4bfc05633dc82a101, pandas 3.0.5, Python 3.11.15
Describe the bug
CSVDocumentSplitter.run()raisesRecursionErroron some CSV inputs when bothrow_split_thresholdandcolumn_split_thresholdare set. It does not fail on every multi-block CSV, only certain layouts where a nested split is needed on a later row block.Error message
Expected behavior
The splitter returns a list of sub-table
Documents (or at least does not crash) for any well-formed CSV input, regardless of how many row/column blocks it contains.Additional context
Only reproduces with both
row_split_thresholdandcolumn_split_thresholdset together, on a CSV with more than one row block where a later block also needs a column split. A single-threshold splitter (row-only or column-only) does not hit this on the same input.To Reproduce
Input CSV:
Script:
Running this raises the
RecursionErrorabove instead of returning sub-table documents.FAQ Check
System:
python:3.11-slim, verified on Linux x86_64b53e614d1c8f826fe348b3b4bfc05633dc82a101, pandas 3.0.5, Python 3.11.15