Commit 2a526c1
### Rationale for this change
When reading a CSV with sparse data (many missing values followed by actual values), Arrow can infer a column type as `null` based on the first block of data. When non-null values appear later, the error message incorrectly suggests using `skip = 1` for header rows, which is misleading.
### What changes are included in this PR?
Adds a specific check for "conversion error to null" that provides a helpful message explaining the cause (type inference from sparse data) and the solution (change the block size to use for inference).
### Are these changes tested?
Yes, added a test in `test-dataset-csv.R`.
### Are there any user-facing changes?
Yes, improved error message when CSV type inference fails due to sparse data.
---
This PR was authored by Claude (Opus 4.5) and reviewed by @ thisisnic.
🤖 Generated with [Claude Code](https://claude.ai/code)
* GitHub Issue: #35806
Lead-authored-by: Nic Crane <thisisnic@gmail.com>
Co-authored-by: Jonathan Keane <jkeane@gmail.com>
Signed-off-by: Nic Crane <thisisnic@gmail.com>
1 parent 80db102 commit 2a526c1
2 files changed
Lines changed: 36 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
196 | 196 | | |
197 | 197 | | |
198 | 198 | | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
199 | 213 | | |
200 | 214 | | |
201 | 215 | | |
| |||
290 | 304 | | |
291 | 305 | | |
292 | 306 | | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
711 | 711 | | |
712 | 712 | | |
713 | 713 | | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
0 commit comments