Fix auto CSV fallback

[koxudaxi]

Summary by CodeRabbit

• Bug Fixes
  • Improved automatic CSV input detection using CSV-like patterns, reducing incorrect matches.
  • Enhanced auto-detection failure messaging with clearer instructions (including the input-type hint) and safer handling of malformed content.
• Documentation
  • Updated supported-format/data-type guidance to reflect the revised CSV auto-detection rules.
• Tests
  • Added/expanded CLI and input-type inference tests to cover valid CSV detection and rejection of malformed non-CSV inputs.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 41718c52-8601-47c8-8712-5517a0c015b4

📥 Commits

Reviewing files that changed from the base of the PR and between e83957d and 1b96f0b.

⛔ Files ignored due to path filters (2)

• docs/llms-full.txt is excluded by none and included by none
• tests/data/expected/schema_docs/input_format_guide.txt is excluded by !tests/data/**/*.txt and included by none

📒 Files selected for processing (6)

• docs/supported-data-types.md
• docs/supported_formats.md
• scripts/build_schema_docs.py
• src/datamodel_code_generator/__init__.py
• tests/test_infer_input_type.py
• tests/test_yaml_backend.py

✅ Files skipped from review due to trivial changes (2)

• docs/supported-data-types.md
• scripts/build_schema_docs.py

🚧 Files skipped from review as they are similar to previous changes (4)

• docs/supported_formats.md
• tests/test_infer_input_type.py
• src/datamodel_code_generator/init.py
• tests/test_yaml_backend.py

---

📝 Walkthrough

Walkthrough

Modifies infer_input_type() to detect CSV input via a text heuristic instead of defaulting to CSV whenever YAML parsing fails, adds centralized error messaging, and updates tests and documentation to match.

Changes

CSV auto-detection

Layer / File(s)	Summary
CSV heuristic and error handling in infer_input_type src/datamodel_code_generator/__init__.py	Adds _looks_like_csv_text() and _infer_input_type_error_message(); infer_input_type() now checks CSV-likeness before falling back to CSV on YAML failure, and raises Error with a detailed message when detection fails.
Unit tests for detection logic tests/test_infer_input_type.py, tests/test_yaml_backend.py	Adds and updates tests for _looks_like_csv_text, CSV inference success cases, malformed-input rejection, and ryaml-specific parse error scenarios.
CLI-level auto-detection tests tests/main/test_main_csv.py	Adds tests verifying CLI auto-detects CSV input and reports it via stderr, and rejects malformed non-CSV inputs with an error exit and explicit-type hint.
Documentation updates for CSV auto detection docs/supported-data-types.md, docs/supported_formats.md, scripts/build_schema_docs.py	Updates the CSV auto-detection description from a YAML-parse-fallback explanation to a CSV-specific text-pattern explanation.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

• koxudaxi/datamodel-code-generator#3055: Both PRs modify infer_input_type in src/datamodel_code_generator/__init__.py to change how YAML parsing failures are handled for CSV inference.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and directly matches the main change: improving automatic CSV fallback detection.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix-auto-csv-fallback

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

📚 Docs Preview: https://pr-3536.datamodel-code-generator.pages.dev

[codspeed-hq]

Merging this PR will improve performance by 48.32%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 11 improved benchmarks
⏩ 98 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	WallTime	test_perf_large_models_pydantic_v2	3.6 s	2.4 s	+51.93%
⚡	WallTime	test_perf_duplicate_names	1,057.9 ms	700 ms	+51.13%
⚡	WallTime	test_perf_all_options_enabled	5.5 s	3.6 s	+50.44%
⚡	WallTime	test_perf_deep_nested	5.2 s	3.4 s	+50.22%
⚡	WallTime	test_perf_complex_refs	2 s	1.3 s	+50.09%
⚡	WallTime	test_perf_openapi_large	3 s	2 s	+49.93%
⚡	WallTime	test_perf_graphql_style_pydantic_v2	803 ms	537.7 ms	+49.34%
⚡	WallTime	test_perf_aws_style_openapi_pydantic_v2	1.9 s	1.3 s	+46.19%
⚡	WallTime	test_perf_stripe_style_pydantic_v2	2 s	1.4 s	+45.45%
⚡	WallTime	test_perf_kubernetes_style_pydantic_v2	2.6 s	1.8 s	+44.84%
⚡	WallTime	test_perf_multiple_files_input	3.4 s	2.4 s	+42.33%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing fix-auto-csv-fallback (1b96f0b) with main (4f20c04)}

Footnotes

1. 98 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (cab75a2) to head (1b96f0b).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main     #3536   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          159       160    +1     
  Lines        34268     34344   +76     
  Branches      4001      4011   +10     
=========================================
+ Hits         34268     34344   +76     

Flag	Coverage Δ
unittests	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/build_schema_docs.py`:
- Line 157: The CSV auto-detection description in the InputFileType.CSV entry is
too narrow and should match the actual heuristic used by the detector. Update
the wording in the schema docs generator so it describes any text with at least
two non-empty lines that share the same comma count, and keep the explicit `csv`
mention; use the InputFileType.CSV mapping in build_schema_docs.py as the place
to rephrase it.

In `@src/datamodel_code_generator/__init__.py`:
- Around line 1570-1584: The _looks_like_csv_text helper returns too early after
the first pair of matching comma counts, so it can misclassify multi-line text
as CSV-like before checking the rest. Update the logic in _looks_like_csv_text
to keep scanning all non-empty lines and require every line to match the same
comma count established from the first non-empty line. Preserve the existing
zero-comma rejection and use the current line-count tracking in the function to
validate consistency across the full input.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b9d69192-a6b7-4ff6-9da3-076ca3234450

📥 Commits

Reviewing files that changed from the base of the PR and between cab75a2 and e83957d.

⛔ Files ignored due to path filters (3)

• docs/llms-full.txt is excluded by none and included by none
• tests/data/expected/schema_docs/input_format_guide.txt is excluded by !tests/data/**/*.txt and included by none
• tests/data/yaml/broken.yaml is excluded by !tests/data/**/*.yaml and included by none

📒 Files selected for processing (7)

• docs/supported-data-types.md
• docs/supported_formats.md
• scripts/build_schema_docs.py
• src/datamodel_code_generator/__init__.py
• tests/main/test_main_csv.py
• tests/test_infer_input_type.py
• tests/test_yaml_backend.py

[github-actions]

Breaking Change Analysis

Result: Breaking changes detected

Reasoning: The core change alters the observable behavior of auto input-type inference. Previously any YAML parse error routed to CSV; now non-CSV-looking malformed input (and ragged CSV) raises an Error instead. This changes both default behavior and error handling for existing auto users — inputs that previously produced output can now fail. No public CLI options, API signatures, enum values, or functions were removed (new helpers are private, underscore-prefixed additions); remaining changes are docs and tests.

Content for Release Notes

Default Behavior Changes

• Auto input-type CSV fallback is now conditional - When --input-file-type auto is used, infer_input_type() previously treated any YAML parse failure as CSV data. It now falls back to CSV only when the text looks like CSV (non-empty lines with consistent, non-zero comma counts). Inputs that were previously inferred as csv — including malformed JSON/YAML and ragged CSV with inconsistent column counts (e.g. id,name,tel\n1,taro\n) — will no longer be inferred as CSV. (Fix auto CSV fallback #3536)

Error Handling Changes

• Malformed auto-detected input now raises an error instead of silently inferring CSV - Under auto inference, input that fails YAML parsing and does not look like CSV now raises Error rather than falling back to CSV and attempting generation. The error message was also expanded to include the underlying YAML parser error, e.g. Can't infer input file type from the input data. YAML parser error: <Type>: <detail>. Please specify the input file type explicitly with --input-file-type option. Workflows depending on the old blanket-CSV fallback must now pass --input-file-type csv explicitly. (Fix auto CSV fallback #3536)

---

This analysis was performed by Claude Code Action

[github-actions]

🎉 Released in 0.67.0

This PR is now available in the latest release. See the release notes for details.