Support local HTTP ref paths#3116
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughAdds a new ChangesLocal HTTP Reference Resolution
Sequence DiagramsequenceDiagram
participant CLI as CLI Argument Parser
participant Config as Config / Generate
participant Parser as JsonSchema Parser
participant RefLoader as Ref Body Loader
participant FileSystem as Local Filesystem
participant Cache as Remote Object Cache
CLI->>Config: supply --http-local-ref-path
Config->>Parser: include http_local_ref_path in config
Parser->>Parser: set self.http_local_ref_path
Parser->>RefLoader: encounter HTTP(S) $ref URL
RefLoader->>RefLoader: is http_local_ref_path set?
alt set
RefLoader->>RefLoader: parse URL (netloc, path)
RefLoader->>FileSystem: map netloc/path → candidate files
FileSystem-->>RefLoader: check existence
alt candidate exists
FileSystem-->>RefLoader: read file content
RefLoader->>Cache: store in remote cache
Cache-->>RefLoader: return schema dict
else try .json fallback
FileSystem-->>RefLoader: read suffixed file or not found
alt found
RefLoader->>Cache: store and return
else
RefLoader->>RefLoader: raise Error (include URL)
end
end
else not set
RefLoader->>RefLoader: perform normal remote fetch logic
end
RefLoader-->>Parser: return schema dict or raise
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 2/8 reviews remaining, refill in 38 minutes and 34 seconds.Comment |
|
📚 Docs Preview: https://pr-3116.datamodel-code-generator.pages.dev |
Merging this PR will not alter performance
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3116 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 88 88
Lines 18325 18419 +94
Branches 2117 2126 +9
=========================================
+ Hits 18325 18419 +94
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
docs/cli-reference/model-customization.md (1)
5275-5277: ⚡ Quick winClarify dependency on
--strict-nullablefor nullability preservation.The documentation states that field types follow the schema's nullability, but based on the code evidence this behavior only occurs when
--strict-nullableis enabled. Without--strict-nullable, fields with defaults receivenullable=None, making them Optional regardless of the schema's nullability specification.Consider adding a brief mention of this dependency in the main explanation text, for example:
The field type still follows the schema's nullability. For example, a required string field with a default is generated as `str = 'value'`, not -`str | None = 'value'`, unless the schema allows null. +`str | None = 'value'`, unless the schema allows null. This behavior is +controlled by the `--strict-nullable` flag.Alternatively, you could phrase it as: "When combined with
--strict-nullable, the field type follows..." to make the dependency more explicit.As per coding guidelines, the code snippet from
src/datamodel_code_generator/parser/jsonschema.py:1239shows that nullable preservation depends on thestrict_nullableconfiguration:nullable=... (False if self.strict_nullable and (has_default or required) else None).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/cli-reference/model-customization.md` around lines 5275 - 5277, The docs claim field types follow the schema's nullability but omit that this only happens when the strict_nullable config/flag is enabled; update the explanatory text around the example to explicitly state that nullability preservation depends on --strict-nullable (or the strict_nullable setting) — e.g. add a sentence like "When --strict-nullable (strict_nullable) is enabled, the field type follows the schema's nullability; otherwise fields with defaults get nullable=None and will be treated as Optional." Reference the strict_nullable configuration and the nullable=... expression in the JSONSchema parser (nullable logic in jsonschema.py) so readers can correlate the behavior with the flag.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/cli-reference/general-options.md`:
- Around line 1653-1690: The example for --http-local-ref-path is misleading
because the JSON Schema has no HTTP $ref; update the example to include a schema
that uses an HTTP $ref (for example a root schema that $ref's
"https://api.example.com/schemas/pet.json") and show the corresponding local
schema-store layout and filenames that mirror that HTTP path (e.g.,
schema-store/schemas/pet.json) so readers can see how the CLI redirects the HTTP
reference to the local file; update the Input Schema block to include the HTTP
$ref, adjust the generated Output block accordingly (e.g., still showing class
Pet produced by datamodel-codegen), and add a brief note or inline listing
demonstrating the schema-store directory structure to make the relocation
behavior explicit.
In `@src/datamodel_code_generator/parser/jsonschema.py`:
- Around line 3852-3858: The current local HTTP $ref handling (involving parsed,
parts, unquote, Path, relative_path and file_paths) allows URL-decoded
backslashes to act as separators on Windows; update the validation to reject any
path segment that contains '/' or '\\' (in addition to "." and "..") after
unquoting, then construct relative_path and compute resolved =
(self.http_local_ref_path / relative_path).resolve() and ensure
resolved.is_relative_to(self.http_local_ref_path.resolve()) before using it; if
the check fails raise Error with the original ref to block path-traversal or
symlink-based escapes.
---
Nitpick comments:
In `@docs/cli-reference/model-customization.md`:
- Around line 5275-5277: The docs claim field types follow the schema's
nullability but omit that this only happens when the strict_nullable config/flag
is enabled; update the explanatory text around the example to explicitly state
that nullability preservation depends on --strict-nullable (or the
strict_nullable setting) — e.g. add a sentence like "When --strict-nullable
(strict_nullable) is enabled, the field type follows the schema's nullability;
otherwise fields with defaults get nullable=None and will be treated as
Optional." Reference the strict_nullable configuration and the nullable=...
expression in the JSONSchema parser (nullable logic in jsonschema.py) so readers
can correlate the behavior with the flag.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: a7d642b9-ec18-4a20-949a-6252943ddee7
⛔ Files ignored due to path filters (7)
tests/data/expected/main/help/color.txtis excluded by!tests/data/**/*.txtand included by nonetests/data/expected/main/help/no_color.txtis excluded by!tests/data/**/*.txtand included by nonetests/data/expected/main_kr/generate_prompt/basic.txtis excluded by!tests/data/**/*.txtand included by nonetests/data/expected/main_kr/generate_prompt/with_list_options.txtis excluded by!tests/data/**/*.txtand included by nonetests/data/expected/main_kr/generate_prompt/with_options.txtis excluded by!tests/data/**/*.txtand included by nonetests/data/expected/main_kr/generate_prompt/with_question.txtis excluded by!tests/data/**/*.txtand included by nonetests/data/expected/main_kr/help_shows_new_options.txtis excluded by!tests/data/**/*.txtand included by none
📒 Files selected for processing (17)
docs/cli-reference/general-options.mddocs/cli-reference/index.mddocs/cli-reference/model-customization.mddocs/cli-reference/quick-reference.mdsrc/datamodel_code_generator/__main__.pysrc/datamodel_code_generator/_types/generate_config_dict.pysrc/datamodel_code_generator/_types/parser_config_dicts.pysrc/datamodel_code_generator/arguments.pysrc/datamodel_code_generator/cli_options.pysrc/datamodel_code_generator/config.pysrc/datamodel_code_generator/parser/base.pysrc/datamodel_code_generator/parser/jsonschema.pysrc/datamodel_code_generator/prompt_data.pytests/data/expected/main/input_model/config_class.pytests/main/test_public_api_signature_baseline.pytests/parser/test_jsonschema.pytests/test_main_kr.py
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/datamodel_code_generator/parser/jsonschema.py`:
- Around line 3893-3894: The code currently redirects all non-file refs to the
local-HTTP resolver when self.http_local_ref_path is set; update the conditional
to only call _get_ref_body_from_local_http_path(ref) when the ref's URL scheme
is 'http' or 'https' (e.g., parse ref with urllib.parse.urlparse and check
scheme in ('http','https')), leaving other schemes untouched so they follow the
existing resolution path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 9aac966d-f76c-4c8b-b835-705458331419
⛔ Files ignored due to path filters (1)
tests/data/jsonschema/http_local_ref_path_root.jsonis excluded by!tests/data/**/*.jsonand included by none
📒 Files selected for processing (7)
docs/cli-reference/general-options.mddocs/cli-reference/model-customization.mdsrc/datamodel_code_generator/parser/jsonschema.pytests/data/expected/main_kr/http_local_ref_path/output.pytests/main/jsonschema/test_main_jsonschema.pytests/parser/test_jsonschema.pytests/test_main_kr.py
✅ Files skipped from review due to trivial changes (3)
- tests/data/expected/main_kr/http_local_ref_path/output.py
- docs/cli-reference/general-options.md
- tests/main/jsonschema/test_main_jsonschema.py
🚧 Files skipped from review as they are similar to previous changes (3)
- tests/test_main_kr.py
- docs/cli-reference/model-customization.md
- tests/parser/test_jsonschema.py
Breaking Change AnalysisResult: No breaking changes detected Reasoning: PR #3116 is a purely additive feature. It adds a new --http-local-ref-path CLI option that defaults to None. When not provided, all code paths remain identical to previous behavior. No existing CLI options were removed or renamed, no generated output format changed, no default behavior changed, and no existing API signatures were modified. The new config fields all use Optional with None defaults or NotRequired in TypedDicts. The only behavioral change (bypassing allow_remote_refs checks for HTTP refs) only activates when the new option is explicitly provided, so it cannot affect existing users. This analysis was performed by Claude Code Action |
|
🎉 Released in 0.57.0 This PR is now available in the latest release. See the release notes for details. |
1 participant
fixes: #3103
Summary by CodeRabbit
New Features
Documentation
Tests