Skip to content

refactor(audit-r3-soft): implement 27 triaged soft recommendations (B1–B9)#79

Merged
yilibinbin merged 1 commit into
mainfrom
fix/audit-r3-soft
Jul 2, 2026
Merged

refactor(audit-r3-soft): implement 27 triaged soft recommendations (B1–B9)#79
yilibinbin merged 1 commit into
mainfrom
fix/audit-r3-soft

Conversation

@yilibinbin

@yilibinbin yilibinbin commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Audit R3 — soft-recommendation implementation (27 KEEP items, B1–B9)

Triaged the R3 review's 68 soft recommendations (27 KEEP / 1 DECIDE / 37 SKIP-with-reason) and implemented the KEEP set in 9 themed batches. Each batch was built in an isolated worktree, then integrated and independently re-verified on the composed tree (not trusting per-batch self-reports).

Batch What
B1 docs deploy.en.md gains the 5 LaTeX-sandbox env vars + DATALAB_SSE_DISABLE_RATE_LIMIT + DATALAB_TRUST_PROXY_HEADERS; CLAUDE.md env list expanded; ARCHITECTURE.md gains a datalab_core service-layer section. (DATALAB_LATEX_ENGINE deliberately not documented — tools-only.)
B2 docstrings fit_custom_model (all params/errors), precision_guard (clamp params, constants, mp.dps contract).
B3 bilingual mcmc_fitter.run_mcmc's 4 validation ValueErrors + constraints._parse_expr_safe now use _dual_msg; separator tests.
B4 type-safety drop defensive getattr on required fields; TypeAlias on MpfCallable; real typed ModelSpecification implicit fields (set as kwargs, not setattr). mypy --strict clean.
B5 perf J^T J via mp.matrix multiply; hoisted mp.mpf("0") — behavior-preserving.
B6 tests new suites for precision/_ast_metrics/_generate_seed_variants/ParameterState.compose — plus a loud-fail guard in compose() for a real length-mismatch truncation.
B7 expr-engine ZeroDivisionError from / and % wrapped in a bilingual ValueError (honors the ValueError-only contract) + tests.
B8 statistics deduplicate _bool_option/_string_option into datalab_core/statistics_helpers.py (strict fail-fast; verified no caller needs string variants).
B9 maintainability hoist a per-iteration combo read; consolidate a 3-line trim_visible; delete the empty WindowFittingParamsMixin placeholder; drop the single-use _latex_escape_text wrapper.

Also: consciously re-baselined fitting/hp_fitter.py in the file-size ratchet (819→890, almost all docstrings).

Test plan

  • TDD / new tests per batch; independent re-verify on the integrated tree.
  • ruff + mypy --strict (shared/fitting/extrapolation_methods/datalab_latex) clean.
  • Full suite: 3808 passed, 9 skipped, 0 failed.

Deferred (not in this PR): 1 DECIDE (_refresh_display_format refactor — needs a maintainer call) and Batch 10 (ExtrapolationWindow 3196-line decomposition — its own staged track).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added clearer configuration guidance for web deployments, including proxy support, SSE rate limiting controls, and LaTeX resource limits.
    • Expanded end-user documentation with a more complete overview of the app’s service layer.
  • Bug Fixes

    • Improved error handling so invalid inputs and divide-by-zero cases now show clearer bilingual messages.
    • Tightened fitting and statistics validation to catch mismatched or invalid values earlier.
  • Documentation

    • Updated precision and deployment docs with more examples and clearer usage notes.

…1–B9)

From the R3 full-package review's 68 soft recommendations, triaged (27 KEEP /
1 DECIDE / 37 SKIP-with-reason) and implemented the KEEP set in 9 themed batches,
each built in an isolated worktree and integrated here with ruff + targeted tests
re-verified on the composed tree.

B1 docs: deploy.en.md gains the 5 LaTeX-sandbox env vars + DATALAB_SSE_DISABLE_
   RATE_LIMIT + DATALAB_TRUST_PROXY_HEADERS; CLAUDE.md env list expanded;
   ARCHITECTURE.md gains a datalab_core service-layer section.
   (DATALAB_LATEX_ENGINE deliberately NOT documented as a deploy var — it's
   tools-only, not on the web runtime path.)
B2 docstrings: fit_custom_model (all params/errors), precision_guard (clamp
   params, constants, mp.dps contract, example).
B3 bilingual: mcmc_fitter run_mcmc's 4 validation ValueErrors + constraints
   _parse_expr_safe parse error now use _dual_msg; bilingual-separator tests.
B4 type-safety: drop defensive getattr on required fields (hp_fitter), TypeAlias
   on MpfCallable, real typed ModelSpecification fields for implicit metadata
   (implicit_model sets them as kwargs, not setattr). mypy --strict clean.
B5 perf: J^T J via mp.matrix multiply; hoisted mp.mpf("0") — behavior-preserving.
B6 tests: new suites for precision (_coerce_int/precision_guard/clamps/validators),
   _ast_metrics, _generate_seed_variants, ParameterState.compose — plus a
   loud-fail guard in compose() for the real length-mismatch truncation.
B7 expression_engine: ZeroDivisionError from / and % now wrapped in a bilingual
   ValueError (honors the module's ValueError-only contract) + tests.
B8 statistics: deduplicate _bool_option/_string_option into datalab_core/
   statistics_helpers.py (strict fail-fast default; verified no caller needs
   string variants).
B9 maintainability: hoist a per-iteration combo read; consolidate a 3-line
   trim_visible; delete the empty WindowFittingParamsMixin placeholder; drop the
   single-use _latex_escape_text wrapper.

360+ tests across all touched areas pass; ruff + mypy --strict clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR bundles bilingual (Chinese/English) error messaging across constraint parsing, MCMC validation, and expression-engine division-by-zero handling; deduplicates statistics option-parsing helpers into a new shared module; removes an unused desktop mixin; refactors implicit-model field wiring and hp_fitter internals; updates deployment/architecture docs; and adds extensive new tests.

Changes

Bilingual error messages and expression engine

Layer / File(s) Summary
Constraint compose validation and parse errors
fitting/constraints.py, tests/test_constraints_compose.py, tests/test_bilingual_errors.py
ParameterState.compose validates free-vector length against free_params and raises bilingual errors; _parse_expr_safe wraps SymPy failures into bilingual ValueErrors; new tests cover compose edge cases and bilingual messaging.
MCMC validation messages
fitting/mcmc_fitter.py, tests/test_bilingual_errors.py
run_mcmc argument-validation errors now use _dual_msg for bilingual text; new tests exercise each invalid-argument path.
Expression engine division-by-zero
shared/expression_engine.py, tests/test_expression_engine_zerodiv.py, tests/test_expression_engine_ast_metrics.py
_evaluate_ast converts ZeroDivisionError into bilingual ValueError; new tests cover zero-division and _ast_metrics behavior.

Statistics helper deduplication

Layer / File(s) Summary
Shared helpers module
datalab_core/statistics_helpers.py
New module defines strict _bool_option/_string_option parsers with bilingual errors.
Module wiring and updated test
datalab_core/statistics.py, statistics_grouped.py, statistics_matrix.py, statistics_time_series.py, tests/test_datalab_core_statistics.py
Local helper implementations removed; modules import from statistics_helpers; one test assertion updated for bilingual message.

Fitting internals

Layer / File(s) Summary
ModelSpecification implicit fields
fitting/model_parser.py, fitting/implicit_model.py
ModelSpecification gains optional implicit fields set directly at construction instead of via setattr.
hp_fitter covariance and docs
fitting/hp_fitter.py, tests/test_hp_fitter_seed_variants.py, tests/test_file_size_ratchet.py
Covariance/error-propagation code uses a shared zero constant and mp.matrix transpose; docstrings expanded; new seed-variant tests added and baseline size updated.

Desktop app cleanup

Layer / File(s) Summary
Mixin removal
app_desktop/window_fitting_params_mixin.py, window_fitting_mixin.py
Empty WindowFittingParamsMixin removed; WindowFittingMixin composition and docstrings updated.
LaTeX escaping consolidation
app_desktop/fitting_latex_writer.py
_latex_escape_text wrapper removed; call sites use latex_escape() directly.
Stats mode visibility tweaks
app_desktop/window.py
Minor logic reordering and reformatting in _on_stats_mode_change.

Docs and precision guard

Layer / File(s) Summary
Env var and architecture docs
CLAUDE.md, docs/web/deploy.en.md, docs/ARCHITECTURE.md, docs/CODE_REVIEW_2026_R3_SOFT_PLAN.md
Documents proxy/SSE/LaTeX env vars, adds a datalab_core service-layer architecture section, and adds a new review-plan document.
precision_guard docs and tests
shared/precision.py, tests/test_shared_precision.py
Docstring expanded; new tests validate coercion, clamping, and probability/confidence helpers.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Possibly related PRs

  • yilibinbin/DataLab#47: Overlaps in fitting/implicit_model.py's build_implicit_model_specification and ModelSpecification construction wiring.
  • yilibinbin/DataLab#52: Touches the same implicit-fitting core (fitting/implicit_model.py, fitting/hp_fitter.py details wiring).
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: implementing the triaged R3 soft recommendations across batches B1–B9.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/audit-r3-soft

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
app_desktop/window.py (1)

1366-1373: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Duplicate combo-box query: reuse method instead of re-fetching time_series_method later.

method here and time_series_method (further down, unchanged, lines 1433-1437) both call self.stats_time_series_method_combo.currentData() and hold the same value within this same function call. Consider consolidating to a single variable now that this call has been hoisted.

♻️ Proposed consolidation
         method = (
             self.stats_time_series_method_combo.currentData()
             if hasattr(self, "stats_time_series_method_combo")
             else None
         )
@@
-        time_series_method = (
-            self.stats_time_series_method_combo.currentData()
-            if hasattr(self, "stats_time_series_method_combo")
-            else None
-        )
-        is_ewma = time_series_method == "ewma"
-        is_rolling_std = time_series_method == "rolling_std"
+        is_ewma = method == "ewma"
+        is_rolling_std = method == "rolling_std"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app_desktop/window.py` around lines 1366 - 1373, The time-series method is
being queried twice in the same update path, once into method and again later
into time_series_method, which is redundant. In the relevant window update logic
in window.py, reuse the already hoisted method value instead of calling
self.stats_time_series_method_combo.currentData() again, and make sure the later
visibility/branching code references that single variable consistently.
fitting/hp_fitter.py (1)

742-746: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Inconsistent getattr usage for a now-guaranteed field.

implicit_definition is a proper ModelSpecification dataclass field (default None) as of this same PR, yet line 745 still reads it via getattr(current_model, "implicit_definition", None). current_model is always a ModelSpecification here (either model or the model_factory return value, both typed as ModelSpecification), so this can be simplified to direct attribute access — consistent with the dependent_defs/expression cleanups made elsewhere in this same diff (lines 256, 691), and it lets mypy actually type-check the access instead of returning Any.

♻️ Proposed simplification
             can_parallelize = (
                 model_factory is None
                 and _dependent_defs_are_picklable(parameter_state)
-                and getattr(current_model, "implicit_definition", None) is None
+                and current_model.implicit_definition is None
             )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fitting/hp_fitter.py` around lines 742 - 746, The parallelization check in
the hp_fitter logic still uses getattr for a field that is now guaranteed on
ModelSpecification. Update the can_parallelize expression to read
implicit_definition directly from current_model, since current_model is always a
ModelSpecification here and this should match the dependent_defs/expression
direct-access cleanup elsewhere in the file. Keep the change localized around
the can_parallelize assignment so mypy can type-check the field access properly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/CODE_REVIEW_2026_R3_SOFT_PLAN.md`:
- Around line 11-12: The plan still points to stale paths and env-var names, so
update the affected notes to match the shipped tree. In the CLAUDE.md and
docs/ARCHITECTURE.md references, use the current DATALAB_LATEX_* set exposed by
app_web/latex_security.py, remove or replace the outdated DATALAB_LATEX_ENGINE
mention unless it is truly present, and change any statistics/_helpers.py
reference to datalab_core/statistics_helpers.py. Keep the plan wording aligned
with the current codebase and cross-reference the existing CLAUDE.md and
architecture sections instead of duplicating outdated location details.

---

Nitpick comments:
In `@app_desktop/window.py`:
- Around line 1366-1373: The time-series method is being queried twice in the
same update path, once into method and again later into time_series_method,
which is redundant. In the relevant window update logic in window.py, reuse the
already hoisted method value instead of calling
self.stats_time_series_method_combo.currentData() again, and make sure the later
visibility/branching code references that single variable consistently.

In `@fitting/hp_fitter.py`:
- Around line 742-746: The parallelization check in the hp_fitter logic still
uses getattr for a field that is now guaranteed on ModelSpecification. Update
the can_parallelize expression to read implicit_definition directly from
current_model, since current_model is always a ModelSpecification here and this
should match the dependent_defs/expression direct-access cleanup elsewhere in
the file. Keep the change localized around the can_parallelize assignment so
mypy can type-check the field access properly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d868b04a-9bdf-4661-91cd-45c86934366a

📥 Commits

Reviewing files that changed from the base of the PR and between 44701c6 and 7f2da04.

📒 Files selected for processing (28)
  • CLAUDE.md
  • app_desktop/fitting_latex_writer.py
  • app_desktop/window.py
  • app_desktop/window_fitting_mixin.py
  • app_desktop/window_fitting_params_mixin.py
  • datalab_core/statistics.py
  • datalab_core/statistics_grouped.py
  • datalab_core/statistics_helpers.py
  • datalab_core/statistics_matrix.py
  • datalab_core/statistics_time_series.py
  • docs/ARCHITECTURE.md
  • docs/CODE_REVIEW_2026_R3_SOFT_PLAN.md
  • docs/web/deploy.en.md
  • fitting/constraints.py
  • fitting/hp_fitter.py
  • fitting/implicit_model.py
  • fitting/mcmc_fitter.py
  • fitting/model_parser.py
  • shared/expression_engine.py
  • shared/precision.py
  • tests/test_bilingual_errors.py
  • tests/test_constraints_compose.py
  • tests/test_datalab_core_statistics.py
  • tests/test_expression_engine_ast_metrics.py
  • tests/test_expression_engine_zerodiv.py
  • tests/test_file_size_ratchet.py
  • tests/test_hp_fitter_seed_variants.py
  • tests/test_shared_precision.py
💤 Files with no reviewable changes (1)
  • app_desktop/window_fitting_params_mixin.py

Comment on lines +11 to +12
- `CLAUDE.md:48` — expand env-var list to include the 8 missing vars (`DATALAB_TRUST_PROXY_HEADERS`, the 5 LaTeX vars, `DATALAB_LATEX_ENGINE`), or add a one-line pointer to `deploy.en.md`. `[_idx 27]`
- `docs/ARCHITECTURE.md:78` — add a `datalab_core` service-layer section between "Computation modules" and "Cross-cutting conventions"; cross-reference CLAUDE.md rather than duplicating. `[_idx 26]`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Keep this plan aligned with the shipped paths.

The batch notes still mention DATALAB_LATEX_ENGINE and statistics/_helpers.py, but the current tree exposes only the five DATALAB_LATEX_* vars in app_web/latex_security.py and the helper module now lives at datalab_core/statistics_helpers.py. Please update the plan so it doesn't point readers at stale locations.

Also applies to: 50-52

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/CODE_REVIEW_2026_R3_SOFT_PLAN.md` around lines 11 - 12, The plan still
points to stale paths and env-var names, so update the affected notes to match
the shipped tree. In the CLAUDE.md and docs/ARCHITECTURE.md references, use the
current DATALAB_LATEX_* set exposed by app_web/latex_security.py, remove or
replace the outdated DATALAB_LATEX_ENGINE mention unless it is truly present,
and change any statistics/_helpers.py reference to
datalab_core/statistics_helpers.py. Keep the plan wording aligned with the
current codebase and cross-reference the existing CLAUDE.md and architecture
sections instead of duplicating outdated location details.

@yilibinbin yilibinbin merged commit a7e230a into main Jul 2, 2026
6 checks passed
@yilibinbin yilibinbin deleted the fix/audit-r3-soft branch July 2, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants