Fix master CI: DataDrivenLux downgrade resolution + docs linkcheck (#624)

* Fix master CI: DataDrivenLux downgrade resolution and docs linkcheck

Two unrelated master CI failures.

1. downgrade-sublibraries / test (lib/DataDrivenLux): the test target listed
   `OrdinaryDiffEq` as a dependency, but it is not used anywhere in
   DataDrivenLux's sources or tests. Under the strict downgrade resolver it
   forced the entire OrdinaryDiffEq solver split-package ecosystem
   (OrdinaryDiffEqBDF, OrdinaryDiffEqNonlinearSolve, ...) to its ancient floor
   (v6/7.0.x), which no longer co-resolves with the modern SciMLBase 3 /
   ModelingToolkit 11 / Symbolics 7 stack that DataDrivenDiffEq pins, yielding
   "Unsatisfiable requirements". Removing the unused dependency lets the
   downgrade `Pkg.test` resolve and the suite pass on Julia 1.10.

2. Documentation / Build and Deploy: `linkcheck` errored on
   docs.sciml.ai/ModelingToolkit/.../AbstractSystem (HTTP 403 anti-crawler) and
   on dead ai.damtp.cam.ac.uk/symbolicregression/.../api/#Options and
   #EquationSearch anchors (404 after the SymbolicRegression docs reorganized).
   Repoint the dead API links to the maintained canonical SciML-hosted docs
   (#EquationSearch -> #equation_search, the current function name) and add a
   linkcheck_ignore regex for docs.sciml.ai, which rejects the linkcheck bot
   with 403 even though the cross-doc links resolve in a browser.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix DataDrivenLux QA: real Aqua/JET findings + local-source resolution

The sublibraries QA group (Aqua + JET) for DataDrivenLux has failed since it
was added (#618). The findings are genuine source bugs:

Aqua (unbound type parameter):
- update_path(f, id, states::PathState{T}...) left T unbound when called with
  zero states. Require at least one state (state1::PathState{T}, states...) so
  T always binds.

JET (5 possible errors):
- custom_priors.jl summary referenced an undefined `E` ($E) in its interpolated
  string; print the error-model name (nameof(typeof(d.errormodel))) instead.
- dataset.jl built the empty t-interval with Interval{T}(zero(T), zero(T)),
  which is no longer a valid IntervalArithmetic constructor (errors at runtime);
  use interval(zero(T), zero(T)), matching the other branch.
- The DataDrivenFunction OOP/IIP *matrix* dispatch was missing the
  {true, true} (implicit + controls) methods in DataDrivenDiffEq's
  build_function.jl, so a Basis{true,true}(::Dataset) call had no matching
  method. Add the two missing matrix methods (additive, mirrors the existing
  {true,false} pair).
- Candidate / CommonAlgOptions: the @concrete field type-constraints made the
  generated inner constructors unprovable to JET when fed Any-typed values.
  Drop the (non-load-bearing) Candidate field bounds, and give CommonAlgOptions
  an explicit type-asserting keyword constructor in place of @kwdef so the
  positional @concrete constructor receives concretely-typed arguments.

QA now resolves the in-repo DataDrivenDiffEq (not the registered release): add
DataDrivenDiffEq to the qa env [deps]/[sources], and in activate_qa_env develop
the local paths on Julia < 1.11 (where [sources] is ignored), mirroring
.github's develop_sources helper.

Verified locally: DataDrivenLux QA (Aqua + JET) passes 12/12 on both Julia 1.10
(lts) and 1.12, and the DataDrivenLux Core test suite passes on 1.12.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: repoint remaining dead SymbolicRegression api link in DataDrivenSR docstring

PR #624 repointed the dead ai.damtp.cam.ac.uk/symbolicregression/.../api/#Options
links in the datadrivensr examples, but missed the same dead link in the
DataDrivenSR `Options` docstring (lib/DataDrivenSR/src/DataDrivenSR.jl), which is
rendered into the docs and linkchecked. That URL 404s, failing the docs build.
Repoint it to the maintained docs.sciml.ai host (already covered by the
linkcheck_ignore added in #624).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix DataDrivenSR QA: coef field bug + local-source resolution

Touching DataDrivenSR's source makes the affected-sublibrary detector run its
QA group, which (like DataDrivenLux's) had never passed. Two real findings:

- JET: coef(x::SRResult) read getfield(x, :k), but SRResult has no :k field
  (the accessor errored on every call and is untested). Return the model
  coefficients — the parameter values of the result's basis,
  get_parameter_values(x.basis) — matching how the parameters are produced in
  the SRResult constructor and used in solution.jl.
- Aqua deps_compat failed because, on Julia < 1.11, the qa env's [sources] is
  ignored and QA analyzed the *registered* DataDrivenSR (whose Project.toml
  lacks compat for the test extras); the in-repo Project already declares them.
  Resolve the in-repo DataDrivenSR/DataDrivenDiffEq by adding DataDrivenDiffEq
  to the qa env [deps]/[sources] and developing the local paths on Julia < 1.11
  in activate_qa_env (no-op effect on >= 1.11), mirroring the DataDrivenLux fix.

Verified locally: DataDrivenSR QA (Aqua + JET) passes 12/12 on Julia 1.12.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Revert DataDrivenLux OrdinaryDiffEq removal; downgrade waits on upstream release

Restores `OrdinaryDiffEq = "6, 7"` to [compat], the OrdinaryDiffEq UUID to
[extras], and OrdinaryDiffEq to [targets].test in lib/DataDrivenLux/Project.toml
(reverts the 10af37c removal).

The DataDrivenLux downgrade lane does NOT actually require dropping
OrdinaryDiffEq: with OrdinaryDiffEq removed, the lane is still red, and the
failure is identical with it present. The CI failing step is
`julia-runtest` (step 11), not the resolver step (8) or buildpkg (10): the
downgrade-compat resolve and buildpkg both succeed; the `Pkg.test()` test-env
re-resolve is what fails Unsatisfiable. Reproduced locally on Julia 1.10 with
the exact CI sequence (downgrade.jl alldeps 1.10 -> Pkg.test allow_reresolve=false):

  Unsatisfiable requirements detected for package ModelingToolkitBase:
   ModelingToolkit [restricted to 11.21.0-11 by DataDrivenDiffEq]
     -> ModelingToolkitBase 1.30.0-1.45.0
        -> SymbolicUtils 4.23.1-4
   but SymbolicUtils is held at <= 4.21.0 at the downgrade floor -> no versions left

Root cause (release-dependent, not test-dep-dependent): DataDrivenDiffEq's
master floor `ModelingToolkit = "11.21.0"` (set in 24ae52c for the Julia 1.10
precompile) transitively requires ModelingToolkitBase >= 1.30.0, whose
SymbolicUtils floor is 4.23.1 (monotone: every MTKBase >= 1.30 needs >= 4.23.1).
Under the strict downgrade min-resolve the symbolic stack is held at its floor
(Symbolics 7.0-7.21 guarantee only SymbolicUtils >= 4.21.0), so SymbolicUtils
cannot reach 4.23.1 and there is no overlap. This is the broader SciML
symbolic-stack downgrade-floor wall; it resolves once an upstream
ModelingToolkit 11.x release raises its `Symbolics` compat floor to >= 7.22.0
(the first Symbolics whose SymbolicUtils floor, 4.27.0, satisfies MTKBase 1.30),
forcing the downgrade min to a SymbolicUtils >= 4.23.1. Verified: pinning the
test floor to Symbolics >= 7.22.0 (or SymbolicUtils >= 4.23.1) resolves cleanly.

Per the directive: do not drop a real test dependency to dodge the downgrade
resolver. The DataDrivenLux downgrade lane stays red until that upstream release
lands; the rest of #624 (DataDrivenSR coef fix, docs linkcheck, QA source fixes,
qa [sources]) is release-independent and ready.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: ChrisRackauckas-Claude <accounts@chrisrackauckas.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: 3 people

docs/make.jl

@@ -87,6 +87,9 @@ makedocs(
         "https://royalsocietypublishing.org/doi/10.1098/rspa.2020.0279",
         "https://www.pnas.org/doi/10.1073/pnas.1517384113",
         "https://link.springer.com/article/10.1007/s00332-015-9258-5",
+        # SciML's hosted docs reject Documenter's linkcheck crawler with HTTP 403,
+        # though the cross-doc links resolve fine in a browser.
+        r"^https://docs\.sciml\.ai/.*",
     ],
     format = Documenter.HTML(
         assets = ["assets/favicon.ico"],

docs/src/libs/datadrivensr/example_01.jl

@@ -33,7 +33,7 @@ prob = ContinuousDataDrivenProblem(X, t, U = U)
 
 #md plot(prob)
 
-# To solve our problem, we will use [`EQSearch`](@ref), which provides a wrapper for the [symbolic regression interface](https://ai.damtp.cam.ac.uk/symbolicregression/stable/api/#Options).
+# To solve our problem, we will use [`EQSearch`](@ref), which provides a wrapper for the [symbolic regression interface](https://docs.sciml.ai/SymbolicRegression/stable/api/#Options).
 # We will stick to simple operations, use a `L1DistLoss`, and limit the verbosity of the algorithm.
 
 eqsearch_options = SymbolicRegression.Options(
@@ -46,7 +46,7 @@ eqsearch_options = SymbolicRegression.Options(
 alg = EQSearch(eq_options = eqsearch_options)
 
 # Again, we `solve` the problem to obtain a [`DataDrivenSolution`](@ref). Note that any additional keyword arguments are passed onto
-# symbolic regressions [`EquationSearch`](https://ai.damtp.cam.ac.uk/symbolicregression/stable/api/#EquationSearch) with the exception of `niterations` which
+# symbolic regressions [`equation_search`](https://docs.sciml.ai/SymbolicRegression/stable/api/#equation_search) with the exception of `niterations` which
 # is `maxiters`
 
 res = solve(prob, alg, options = DataDrivenCommonOptions(maxiters = 100))

docs/src/libs/datadrivensr/example_02.jl

@@ -27,7 +27,7 @@ prob = DataDrivenProblem(sol)
 
 #md plot(prob)
 
-# To solve our problem, we will use [`EQSearch`](@ref), which provides a wrapper for the [symbolic regression interface](https://ai.damtp.cam.ac.uk/symbolicregression/stable/api/#Options).
+# To solve our problem, we will use [`EQSearch`](@ref), which provides a wrapper for the [symbolic regression interface](https://docs.sciml.ai/SymbolicRegression/stable/api/#Options).
 # We will stick to simple operations, use a `L1DistLoss`, and limit the verbosity of the algorithm.
 # Note that we do not include `sin`, but rather lift the search space of variables.
 

lib/DataDrivenLux/src/algorithms/common.jl

@@ -1,19 +1,45 @@
-@kwdef @concrete struct CommonAlgOptions
-    populationsize::Int = 100
-    functions = (sin, exp, cos, log, +, -, /, *)
-    arities = (1, 1, 1, 1, 2, 2, 2, 2)
-    n_layers::Int = 1
-    skip::Bool = true
-    simplex <: AbstractSimplex = Softmax()
-    loss = aicc
-    keep <: Union{Real, Int} = 0.1
-    use_protected::Bool = true
-    distributed::Bool = false
-    threaded::Bool = false
-    rng <: AbstractRNG = Random.default_rng()
-    optimizer = LBFGS()
-    optim_options <: Optim.Options = Optim.Options()
-    optimiser <: Union{Nothing, Optimisers.AbstractRule} = nothing
-    observed <: Union{ObservedModel, Nothing} = nothing
-    alpha::Real = 0.999f0
+@concrete struct CommonAlgOptions
+    populationsize::Int
+    functions
+    arities
+    n_layers::Int
+    skip::Bool
+    simplex <: AbstractSimplex
+    loss
+    keep <: Union{Real, Int}
+    use_protected::Bool
+    distributed::Bool
+    threaded::Bool
+    rng <: AbstractRNG
+    optimizer
+    optim_options <: Optim.Options
+    optimiser <: Union{Nothing, Optimisers.AbstractRule}
+    observed <: Union{ObservedModel, Nothing}
+    alpha::Real
+end
+
+function CommonAlgOptions(;
+        populationsize::Int = 100,
+        functions = (sin, exp, cos, log, +, -, /, *),
+        arities = (1, 1, 1, 1, 2, 2, 2, 2),
+        n_layers::Int = 1,
+        skip::Bool = true,
+        simplex::AbstractSimplex = Softmax(),
+        loss = aicc,
+        keep::Union{Real, Int} = 0.1,
+        use_protected::Bool = true,
+        distributed::Bool = false,
+        threaded::Bool = false,
+        rng::AbstractRNG = Random.default_rng(),
+        optimizer = LBFGS(),
+        optim_options::Optim.Options = Optim.Options(),
+        optimiser::Union{Nothing, Optimisers.AbstractRule} = nothing,
+        observed::Union{ObservedModel, Nothing} = nothing,
+        alpha::Real = 0.999f0
+    )
+    return CommonAlgOptions(
+        populationsize, functions, arities, n_layers, skip, simplex, loss, keep,
+        use_protected, distributed, threaded, rng, optimizer, optim_options,
+        optimiser, observed, alpha
+    )
 end

lib/DataDrivenLux/src/caches/candidate.jl

@@ -50,27 +50,27 @@ $(FIELDS)
 """
 @concrete struct Candidate <: StatsBase.StatisticalModel
     "Random seed"
-    rng <: AbstractRNG
+    rng
     "The current state"
-    st <: NamedTuple
+    st
     "The current parameters"
-    ps <: AbstractVector
+    ps
     "Incoming paths"
-    incoming_path <: Vector{<:AbstractPathState}
+    incoming_path
     "Outgoing path"
-    outgoing_path <: Vector{<:AbstractPathState}
+    outgoing_path
     "Statistics"
-    statistics <: PathStatistics
+    statistics
     "The observed model"
-    observed <: ObservedModel
+    observed
     "The parameter distribution"
-    parameterdist <: ParameterDistributions
+    parameterdist
     "The optimal scales"
-    scales <: AbstractVector
+    scales
     "The optimal parameters"
-    parameters <: AbstractVector
+    parameters
     "The component model"
-    model <: ComponentModel
+    model
 end
 
 function (c::Candidate)(dataset::Dataset{T}, ps = c.ps, p = c.parameters) where {T}

lib/DataDrivenLux/src/caches/dataset.jl

@@ -25,7 +25,7 @@ function Dataset(
     x_intervals = interval.(map(extrema, eachrow(X)))
     y_intervals = interval.(map(extrema, eachrow(Y)))
     u_intervals = interval.(map(extrema, eachrow(U)))
-    t_intervals = isempty(t) ? Interval{T}(zero(T), zero(T)) : interval(extrema(t))
+    t_intervals = isempty(t) ? interval(zero(T), zero(T)) : interval(extrema(t))
     return Dataset{T}(X, Y, U, t, x_intervals, y_intervals, u_intervals, t_intervals)
 end
 

lib/DataDrivenLux/src/custom_priors.jl

@@ -47,8 +47,11 @@ function ObservedDistribution(
     return ObservedDistribution{fixed, D}(errormodel, latent_scale, transform)
 end
 
-function Base.summary(io::IO, ::ObservedDistribution{fixed, D}) where {fixed, D}
-    return print(io, "$E : $D() with $(fixed ? "fixed" : "variable") scale.")
+function Base.summary(io::IO, d::ObservedDistribution{fixed, D}) where {fixed, D}
+    return print(
+        io,
+        "$(nameof(typeof(d.errormodel))) : $D() with $(fixed ? "fixed" : "variable") scale."
+    )
 end
 
 get_init(d::ObservedDistribution) = d.latent_scale

lib/DataDrivenLux/src/lux/path_state.jl

@@ -48,11 +48,13 @@ function update_path(::Nothing, id::Tuple{Int, Int}, state::PathState{T}) where
 end
 
 function update_path(
-        f::F where {F <: Function}, id::Tuple{Int, Int}, states::PathState{T}...
+        f::F where {F <: Function}, id::Tuple{Int, Int},
+        state1::PathState{T}, states::PathState{T}...
     ) where {T}
+    allstates = (state1, states...)
     return PathState{T}(
-        f(get_interval.(states)...), (f, tuplejoin(map(get_operators, states)...)...),
-        (id, tuplejoin(map(get_nodes, states)...)...)
+        f(get_interval.(allstates)...), (f, tuplejoin(map(get_operators, allstates)...)...),
+        (id, tuplejoin(map(get_nodes, allstates)...)...)
     )
 end
 

lib/DataDrivenLux/test/qa/Project.toml

@@ -1,10 +1,12 @@
 [deps]
 Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595"
+DataDrivenDiffEq = "2445eb08-9709-466a-b3fc-47e12bd697a2"
 DataDrivenLux = "47881146-99d0-492a-8425-8f2f33327637"
 JET = "c3a54625-cd67-489e-a8e7-0a5a0ff4e31b"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
 
 [sources]
+DataDrivenDiffEq = {path = "../../../.."}
 DataDrivenLux = {path = "../.."}
 
 [compat]

lib/DataDrivenLux/test/runtests.jl

@@ -10,6 +10,18 @@ const GROUP = get(ENV, "DATADRIVENDIFFEQ_TEST_GROUP", get(ENV, "GROUP", "All"))
 
 function activate_qa_env()
     Pkg.activate(joinpath(@__DIR__, "qa"))
+    # On Julia < 1.11 the qa env's [sources] table is ignored, so the in-repo
+    # DataDrivenLux/DataDrivenDiffEq would resolve as registered packages and QA
+    # would analyze stale released code. Develop the local paths to restore the
+    # 1.11+ [sources] behavior (no-op effect on >= 1.11, which honors [sources]).
+    if VERSION < v"1.11.0-DEV.0"
+        Pkg.develop(
+            [
+                Pkg.PackageSpec(path = joinpath(@__DIR__, "..", "..", "..")),
+                Pkg.PackageSpec(path = joinpath(@__DIR__, "..")),
+            ]
+        )
+    end
     return Pkg.instantiate()
 end