Implement closed form strategy

[Nimrais]

Feature: ClosedFormStrategy for Exact Gradient Projections

Overview

This PR introduces a new projection strategy, ClosedFormStrategy, which leverages ClosedFormExpectations.jl to compute exact, analytic gradients for KL divergence minimization. By replacing Monte Carlo sampling with symbolic-numeric integration, this strategy offers "Zero-Variance" gradients, improving both accuracy and convergence speed for supported distributions.

Key Changes

1. New Strategy

• Added ClosedFormStrategy struct in src/strategies/closed_form.jl.
• Implemented compute_gradient! logic within a new package extension: ext/ClosedFormExpectationsExt.
• This design ensures ExponentialFamilyProjection.jl maintains a weak dependency on ClosedFormExpectations.jl (only loaded when the user explicitly uses it).

2. Analytic Gradient Computation

• Implemented gradient calculation using ClosedWilliamsProduct from ClosedFormExpectations.jl.

Design Choices

• Extension Mechanism: utilized Julia’s [extensions] to isolate heavy symbolic dependencies. The core package remains lightweight.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.41%. Comparing base (3f77bab) to head (c92114f).
⚠️ Report is 20 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #86      +/-   ##
==========================================
+ Coverage   99.38%   99.41%   +0.03%     
==========================================
  Files          12       14       +2     
  Lines         486      517      +31     
==========================================
+ Hits          483      514      +31     
  Misses          3        3              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[albertpod]

Tests are failing, but LGTM overall