Skip to content

Commit 7a3ecb9

Browse files
authored
Merge pull request #1476 from PolicyEngine/rollback/v0.13.13-pin-core-urllib
Roll back API to 0.13.13 and pin core/urllib3
2 parents d296e67 + d11fa23 commit 7a3ecb9

28 files changed

Lines changed: 76 additions & 1004 deletions

.env-example

Lines changed: 1 addition & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,2 @@
11
FLASK_DEBUG=1
2-
CACHE_REDIS_HOST=redis
3-
4-
# Optional: wipe the local sqlite analytics DB on startup. Only
5-
# consulted when FLASK_DEBUG=1 and analytics is enabled. Default off
6-
# so captured debug data is not lost across restarts.
7-
# RESET_ANALYTICS=1
8-
9-
# Optional: comma-separated list of origins (or regex patterns) allowed
10-
# by CORS. If unset, the default allowlist is:
11-
# - https://policyengine.org
12-
# - https://*.policyengine.org (anchored regex)
13-
# - http://localhost[:port] (any port)
14-
# - http://127.0.0.1[:port]
15-
# Example override:
16-
# CORS_ALLOWED_ORIGINS=https://foo.example.com,https://bar.example.com
2+
CACHE_REDIS_HOST=redis

CHANGELOG.md

Lines changed: 0 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -5,28 +5,6 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8-
## [0.13.15] - 2026-04-22 22:45:54
9-
10-
### Changed
11-
12-
- Update PolicyEngine US to 1.663.0
13-
14-
## [0.13.14] - 2026-04-17 16:45:19
15-
16-
### Fixed
17-
18-
- Flatten every (entity, variable, period) triple in flatten_variables_from_household (#1462).
19-
- Tighten /calculate_demo rate limit from 1/second to 1/10 seconds (#1463).
20-
- Stop unconditionally wiping the analytics SQLite DB and fix the sqlite:// URI (#1464).
21-
- Restrict CORS to PolicyEngine origins by default, anchored so attacker subdomains can't bypass (#1465).
22-
- Replace invalid ConnectionError(description=...) with a GCPError class (#1466).
23-
- Keep "0"/"1" env-var values as integers instead of collapsing to False/True (#1467).
24-
- Verify JWT signatures in the analytics decorator and drop datetime.utcnow (#1468).
25-
- Re-raise tracer failures in PolicyEngineCountry.calculate so the endpoint can return a real 500 (#1469).
26-
- Validate /calculate payloads and cap axes scans; add per-endpoint rate limit (#1470).
27-
- Time-bound and lazy-load the Auth0 JWKS fetch so a startup outage doesn't crash the API, caching only successes so the lazy retry actually retries (#1471).
28-
- Replace deprecated dpath.util.search with dpath.search (#1472).
29-
308
## [0.13.13] - 2026-04-12 01:07:14
319

3210
### Added
@@ -1701,8 +1679,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
17011679

17021680

17031681

1704-
[0.13.15]: https://github.com/PolicyEngine/policyengine-household-api/compare/0.13.14...0.13.15
1705-
[0.13.14]: https://github.com/PolicyEngine/policyengine-household-api/compare/0.13.13...0.13.14
17061682
[0.13.13]: https://github.com/PolicyEngine/policyengine-household-api/compare/0.13.12...0.13.13
17071683
[0.13.12]: https://github.com/PolicyEngine/policyengine-household-api/compare/0.13.11...0.13.12
17081684
[0.13.11]: https://github.com/PolicyEngine/policyengine-household-api/compare/0.13.10...0.13.11

changelog.yaml

Lines changed: 0 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1417,30 +1417,3 @@
14171417
added:
14181418
- Return PolicyEngine bundle metadata from household calculate responses.
14191419
date: 2026-04-12 01:07:14
1420-
- bump: patch
1421-
changes:
1422-
fixed:
1423-
- Flatten every (entity, variable, period) triple in flatten_variables_from_household
1424-
(#1462).
1425-
- Tighten /calculate_demo rate limit from 1/second to 1/10 seconds (#1463).
1426-
- Stop unconditionally wiping the analytics SQLite DB and fix the sqlite:// URI
1427-
(#1464).
1428-
- Restrict CORS to PolicyEngine origins by default, anchored so attacker subdomains
1429-
can't bypass (#1465).
1430-
- Replace invalid ConnectionError(description=...) with a GCPError class (#1466).
1431-
- Keep "0"/"1" env-var values as integers instead of collapsing to False/True
1432-
(#1467).
1433-
- Verify JWT signatures in the analytics decorator and drop datetime.utcnow (#1468).
1434-
- Re-raise tracer failures in PolicyEngineCountry.calculate so the endpoint can
1435-
return a real 500 (#1469).
1436-
- Validate /calculate payloads and cap axes scans; add per-endpoint rate limit
1437-
(#1470).
1438-
- Time-bound and lazy-load the Auth0 JWKS fetch so a startup outage doesn't crash
1439-
the API, caching only successes so the lazy retry actually retries (#1471).
1440-
- Replace deprecated dpath.util.search with dpath.search (#1472).
1441-
date: 2026-04-17 16:45:19
1442-
- bump: patch
1443-
changes:
1444-
changed:
1445-
- Update PolicyEngine US to 1.663.0
1446-
date: 2026-04-22 22:45:54

changelog_entry.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
- bump: patch
2+
changes:
3+
changed:
4+
- Restore the household API codebase to the 0.13.13 baseline
5+
- Pin policyengine_core to <=3.23.6 and urllib3 to <=1.26.20

config/README.md

Lines changed: 0 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -270,39 +270,6 @@ The following endpoints remain unprotected:
270270
- When enabled, all protected endpoints validate JWT tokens against Auth0's JWKS
271271
- The Auth0 domain and audience must match the configured values
272272

273-
## CORS Configuration
274-
275-
Browsers enforce CORS against the API. The default allowlist accepts:
276-
277-
- `https://policyengine.org`
278-
- Any `https://*.policyengine.org` host (anchored regex)
279-
- `http://localhost` on any port (dev servers)
280-
- `http://127.0.0.1` on any port
281-
282-
Override with `CORS_ALLOWED_ORIGINS` (comma-separated strings or
283-
regexes) or `cors.allowed_origins` in YAML:
284-
285-
```bash
286-
CORS_ALLOWED_ORIGINS=https://app.example.com,https://admin.example.com
287-
```
288-
289-
```yaml
290-
cors:
291-
allowed_origins:
292-
- https://app.example.com
293-
- 'https://.*\.example\.com$'
294-
```
295-
296-
Always terminate regex patterns with `$` — Flask-CORS matches with
297-
`re.match`, so an unanchored pattern like `https://.*\.example\.com`
298-
would accept `https://example.com.attacker.com`.
299-
300-
## Analytics reset (debug only)
301-
302-
`RESET_ANALYTICS=1` (or `analytics.reset: true` in YAML) wipes the
303-
local SQLite analytics DB on startup. This is **only** consulted when
304-
`FLASK_DEBUG=1`; production never resets the analytics DB.
305-
306273
## Usage Examples
307274

308275
### Production Deployment (Current)

config/default.yaml

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -46,11 +46,3 @@ ai:
4646
enabled: false
4747
anthropic:
4848
api_key: "" # Override with ANTHROPIC_API_KEY
49-
50-
# CORS configuration
51-
cors:
52-
# List of allowed origins (strings or regex patterns). If left null
53-
# the API defaults to PolicyEngine production domains. Override with
54-
# CORS_ALLOWED_ORIGINS (comma-separated) in environments that serve
55-
# additional frontends.
56-
allowed_origins: null

policyengine_household_api/api.py

Lines changed: 2 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,6 @@
2424
from policyengine_household_api.decorators.analytics import (
2525
log_analytics_if_enabled,
2626
)
27-
from policyengine_household_api.utils.config_loader import get_config_value
2827

2928
# Endpoints
3029
from .endpoints import (
@@ -41,52 +40,7 @@
4140

4241
app = application = flask.Flask(__name__)
4342

44-
# Reject absurdly large request bodies before any view runs. 10 MiB is
45-
# well above the largest legitimate household payload we have seen
46-
# (axes scans push a few hundred KiB) while still capping the memory a
47-
# single attacker can force us to allocate. Overridable via the
48-
# ``MAX_CONTENT_LENGTH`` env var (bytes).
49-
app.config["MAX_CONTENT_LENGTH"] = int(
50-
os.getenv("MAX_CONTENT_LENGTH", 10 * 1024 * 1024)
51-
)
52-
53-
54-
def _resolve_cors_origins():
55-
"""
56-
Resolve the CORS allowed origins list.
57-
58-
Priority:
59-
1. CORS_ALLOWED_ORIGINS env var (comma-separated list)
60-
2. config value "cors.allowed_origins" (list or comma string)
61-
3. Safe default: the PolicyEngine production domains
62-
63-
Use regex patterns so that wildcard subdomains work with
64-
Flask-CORS's `origins` kwarg.
65-
"""
66-
raw = os.getenv("CORS_ALLOWED_ORIGINS") or get_config_value(
67-
"cors.allowed_origins", None
68-
)
69-
70-
if raw is None:
71-
# Flask-CORS uses re.match, which is a prefix match; anchor with
72-
# ``$`` so a hostile host like ``policyengine.org.attacker.com``
73-
# cannot satisfy the wildcard pattern. Include ``localhost:*``
74-
# so local dev servers can hit the API without extra setup.
75-
origins = [
76-
"https://policyengine.org",
77-
r"https://.*\.policyengine\.org$",
78-
r"http://localhost(:[0-9]+)?$",
79-
r"http://127\.0\.0\.1(:[0-9]+)?$",
80-
]
81-
elif isinstance(raw, str):
82-
origins = [o.strip() for o in raw.split(",") if o.strip()]
83-
else:
84-
origins = list(raw)
85-
86-
return origins
87-
88-
89-
CORS(app, origins=_resolve_cors_origins())
43+
CORS(app)
9044

9145
# Use in-memory storage for rate limiting
9246
# Note that this provides limits per-instance;
@@ -105,7 +59,6 @@ def _resolve_cors_origins():
10559

10660
@app.route("/<country_id>/calculate", methods=["POST"])
10761
@require_auth_if_enabled()
108-
@limiter.limit("60 per minute")
10962
@log_analytics_if_enabled
11063
def calculate(country_id):
11164
return get_calculate(country_id)
@@ -131,11 +84,8 @@ def readiness_check():
13184
)
13285

13386

134-
# Note: `/calculate_demo` is intentionally public (documented in
135-
# config/README.md). It is guarded by a conservative rate limit rather
136-
# than JWT authentication.
13787
@app.route("/<country_id>/calculate_demo", methods=["POST"])
138-
@limiter.limit("1 per 10 seconds")
88+
@limiter.limit("1 per second")
13989
def calculate_demo(country_id):
14090
return get_calculate(country_id)
14191

Lines changed: 2 additions & 98 deletions
Original file line numberDiff line numberDiff line change
@@ -1,114 +1,18 @@
11
import json
2-
import logging
3-
import time
4-
from threading import Lock
52
from urllib.request import urlopen
63

74
from authlib.oauth2.rfc7523 import JWTBearerTokenValidator
85
from authlib.jose.rfc7517.jwk import JsonWebKey
96

10-
logger = logging.getLogger(__name__)
11-
12-
JWKS_FETCH_TIMEOUT = 10 # seconds
13-
# Minimum wait between back-to-back lazy retries after a failure.
14-
# Keeps us from hammering Auth0 when it is actively degraded.
15-
JWKS_RETRY_INTERVAL_SECONDS = 30
16-
17-
18-
# Module-level cache of successful JWKS fetches, keyed by issuer. Only
19-
# successes are cached so that a transient failure is retried on the
20-
# next authenticated request (``lru_cache`` would have memoised the
21-
# ``None`` return, making the "lazy retry" dead code).
22-
_jwks_cache: dict = {}
23-
# Records the monotonic timestamp of the most recent *failed* fetch
24-
# per-issuer so we can rate-limit retries without caching the failure
25-
# itself.
26-
_jwks_last_failure: dict = {}
27-
_jwks_lock = Lock()
28-
29-
30-
def _fetch_jwks_uncached(issuer: str):
31-
"""Fetch the JWKS for an Auth0 issuer, bypassing the cache.
32-
33-
Returns an authlib key set on success, ``None`` on failure. Errors
34-
are logged rather than raised so that a transient Auth0 outage
35-
doesn't crash the process at import time.
36-
"""
37-
jwks_url = f"{issuer}.well-known/jwks.json"
38-
try:
39-
with urlopen(jwks_url, timeout=JWKS_FETCH_TIMEOUT) as response:
40-
return JsonWebKey.import_key_set(json.loads(response.read()))
41-
except Exception as e:
42-
logger.warning(f"Failed to fetch JWKS from {jwks_url}: {e}")
43-
return None
44-
45-
46-
def _fetch_jwks(issuer: str):
47-
"""Fetch JWKS, caching only successful results.
48-
49-
On failure we record the time but do not memoise the ``None`` — a
50-
later call will retry (subject to ``JWKS_RETRY_INTERVAL_SECONDS``
51-
backoff) so that the validator self-heals once Auth0 recovers.
52-
"""
53-
with _jwks_lock:
54-
cached = _jwks_cache.get(issuer)
55-
if cached is not None:
56-
return cached
57-
last_failure = _jwks_last_failure.get(issuer)
58-
if (
59-
last_failure is not None
60-
and time.monotonic() - last_failure < JWKS_RETRY_INTERVAL_SECONDS
61-
):
62-
# Too soon after the last failure — don't hammer Auth0.
63-
return None
64-
65-
# Fetch outside the lock so a slow network call doesn't block
66-
# other threads that might be serving requests with a cached key.
67-
key_set = _fetch_jwks_uncached(issuer)
68-
69-
with _jwks_lock:
70-
if key_set is not None:
71-
_jwks_cache[issuer] = key_set
72-
_jwks_last_failure.pop(issuer, None)
73-
else:
74-
_jwks_last_failure[issuer] = time.monotonic()
75-
return key_set
76-
77-
78-
def _clear_jwks_cache():
79-
"""Test helper: wipe the success/failure caches."""
80-
with _jwks_lock:
81-
_jwks_cache.clear()
82-
_jwks_last_failure.clear()
83-
847

858
class Auth0JWTBearerTokenValidator(JWTBearerTokenValidator):
869
def __init__(self, domain, audience):
8710
issuer = f"https://{domain}/"
88-
89-
public_key = _fetch_jwks(issuer)
90-
if public_key is None:
91-
# Retry on next token validation rather than failing hard
92-
# at construction time. A missing key set means token
93-
# validation will fail cleanly inside authlib.
94-
logger.warning(
95-
"JWKS unavailable at construction; will retry on first "
96-
"token validation."
97-
)
98-
11+
jsonurl = urlopen(f"{issuer}.well-known/jwks.json")
12+
public_key = JsonWebKey.import_key_set(json.loads(jsonurl.read()))
9913
super(Auth0JWTBearerTokenValidator, self).__init__(public_key)
100-
self._issuer = issuer
10114
self.claims_options = {
10215
"exp": {"essential": True},
10316
"aud": {"essential": True, "value": audience},
10417
"iss": {"essential": True, "value": issuer},
10518
}
106-
107-
def authenticate_token(self, token_string):
108-
# Lazy-refresh the JWKS if the initial fetch failed. Because
109-
# ``_fetch_jwks`` only caches successes, this call will retry
110-
# the network fetch (subject to a short backoff) until Auth0
111-
# responds.
112-
if self.public_key is None:
113-
self.public_key = _fetch_jwks(self._issuer)
114-
return super().authenticate_token(token_string)

policyengine_household_api/constants.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
POST = "POST"
77
UPDATE = "UPDATE"
88
LIST = "LIST"
9-
VERSION = "0.13.15"
9+
VERSION = "0.13.13"
1010
COUNTRIES = ("uk", "us", "ca", "ng", "il")
1111
COUNTRY_PACKAGE_NAMES = (
1212
"policyengine_uk",

policyengine_household_api/country.py

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
import importlib
2-
import logging
32
from flask import Response
43
import json
54
from policyengine_core.taxbenefitsystems import TaxBenefitSystem
@@ -433,12 +432,8 @@ def calculate(
433432

434433
return household, None
435434

436-
except Exception:
437-
# Re-raise so endpoints/household.py (which unpacks
438-
# ``(result, computation_tree_uuid)``) can surface a real
439-
# 500 instead of a TypeError on ``None`` unpacking.
440-
logging.exception("Tracer failed while computing household")
441-
raise
435+
except Exception as e:
436+
print(f"Error computing tracer output: {e}")
442437

443438

444439
def create_policy_reform(policy_data: dict) -> dict:
@@ -483,7 +478,7 @@ def apply(self):
483478

484479

485480
def get_requested_computations(household: dict):
486-
requested_computations = dpath.search(
481+
requested_computations = dpath.util.search(
487482
household,
488483
"*/*/*/*",
489484
afilter=lambda t: t is None,

0 commit comments

Comments
 (0)