RAA-7897: Replace S3 routing with EPC API Gateway lookup#324
Conversation
Replaces the static S3 targets.json lookup with a dynamic call to the EPC API Gateway. The proxy now queries GET /Endpoint?HealthcareService.identifier using the decoded NHSD-Target-Identifier header, extracts the address field from the first active Endpoint in the returned FHIR Bundle, and routes to it via mTLS. Adds distinct OperationOutcome error responses for EPC 404 (no endpoint found), 5xx/timeout (unavailable), and malformed response scenarios.
geberele
force-pushed
the
raa-7897-epc-routing
branch
from
3f17f0c to
0a9d552
Compare
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
pyyaml 5.4.1 was released before Python 3.10 existed and has no cp310 binary wheels. Building from source fails on modern setuptools because the old setup.py references build_ext.cython_sources which no longer exists. Upgrade to 6.0.2 which has cp310 wheels and proper PEP 517 support. awscli 1.25.24 constrained PyYAML < 5.5, blocking the pyyaml upgrade. Upgrade awscli to 1.32.0 (PyYAML < 6.1) and its pinned botocore to 1.34.0 and s3transfer to 0.9.0. All locked transitive deps (urllib3 1.26.9, colorama 0.4.4, rsa 4.7.2, docutils 0.16) still satisfy the new version constraints.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
The wheel URL for api-test-utils is on a private GitHub repo that the CI runner cannot access, causing poetry install to fail. The package is only needed for running Apigee e2e tests (tests/conftest.py, tests/api_test.py), not for the CI build or version calculation. Moving it to an optional group means `poetry install` skips it by default. To run the e2e tests locally: `poetry install --with e2e`.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
DecodeBase64String.py references the Apigee-injected 'flow' global, which pyflakes reports as F821 (undefined name). The .flake8 exclude entry is bypassed because the Makefile passes files explicitly via find|xargs. Mirror the existing sandbox exclusion in the find command.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
Poetry 2.x validates the PEP 621 [project] table and requires 'name'. The leftover stub contained only an invalid 'python' key, so 'poetry run' (used by make lint) failed with 'project must contain [name] properties'. All real metadata already lives in [tool.poetry], so drop the stub.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
…hash)
The committed lock was lock-version 2.0 with no per-package group info, so Poetry 2.x's 'poetry install' could not resolve the dev group and skipped flake8 ('Command not found: flake8' in make lint). Regenerated to lock-version 2.1 with explicit groups; no dependency versions changed.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
The shared apigee-build template defaults python_version to 3.13. The pinned deps (aiohttp 3.8.1 -> frozenlist 1.3.0, etc.) use CPython internals removed in 3.12+ and fail to compile, so 'make install' (poetry install) failed and flake8 was never installed, breaking 'make lint'. Pin to 3.10 to match the intended runtime and the pinned dependency set.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
The deploy post-deploy step ran 'make install-python' (poetry install) without a Python version pin, so it picked up the agent default (3.13), against which the pinned C-extension deps (frozenlist, aiohttp, typed-ast) fail to compile. Pin 3.10 to match the build pipeline.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
SC.CallEpc used a fully-variable <URL>{private.epcBaseUrl}</URL>, which Apigee rejects at import with ProtocolMissingInURL. Hardcode the https:// scheme (the callout is mTLS) and derive private.epcHost in BuildEpcPath by stripping any scheme the KVM value may carry, so the target resolves correctly regardless of how epc_base_url is stored.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
Apigee's Rhino engine rejects `return` at script top level (javascript.exec.CompilationError), failing deployment. Wrap the body in an IIFE so the early-exit returns are valid; behaviour is unchanged since the script communicates only via context variables.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
The dist artifact copies pyproject.toml (which declares readme = 'README.md') but not README.md, so 'poetry install' in the post-deploy step fails resolving the root project's readme path. Add README.md to _dist_include so the packaged artifact is self-consistent.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
The project is not a Python package (no package directory, no packages config), so 'poetry install' fails trying to install the root project. The post-deploy step only needs dependencies to run pytest, so install deps without the root package, as Poetry itself recommends.
|
This branch is work on a ticket in the NHS Digital APM JIRA Project. Here's a handy link to the ticket: RAA-7897 |
1 participant
Jira
RAA-7897
Summary
targets.jsonrouting pipeline (KVM credentials fetch → AWS Sig V4 signing → S3 ServiceCallout) from the BaRS proxy target endpointGET /Endpoint?HealthcareService.identifier={system}|{value}using the decodedNHSD-Target-Identifierheaderbars-client/certkeystore) — auth method TBD, see open decisions belowaddressfield from the first activeEndpointin the returned FHIR Bundle and uses it to route the request404ProxyNotFound), 5xx/timeout (new503EpcUnavailable), and malformed response (new500EpcInvalidResponse)Python.DecodeBase64,Javascript.CheckTarget) now run before the EPC call to fail fast on bad identifier formatFiles changed
Deleted (S3 pipeline):
KVM.GetS3Credentials,AM-S3Request,JC-AWSSignV4,SC-CallS3,AssignMessage.Targets,Javascript.UnpackVars,Javascript.setTargetUrl,UnpackVars.js,SetTargetUrl.jsCreated (EPC pipeline):
KVM.GetEpcConfig,Javascript.BuildEpcPath+BuildEpcPath.js,AM.PrepareEpcRequest,SC.CallEpc,Javascript.ParseEpcResponse+ParseEpcResponse.js,RaiseFault.503EpcUnavailable,RaiseFault.500EpcInvalidResponseModified:
bref-target.xml,CheckTargets.js,BarsLibrary.js,DecodeBase64String.py,HandleErrors.js,ErrorRepository.jsOpen decisions (pending platform team confirmation)
Pre-deploy prerequisites
A new Apigee KVM
booking-and-referral-epc-configmust be created with:epc_base_url— EPC API Gateway base URLepc_organisation— base64-encoded FHIR Organization JSON for the proxy service identityepc_api_key— (if API-Key auth is chosen) API key for EPC GatewayAdditional pending confirmations:
bars-client/certis reused or a separate EPC keystore is neededepc_organisationservice identityTest plan
pytest tests/suite — pre-flight and auth flow tests should pass unchangedtests/test_target_identifier.py: EPC 200 active endpoint → correct routing, EPC 404 → 404 OperationOutcome, EPC 503 → 503 OperationOutcome, EPC malformed JSON → 500 OperationOutcomeGET /Slotwith validNHSD-Target-Identifierand confirm request reaches expected backendSC.CallEpc.xml/AM.PrepareEpcRequest.xmlaccordingly