feat: Python 3.14 support — langchain PEP 649 compatibility

[cluster2600]

Summary

Add comprehensive Python 3.14 support to NeMo Guardrails, addressing PEP 649 (deferred annotation evaluation), asyncio lifecycle changes, native dependency compatibility, and experimental free-threaded (no-GIL) support.

Key changes

• PEP 649 langchain compatibility shim (_langchain_compat.py) — patches builtins.dict and builtins.list into affected langchain/langchain_core modules before pydantic triggers annotation evaluation
• PEP 649 OpenTelemetry safety (_otel_compat.py) — catches TypeError alongside ImportError on OTel imports so tracing degrades gracefully
• asyncio lifecycle fixes — replace deprecated get_event_loop() with get_or_create_event_loop() / get_running_loop() across 4 modules
• Native dependency skip guards — graceful fallback when annoy or yara-python lack 3.14 wheels (77% of initial cascading failures)
• CI integration — reusable _test-py314.yml workflow, Dockerfile.py314 multi-stage build (test + bench targets)
• Performance benchmark harness — latency, throughput, streaming, import, and free-threaded benchmarks with CI regression gate
• Migration guide — 425-line docs/python-3.14-migration.md covering all changes, Docker usage, and troubleshooting

Test results (Python 3.14.3)

Metric	Before	After	Delta
Passed	2,105	2,583	+478
Failed	200	0	-200 (100%)
Skipped	0	172	+172 (proper skip guards)

Remaining skipped tests:

• 10 Pangea AI Guard + 6 Trend AI Guard: upstream httpcore weak-ref bug on Python 3.14
• 1 passthrough mode: upstream asyncio.Timeout strictness in IsolatedAsyncioTestCase teardown
• 155 tests using optional native deps without 3.14 wheels (annoy, yara, pytorch, etc.)

Commits (15)

1. 53917c3 feat: add Python 3.14 compatibility shim for langchain PEP 649
2. 003638e fix: handle closed event loops on Python 3.14
3. dae295e bench: add threading benchmark for GIL vs free-threaded comparison
4. 4ef48cb perf: add benchmark harness and CI regression gate
5. c3bca1c fix(tracing): add PEP 649 safety for OpenTelemetry imports
6. 868fd0e fix(compat): harden langchain shim with list patching and langchain_core
7. fb21866 fix(asyncio): replace deprecated get_event_loop() calls
8. c81848d fix(deps): add skip guards for annoy and yara
9. 8ee7159 ci: add Python 3.14 CI workflow and multi-stage Dockerfile
10. a7e48f6 docs: add migration guide, compat tests, and triage report
11. 75b5a4b fix(docker): add pytest-timeout to test stage
12. 570856d fix(docker): add fastembed, fix package metadata
13. dd7bd69 fix(tests): add importorskip guards for pytest-httpx
14. 164203c docs: add rich dependency note and Pydantic V1 warning
15. b3a3ebe fix(tests): skip tests hitting upstream Python 3.14 bugs

Related

• Issue: Python 3.14 support: langchain Chain.dict() shadows builtin under PEP 649 #1720 (Python 3.14 compatibility tracking)
• langchain upstream fix: fix: support python 3.14 in various projects langchain-ai/langchain#33575
• PEP 649: https://peps.python.org/pep-0649/

Test plan

• Docker smoke test — all 8 latency scenarios pass on Python 3.14.3
• Full pytest suite — 2,583 passed, 0 failed, 172 skipped
• Unit tests for _langchain_compat.py — 16 tests covering patching, idempotency, edge cases
• Benchmark harness validation — latency, import, threading benchmarks run cleanly
• Nightly CI with free-threaded Python 3.14t (pending 3.14t availability in GitHub Actions)

[cluster2600]

Closing in favour of #1718, which correctly scopes the changes to Python 3.13 support. Python 3.14 is blocked upstream by langchain 0.3.x (PEP 649 / pydantic type evaluation failurs).