Add generic /api/v1/webhooks/learning_resources/ endpoint (Cohort 2 webhook delivery)

[blarghmatey]

What are the relevant tickets?

Part of the MIT Learn ETL → OL Data Platform migration, Cohort 2 (REST API sources via dlt + webhook): https://github.com/mitodl/hq/issues/11511

Description (What does it do?)

Adds the single generic webhook endpoint that the OL Data Platform pushes pre-computed canonical LearningResource payloads to. This is the MIT Learn receiving half of the "webhook push" delivery pattern; the platform side is ol-data-platform PR mitodl/ol-data-platform#2277 (MITLearnApiClient.notify_learning_resources()).

• POST /api/v1/webhooks/learning_resources/ — LearningResourceWebhookView(BaseWebhookView). HMAC signature validation (X-MITLearn-Signature, shared WEBHOOK_SECRET) is inherited from BaseWebhookView.
• Body: {"resources": [ <canonical LearningResource dict>, ... ]}. Each resource must carry at minimum readable_id, etl_source, resource_type; all other keys pass through untouched to the loaders.
• Routing: process_learning_resources_webhook() groups resources by (etl_source, resource_type) and dispatches each group to the existing loaders — course → load_courses(etl_source, items), program → load_programs(etl_source, items), video → load_videos(items), podcast → load_podcasts(items).
• OpenSearch: no separate reindex — the batch loaders call the per-item loaders which call update_index(), so indexing happens automatically (same path as the existing Celery ETL). clear_views_cache() is called after processing, mirroring the OVS webhook view.
• Resilience: a resource_type with no loader (e.g. mit_climate currently sends "article", which has no LearningResourceType member) is logged and skipped rather than 500-ing, so one unsupported source can't reject an otherwise-valid mixed batch.

All 4 Cohort 2 sources (MIT Climate, MIT PE, OLL, edX programs) plus YouTube and Podcasts (Cohort 3) will deliver through this one endpoint.

Files: webhooks/views.py, webhooks/serializers.py, webhooks/urls.py, webhooks/learning_resources_webhook_test.py.

How can this be tested?

Automated (added in this PR): webhooks/learning_resources_webhook_test.py covers course routing, grouping by (etl_source, resource_type), video/podcast routing, unsupported-type skip, missing-required-field → 400, invalid-signature rejection, and malformed-JSON → 400.

Run locally (Postgres + Redis via docker-compose.services.yml, env per .github/workflows/ci.yml Tests step):

```
uv run pytest webhooks/ --no-cov -q
```

Result on this branch: full webhooks/ suite 38 passed; pre-commit (ruff, ruff-format, detect-secrets, DRF Serializer ORM Check) all pass.

Manual: POST a signed {"resources":[...]} batch and confirm resources are grouped and passed to the matching loader.

Additional Context

Draft — one open product decision before this should merge: mit_climate delivers resource_type: "article", which has no MIT Learn LearningResourceType / loader. This handler currently logs-and-skips unsupported types, so Climate articles would not ingest until "article" is mapped to a real resource type (or a loader is added). Flagging for platform/product input.

Depends on the delivery contract in mitodl/ol-data-platform#2277 (notify_learning_resources() → POST /api/v1/webhooks/learning_resources/, HMAC-signed {"resources":[...]}).

[github-actions]

OpenAPI Changes

25 changes: 0 error, 9 warning, 16 info

View full changelog

Unexpected changes? Ensure your branch is up-to-date with main (consider rebasing).

[blarghmatey]

Update — the mit_climate "article" mapping question is resolved.

MIT Learn has no article resource_type; the legacy Celery ETL loads MIT Climate articles via load_documents as LearningResourceType.document with resource_category "Article" (learning_resources/etl/pipelines.py). So:

• This PR now routes document → load_documents(etl_source, items) (commit 76b8509ae) + test.
• Platform side: ol-data-platform PR chore(deps): update redis docker tag to v7.4.4 #2277 updated to emit resource_type: "document" + resource_category: "Article" instead of "article" (commit a37cbbbe).

Net: MIT Climate articles ingest through the same load_documents path as today — no new type/migration. The skip-unsupported-type behavior remains for genuinely unknown types.