ci(e2e-http): split into 2 PR-triggered workflows (lite + full) sharing reusable shape workflow (PR-2 of 2) (#1744)

* ci(e2e-http): split into 2 PR-triggered workflows (lite + full) sharing reusable shape workflow

Follow-up to #1740 / #1743. Replaces #1740's intra-job matrix on
``e2e-http-smoke`` with two separate top-level workflows, each
PR-triggered and each pinned to one deployment shape. The matrix in
#1740 ran identical lanes through the smoke layer; the only difference
between lanes was which lazy-init backend the api picked, and the smoke
hurl files all set ``enable_*: false`` so neither lane actually
exercised pgvector / PG-graph / qdrant / neo4j data paths.

This PR moves the shape variation up one layer so each PR runs *both*
the smoke layer *and* the provider-aware layer under each shape — the
provider layer is where real ingest (embedding + KG extraction) and
recall happen, so under this structure the lite job actually exercises
the pgvector + PG-graph data path and the full job actually exercises
qdrant + neo4j. LLM token cost per PR doubles; this was confirmed
acceptable.

Files:
- ``e2e-http-shape.yml`` (NEW, reusable) — takes ``shape`` input, runs
  smoke + provider-preflight + provider-aware suite for that shape.
  ``SHAPE`` env var threads through to ``up.sh``, which sources the
  matching ``tests/e2e_http/shapes/<shape>.env`` (introduced in #1743).
  Artifact upload names include the shape so multi-shape runs do not
  clash.
- ``e2e-http-lite.yml`` (NEW, top-level) — PR-triggered + manual; calls
  ``e2e-http-shape.yml`` with ``shape: lite`` (pgvector + PG-graph).
- ``e2e-http-full.yml`` (NEW, top-level) — PR-triggered + manual; calls
  ``e2e-http-shape.yml`` with ``shape: full-neo4j`` (Qdrant + Neo4j).
- ``e2e-http-smoke.yml`` (DELETED) — superseded.
- ``cicd-pull-request.yml`` (DELETED) — superseded; the lite/full
  workflows now PR-trigger directly.

Adding a new shape (e.g. nebula) is one new ``e2e-http-<name>.yml``
caller file plus the matching ``shapes/<name>.env`` from #1743.

Doc references to the deleted workflow file names (e.g.
``docs/modularization/hurl-coverage-matrix.md``) are deferred to a
follow-up sweep — the ``e2e-http-smoke`` *job* name still exists inside
``e2e-http-shape.yml``, so most of those references remain
semantically correct.

* ci/test: rename shapes to <vector>-<graph>; add qdrant-nebula workflow

"full" was misleading because Qdrant+Neo4j and Qdrant+Nebula are equally
"full" deployments. Switch to ``<vector>-<graph>`` naming with ``lite``
kept as the special name for the single-PG (pgvector + PG-graph)
ApeRAG-Lite deployment:

- ``full-neo4j``  → ``qdrant-neo4j``
- ``full-nebula`` → ``qdrant-nebula``
- ``lite``        unchanged

The new naming scales: a future hybrid like ``pgvector-neo4j`` slots
in cleanly without needing a "full / fuller / fullest" hierarchy.

Also wires Nebula into PR-triggered CI per request — every PR now runs
all three shapes through both the smoke and the provider-aware layer.
LLM token cost per PR triples; this is acceptable for now and a fake
provider is on the roadmap to drop the cost back down.

Files:
- ``tests/e2e_http/shapes/qdrant-{neo4j,nebula}.env`` (renamed; header
  comments updated)
- ``.github/workflows/e2e-http-qdrant-neo4j.yml`` (renamed from
  ``e2e-http-full.yml``; display name "E2E HTTP Qdrant + Neo4j";
  ``shape: qdrant-neo4j``)
- ``.github/workflows/e2e-http-qdrant-nebula.yml`` (NEW; display name
  "E2E HTTP Qdrant + Nebula"; ``shape: qdrant-nebula``)
- ``Makefile`` shortcuts: ``test-http-smoke-compose-qdrant-neo4j``
  / ``-qdrant-nebula`` (old ``-full`` target dropped — only existed
  in #1743 / origin/main, not yet referenced anywhere external)
- ``tests/e2e_http/runners/compose/README.md`` reflects new names

* test(unit): update e2e-http workflow contract test for new shape structure

Author: earayu

.github/workflows/cicd-pull-request.yml

@@ -0,0 +1,20 @@
+name: E2E HTTP Lite
+
+# ApeRAG-Lite deployment shape: pgvector + PG-graph (single-PG).
+# Runs the smoke + provider-aware HTTP suite against this shape.
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  e2e-http-compose:
+    if: ${{ !github.event.pull_request.draft }}
+    uses: ./.github/workflows/e2e-http-shape.yml
+    secrets: inherit
+    with:
+      shape: lite

.github/workflows/e2e-http-lite.yml

@@ -0,0 +1,20 @@
+name: E2E HTTP Qdrant + Nebula
+
+# Distributed deployment shape: Qdrant (vector) + Nebula (graph).
+# Runs the smoke + provider-aware HTTP suite against this shape.
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  e2e-http-compose:
+    if: ${{ !github.event.pull_request.draft }}
+    uses: ./.github/workflows/e2e-http-shape.yml
+    secrets: inherit
+    with:
+      shape: qdrant-nebula

.github/workflows/e2e-http-qdrant-nebula.yml

@@ -0,0 +1,20 @@
+name: E2E HTTP Qdrant + Neo4j
+
+# Distributed deployment shape: Qdrant (vector) + Neo4j (graph).
+# Runs the smoke + provider-aware HTTP suite against this shape.
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  e2e-http-compose:
+    if: ${{ !github.event.pull_request.draft }}
+    uses: ./.github/workflows/e2e-http-shape.yml
+    secrets: inherit
+    with:
+      shape: qdrant-neo4j

.github/workflows/e2e-http-qdrant-neo4j.yml

@@ -1,38 +1,33 @@
-name: E2E HTTP
+name: E2E HTTP (shape)
+
+# Reusable workflow that runs the HTTP E2E suite (smoke + provider-aware)
+# against a single deployment SHAPE. Top-level callers (e2e-http-lite.yml,
+# e2e-http-full.yml) bind the shape so each shape produces its own status
+# check, and adding a new shape is a one-line caller file.
 
 on:
   workflow_call:
-  # ``workflow_dispatch`` lets a diagnostic task fire the full suite against
-  # the current ref without opening a dummy PR. See task #11: after Phase A
-  # (log preservation) merges, Phase B (causation verdict) may need to
-  # replay the provider-aware suite on demand rather than waiting for a
-  # natural PR failure.
+    inputs:
+      shape:
+        description: "Deployment shape name (must match tests/e2e_http/shapes/<shape>.env)"
+        required: true
+        type: string
   workflow_dispatch:
+    inputs:
+      shape:
+        description: "Deployment shape name (must match tests/e2e_http/shapes/<shape>.env)"
+        required: true
+        type: string
+        default: lite
 
 jobs:
   e2e-http-smoke:
     runs-on: ubuntu-latest
     timeout-minutes: 45
     permissions:
       contents: read
-    strategy:
-      fail-fast: false
-      # Two deployment-shape combos. Connector-level "swap one DB" coverage
-      # already lives in compat-test.yml's matrix (Qdrant/pgvector,
-      # PG/Neo4j/Nebula); this matrix exists to prove the full HTTP stack
-      # boots and runs end-to-end under each shape.
-      matrix:
-        combo:
-          - name: lite
-            vector: pgvector
-            graph: postgresql
-          - name: full
-            vector: qdrant
-            graph: neo4j
-    name: e2e-http-smoke (${{ matrix.combo.name }})
     env:
-      VECTOR_DB_TYPE: ${{ matrix.combo.vector }}
-      GRAPH_DB_TYPE: ${{ matrix.combo.graph }}
+      SHAPE: ${{ inputs.shape }}
     steps:
       - uses: actions/checkout@v4
 
@@ -71,7 +66,7 @@ jobs:
         if: failure()
         uses: actions/upload-artifact@v4
         with:
-          name: e2e-http-bootstrap-artifacts-${{ matrix.combo.name }}
+          name: e2e-http-bootstrap-artifacts-${{ inputs.shape }}-smoke
           path: tests/e2e_http/bootstrap/.generated
           if-no-files-found: ignore
           # The bootstrap directory name starts with "." — actions/upload-artifact@v4
@@ -122,6 +117,7 @@ jobs:
     permissions:
       contents: read
     env:
+      SHAPE: ${{ inputs.shape }}
       E2E_ALIBABACLOUD_API_KEY: ${{ secrets.E2E_ALIBABACLOUD_API_KEY }}
       E2E_OPENROUTER_API_KEY: ${{ secrets.E2E_OPENROUTER_API_KEY }}
     steps:
@@ -162,7 +158,7 @@ jobs:
         if: failure()
         uses: actions/upload-artifact@v4
         with:
-          name: e2e-http-provider-bootstrap-artifacts
+          name: e2e-http-bootstrap-artifacts-${{ inputs.shape }}-provider
           path: tests/e2e_http/bootstrap/.generated
           if-no-files-found: ignore
           include-hidden-files: true
@@ -176,7 +172,7 @@ jobs:
         if: failure()
         uses: actions/upload-artifact@v4
         with:
-          name: e2e-http-provider-diagnostic-artifacts
+          name: e2e-http-provider-diagnostic-artifacts-${{ inputs.shape }}
           path: tests/e2e_http/.diagnostic-artifacts
           if-no-files-found: ignore
           # ``.diagnostic-artifacts`` starts with a dot (hidden). v4 default skips
@@ -187,8 +183,8 @@ jobs:
       - name: Dump Compose diagnostics on failure
         if: failure()
         run: |
-          docker compose -f docker-compose.yml ps || true
-          docker compose -f docker-compose.yml logs --no-color api postgres redis qdrant es || true
+          docker compose --profile neo4j --profile nebula -f docker-compose.yml ps || true
+          docker compose --profile neo4j --profile nebula -f docker-compose.yml logs --no-color || true
 
       - name: Stop Compose stack
         if: always()

.github/workflows/e2e-http-smoke.yml .github/workflows/e2e-http-shape.yml.github/workflows/e2e-http-smoke.yml renamed to .github/workflows/e2e-http-shape.yml

@@ -51,7 +51,7 @@ help:
 	@printf "  make test-http-full       Run full HTTP suite against an existing target\n"
 	@printf "  make test-http-up-compose / test-http-down-compose\n"
 	@printf "  make test-http-smoke-compose / test-http-full-compose\n"
-	@printf "  make test-http-smoke-compose-lite / test-http-smoke-compose-full   (DB-combo shortcuts)\n"
+	@printf "  make test-http-smoke-compose-{lite,qdrant-neo4j,qdrant-nebula}   (per-shape shortcuts)\n"
 	@printf "  make test-http-up-k8s / test-http-down-k8s\n"
 	@printf "  make test-http-smoke-k8s / test-http-full-k8s\n\n"
 	@printf "Build / API\n"
@@ -205,7 +205,7 @@ add-license:
 .PHONY: test-all test-unit test-integration test-e2e test-e2e-perf \
 	test-http-bootstrap test-http-smoke test-http-full \
 	test-http-up-compose test-http-down-compose test-http-smoke-compose test-http-full-compose \
-	test-http-smoke-compose-lite test-http-smoke-compose-full \
+	test-http-smoke-compose-lite test-http-smoke-compose-qdrant-neo4j test-http-smoke-compose-qdrant-nebula \
 	test-http-up-k8s test-http-down-k8s test-http-smoke-k8s test-http-full-k8s
 test-all: test-unit test-integration test-e2e
 
@@ -286,8 +286,11 @@ test-http-full-compose:
 test-http-smoke-compose-lite:
 	@SHAPE=lite ./tests/e2e_http/scripts/run_compose_smoke.sh
 
-test-http-smoke-compose-full:
-	@SHAPE=full-neo4j ./tests/e2e_http/scripts/run_compose_smoke.sh
+test-http-smoke-compose-qdrant-neo4j:
+	@SHAPE=qdrant-neo4j ./tests/e2e_http/scripts/run_compose_smoke.sh
+
+test-http-smoke-compose-qdrant-nebula:
+	@SHAPE=qdrant-nebula ./tests/e2e_http/scripts/run_compose_smoke.sh
 
 test-http-up-k8s:
 	@./tests/e2e_http/runners/k8s/up.sh

Makefile

@@ -20,8 +20,9 @@ This runner is intentionally replaceable. The suite must remain runnable against
 ApeRAG supports several real production deployment forms — different choices of vector backend (Qdrant or pgvector) and graph backend (PostgreSQL, Neo4j, or Nebula). Each named combination is a *shape*.
 
 ```
-SHAPE=lite       ./tests/e2e_http/runners/compose/up.sh
-SHAPE=full-neo4j ./tests/e2e_http/runners/compose/up.sh
+SHAPE=lite          ./tests/e2e_http/runners/compose/up.sh
+SHAPE=qdrant-neo4j  ./tests/e2e_http/runners/compose/up.sh
+SHAPE=qdrant-nebula ./tests/e2e_http/runners/compose/up.sh
 ```
 
 Each shape file under `tests/e2e_http/shapes/<shape>.env` declares:
@@ -37,9 +38,11 @@ Adding a new shape = adding one new file. The runner / Makefile / CI all referen
 
 ### Available shapes (today)
 
-- **lite** — single-PG ApeRAG-Lite: pgvector + PG-graph, no qdrant/neo4j containers
-- **full-neo4j** — distributed: Qdrant + Neo4j
-- **full-nebula** — distributed: Qdrant + Nebula
+Shapes are named `<vector>-<graph>`, with **lite** as the special name for the single-PG (pgvector + PG-graph) ApeRAG-Lite deployment:
+
+- **lite** — single-PG ApeRAG-Lite: pgvector + PG-graph, no qdrant/neo4j/nebula containers
+- **qdrant-neo4j** — distributed: Qdrant + Neo4j
+- **qdrant-nebula** — distributed: Qdrant + Nebula
 
 ### Backward compatibility
 

tests/e2e_http/runners/compose/README.md

@@ -0,0 +1,8 @@
+# SHAPE: qdrant-nebula — distributed deployment with Qdrant + Nebula.
+#
+# Same as qdrant-neo4j but with Nebula as the graph backend (activated
+# via the "nebula" compose profile).
+SHAPE_VECTOR_DB_TYPE=qdrant
+SHAPE_GRAPH_DB_TYPE=nebula
+SHAPE_COMPOSE_SERVICES="postgres redis es qdrant api"
+SHAPE_COMPOSE_PROFILES="--profile nebula"

tests/e2e_http/shapes/full-nebula.env

@@ -1,4 +1,4 @@
-# SHAPE: full-neo4j — distributed deployment with Qdrant + Neo4j.
+# SHAPE: qdrant-neo4j — distributed deployment with Qdrant + Neo4j.
 #
 # Postgres holds relational data only. Vector data lives in Qdrant.
 # Graph data lives in Neo4j (activated via the "neo4j" compose profile).