ci: retire cross-version test jobs (openai + langchain)

[jennsun]

Summary

Removes the langchain_cross_version_test and openai_cross_version_test jobs from .github/workflows/main.yml. Both jobs are permanently red after databricks-vectorsearch>=0.74 shipped (with the rename to databricks-ai-search and the removal of the VectorSearchIndex re-export from databricks.vector_search.client).

Targeted at unblocking the next release.

Background

The two retired jobs check out historical bridge tags (databricks-ai-v0.2.0 through v0.5.0), copy that tag's integrations/openai or integrations/langchain source over HEAD, and run a small smoke test against current PyPI. The implicit contract: "PyPI deps stay backward-compatible, so old bridge sources keep importing."

That contract broke when databricks-vectorsearch>=0.74 removed the VectorSearchIndex re-export from databricks.vector_search.client. Every historical bridge tag hard-codes:

from databricks.vector_search.client import VectorSearchIndex

That import now fails on the version of vectorsearch that the matrix install pulls. The frozen tag sources can't be patched — they're sealed in the wheel. No PR to HEAD can turn the matrix green again.

We saw this exact failure on:

• Import VectorSearchIndex from its canonical module #435 (canonical-path fix)
• Migrate from databricks-vectorsearch to databricks-ai-search #437 (ai-search migration)
• And on a no-op PR off main ([do-not-merge] empty PR off main  #436) — proving the failures are upstream PyPI drift, not anything in-flight.

What this changes

• Delete the langchain_cross_version_test job (lines 109–154 on main).
• Delete the openai_cross_version_test job (lines 201–241 on main).
• Add an inline comment in the workflow explaining why both were retired and the plan for re-introducing them.

YAML validity confirmed locally via yaml.safe_load. Top-level jobs after the edit: core_test, core_test_with_extras, langchain_test, langchain_test_with_extras, openai_test, openai_test_with_extras, llamaindex_test, mcp_test, dspy_test. None of the removed jobs were marked Required.

Plan going forward

Per Ann: "we can restart them from the version we are deploying." Once a release containing the ai-search migration (#437 merged as d3a4f93) ships, we can re-introduce a cross-version matrix using that release as the new stable baseline.

The bridge's own unit tests + integration tests still cover regression risk for HEAD. This removes a non-Required CI signal that has become noise rather than information.

Test plan

• CI on this branch turns green (no cross-version jobs to fail).
• Verify no other workflow references cross_version_test (grep clean).
• After merge: subsequent PRs no longer surface the three openai_test (v0.x) / four langchain_test (v0.x) failures.