Revert "Retry embeddings tasks on Qdrant gRPC DEADLINE_EXCEEDED (#3520)"

[mbertrand]

This reverts a previous commit that rertried grpc errors and raise after final retry. Reverting because the embed_run_content_files task creates a chain of generate_embeddings tasks and if one fails, the entire rest of the chain is halted. Will figure out an alternative approach later.

What are the relevant tickets?

Related to https://github.com/mitodl/hq/issues/11987

Description (What does it do?)

Just reverts the commit

How can this be tested?

Tests should pass

[github-actions]

OpenAPI Changes

No changes detected

View full changelog

Unexpected changes? Ensure your branch is up-to-date with main (consider rebasing).

[Copilot]

Pull request overview

This PR reverts prior changes that retried Qdrant gRPC DEADLINE_EXCEEDED errors in the vector-search embedding/removal Celery tasks, to avoid halting long Celery chains (notably those created by embed_run_content_files) when a single task fails.

Changes:

• Remove gRPC-specific retry/raise behavior from generate_embeddings / remove_embeddings and revert to “swallow and log” error handling.
• Drop the gRPC/RetryError-focused unit tests that enforced the prior retry/raise behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
vector_search/tasks.py	Reverts task-level exception handling from gRPC-aware retry/raise back to a catch-all “log and return” approach.
vector_search/tasks_test.py	Removes tests that asserted gRPC deadline behavior and related imports/helpers.

Comments suppressed due to low confidence (1)

vector_search/tasks_test.py:37

• With the gRPC-specific tests removed, there’s no longer any coverage for the current behavior that generate_embeddings/remove_embeddings swallow unexpected exceptions and return an error string. Adding a small regression test for the swallow-and-log behavior would help ensure future changes don’t reintroduce chain-halting failures unintentionally.

from vector_search.tasks import (
    embed_learning_resources_by_id,
    embed_new_content_files,
    embed_new_learning_resources,
    embed_run_content_files,
    embeddings_healthcheck,
    remove_run_content_files,
    remove_unpublished_run_content_files,
    start_embed_resources,
)

[shanbady]

just a revert. approved