Skip to content

LCORE-2802: Bake embedding model into image and make HF_HUB_OFFLINE o… #2008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alessandralanz wants to merge 3 commits into from
Closed

LCORE-2802: Bake embedding model into image and make HF_HUB_OFFLINE o… #2008

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
acac105
LCORE-2802: Bake embedding model into image and make HF_HUB_OFFLINE o…
alessandralanz Jun 26, 2026
4484530
LCORE-2802: Fix HF_HOME to be build-time only, not global ENV
alessandralanz Jun 26, 2026
9a9de52
LCORE-2802: Remove HF_HOME pass-through that breaks default HF cache …
alessandralanz Jun 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions deploy/lightspeed-stack/Containerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,13 @@ ENV PATH="/app-root/.venv/bin:$PATH"
# We place them at /app-root/providers.d. YAMLs there reference lightspeed_stack_providers.*, so that package must be on PYTHONPATH.
ENV PYTHONPATH="/app-root"

# Pre-download embedding model for OKP/Solr vector search (~61MB, baked into image).
# Uses a dedicated path so the docker-compose HF cache volume mount does not shadow it.
ENV HF_HOME=/app-root/.hf-models
RUN mkdir -p /app-root/.hf-models && \
python3.12 -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('ibm-granite/granite-embedding-30m-english')" && \

@coderabbitai coderabbitai Bot Jun 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Bake the same model ID the runtime defaults to.

Line 137 downloads ibm-granite/granite-embedding-30m-english, but src/constants.py:235-238 sets the default Solr embedding model to sentence-transformers/ibm-granite/granite-embedding-30m-english. With HF_HUB_OFFLINE=1 as the default path, OKP/Solr can still miss the baked artifact and fail when it resolves the configured default model ID. Use one canonical identifier in both places.

Suggested fix 
 ENV HF_HOME=/app-root/.hf-models
 RUN mkdir -p /app-root/.hf-models && \
-    python3.12 -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('ibm-granite/granite-embedding-30m-english')" && \
+    python3.12 -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('sentence-transformers/ibm-granite/granite-embedding-30m-english')" && \
     chown -R 1001:1001 /app-root/.hf-models
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
ENV HF_HOME=/app-root/.hf-models
RUN mkdir -p /app-root/.hf-models && \
python3.12 -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('ibm-granite/granite-embedding-30m-english')" && \
ENV HF_HOME=/app-root/.hf-models
RUN mkdir -p /app-root/.hf-models && \
python3.12 -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('sentence-transformers/ibm-granite/granite-embedding-30m-english')" && \
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/lightspeed-stack/Containerfile` around lines 135 - 137, The baked
Hugging Face model ID in the Containerfile does not match the runtime default
used by the Solr embedding configuration. Update the model download step to use
the same canonical identifier as the default in constants so the artifact baked
into the image matches what runtime resolves when HF_HUB_OFFLINE=1 is set. Keep
the identifiers aligned between the Containerfile model fetch and the default
embedding model constant.

chown -R 1001:1001 /app-root/.hf-models

# Run the application
EXPOSE 8080
ENTRYPOINT ["python3.12", "src/lightspeed_stack.py"]
Expand Down
4 changes: 3 additions & 1 deletion docker-compose-library.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,10 @@ services:
- LLAMA_STACK_LOGGING=${LLAMA_STACK_LOGGING:-}
# FAISS test and inline RAG config
- FAISS_VECTOR_STORE_ID=${FAISS_VECTOR_STORE_ID:-}
# Pass env var from shell into container to access OKP
- RH_SERVER_OKP=${RH_SERVER_OKP:-}
# Prevent HuggingFace Hub update checks (HTTP 429 rate-limiting in CI from parallel jobs).
- HF_HUB_OFFLINE=1
- HF_HUB_OFFLINE=${HF_HUB_OFFLINE:-1}
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/liveness"]
interval: 10s # how often to run the check
Expand Down
Loading