-
Notifications
You must be signed in to change notification settings - Fork 64
RHIDP-13359: Lightspeed 1.10 changes #2210
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
3fe416e
21f8240
dbb95b9
6f3cb0c
2fe8e88
4f7130c
0403249
65c37eb
6576038
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| :_mod-docs-content-type: ASSEMBLY | ||
| ifdef::context[:parent-context: {context}] | ||
|
|
||
| [id="appendix-manage-user-data-security_{context}"] | ||
| = Appendix: Manage user data security | ||
|
|
||
| :context: appendix-about-user-data-security | ||
|
|
||
| [role="_abstract"] | ||
| Review data handling practices, feedback storage protocols, and model configuration architectures, such as the _Bring Your Own Model_ approach, to evaluate and enforce information security standards for your organization. | ||
|
|
||
| include::../modules/shared/con-data-use-and-privacy-practices.adoc[leveloffset=+1] | ||
|
|
||
| include::../modules/shared/con-user-feedback-collection.adoc[leveloffset=+1] | ||
|
|
||
| include::../modules/shared/con-bring-your-own-model-integration.adoc[leveloffset=+1] | ||
|
|
||
| include::../modules/shared/con-your-compliance-and-data-sharing-responsibility.adoc[leveloffset=+1] | ||
|
|
||
| ifdef::parent-context[:context: {parent-context}] | ||
| ifndef::parent-context[:!context:] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| :_mod-docs-content-type: ASSEMBLY | ||
| ifdef::context[:parent-context: {context}] | ||
|
|
||
| [id="configure-to-initialize-the-ai-assistant_{context}"] | ||
| = Configure {ls-short} to initialize the AI assistant | ||
|
|
||
| :context: configure-to-initialize-the-ai-assistant | ||
|
|
||
| [role="_abstract"] | ||
| {ls-brand-name} is enabled by default on {product} ({product-very-short}) instances. To provide developers with chat assistance, configure your deployment settings by using either the Operator or the Helm chart. | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| Perform a fresh installation to ensure compatibility with the updated system architecture. Do not update directly from the previous version. Direct updates cause operational errors because the Helm `values.yaml` file structure has changed. | ||
| ==== | ||
|
|
||
| include::../modules/shared/proc-configure-by-using-the-operator.adoc[leveloffset=+1] | ||
|
|
||
| include::../modules/shared/proc-configure-by-using-the-helm-chart.adoc[leveloffset=+1] | ||
|
|
||
| include::../modules/shared/proc-mirror-images-for-air-gapped-environments.adoc[leveloffset=+1] | ||
|
|
||
| ifdef::parent-context[:context: {parent-context}] | ||
| ifndef::parent-context[:!context:] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| :_mod-docs-content-type: ASSEMBLY | ||
| ifdef::context[:parent-context: {context}] | ||
|
|
||
| [id="customize-ai-responses_{context}"] | ||
| = Customize {ls-short} AI responses | ||
|
|
||
| :context: customize | ||
|
pabel-rh marked this conversation as resolved.
|
||
|
|
||
| [role="_abstract"] | ||
| You can customize {ls-short} to align model behavior with your operational goals, enhance developer productivity, and ensure secure data retention. | ||
|
|
||
| Customize {ls-short} by enabling user feedback, persisting chat history, and {model-context-protocol-book-link}#proc-configure-mcp-tools-for-developer-lightspeed_assembly-model-context-protocol-tools[configuring Model Context Protocol (MCP) tools]. | ||
|
|
||
| include::../modules/shared/proc-enable-user-feedback-to-improve-model-performance.adoc[leveloffset=+1] | ||
|
|
||
| include::../modules/shared/proc-customize-ai-responses-by-using-system-prompts.adoc[leveloffset=+1] | ||
|
|
||
| include::../modules/shared/proc-customize-chat-history-storage.adoc[leveloffset=+1] | ||
|
|
||
| include::../modules/shared/proc-enable-secure-ai-research-with-developer-lightspeed-notebooks.adoc[leveloffset=+1] | ||
|
|
||
| // include::../modules/shared/proc-changing-your-llm-provider.adoc[leveloffset=+1] | ||
| ifdef::parent-context[:context: {parent-context}] | ||
| ifndef::parent-context[:!context:] | ||
|
|
||
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| :_mod-docs-content-type: CONCEPT | ||
|
|
||
| [id="ai-reference-and-tool-calling-capabilities-through_{context}"] | ||
| = AI reference and tool-calling capabilities through {lcs-name} | ||
|
|
||
| [role="_abstract"] | ||
| Review the core components managed by the {lcs-name}({lcs-short}) sidecar container to plan integrations with large language models (LLM) and tool runtime providers. | ||
|
|
||
| The {lcs-short} container deploys as a sidecar to extend {product-very-short} functionality. The container integrates and manages the following core architectural components: | ||
|
|
||
| * Large language model (LLM) inference providers | ||
|
|
||
| * Model Context Protocol (MCP) or Retrieval Augmented Generation (RAG) tool runtime providers | ||
| + | ||
| [IMPORTANT] | ||
| ==== | ||
| You must verify that your model supports tool calling before you enable MCP features. Using an incompatible model results in error messages. | ||
| ==== | ||
|
|
||
| * Safety providers | ||
|
|
||
| * Vector database settings | ||
|
|
||
| {lcs-short} also manages critical operational configuration and key data, specifically: | ||
|
|
||
| * User feedback collection | ||
|
|
||
| * MCP server configuration | ||
|
|
||
| * Chat history | ||
|
|
||
| {ls-short} sends prompts and receives LLM responses through the {lcs-short} sidecar. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| :_mod-docs-content-type: CONCEPT | ||
|
|
||
| [id="architecture-for-your-ai-backend-deployment_{context}"] | ||
| = {ls-brand-name} architecture for your AI backend deployment | ||
|
|
||
| [role="_abstract"] | ||
| Review the {ls-short} component architecture to plan your system layout and coordinate connections with your artificial intelligence (AI) backend deployment. | ||
|
|
||
| The architecture relies on the {lcs-name} ({lcs-short}) container, which operates as the primary intermediary layer to manage {ls-short} functionality and console user interactions. By default, the interface appears as a floating action button (FAB) on all platforms that host {product-very-short}. | ||
|
|
||
| //Need to replace this screenshot | ||
| // image::integrate_interacting-with-developer-lightspeed-for-rhdh/developer-lightspeed-architecture-1-8-0.png["Lightspeed supported architecture diagram"] | ||
|
|
||
| .Additional resources | ||
| * link:https://access.redhat.com/support/policy/updates/developerhub[{product} Life Cycle and supported platforms] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| :_mod-docs-content-type: CONCEPT | ||
|
|
||
| [id="bring-your-own-model-integration_{context}"] | ||
| = Bring Your Own Model integration | ||
|
|
||
| [role="_abstract"] | ||
| Review _Bring Your Own Model (BYOM)_ requirements to select and integrate an OpenAI API-compatible inference service with {lcs-name}. | ||
|
|
||
| {ls-short} relies on a BYOM architecture that let you connect the {lcs-name} ({lcs-short}) layer to any OpenAI API-compatible inference platforms. To establish connection compatibility, your chosen inference service must satisfy the following technical criteria: | ||
|
|
||
| * The service must conform to the OpenAI API specification for chat completions. | ||
| * The host environment must match the specified infrastructure configuration and installation instructions. | ||
|
|
||
| Various commercial and open-source inference services support the OpenAI API specification. Because operational costs, performance metrics, and data security controls vary by provider, you must evaluate and test prospective platforms locally to select the service that best meets your organizational requirements. | ||
|
|
||
| .Additional resources | ||
| * link:https://github.com/openai/openai-openapi/tree/manual_spec[OpenAI API specification] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| :_mod-docs-content-type: CONCEPT | ||
|
|
||
| [id="data-use-and-privacy-practices_{context}"] | ||
| = Data use and privacy practices | ||
|
|
||
| [role="_abstract"] | ||
| Review data routing and privacy practices to evaluate how {ls-short} handles chat messages and operational information transmitted to large language model (LLM) providers. | ||
|
|
||
| {ls-short} sends your chat messages directly to your configured large language model (LLM) provider. Because these messages can contain sensitive operational data regarding your cluster, users, or business environment, ensure that your provider compliance policies align with your organizational security standards. | ||
|
|
||
| {ls-short} has limited capabilities to filter or redact the information you submit during user interactions. To mitigate data exposure risks, do not enter proprietary or confidential information into {ls-short}. To encourage user compliance, {ls-short} displays a mandatory warning at the start of each sessions, reminding users to omit personal or sensitive details. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,13 +4,17 @@ | |
| = Large language model (LLM) requirements | ||
|
|
||
| [role="_abstract"] | ||
| {ls-short} follows a _Bring Your Own Model_ approach, requiring you to provide access to a large language model (LLM). You must configure your preferred LLM provider during installation. | ||
| To plan your {ls-short} deployment, you must determine which compatible large language model (LLM) inference provider fits your infrastructure. | ||
|
|
||
| LLMs are usually provided by a service or server. Because {ls-short} does not provide an LLM for you, you must configure your preferred LLM provider during installation. You can configure the underlying Llama Stack server to integrate with several LLM `providers` that offer compatibility with the OpenAI API including the following inference providers: | ||
| {ls-short} operates on a _Bring Your Own Model (BYOM)_ architecture. Because the service does not include a native model, you must connect a compatible inference provider during installation. | ||
|
|
||
| * OpenAI (cloud-based inference service) | ||
| * {rhoai-brand-name} (enterprise model builder and inference server) | ||
| * {rhel} AI (enterprise inference server) | ||
| * Ollama (popular desktop inference server) | ||
| * vLLM (popular enterprise inference server) | ||
| * Gemini (available through Vertex AI) | ||
| The underlying {lcs-short} service integrates with platforms that support the OpenAI API specification or utilize the vLLM inference engine. Because there is no explicit {rhoai-brand-name} provider option in the configuration, you must route those deployments through the vLLM or OpenAI-compatible provider settings. | ||
|
|
||
| The `vllm` provider type communicates with endpoints that conform to the OpenAI API schema by automatically appending `/v1` to the configured provider URL. This mechanism allows you to use the `vllm` configuration for other hosted, OpenAI-compliant inference providers. | ||
|
|
||
| {ls-short} supports the following inference provider configurations: | ||
|
|
||
| * OpenAI cloud-based inference services | ||
| * vLLM enterprise inference servers, which includes models hosted on {rhoai-brand-name} and {rhel} AI | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Possibly here is where that explanation about it probably working via There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not familiar enough with those 2 services, I think maybe @gabemontero or @johnmcollier would know more about their workings?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This looks good to me |
||
| * Ollama desktop inference servers | ||
| * Gemini services through Vertex AI | ||
Uh oh!
There was an error while loading. Please reload this page.