Clarify creating a standalone connector#12480
Conversation
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
kolchfa-aws
requested review from
cwperks,
dlvenable,
epugh,
mgodwan,
peterzhuamazon and
sumobrian
as code owners
|
Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Merged. Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer. When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). |
| You can provision connectors in two ways: | ||
|
|
||
| 1. [Create a standalone connector](#creating-a-standalone-connector): A standalone connector can be reused and shared by multiple models but requires access to both the model and connector in OpenSearch and the third-party platform, such as OpenAI or Amazon SageMaker, that the connector is accessing. Standalone connectors are saved in a connector index. | ||
| 1. [Create a standalone connector](#creating-a-standalone-connector): A standalone connector can be reused by multiple model registrations in OpenSearch that share the same external endpoint and configuration. For example, you can register several models that all use the same OpenAI GPT-3.5 connector. However, if you need to connect to a different external model (for example, switching from `gpt-3.5-turbo` to `gpt-4`), you must create a separate connector. Standalone connectors require access to both the connector and the model in OpenSearch as well as the third-party platform. Standalone connectors are saved in a connector index. |
There was a problem hiding this comment.
High-priority — fix is incomplete: same wording remains in two sibling docs.
The exact phrase that motivated #12474 — "You can create a standalone connector that can be reused for multiple models." — still appears verbatim in:
_ml-commons-plugin/remote-models/index.md:115_ml-commons-plugin/remote-models/batch-ingestion.md:53
A user (or LLM) landing on either of those pages first will hit the same trap that this PR is trying to close. Suggest mirroring the new wording on both lines, e.g.: "You can create a standalone connector that can be reused by multiple model registrations in OpenSearch that share the same external endpoint and configuration."
| You can provision connectors in two ways: | ||
|
|
||
| 1. [Create a standalone connector](#creating-a-standalone-connector): A standalone connector can be reused and shared by multiple models but requires access to both the model and connector in OpenSearch and the third-party platform, such as OpenAI or Amazon SageMaker, that the connector is accessing. Standalone connectors are saved in a connector index. | ||
| 1. [Create a standalone connector](#creating-a-standalone-connector): A standalone connector can be reused by multiple model registrations in OpenSearch that share the same external endpoint and configuration. For example, you can register several models that all use the same OpenAI GPT-3.5 connector. However, if you need to connect to a different external model (for example, switching from `gpt-3.5-turbo` to `gpt-4`), you must create a separate connector. Standalone connectors require access to both the connector and the model in OpenSearch as well as the third-party platform. Standalone connectors are saved in a connector index. |
There was a problem hiding this comment.
Medium — "you must create a separate connector" is slightly stronger than the actual constraint.
This same repo's _ml-commons-plugin/remote-models/blueprints.md:64 documents that connector parameters can be overridden at predict time:
"Any parameters indicated in this field can be overridden by parameters specified in a predict request."
So a caller can point a single connector at multiple external models by passing parameters.model per predict request, provided the connector references it through a placeholder. The recommendation in the new sentence is sound, but the absolute "must" overstates it.
Suggested softening: "...we recommend creating a separate connector" or "...the simplest path is to create a separate connector". Optional: a one-line pointer to the predict-time override behavior for advanced users.
| You can provision connectors in two ways: | ||
|
|
||
| 1. [Create a standalone connector](#creating-a-standalone-connector): A standalone connector can be reused and shared by multiple models but requires access to both the model and connector in OpenSearch and the third-party platform, such as OpenAI or Amazon SageMaker, that the connector is accessing. Standalone connectors are saved in a connector index. | ||
| 1. [Create a standalone connector](#creating-a-standalone-connector): A standalone connector can be reused by multiple model registrations in OpenSearch that share the same external endpoint and configuration. For example, you can register several models that all use the same OpenAI GPT-3.5 connector. However, if you need to connect to a different external model (for example, switching from `gpt-3.5-turbo` to `gpt-4`), you must create a separate connector. Standalone connectors require access to both the connector and the model in OpenSearch as well as the third-party platform. Standalone connectors are saved in a connector index. |
There was a problem hiding this comment.
Nit — bullet-length imbalance.
Bullet 1 is now ~4 sentences, bullet 2 is still ~2. Scanability suffers a bit. Consider lifting the example and the "switching to gpt-4" exception out of the bullet into a one-paragraph note immediately after the list, so the two list items stay parallel in length.
| ## Creating a standalone connector | ||
|
|
||
| Standalone connectors can be used by multiple models. To create a standalone connector, send a request to the `connectors/_create` endpoint and provide all of the parameters described in [Connector blueprints]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/blueprints/): | ||
| Standalone connectors can be reused by multiple model registrations in OpenSearch that share the same external endpoint and configuration. To connect to a different external model, create a separate connector. To create a standalone connector, send a request to the `connectors/_create` endpoint and provide all of the parameters described in [Connector blueprints]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/blueprints/): |
There was a problem hiding this comment.
Nit — light repetition with line 39.
The overview bullet at line 39 now carries the "reuse" + "separate connector for a different model" idea. Restating both ideas again here within ~10 lines is a touch repetitive. The section intro could be trimmed to just the action sentence, e.g.: "To create a standalone connector, send a request to the connectors/_create endpoint and provide all of the parameters described in [Connector blueprints]..." and let the bullet at line 39 do the conceptual work. Not blocking.
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
|
Thank you, @ylwu-amzn! Comments addressed. |
3 participants
Closes #12474
Checklist
For more information on following Developer Certificate of Origin and signing off your commits, please check here.