You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/docs/configuration/librechat_yaml/object_structure/aws_bedrock.mdx
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,6 +19,7 @@ endpoints:
19
19
guardrailIdentifier: "your-guardrail-id"
20
20
guardrailVersion: "1"
21
21
trace: "enabled"
22
+
streamProcessingMode: "sync"
22
23
```
23
24
24
25
> **Note:** AWS Bedrock endpoint supports all [Shared Endpoint Settings](/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings), including `streamRate`, `titleModel`, `titleMethod`, `titlePrompt`, `titlePromptTemplate`, and `titleEndpoint`. The settings shown below are specific to Bedrock or have Bedrock-specific defaults.
@@ -148,6 +149,7 @@ endpoints:
148
149
['guardrailIdentifier', 'String', 'The unique identifier of the guardrail to apply.', 'Required when using guardrails.'],
149
150
['guardrailVersion', 'String', 'The version of the guardrail to use.', 'Required when using guardrails.'],
150
151
['trace', 'String', 'Controls guardrail trace output for debugging. Options: "enabled", "enabled_full", or "disabled".', 'Optional. Default: "disabled"'],
- Guardrails help ensure responsible AI usage by filtering harmful content, PII, and other sensitive information
166
169
- The `guardrailIdentifier` can be found in the AWS Bedrock console under Guardrails
167
170
- Set `trace` to `"enabled"` or `"enabled_full"` during development to see which guardrail policies are triggered
171
+
- Set `streamProcessingMode` to `"async"` to stream responses faster (at the cost of guardrail possibly allowing inappropriate content through until its scan completes)
168
172
- For production, set `trace` to `"disabled"` to reduce response payload size
Copy file name to clipboardExpand all lines: content/docs/configuration/pre_configured_ai/bedrock.mdx
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -64,6 +64,7 @@ endpoints:
64
64
guardrailIdentifier: "abc123xyz"
65
65
guardrailVersion: "1"
66
66
trace: "enabled"
67
+
streamProcessingMode: "sync"
67
68
```
68
69
69
70
- `streamRate`: (Optional) Set the rate of processing each new token in milliseconds.
@@ -81,6 +82,7 @@ endpoints:
81
82
- `guardrailIdentifier`: The guardrail ID or ARN from your AWS Bedrock Console.
82
83
- `guardrailVersion`: The guardrail version number (e.g., `"1"`) or `"DRAFT"`.
83
84
- `trace`: (Optional) Enable trace logging: `"enabled"`, `"disabled"`, or `"enabled_full"`.
85
+
- `streamProcessingMode`: (Optional) Set stream processing mode: `"sync"` or `"async"` (defaults to `"sync"`).
84
86
- See [AWS Bedrock Guardrails documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-how.html) for creating and managing guardrails.
0 commit comments