Skip to content

Commit 9cf024d

Browse files
update: change default maximum token limit for custom LLM from 128000 to 64000
1 parent 23671a1 commit 9cf024d

2 files changed

Lines changed: 4 additions & 2 deletions

File tree

docs/self-hosting/govern/environment-variables.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -210,7 +210,7 @@ Plane AI supports multiple LLM providers. Configure one or more by adding their
210210
| **CUSTOM_LLM_API_KEY** | API key for authenticating with the custom endpoint. Required for `openai` provider; used as the AWS access key ID when `CUSTOM_LLM_PROVIDER=bedrock`. | Optional |
211211
| **CUSTOM_LLM_AWS_REGION** | AWS region for the Bedrock model (e.g. `us-east-1`). Required when `CUSTOM_LLM_PROVIDER=bedrock`. | Optional |
212212
| **CUSTOM_LLM_NAME** | Display name for the custom model shown in the UI. Defaults to `Custom LLM`. | Optional |
213-
| **CUSTOM_LLM_MAX_TOKENS** | Maximum token limit for the custom model. Defaults to `128000`. | Optional |
213+
| **CUSTOM_LLM_MAX_TOKENS** | Maximum token limit for the custom model. Defaults to `64000`. | Optional |
214214

215215
#### Provider base URLs
216216

docs/self-hosting/govern/plane-ai/custom-llm.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ CUSTOM_LLM_MODEL_KEY=your-model-id # model ID as the endpoint expects it
3434
CUSTOM_LLM_BASE_URL=https://your-endpoint/v1
3535
CUSTOM_LLM_API_KEY=your-api-key # use any non-empty string if no key is required
3636
CUSTOM_LLM_NAME=Your Model Name # display name shown to users
37-
CUSTOM_LLM_MAX_TOKENS=128000 # optional, defaults to 128000
37+
CUSTOM_LLM_MAX_TOKENS=64000 # optional; max output tokens per response
3838
```
3939

4040
**Examples:**
@@ -72,6 +72,7 @@ CUSTOM_LLM_API_KEY=your-aws-secret-access-key
7272
CUSTOM_LLM_AWS_REGION=us-east-1
7373
AWS_ACCESS_KEY_ID=your-aws-access-key-id # standard AWS env var, picked up by boto3
7474
CUSTOM_LLM_NAME=Claude via Bedrock
75+
CUSTOM_LLM_MAX_TOKENS=64000 # optional; max output tokens per response
7576
```
7677

7778
:::warning IAM permission required
@@ -90,6 +91,7 @@ CUSTOM_LLM_AWS_REGION=us-east-1
9091
BEDROCK_INFERENCE_PROFILE_ARN=arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/xxxx
9192
# or use BEDROCK_INFERENCE_PROFILE_ID=global.anthropic.claude-sonnet-4-6
9293
CUSTOM_LLM_NAME=Claude via Inference Profile
94+
CUSTOM_LLM_MAX_TOKENS=64000 # optional; max output tokens per response
9395
```
9496

9597
Plane AI activates inference profile mode automatically when a profile ARN or ID is set and ambient AWS credentials are present (`AWS_ROLE_ARN`, `AWS_WEB_IDENTITY_TOKEN_FILE`, `AWS_CONTAINER_CREDENTIALS_FULL_URI`, or `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE`).

0 commit comments

Comments
 (0)