Missing documentation for AWS inference profile setup with GenAIScript

[lemmy]

The documentation currently does not explain how to set up GenAIScript with a AWS inference profile, which is required to access the more powerful models. Could you please provide guidance on how to properly configure GenAIScript in this scenario?

I attempted the following setup, but it failed:

DEBUG=* npx --yes genaiscript@latest run translate --model arn:aws:bedrock:us-east-1:024871859028:inference-profile/global.anthropic.claude-sonnet-4-20250514-v1:0 --provider anthropic_bedrock

genaiscript:models resolving model for 'arn:aws:bedrock:us-east-1:024871859028:inference-profile/global.anthropic.claude-sonnet-4-20250514-v1:0' +297ms
  genaiscript:models candidate arn:aws:bedrock:us-east-1:024871859028:inference-profile/global.anthropic.claude-sonnet-4-20250514-v1:0 +0ms
  genaiscript:models resolving arn:aws:bedrock:us-east-1:024871859028:inference-profile/global.anthropic.claude-sonnet-4-20250514-v1:0 +0ms
  genaiscript:config:env parsing token for 'arn:aws:bedrock:us-east-1:024871859028:inference-profile/global.anthropic.claude-sonnet-4-20250514-v1:0:bedrock:us-east-1:024871859028:inference-profile/global.anthropic.claude-sonnet-4-20250514-v1:0' +297ms
  genaiscript:config:env looking for ARN_AWS_BEDROCK_US_EAST_1_024871859028_INFERENCE_PROFILE_GLOBAL_ANTHROPIC_CLAUDE_SONNET_4_20250514_V1_0_BEDROCK_US_EAST_1_024871859028_INFERENCE_PROFILE_GLOBAL_ANTHROPIC_CLAUDE_SONNET_4_20250514_V1_0_... +1ms
  genaiscript:config:env looking for ARN_AWS_BEDROCK_US_EAST_1_024871859028_INFERENCE_PROFILE_GLOBAL_ANTHROPIC_CLAUDE_SONNET_4_20250514_V1_0_... +0ms
  genaiscript:config:env looking for ARN_... +0ms
  genaiscript:config:env looking for AWS_BEDROCK_US_EAST_1_024871859028_INFERENCE_PROFILE_GLOBAL_ANTHROPIC_CLAUDE_SONNET_4_20250514_V1_0_... +0ms
  genaiscript:config:env no matching provider found, returning undefined +0ms
  genaiscript:host:node no token found for arn:aws:bedrock:us-east-1:024871859028:inference-profile/global.anthropic.claude-sonnet-4-20250514-v1:0 +295ms
  genaiscript:models configuration not found +1ms
  genaiscript:error {
  genaiscript:error   name: 'Error',
  genaiscript:error   message: 'model configuration not found',
  genaiscript:error   stack: 'Error: model configuration not found\n' +
  genaiscript:error     '    at runPrompt (file:///Users/markus/.npm/_npx/4f85e8f904063418/node_modules/@genaiscript/core/dist/esm/runpromptcontext.js:647:23)\n' +
  genaiscript:error     '    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n' +
  genaiscript:error     '    at async <anonymous> (/Users/markus/src/TLA/_specs/TLAiBench/genaisrc/translate.genai.mts:71:24)'
  genaiscript:error } +2ms
  script {
  script   messages: [],
  script   text: '',
  script   reasoning: false,
  script   finishReason: 'fail',
  script   error: {
  script     name: 'Error',
  script     message: 'model configuration not found',
  script     stack: 'Error: model configuration not found\n' +
  script       '    at runPrompt (file:///Users/markus/.npm/_npx/4f85e8f904063418/node_modules/@genaiscript/core/dist/esm/runpromptcontext.js:647:23)\n' +
  script       '    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\n' +
  script       '    at async <anonymous> (/Users/markus/src/TLA/_specs/TLAiBench/genaisrc/translate.genai.mts:71:24)'
  script   }
  script } +0ms

---

Failure when trying without inference profile:

400 Invocation of model ID anthropic.claude-sonnet-4-20250514-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.
Error: 400 Invocation of model ID anthropic.claude-sonnet-4-20250514-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.
    at APIError.generate (/Users/markus/.npm/_npx/4f85e8f904063418/node_modules/@anthropic-ai/sdk/src/core/error.ts:63:14)
    at AnthropicBedrock.makeStatusError (/Users/markus/.npm/_npx/4f85e8f904063418/node_modules/@anthropic-ai/sdk/src/client.ts:410:28)
    at AnthropicBedrock.makeRequest (/Users/markus/.npm/_npx/4f85e8f904063418/node_modules/@anthropic-ai/sdk/src/client.ts:643:24)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
status: 400
headers: {}
requestID: null
error:
  message: Invocation of model ID anthropic.claude-sonnet-4-20250514-v1:0 with
    on-demand throughput isn’t supported. Retry your request with the ID or ARN
    of an inference profile that contains this model.

  genaiscript:error {
  genaiscript:error   status: 400,
  genaiscript:error   headers: {},
  genaiscript:error   requestID: null,
  genaiscript:error   error: {
  genaiscript:error     message: 'Invocation of model ID anthropic.claude-sonnet-4-20250514-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.'
  genaiscript:error   },
  genaiscript:error   name: 'Error',
  genaiscript:error   message: '400 Invocation of model ID anthropic.claude-sonnet-4-20250514-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.',
  genaiscript:error   stack: 'Error: 400 Invocation of model ID anthropic.claude-sonnet-4-20250514-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.\n' +
  genaiscript:error     '    at APIError.generate (/Users/markus/.npm/_npx/4f85e8f904063418/node_modules/@anthropic-ai/sdk/src/core/error.ts:63:14)\n' +
  genaiscript:error     '    at AnthropicBedrock.makeStatusError (/Users/markus/.npm/_npx/4f85e8f904063418/node_modules/@anthropic-ai/sdk/src/client.ts:410:28)\n' +
  genaiscript:error     '    at AnthropicBedrock.makeRequest (/Users/markus/.npm/_npx/4f85e8f904063418/node_modules/@anthropic-ai/sdk/src/client.ts:643:24)\n' +
  genaiscript:error     '    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)'
  genaiscript:error } +1ms


╰─🏁  anthropic_bedrock:anthropic.claude-sonnet-4-20250514-v1:0 ✉ 2 495ms ⇅ 0t ↑0t ↓0t

[pelikhan]

let me review and add the code and you'll have to try again.

[pelikhan]

#1919