Skip to content

Commit cc394e8

Browse files
julianwoodjulianwood
authored
Merge pull request #3054 from NithinChandranR-AWS/NithinChandranR-AWS-feature-apigw-streaming-lambda-bedrock-cdk
New serverless pattern - apigw-streaming-lambda-bedrock-cdk
2 parents 2792d30 + 41a34c1 commit cc394e8

10 files changed

Lines changed: 434 additions & 0 deletions

File tree

‎apigw-streaming-lambda-bedrock-cdk/.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
node_modules
2+
cdk.out
3+
*.js
4+
!src/**/*.js
5+
*.d.ts
6+
package-lock.json

‎apigw-streaming-lambda-bedrock-cdk/README.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Amazon API Gateway REST Streaming with AWS Lambda and Amazon Bedrock
2+
3+
This pattern deploys an Amazon API Gateway REST API with response streaming enabled, backed by a streaming AWS Lambda function that returns real-time LLM responses from Amazon Bedrock using Server-Sent Events (SSE).
4+
5+
Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/apigw-streaming-lambda-bedrock-cdk
6+
7+
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
8+
9+
## Requirements
10+
11+
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in.
12+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
13+
* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
14+
* [Node.js 18+](https://nodejs.org/en/download/) installed
15+
* [AWS CDK v2](https://docs.aws.amazon.com/cdk/v2/guide/getting-started.html) installed
16+
* [Amazon Bedrock model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) enabled for Anthropic Claude Sonnet in your target region
17+
18+
## Architecture
19+
20+
```
21+
┌──────────┐ ┌──────────────────┐ ┌──────────────────┐ ┌─────────────┐
22+
│ Client │────▶│ API Gateway │────▶│ Lambda │────▶│ Bedrock │
23+
│ (curl) │◀────│ REST (Stream) │◀────│ (Streaming) │◀────│ (Claude) │
24+
└──────────┘ SSE └──────────────────┘ └──────────────────┘ └─────────────┘
25+
```
26+
27+
## How it works
28+
29+
> **Note:** This sample does not include authentication/authorization for simplicity. In a production setting, enable IAM authorization, Cognito, or API keys on the API Gateway method.
30+
31+
1. Client sends a POST request with a prompt to the API Gateway REST endpoint.
32+
2. API Gateway forwards the request to a Lambda function using **streaming invocation** (`InvokeWithResponseStreaming`).
33+
3. The Lambda function calls Amazon Bedrock's `InvokeModelWithResponseStream` API.
34+
4. As Bedrock generates tokens, Lambda writes each chunk to the response stream.
35+
5. API Gateway streams the response back to the client in real-time using chunked transfer encoding.
36+
6. The client receives tokens as they are generated — time-to-first-byte is typically under 1 second.
37+
38+
## Deployment Instructions
39+
40+
1. Clone the repository:
41+
```bash
42+
git clone https://github.com/aws-samples/serverless-patterns
43+
cd serverless-patterns/apigw-streaming-lambda-bedrock-cdk
44+
```
45+
46+
2. Install dependencies:
47+
```bash
48+
npm install
49+
```
50+
51+
3. Deploy the stack:
52+
```bash
53+
cdk deploy
54+
```
55+
56+
4. Note the API endpoint URL from the stack outputs.
57+
58+
## Testing
59+
60+
1. Stream a response using curl:
61+
```bash
62+
curl --no-buffer -X POST \
63+
'<ApiEndpoint>/prod/chat' \
64+
-H 'Content-Type: application/json' \
65+
-d '{"prompt": "Explain serverless computing in 3 paragraphs"}'
66+
```
67+
68+
2. The API streams the response in real-time, token by token.
69+
70+
## Cleanup
71+
72+
```bash
73+
cdk destroy
74+
```
75+
76+
----
77+
Copyright 2026 Amazon.com, Inc. or its affiliates. All Rights Reserved.
78+
79+
SPDX-License-Identifier: MIT-0

‎apigw-streaming-lambda-bedrock-cdk/apigw-streaming-lambda-bedrock-cdk.json‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
{
2+
"title": "Stream Amazon Bedrock responses with Amazon API Gateway",
3+
"description": "Stream LLM responses from Amazon Bedrock through Amazon API Gateway REST API using AWS Lambda response streaming for real-time AI chat experiences.",
4+
"language": "TypeScript",
5+
"level": "300",
6+
"framework": "AWS CDK",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This pattern deploys an Amazon API Gateway REST API with response streaming enabled, backed by an AWS Lambda function that streams responses from Amazon Bedrock (Claude) in real-time using Server-Sent Events (SSE).",
11+
"When a client sends a prompt, API Gateway forwards it to a streaming Lambda function. The function invokes Amazon Bedrock and progressively writes each text chunk to the response stream. The client receives tokens as they are generated, dramatically reducing time-to-first-byte.",
12+
"This pattern is ideal for building AI chatbots, copilots, and interactive assistants where real-time streaming improves user experience."
13+
]
14+
},
15+
"gitHub": {
16+
"template": {
17+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/apigw-streaming-lambda-bedrock-cdk",
18+
"templateURL": "serverless-patterns/apigw-streaming-lambda-bedrock-cdk",
19+
"projectFolder": "apigw-streaming-lambda-bedrock-cdk",
20+
"templateFile": "lib/apigw-streaming-lambda-bedrock-stack.ts"
21+
}
22+
},
23+
"resources": {
24+
"bullets": [
25+
{
26+
"text": "Building responsive APIs with Amazon API Gateway response streaming",
27+
"link": "https://aws.amazon.com/blogs/compute/building-responsive-apis-with-amazon-api-gateway-response-streaming/"
28+
},
29+
{
30+
"text": "Serverless strategies for streaming LLM responses",
31+
"link": "https://aws.amazon.com/blogs/compute/serverless-strategies-for-streaming-llm-responses/"
32+
},
33+
{
34+
"text": "Amazon Bedrock",
35+
"link": "https://aws.amazon.com/bedrock/"
36+
}
37+
]
38+
},
39+
"patternArch": {
40+
"icon1": {
41+
"x": 20,
42+
"y": 50,
43+
"service": "apigw",
44+
"label": "API Gateway REST API"
45+
},
46+
"icon2": {
47+
"x": 50,
48+
"y": 50,
49+
"service": "lambda",
50+
"label": "Lambda (Streaming)"
51+
},
52+
"icon3": {
53+
"x": 80,
54+
"y": 50,
55+
"service": "bedrock",
56+
"label": "Bedrock (Claude)"
57+
},
58+
"line1": {
59+
"from": "icon1",
60+
"to": "icon2"
61+
},
62+
"line2": {
63+
"from": "icon2",
64+
"to": "icon3"
65+
}
66+
},
67+
"deploy": {
68+
"text": [
69+
"cdk deploy"
70+
]
71+
},
72+
"testing": {
73+
"text": [
74+
"See the GitHub repo for detailed testing instructions."
75+
]
76+
},
77+
"cleanup": {
78+
"text": [
79+
"Delete the stack: <code>cdk destroy</code>."
80+
]
81+
},
82+
"authors": [
83+
{
84+
"name": "Nithin Chandran R",
85+
"bio": "Technical Account Manager at AWS",
86+
"linkedin": "nithin-chandran-r"
87+
}
88+
]
89+
}

‎apigw-streaming-lambda-bedrock-cdk/bin/app.ts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#!/usr/bin/env node
2+
import "source-map-support/register";
3+
import * as cdk from "aws-cdk-lib";
4+
import { ApigwStreamingLambdaBedrockStack } from "../lib/apigw-streaming-lambda-bedrock-stack";
5+
6+
const description="Sample app (uksb-1tthgi812) (tag:apigw-streaming-lambda-bedrock-cdk"
7+
const app = new cdk.App();
8+
new ApigwStreamingLambdaBedrockStack(app, "ApigwStreamingLambdaBedrockStack",{description});

‎apigw-streaming-lambda-bedrock-cdk/cdk.json‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"app": "npx ts-node --prefer-ts-exts bin/app.ts",
3+
"watch": {
4+
"include": ["**"],
5+
"exclude": ["README.md", "cdk*.json", "**/*.d.ts", "**/*.js", "node_modules", "src"]
6+
},
7+
"context": {
8+
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
9+
"@aws-cdk/core:checkSecretUsage": true
10+
}
11+
}

‎apigw-streaming-lambda-bedrock-cdk/example-pattern.json‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
{
2+
"title": "API Gateway REST Streaming with Lambda and Amazon Bedrock",
3+
"description": "Stream LLM responses from Amazon Bedrock through API Gateway REST API using Lambda response streaming for real-time AI chat experiences.",
4+
"language": "TypeScript",
5+
"level": "300",
6+
"framework": "AWS CDK",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This pattern deploys an API Gateway REST API with response streaming enabled, backed by a Lambda function that streams responses from Amazon Bedrock (Claude) in real-time using Server-Sent Events (SSE).",
11+
"When a client sends a prompt, API Gateway forwards it to a streaming Lambda function. The Lambda invokes Bedrock with streaming enabled and progressively writes each text chunk to the response stream. The client receives tokens as they are generated, dramatically reducing time-to-first-byte.",
12+
"This pattern is ideal for building AI chatbots, copilots, and interactive assistants where real-time streaming improves user experience."
13+
]
14+
},
15+
"gitHub": {
16+
"template": {
17+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/apigw-streaming-lambda-bedrock-cdk",
18+
"templateURL": "serverless-patterns/apigw-streaming-lambda-bedrock-cdk",
19+
"projectFolder": "apigw-streaming-lambda-bedrock-cdk",
20+
"templateFile": "lib/apigw-streaming-lambda-bedrock-stack.ts"
21+
}
22+
},
23+
"resources": {
24+
"bullets": [
25+
{
26+
"text": "Building responsive APIs with Amazon API Gateway response streaming",
27+
"link": "https://aws.amazon.com/blogs/compute/building-responsive-apis-with-amazon-api-gateway-response-streaming/"
28+
},
29+
{
30+
"text": "Serverless strategies for streaming LLM responses",
31+
"link": "https://aws.amazon.com/blogs/compute/serverless-strategies-for-streaming-llm-responses/"
32+
},
33+
{
34+
"text": "Amazon Bedrock",
35+
"link": "https://aws.amazon.com/bedrock/"
36+
}
37+
]
38+
},
39+
"deploy": {
40+
"text": [
41+
"cdk deploy"
42+
]
43+
},
44+
"testing": {
45+
"text": [
46+
"See the GitHub repo for detailed testing instructions."
47+
]
48+
},
49+
"cleanup": {
50+
"text": [
51+
"Delete the stack: <code>cdk destroy</code>."
52+
]
53+
},
54+
"authors": [
55+
{
56+
"name": "Nithin Chandran R",
57+
"bio": "Technical Account Manager at AWS",
58+
"linkedin": "nithin-chandran-r"
59+
}
60+
]
61+
}

‎apigw-streaming-lambda-bedrock-cdk/lib/apigw-streaming-lambda-bedrock-stack.ts‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
import * as cdk from "aws-cdk-lib";
2+
import * as lambda from "aws-cdk-lib/aws-lambda";
3+
import * as apigateway from "aws-cdk-lib/aws-apigateway";
4+
import * as iam from "aws-cdk-lib/aws-iam";
5+
import * as logs from "aws-cdk-lib/aws-logs";
6+
import { Construct } from "constructs";
7+
8+
export class ApigwStreamingLambdaBedrockStack extends cdk.Stack {
9+
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
10+
super(scope, id, props);
11+
12+
const modelId = new cdk.CfnParameter(this, "BedrockModelId", {
13+
type: "String",
14+
default: "us.anthropic.claude-sonnet-4-20250514-v1:0",
15+
description: "Bedrock model ID (inference profile) to use",
16+
});
17+
18+
// Streaming Lambda function
19+
const fn = new lambda.Function(this, "StreamingBedrockFn", {
20+
runtime: lambda.Runtime.NODEJS_22_X,
21+
handler: "index.handler",
22+
code: lambda.Code.fromAsset("src"),
23+
timeout: cdk.Duration.minutes(5), // Must be >= API Gateway integration timeout (5 min) to avoid premature termination
24+
memorySize: 256,
25+
environment: {
26+
MODEL_ID: modelId.valueAsString,
27+
},
28+
logRetention: logs.RetentionDays.ONE_WEEK,
29+
});
30+
31+
fn.addToRolePolicy(
32+
new iam.PolicyStatement({
33+
actions: ["bedrock:InvokeModelWithResponseStream"],
34+
resources: [
35+
`arn:aws:bedrock:${this.region}:${this.account}:inference-profile/${modelId.valueAsString}`,
36+
"arn:aws:bedrock:*::foundation-model/*",
37+
],
38+
})
39+
);
40+
41+
// REST API with streaming integration
42+
const api = new apigateway.RestApi(this, "StreamingApi", {
43+
restApiName: "Bedrock Streaming API",
44+
description: "API Gateway REST API with response streaming to Bedrock",
45+
deployOptions: { stageName: "prod" },
46+
});
47+
48+
const chatResource = api.root.addResource("chat");
49+
50+
// Add POST method with standard Lambda proxy integration
51+
const method = chatResource.addMethod(
52+
"POST",
53+
// Override default 29s API Gateway timeout to allow streaming responses to complete
54+
new apigateway.LambdaIntegration(fn, { timeout: cdk.Duration.minutes(5) })
55+
);
56+
57+
// Override integration URI to use response-streaming-invocations path
58+
const cfnMethod = method.node.defaultChild as apigateway.CfnMethod;
59+
cfnMethod.addPropertyOverride(
60+
"Integration.Uri",
61+
`arn:aws:apigateway:${this.region}:lambda:path/2021-11-15/functions/${fn.functionArn}/response-streaming-invocations`
62+
);
63+
cfnMethod.addPropertyOverride(
64+
"Integration.ResponseTransferMode",
65+
"STREAM"
66+
);
67+
cfnMethod.addPropertyOverride("Integration.TimeoutInMillis", 300000);
68+
69+
new cdk.CfnOutput(this, "ApiEndpoint", {
70+
value: api.urlForPath("/chat"),
71+
description: "POST your prompt to this URL to stream Bedrock responses",
72+
});
73+
new cdk.CfnOutput(this, "FunctionName", { value: fn.functionName });
74+
}
75+
}

‎apigw-streaming-lambda-bedrock-cdk/package.json‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"name": "apigw-streaming-lambda-bedrock-cdk",
3+
"version": "1.0.0",
4+
"bin": {
5+
"app": "bin/app.ts"
6+
},
7+
"scripts": {
8+
"build": "tsc",
9+
"cdk": "cdk"
10+
},
11+
"dependencies": {
12+
"aws-cdk-lib": "2.180.0",
13+
"constructs": "10.4.2"
14+
},
15+
"devDependencies": {
16+
"@types/node": "^22.0.0",
17+
"ts-node": "^10.9.0",
18+
"typescript": "~5.7.0"
19+
}
20+
}

0 commit comments

Comments
 (0)