Skip to content

Add optional timeout configurations for AWS Lambda plugin#6413

Merged
dinujoh merged 3 commits into
opensearch-project:mainfrom
ashrao94:add-lambda-read-timeout
Jan 28, 2026
Merged

Add optional timeout configurations for AWS Lambda plugin#6413
dinujoh merged 3 commits into
opensearch-project:mainfrom
ashrao94:add-lambda-read-timeout

Conversation

@ashrao94

@ashrao94 ashrao94 commented Jan 15, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds optional timeout configurations to the AWS Lambda plugin in Data Prepper, addressing timeout issues when Lambda functions take longer than the default SDK timeouts.

Changes Made

  • Add api_call_attempt_timeout configuration for per-attempt timeouts
  • Make read_timeout optional (only applied when specified)
  • Add comprehensive unit tests for timeout configurations
  • Maintain backward compatibility with existing configurations

Both timeout parameters are now optional and only configured when explicitly set, allowing users to fine-tune timeout behavior for their Lambda functions.

Configuration Options

yaml
aws:
lambda:
client:

Optional timeout configurations

api_call_attempt_timeout: PT5M # Per-attempt timeout (NEW)
read_timeout: PT15M # HTTP read timeout (now optional)

api_call_timeout: PT15M
connection_timeout: PT60S
max_concurrency: 200

Issues Resolved

#6257

Check List

  • [ yes] New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • [yes ] New functionality has javadoc added
  • [ yes] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

graytaylor0
graytaylor0 previously approved these changes Jan 15, 2026
srikanthjg
srikanthjg previously approved these changes Jan 21, 2026
@srikanthjg srikanthjg dismissed their stale review January 21, 2026 20:16

build failing, please fix the build build

@ashrao94 ashrao94 force-pushed the add-lambda-read-timeout branch 3 times, most recently from 6179285 to 5dc16ef Compare January 22, 2026 16:03
@ashrao94 ashrao94 requested a review from dinujoh January 22, 2026 16:08
Aiswarya Sadananda Rao added 2 commits January 22, 2026 16:11
- Add api_call_attempt_timeout configuration for per-attempt timeouts
- Make read_timeout optional (only applied when specified)
- Add comprehensive unit tests for timeout configurations
- Maintain backward compatibility with existing configurations
- Follow AWS SDK timeout hierarchy best practices

Both timeout parameters are now optional and only configured when explicitly set,
allowing users to fine-tune timeout behavior for their Lambda functions.

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>
- Add required OpenSearch Contributors license header
- Fixes license header violation in new test file

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>
@ashrao94 ashrao94 force-pushed the add-lambda-read-timeout branch from 5dc16ef to 17b02d5 Compare January 22, 2026 16:11
dinujoh
dinujoh previously approved these changes Jan 22, 2026
srikanthjg
srikanthjg previously approved these changes Jan 22, 2026
Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>
@ashrao94 ashrao94 dismissed stale reviews from srikanthjg and dinujoh via ab3b76d January 28, 2026 08:53
@ashrao94 ashrao94 force-pushed the add-lambda-read-timeout branch from 17b02d5 to ab3b76d Compare January 28, 2026 08:53
@dinujoh dinujoh merged commit e1483c7 into opensearch-project:main Jan 28, 2026
55 of 56 checks passed
simonelbaz pushed a commit to simonelbaz/data-prepper that referenced this pull request Jan 31, 2026
…-project#6413)

* Add optional timeout configurations for AWS Lambda plugin

- Add api_call_attempt_timeout configuration for per-attempt timeouts
- Make read_timeout optional (only applied when specified)
- Add comprehensive unit tests for timeout configurations
- Maintain backward compatibility with existing configurations
- Follow AWS SDK timeout hierarchy best practices

Both timeout parameters are now optional and only configured when explicitly set,
allowing users to fine-tune timeout behavior for their Lambda functions.

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>

* Add license header to ClientOptionsTest.java

- Add required OpenSearch Contributors license header
- Fixes license header violation in new test file

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>

* Remove unused Duration import from ClientOptionsTest

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>

---------

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>
Co-authored-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>
Signed-off-by: Simon ELBAZ <elbazsimon9@gmail.com>
simonelbaz pushed a commit to simonelbaz/data-prepper that referenced this pull request Jan 31, 2026
…-project#6413)

* Add optional timeout configurations for AWS Lambda plugin

- Add api_call_attempt_timeout configuration for per-attempt timeouts
- Make read_timeout optional (only applied when specified)
- Add comprehensive unit tests for timeout configurations
- Maintain backward compatibility with existing configurations
- Follow AWS SDK timeout hierarchy best practices

Both timeout parameters are now optional and only configured when explicitly set,
allowing users to fine-tune timeout behavior for their Lambda functions.

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>

* Add license header to ClientOptionsTest.java

- Add required OpenSearch Contributors license header
- Fixes license header violation in new test file

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>

* Remove unused Duration import from ClientOptionsTest

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>

---------

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>
Co-authored-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>
simonelbaz pushed a commit to simonelbaz/data-prepper that referenced this pull request Jan 31, 2026
…-project#6413)

* Add optional timeout configurations for AWS Lambda plugin

- Add api_call_attempt_timeout configuration for per-attempt timeouts
- Make read_timeout optional (only applied when specified)
- Add comprehensive unit tests for timeout configurations
- Maintain backward compatibility with existing configurations
- Follow AWS SDK timeout hierarchy best practices

Both timeout parameters are now optional and only configured when explicitly set,
allowing users to fine-tune timeout behavior for their Lambda functions.

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>

* Add license header to ClientOptionsTest.java

- Add required OpenSearch Contributors license header
- Fixes license header violation in new test file

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>

* Remove unused Duration import from ClientOptionsTest

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>

---------

Signed-off-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>
Co-authored-by: Aiswarya Sadananda Rao <aiswarao@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants