Skip to content

unwrap() header peek in SSLEngine to fix BUFFER_UNDERFLOW#352

Merged
cconlon merged 2 commits intowolfSSL:masterfrom
rlm2002:sunJSSE_ServerName
Apr 17, 2026
Merged

unwrap() header peek in SSLEngine to fix BUFFER_UNDERFLOW#352
cconlon merged 2 commits intowolfSSL:masterfrom
rlm2002:sunJSSE_ServerName

Conversation

@rlm2002
Copy link
Copy Markdown
Contributor

@rlm2002 rlm2002 commented Apr 6, 2026

Adds a pre-check in unwrap() that inspects the TLS record header before calling into wolfSSL, returning BUFFER_UNDERFLOW when the buffer holds a partial record.

Also adds on to the testHandshakeUnwrapConsumedNotBufferUnderflow regression test.

@rlm2002 rlm2002 self-assigned this Apr 6, 2026
@cconlon cconlon requested a review from Copilot April 6, 2026 23:26
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR adjusts SSLEngine.unwrap() handshake behavior to better comply with the JSSE spec on BUFFER_UNDERFLOW, and updates a regression test to match the expected bytesConsumed() semantics.

Changes:

  • Added a client-mode pre-check for partial TLS records during handshake to return BUFFER_UNDERFLOW with bytesConsumed() == 0.
  • Guarded post-handshake/close-status logic to avoid running when the handshake underflow pre-check triggers.
  • Updated the handshake regression test to only fail when BUFFER_UNDERFLOW is returned with bytesConsumed() > 0.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Adds client-side TLS record header pre-check to drive spec-compliant BUFFER_UNDERFLOW behavior during handshake.
src/test/com/wolfssl/provider/jsse/test/WolfSSLEngineTest.java Adjusts regression test assertions to align with JSSE “no data consumed on underflow” requirement.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
@rlm2002 rlm2002 force-pushed the sunJSSE_ServerName branch 2 times, most recently from e593e63 to 34e923d Compare April 7, 2026 22:44
@rlm2002 rlm2002 marked this pull request as ready for review April 8, 2026 21:45
@rlm2002
Copy link
Copy Markdown
Contributor Author

rlm2002 commented Apr 9, 2026

retest this please Jenkins

@rlm2002 rlm2002 assigned cconlon and unassigned rlm2002 Apr 9, 2026
@cconlon cconlon requested a review from Copilot April 9, 2026 23:13
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
Comment thread src/test/com/wolfssl/provider/jsse/test/WolfSSLEngineTest.java Outdated
Comment thread src/test/com/wolfssl/provider/jsse/test/WolfSSLEngineTest.java Outdated
@cconlon cconlon assigned rlm2002 and unassigned cconlon Apr 9, 2026
@rlm2002 rlm2002 force-pushed the sunJSSE_ServerName branch 2 times, most recently from 0e2f0aa to ed375eb Compare April 15, 2026 23:46
@cconlon cconlon requested a review from Copilot April 15, 2026 23:49
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java Outdated
@rlm2002 rlm2002 changed the title unwrap() Client-side pre-check in SSLEngine to fix BUFFER_UNDERFLOW unwrap() header peek in SSLEngine to fix BUFFER_UNDERFLOW Apr 16, 2026
@rlm2002 rlm2002 force-pushed the sunJSSE_ServerName branch 2 times, most recently from 1aecfdd to fc57f89 Compare April 16, 2026 23:25
@cconlon cconlon requested a review from Copilot April 16, 2026 23:29
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/java/com/wolfssl/provider/jsse/WolfSSLEngine.java
Comment thread src/test/com/wolfssl/provider/jsse/test/WolfSSLEngineTest.java Outdated
Comment thread src/java/com/wolfssl/WolfSSL.java Outdated
@rlm2002 rlm2002 force-pushed the sunJSSE_ServerName branch 3 times, most recently from 71fa967 to 23072bc Compare April 17, 2026 14:31
skipping TLS record peek if continuing with the same TLS record. Update
bufferUnderflow guards.

move/add TLS record header constants to WolfSSL

check header bytes for plausible record header before peeking for
bufferUnderflow
…n additional continuation bytes testing

add partial header unwrap test and oversized record test
@rlm2002 rlm2002 force-pushed the sunJSSE_ServerName branch from 23072bc to 099a55a Compare April 17, 2026 18:04
@rlm2002 rlm2002 assigned cconlon and unassigned rlm2002 Apr 17, 2026
@cconlon cconlon merged commit dbd89ba into wolfSSL:master Apr 17, 2026
102 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants