Add support for continuation frame in WebSocketMessageParser#2320
Add support for continuation frame in WebSocketMessageParser#2320an-tao merged 4 commits intodrogonframework:masterfrom
Conversation
|
@dm385 Thanks so much for your patch, I changed some code to use loops instead of recursion. Please check if it is correct. |
|
Seems like it's coming to an infinite loop in my case now, but haven't had the time to look at it more closely. |
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for parsing continuation frames within the WebSocketMessageParser, addressing issue #1647.
- Introduces looping to process frames as long as sufficient data is available.
- Adds boundary checks and overflow protection when reading extended payload lengths.
- Adjusts buffer retrieval logic for both masked and unmasked frames based on FIN flags.
| // According to the rfc6455 | ||
| gotAll_ = false; | ||
| if (buffer->readableBytes() >= 2) | ||
| while (buffer->readableBytes() >= 2) |
There was a problem hiding this comment.
Consider adding a comment to clarify that the while loop is intentionally used to process multiple frames in a single parse call, ensuring that continuation frames are handled correctly.
| length = 0; | ||
| for (int i = 2; i <= 9; ++i) | ||
| { | ||
| if (length > ((std::numeric_limits<size_t>::max)() >> 8)) |
There was a problem hiding this comment.
Add a comment clarifying that this check prevents an overflow when accumulating the extended payload length, ensuring safe processing of large payloads.
| @@ -380,9 +389,16 @@ bool WebSocketMessageParser::parse(trantor::MsgBuffer *buffer) | |||
| { | |||
| message_[oldLen + i] = (rawData[i] ^ masks[i % 4]); | |||
| } | |||
There was a problem hiding this comment.
[nitpick] Review whether consuming the entire frame, regardless of the FIN flag, is the intended design for continuation frames. A brief inline comment explaining this decision may improve clarity for future maintainers.
| } | |
| } | |
| // Consume the entire frame from the buffer, regardless of the FIN flag. | |
| // This is necessary to process subsequent frames correctly, as the WebSocket | |
| // protocol allows fragmented messages. The `message_` buffer accumulates | |
| // data from all frames, and the `isFin` flag determines when the message | |
| // is complete. |
hwc0919
left a comment
There was a problem hiding this comment.
Just make sure that illegal data won't end up in an infinite loop.
| length = 0; | ||
| for (int i = 2; i <= 9; ++i) | ||
| { | ||
| if (length > ((std::numeric_limits<size_t>::max)() >> 8)) |
There was a problem hiding this comment.
This does not feel right. size_t is different on 32 and 64bit systems. So the max size will be 24 and 48bits respectively. Do you intend to use uint64_t?
There was a problem hiding this comment.
The judgment is followed by an 8-bit left shift operation, so the maximum values are 32-bit and 64-bit respectively. I think this judgment protects the 32-bit system from overflow.
There was a problem hiding this comment.
On 32bit system this is 16M. Which is not big in modern standards. Is this really ok? And on 64bits 2^56 is, a lot. I think we should have a consistent limit instead of "it depends on the processor"
|
The infinite loop does not occur anymore for my case. Thank you! |
@hwc0919 why do you think it is "illegal" data? |
…ramework#2320) Co-authored-by: antao <antao2002@gmail.com>
* Add an example of prometheus (#2076) * Delay parsing json for HttpClient (#2077) * Update README.md (#2080) A few minor typos. * Add setsockopt to HttpServer (#2086) * Support request stream (#2055) * Bump version to 1.9.6 * Allow `MultiPartParser` to be movable (#2107) * add quotes (#2116) * change stoi to stoul (#2115) * Add coroutine mutex (#2095) * Modernize cookies (#2112) * Add requestsBufferSize function (#2124) * Use correct libraries when compiling statically (#2136) When compiling statically, cmake pulls shared libraries as dependencies for drogon e.g. libpq.so instead of libpq.a. This causes linkage errors when compiling the whole program. The flag USE_STATIC_LIBS_ONLY should set the CMAKE_FIND_LIBRARY_SUFFIXES to the appropriate suffix on different platforms, thus find_* commands find the right library. * Change a log level * Refine SQLite3 error types with new exception handling (#2145) Signed-off-by: yuanlu <2573580691@qq.com> * Feature: TcpServer hot reload SSL file (#2150) * Bump version to 1.9.7 * Fix forwarding with space in url by encoding (#2155) * Partially revert commit 93d8fb4 (#2156) * Add in-place base64 encode and decode (#2153) * Revert original path to its initial behavior (#2157) * Fix coroutine continuation handle (#2163) Using coroutines directly by the user e.g. declaring a function with drogon::Task<> return type, will have it's continuation set to nullptr. Returning nullptr causes a segfault and it must be checked before returning e.g. the snippet form https://en.cppreference.com/w/cpp/coroutine/noop_coroutine ```cpp struct final_awaiter { std::coroutine_handle<> await_suspend(std::coroutine_handle<promise_type> h) noexcept { // final_awaiter::await_suspend is called when the execution of the // current coroutine (referred to by 'h') is about to finish. // If the current coroutine was resumed by another coroutine via // co_await get_task(), a handle to that coroutine has been stored // as h.promise().previous. In that case, return the handle to resume // the previous coroutine. // Otherwise, return noop_coroutine(), whose resumption does nothing. if (auto previous = h.promise().previous; previous) return previous; else return std::noop_coroutine(); } }; ``` This commit default initializes the continuation handle to no op coroutine and avoids the check. Signed-off-by: Omar Mohamed <mohamed.omar67492@gmail.com> * Add Hodor whitelists (#2154) * include exception header for std::exception_ptr (#2159) * Update trantor * Add support for escaped identifiers in Postgresql (#2167) * Remove content-length header from 101 Switching Protocols response (#2164) * Remove websocketResponseTest from windows shared library env (#2170) * Optimize query params and allow for empty values (#2171) * Fix a bug after removing content-length header in some responses (#2176) * Update trantor and add docker actions * Fix lint (#2181) * Replace rejection sampling and remove use of rand() (#2180) * Add sending customized http requests to drogon_ctl (#2186) * Fix some bugs in plugin PromExporter. (#2189) * Add check the client connection status (#2191) * Bump version to 1.9.8 * Bump docker/login-action from 1 to 3 (#2197) Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 3. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](docker/login-action@v1...v3) --- updated-dependencies: - dependency-name: docker/login-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix a bug in plugin Redirector. (#2198) * Update FindFilesystem.cmake to check for GNU instead of GCC for CMAKE_CXX_COMPILER_ID (#2211) This patch fixes a build issue encountered in our Amazon Linux 2 environment when integrating Drogon alongside other dependencies. The issue occurred because the filesystem check failed to compile when the compiler ID was set to "GNU", due to missing -std=c++17 flag. The existing CXX_FILESYSTEM_HAVE_FS check was looking for "GCC" instead of "GNU". "GNU" is the correct identifier for gcc according to CMake documentation (see https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER_ID.html). Interestingly, this issue only manifested when Drogon was integrated alongside other dependencies. When Drogon was the sole project added to our build, compilation proceeded without error. * Fix CMAKE issues mentioned in #2144 and a linking problem which manifest with gcc12.3 when building with shared libs (#2208) (#2213) * Respect find_package QUIET * Add policy_max to cmake_minimum_required Avoid deprecation warning introduced by cmake 3.31 * Add missing DROGON_EXPORT * Fixed issues created by the date string being localized (#2217) * Fix: Removed dependency on locales being installed (#2227) * Added Partitioned flag for cookies (#2230) Co-authored-by: Antonio Nesic <anesic@collectivemind.dev> * Chore(workflow/cmake.yml): upgrade macos runner (#2228) * Update README * Added emptiness check to the LogStream &operator<< with std::string_view (#2234) * Bump version to 1.9.9 * ORM:Avoid unnecessary copies when returning search results (#2237) * Add setConnectionCallback (#2204) * Improve the zh-TW README translation (#2239) * Improve the zh-TW README translation * Unify the "View" term in the zh-TW README * Make quit function thread safe (#2247) * Fix the issue in view generation by including the missing header file drogon/utils/Utilities.h (#2248) Co-authored-by: dlinten <david.linten@gmail.com> * Added path_exempt in AccessLogger plugin config to exclude desired path from logging (#2258) * Fix ci: codespell (#2259) * Fix the CI status badge * Bump version to 1.9.10 * fix: Fix a bug in isAutoCreationClass<T>. (#2277) * fix: Do not write to source directory during build (#2288) Fixes #2287 * Improved Postgres connection stability (#2286) * Fix CI on MacOS (#2289) * Added handleFatalError in handleClosed (#2291) * added -o|--output option to drogon_ctl create models (#2304) Co-authored-by: Mouhamad Kebe <mouhamad.kebe@ses.com> * Add qrcode for WeChat official account to the README file * Support for iOS compiling (#2307) * Update trantor * Fix issue with precision loss of double-type parameters in ORM inputs (#2310) * Add a new overload for execSqlCoro (#2314) * Add cors example to demonstrate cross-origin support in drogon (#2323) * Add support for continuation frame in WebSocketMessageParser (#2320) Co-authored-by: antao <antao2002@gmail.com> * dg_ctl: fix segfault when using --output option (#2330) * Add RawParameter API to pass raw SQL parameters to the database directly (#2335) * chore(workflow): upgrade Windows image and re-enable tests on Windows (#2336) * chore(workflow): upgrade Windows image to 2022 * chore(test): re-enable tests on Windows * Fix compile warning (#2337) * Bump version to 1.9.11 * models: add toString method (#2339) * implement when_all coroutine gate (#2342) * Fix errors after database connection interruption (#2350) * Use redisFreeCommand instead of free() function (#2351) * Fix the missing openssl dependency in FindMysql (#2360) * changing std::strlen to a string comparison consistent with multibyte chars on model generation template (#2379) Co-authored-by: discollizard <discollizard@github.com> * Fix warnings - for range loop no copies, unsigned int always >= 0 (#2383) * Fix cross compiling (#2389) Signed-off-by: Dario Binacchi <dario.binacchi@amarulasolutions.com> * Bump github/codeql-action from 3 to 4 (#2400) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3 to 4. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@v3...v4) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Enhance error handling in CacheFile methods (#2403) Add error logging for file operations in CacheFile. * fixed linker errors->mingw64 (#2411) * Manually install g++-9 in workflow (#2416) * fix cert * MacOS 13 deprcated. Remove from CI (#2424) --------- Signed-off-by: yuanlu <2573580691@qq.com> Signed-off-by: Omar Mohamed <mohamed.omar67492@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Dario Binacchi <dario.binacchi@amarulasolutions.com> Co-authored-by: An Tao <antao2002@gmail.com> Co-authored-by: fantasy-peak <82742316+fantasy-peak@users.noreply.github.com> Co-authored-by: Chuck <32836622+chuckn408@users.noreply.github.com> Co-authored-by: Nitromelon <hwc14@qq.com> Co-authored-by: Muhammad <73486659+Mis1eader-dev@users.noreply.github.com> Co-authored-by: Bohdan Tsehelnyk <38397084+Qewby@users.noreply.github.com> Co-authored-by: Omar Mohamed Khallaf <51155980+omarmohamedkh@users.noreply.github.com> Co-authored-by: 元路 <2573580691@qq.com> Co-authored-by: Ponder <42824065+lizhizhuanshu@users.noreply.github.com> Co-authored-by: toge <toge.mail@gmail.com> Co-authored-by: Christopher T <github@dcolt.org> Co-authored-by: Chad Barth <chadebarth@gmail.com> Co-authored-by: Tanglong3bf <tanglong3bf@163.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: hanhuayin <64297391+hanhuayin@users.noreply.github.com> Co-authored-by: Eyal Niv <eyal@pengusystems.com> Co-authored-by: EasyMoney322 <20065717+easymoney322@users.noreply.github.com> Co-authored-by: Antonio Nesic <nesic.antonio@gmail.com> Co-authored-by: Antonio Nesic <anesic@collectivemind.dev> Co-authored-by: Heran Yang <heran55@126.com> Co-authored-by: Alexey Gerasimchuck <a.gerasimchuck@arenadata.io> Co-authored-by: pan93412 <pan93412@gmail.com> Co-authored-by: ereynalabs <development@ereynalabs.com> Co-authored-by: dlinten <david.linten@gmail.com> Co-authored-by: TheEnigmist <andres_1991@hotmail.it> Co-authored-by: Axel Svensson <mail@axelsvensson.com> Co-authored-by: KEBE Mouhamad <kebe.mouhamad@gmail.com> Co-authored-by: Mouhamad Kebe <mouhamad.kebe@ses.com> Co-authored-by: cjserio <cjserio@gmail.com> Co-authored-by: 程憨憨 <9647185+chengxiaosheng@users.noreply.github.com> Co-authored-by: Leonardo Monteiro <leo.monteiro06@live.com> Co-authored-by: 曹梦轩 <114896873+caomengxuan666@users.noreply.github.com> Co-authored-by: dm <daniel.manser@awv-informatik.ch> Co-authored-by: LordMZTE <lord@mzte.de> Co-authored-by: Bai Miao <68489543+ChrisCatCP@users.noreply.github.com> Co-authored-by: discollizard <49206310+discollizard@users.noreply.github.com> Co-authored-by: discollizard <discollizard@github.com> Co-authored-by: sjanel <67502353+sjanel@users.noreply.github.com> Co-authored-by: Dario Binacchi <45538935+passgat@users.noreply.github.com> Co-authored-by: Patrick Colis <85942741+PColis@users.noreply.github.com> Co-authored-by: ADITHYA H K <adixxtya@gmail.com>
This PR fixes #1647 by adding support for parsing continuation frames.