feat(connectors): add BSON payload support to connectors runtime#3299
Draft
Standing-Man wants to merge 6 commits into
Draft
feat(connectors): add BSON payload support to connectors runtime#3299Standing-Man wants to merge 6 commits into
Standing-Man wants to merge 6 commits into
Conversation
|
Thanks for the pull request. It is now waiting for review, labeled You can update that label as the review goes back and forth, with slash commands - each on its own line, in a regular PR comment (not an inline review reply):
Commands take up to ~90s to apply. If no reaction (👍 or 😕) appears on your comment, the apply step likely failed - check the repo's Actions tab for the See CONTRIBUTING.md for details. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3299 +/- ##
=============================================
- Coverage 73.47% 52.69% -20.78%
Complexity 937 937
=============================================
Files 1286 1291 +5
Lines 141300 126392 -14908
Branches 117132 102025 -15107
=============================================
- Hits 103819 66602 -37217
- Misses 34235 56891 +22656
+ Partials 3246 2899 -347
🚀 New features to boost your workflow:
|
Contributor
|
@Standing-Man what's the status for this one? I see that it's a draft, do you have plans to continue? |
Contributor
Author
My schedule has been a bit tight recently. If time permits, I’ll continue working on this PR and move it forward. That said, if the PR remains inactive for an extended period, other contributors are more than welcome to pick up the issue and continue the work. |
Standing-Man
force-pushed
the
bson-payload
branch
3 times, most recently
from
d4c1036 to
4202c70
Compare
Contributor
|
@Standing-Man no worries. PR stay as draft as long as you want. i just wanted to confirm whether you plan to work on it. |
Standing-Man
force-pushed
the
bson-payload
branch
from
4202c70 to
61d3595
Compare
Signed-off-by: StandingMan <jmtangcs@gmail.com>
Standing-Man
force-pushed
the
bson-payload
branch
from
26bcb87 to
8b3ea03
Compare
Signed-off-by: StandingMan <jmtangcs@gmail.com>
Enable the Avro stream encoder to convert BSON documents directly into Avro values according to the configured schema, preserving BSON-specific types and reporting conversion errors with precise field paths. Resolve named Avro schema references during schema loading and cache the resulting name index for reuse during BSON encoding. Refresh or clear the cache on configuration updates while preserving the previous schema and index when an update fails. Add coverage for recursive BSON conversion, logical types, named schema references, field mappings, error paths, and name-cache lifecycle behavior. Signed-off-by: StandingMan <jmtangcs@gmail.com>
Reject BSON documents that omit fields required by the Avro schema instead of silently converting missing fields to null. Preserve support for explicit BSON null values when the corresponding Avro field schema includes a null variant, and report missing fields with their full document path. Signed-off-by: StandingMan <jmtangcs@gmail.com>
Prevent BSON doubles and JSON numbers from silently overflowing or underflowing when converted to Avro float values. Use a shared checked conversion that rejects values outside the f32 range, non-finite values, and non-zero values that underflow to zero. Add coverage for valid boundaries and invalid BSON and JSON inputs. Signed-off-by: StandingMan <jmtangcs@gmail.com>
BSON payloads require protobuf descriptors to determine field numbers, wire types, nested message layouts, maps, enums, and oneof membership. Build dynamic messages directly from BSON documents using runtime descriptors instead of converting BSON through JSON. Support recursive messages, repeated and map fields, enums, oneofs, binary values, BSON object IDs, dates, numeric range validation, and precise field paths. Signed-off-by: StandingMan <jmtangcs@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #1847.
Rationale
What changed?
Local Execution
Passed
Ran
AI Usage