RUBY-3363 Support named KMS providers (type:name format)#3062
Draft
jamis wants to merge 12 commits into
Draft
Conversation
Split the kms_provider identifier on ':' to extract the provider type for dispatch, and store the full identifier so to_document sets the provider field to the complete value (e.g. "aws:name1"). This allows multiple providers of the same type via named identifiers.
Replace fixed attr_readers with a credentials map keyed by the full
provider identifier string. Named providers ("aws:name1", "local:name2")
are parsed by splitting on ':' to determine the provider type while
preserving the full identifier as the map key and in to_document output.
The five legacy accessors (aws, azure, gcp, kmip, local) are retained for
backward compatibility and return the unnamed credential for that type.
Replace Utils.snakeize_hash (which mangled base64 values) with a key-only transform so string values like hardcoded local keys are preserved. Extract credential substitution into resolve_kms_provider_option to keep generate_entities within complexity limits. Dispatch on the base provider type so named providers like aws:name1 match the right credentials. Decode hardcoded base64 string keys for named local providers.
Ruby's Psych YAML parser rejects colons in anchor names. Rename anchors like &aws:name1_key_id to &aws_name1_key_id (and matching aliases) so the file loads correctly.
Ruby's { 'key': value } syntax creates a symbol key, but YAML
deserializes { $$placeholder: 1 } with a string key. The mismatch
caused the placeholder check to always fail, leaving the raw hash
to be passed as a KMS credential, triggering an ArgumentError.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds driver and test-runner support for named KMS providers using the type:name identifier format (e.g. aws:name1, local:name2) across CSFLE/QE surfaces, enabling multiple providers of the same base type in a single kmsProviders map.
Changes:
- Extend CSFLE internals to treat KMS provider identifiers as
type[:name](base-type routing, but preserve full identifier in serialized documents and credential maps). - Update unified test runner + fixtures to support named provider credentials (placeholder resolution and local key decoding), and add new namedKMS unified spec fixtures.
- Add targeted specs for named provider behavior in
Mongo::Crypt::KMS::CredentialsandMasterKeyDocument.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| spec/support/utils.rb | Adds autoEncryptOpts support for named local KMS providers by decoding/overriding the local master key bytes. |
| spec/support/spec_config.rb | Adds secondary AWS credential accessors for named-provider tests. |
| spec/spec_tests/data/client_side_encryption/unified/namedKMS.yml | New unified spec fixture covering automatic encrypt/decrypt with a named local provider. |
| spec/spec_tests/data/client_side_encryption/unified/namedKMS-rewrapManyDataKey.yml | New unified spec fixture covering rewrapManyDataKey across named providers/types. |
| spec/spec_tests/data/client_side_encryption/unified/namedKMS-explicit.yml | New unified spec fixture covering explicit encrypt/decrypt using a named provider. |
| spec/spec_tests/data/client_side_encryption/unified/namedKMS-createDataKey.yml | New unified spec fixture covering createDataKey using named providers. |
| spec/runners/unified/test.rb | Changes KMS provider key-transform to avoid corrupting binary values; adds placeholder resolution + named-provider handling. |
| spec/runners/unified/client_side_encryption_operations.rb | Adds unified operations for encrypt / decrypt. |
| spec/mongo/crypt/kms/credentials_spec.rb | Adds coverage for named provider credentials + master key document provider preservation. |
| lib/mongo/crypt/kms/master_key_document.rb | Routes by base provider type but preserves full identifier in serialized masterKey documents. |
| lib/mongo/crypt/kms/credentials.rb | Reworks credentials storage to a string-keyed map supporting arbitrary type:name identifiers; keeps legacy typed accessors for unnamed providers. |
| lib/mongo/crypt/kms.rb | Adds KMS.provider_base_type helper for splitting type:name identifiers. |
| lib/mongo/crypt/handle.rb | Updates on-demand credential state detection and adds named-provider TLS option fallback. |
| lib/mongo/crypt/context.rb | Updates on-demand credential retrieval to operate over the new credentials map (including named identifiers). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds support for named KMS providers using the
type:nameformat (e.g.aws:name1,local:name2) everywhere a KMS provider identifier is accepted in CSFLE/QE. This allows multiple providers of the same type to coexist in a singlekmsProvidersmap.Changes
Crypt::KMS::MasterKeyDocument: Parsetype:nameidentifiers by splitting on:to determine provider type; preserve the full identifier in the serialized document.Crypt::KMS::Credentials: Replace typed attr-readers with a string-keyed@credentials_mapthat accepts arbitrarytype:namekeys; retain the five typed accessors (aws,azure,gcp,kmip,local) for backward compatibility.Crypt::Handle#kms_tls_options: Add a four-step fallback that tries the full named identifier, then the base type, in both string and symbol forms.Utils.snakeize_hash(which corrupted binary values) with a keys-only transform; add$placeholderresolution for named provider credentials including secondary AWS credentials and hardcoded local master keys.fle_aws_key2/fle_aws_secret2accessors for the secondary AWS account used byaws:name2tests.namedKMSunified test YAML files from the specifications repo; fix YAML anchor names that contained colons (Psych rejects these).