Skip to content

Skip list_secrets call when case_sensitive=True#862

Open
ecerulm wants to merge 5 commits into
pydantic:mainfrom
ecerulm:no_list_secrets
Open

Skip list_secrets call when case_sensitive=True#862
ecerulm wants to merge 5 commits into
pydantic:mainfrom
ecerulm:no_list_secrets

Conversation

@ecerulm
Copy link
Copy Markdown

@ecerulm ecerulm commented May 12, 2026

Summary

Avoid list_secrets() when case_sensitive=True, lowering the required GCP IAM permissions from roles/secretmanager.viewer + roles/secretmanager.secretAccessor down to just roles/secretmanager.secretAccessor.

Closes #861

Changes

  • Skip list_secrets when case_sensitive=True: key lookup goes directly to access_secret_version, so no listing permission is needed.
  • Distinguish NotFound from PermissionDenied: adds _get_secret_value_or_raise() which raises KeyError on google.api_core.exceptions.NotFound (secret does not exist) but returns None for other errors such as PermissionDenied (secret exists but access was denied). This preserves the correct __getitem__ contract without requiring list_secrets to verify key existence.
  • Update tests: mock google.api_core.exceptions.NotFound for missing secrets and google.api_core.exceptions.PermissionDenied for access-denied scenarios instead of using Exception for both.

Permissions required

case_sensitive Permissions needed
False (default) roles/secretmanager.viewer + roles/secretmanager.secretAccessor
True roles/secretmanager.secretAccessor only

Selected Reviewer: @dmontagu

@ecerulm ecerulm force-pushed the no_list_secrets branch 3 times, most recently from af49a11 to e9b9997 Compare May 14, 2026 12:32
When case_sensitive=True, secret names are used as-is without any
name translation, so there's no need to enumerate all secrets via
list_secrets (which requires secretmanager.secrets.list permission).

__getitem__ now calls _get_secret_value directly when case_sensitive=True.
get_field_value likewise uses env_name directly when case_sensitive=True,
avoiding access to _secret_name_map (and the underlying list_secrets call)
even when a SecretVersion is specified.
@ecerulm ecerulm force-pushed the no_list_secrets branch from e9b9997 to 9df7f2c Compare May 14, 2026 12:39
- Add _is_not_found_error() helper using google.api_core.exceptions.NotFound
- Add _get_secret_value_or_raise() which raises KeyError on NotFound but
  returns None for other errors (e.g. PermissionDenied)
- Use _get_secret_value_or_raise() in __getitem__ for case_sensitive=True,
  avoiding the need for list_secrets to check key existence
- Update test fixtures to raise NotFound for missing secrets and
  PermissionDenied for access-denied scenarios
@ecerulm ecerulm marked this pull request as ready for review May 14, 2026 13:10
resp.payload.data.decode.return_value = secret_values[name]
return resp
raise Exception(f'Secret not found or access denied: {name}')
raise NotFound(f'Secret not found: {name}')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it a breaking change?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, those are the actual exceptions that the gcp actually raises. The previous test was just raising the general Exception now it raises the same exception it will be raised by gcloud sdk

@hramezani
Copy link
Copy Markdown
Member

Thanks @ecerulm for the PR.
Could you please add some tests for the new changes?

- Verify list_secrets is not called when case_sensitive=True
- Verify NotFound raises KeyError
- Verify PermissionDenied returns None
@ecerulm
Copy link
Copy Markdown
Author

ecerulm commented May 16, 2026

Thanks @ecerulm for the PR. Could you please add some tests for the new changes?

I added three test cases for caseSensitive=True

  • test_case_sensitive_getitem_skips_list_secrets: checks that list_secrets is not called when the secret exists and we have the permissions to access it
  • test_case_sensitive_getitem_not_found_raises_keyerror: checks that list_secrets is not called when the secret does not exist and check that KeyError is returned which is the same current behaviour (calling list_secrets)
  • test_case_sensitive_getitem_permission_denied_returns_none: Checks that list_secrets operation is not called when the secret exists but we don't have permission to access it and also checks it returns None in this case which is the current behaviour.

@ecerulm ecerulm requested a review from hramezani May 17, 2026 06:04
@hramezani
Copy link
Copy Markdown
Member

Thanks @ecerulm for updating the PR.

@ezwiefel Could you please review the PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GoogleSecretManagerSettingsSource: skip the list secrets step when case_sensitive=True

3 participants