Skip to content

Commit c3bd6c0

Browse files
partheaparthea
authored
chore(migration): Migrate code from googleapis/python-spanner into packages/google-cloud-spanner (#16488)
See #10952. This PR should be merged with a merge-commit, not a squash-commit, in order to preserve the git history.
2 parents fb3c4f1 + b82374a commit c3bd6c0

File tree

495 files changed

+247725
-3
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

495 files changed

+247725
-3
lines changed

‎.librarian/config.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,7 @@ libraries:
1616
# Allow releases for google-cloud-storage once this bug is fixed.
1717
- id: "google-cloud-storage"
1818
release_blocked: true
19+
# TODO(https://github.com/googleapis/google-cloud-python/issues/16490):
20+
# Allow generation for google-cloud-spanner once this bug is fixed.
21+
- id: "google-cloud-spanner"
22+
generate_blocked: true

‎.librarian/generator-input/client-post-processing/spanner-integration.yaml‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
# Copyright 2026 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
description: Integrate Google Cloud Spanner customizations for MetricsInterceptor and gRPC options
15+
# TODO(Fill in issue number below to add more context)
16+
url: https://github.com/googleapis/gapic-generator-python/issues/123
17+
replacements:
18+
- paths: [
19+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/base.py",
20+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/grpc.py",
21+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/grpc_asyncio.py",
22+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/rest.py",
23+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/rest_base.py",
24+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/client.py",
25+
]
26+
before: |
27+
from google.cloud.spanner_v1.types import transaction
28+
after: |
29+
from google.cloud.spanner_v1.types import transaction
30+
from google.cloud.spanner_v1.metrics.metrics_interceptor import MetricsInterceptor
31+
count: 6
32+
33+
- paths: [
34+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/base.py",
35+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/grpc.py",
36+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/grpc_asyncio.py",
37+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/rest.py",
38+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/rest_base.py",
39+
]
40+
before: |
41+
(?P<indent>\s+)api_audience: Optional\[str\] = None,\n\s+\*\*kwargs,\n\s+\) -> None:
42+
after: |
43+
\g<indent>api_audience: Optional[str] = None,
44+
\g<indent> metrics_interceptor: Optional[MetricsInterceptor] = None,
45+
\g<indent> **kwargs,
46+
\g<indent>) -> None:
47+
count: 1 # Using regex with backreferences for correctness
48+
49+
- paths: [
50+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/grpc.py",
51+
]
52+
before: |
53+
\)\n\n self._interceptor = _LoggingClientInterceptor\(\)
54+
after: |
55+
)
56+
57+
# Wrap the gRPC channel with the metric interceptor
58+
if metrics_interceptor is not None:
59+
self._metrics_interceptor = metrics_interceptor
60+
self._grpc_channel = grpc.intercept_channel(
61+
self._grpc_channel, metrics_interceptor
62+
)
63+
64+
self._interceptor = _LoggingClientInterceptor()
65+
count: 1
66+
67+
- paths: [
68+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/grpc.py",
69+
]
70+
before: |
71+
self._stubs: Dict\[str, Callable\] = \{\}\n\n if api_mtls_endpoint:
72+
after: |
73+
self._stubs: Dict[str, Callable] = {}
74+
self._metrics_interceptor = None
75+
76+
if api_mtls_endpoint:
77+
count: 1
78+
79+
- paths: [
80+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/client.py",
81+
]
82+
before: |
83+
# initialize with the provided callable or the passed in class
84+
self._transport = transport_init\(
85+
credentials=credentials,
86+
credentials_file=self._client_options.credentials_file,
87+
host=self._api_endpoint,
88+
scopes=self._client_options.scopes,
89+
client_cert_source_for_mtls=self._client_cert_source,
90+
quota_project_id=self._client_options.quota_project_id,
91+
client_info=client_info,
92+
always_use_jwt_access=True,
93+
api_audience=self._client_options.api_audience,
94+
\)
95+
after: |
96+
# initialize with the provided callable or the passed in class
97+
self._transport = transport_init(
98+
credentials=credentials,
99+
credentials_file=self._client_options.credentials_file,
100+
host=self._api_endpoint,
101+
scopes=self._client_options.scopes,
102+
client_cert_source_for_mtls=self._client_cert_source,
103+
quota_project_id=self._client_options.quota_project_id,
104+
client_info=client_info,
105+
always_use_jwt_access=True,
106+
api_audience=self._client_options.api_audience,
107+
metrics_interceptor=MetricsInterceptor(),
108+
)
109+
count: 1
110+
111+
- paths: [
112+
"packages/google-cloud-spanner/tests/unit/gapic/spanner_v1/test_spanner.py",
113+
]
114+
before: |
115+
api_audience=None,\n(\s+)\)
116+
after: |
117+
api_audience=None,
118+
metrics_interceptor=mock.ANY,
119+
)
120+
count: 12
121+
122+
- paths: [
123+
"packages/google-cloud-spanner/tests/unit/gapic/spanner_v1/test_spanner.py",
124+
]
125+
before: |
126+
api_audience="https://language.googleapis.com"\n(\s+)\)
127+
after: |
128+
api_audience="https://language.googleapis.com",
129+
metrics_interceptor=mock.ANY,
130+
)
131+
count: 1
132+
133+
- paths: [
134+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/grpc.py",
135+
"packages/google-cloud-spanner/google/cloud/spanner_v1/services/spanner/transports/grpc_asyncio.py",
136+
"packages/google-cloud-spanner/tests/unit/gapic/spanner_v1/test_spanner.py",
137+
"packages/google-cloud-spanner/google/cloud/spanner_admin_instance_v1/services/instance_admin/transports/grpc.py",
138+
"packages/google-cloud-spanner/google/cloud/spanner_admin_instance_v1/services/instance_admin/transports/grpc_asyncio.py",
139+
"packages/google-cloud-spanner/tests/unit/gapic/spanner_admin_instance_v1/test_instance_admin.py",
140+
"packages/google-cloud-spanner/google/cloud/spanner_admin_database_v1/services/database_admin/transports/grpc.py",
141+
"packages/google-cloud-spanner/google/cloud/spanner_admin_database_v1/services/database_admin/transports/grpc_asyncio.py",
142+
"packages/google-cloud-spanner/tests/unit/gapic/spanner_admin_database_v1/test_database_admin.py",
143+
]
144+
before: |
145+
^\s+options=\[[\s\S]*?\]
146+
after: |
147+
options=[
148+
("grpc.max_send_message_length", -1),
149+
("grpc.max_receive_message_length", -1),
150+
("grpc.keepalive_time_ms", 120000),
151+
]
152+
count: 9 # One per file roughly, or adjusts based on regex triggers

‎.librarian/state.yaml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3226,6 +3226,74 @@ libraries:
32263226
remove_regex:
32273227
- packages/google-cloud-source-context/
32283228
tag_format: '{id}-v{version}'
3229+
- id: google-cloud-spanner
3230+
version: 3.63.0
3231+
last_generated_commit: 64f1dbf504bab7b58ed96a539a5b26a7ebfc65c7
3232+
apis:
3233+
- path: google/spanner/admin/instance/v1
3234+
service_config: spanner.yaml
3235+
- path: google/spanner/admin/database/v1
3236+
service_config: spanner.yaml
3237+
- path: google/spanner/v1
3238+
service_config: spanner.yaml
3239+
source_roots:
3240+
- packages/google-cloud-spanner
3241+
preserve_regex:
3242+
- packages/google-cloud-spanner/CHANGELOG.md
3243+
- docs/CHANGELOG.md
3244+
remove_regex:
3245+
- ^packages/google-cloud-spanner/.coveragerc
3246+
- ^packages/google-cloud-spanner/.flake8
3247+
- ^packages/google-cloud-spanner/.repo-metadata.json
3248+
- ^packages/google-cloud-spanner/LICENSE
3249+
- ^packages/google-cloud-spanner/MANIFEST.in
3250+
- ^packages/google-cloud-spanner/README.rst
3251+
- ^packages/google-cloud-spanner/mypy.ini
3252+
- ^packages/google-cloud-spanner/noxfile.py
3253+
- ^packages/google-cloud-spanner/setup.py
3254+
- ^packages/google-cloud-spanner/docs/conf.py
3255+
- ^packages/google-cloud-spanner/docs/index.rst
3256+
- ^packages/google-cloud-spanner/docs/summary_overview.md
3257+
- ^packages/google-cloud-spanner/docs/README.rst
3258+
- ^packages/google-cloud-spanner/docs/_static
3259+
- ^packages/google-cloud-spanner/docs/_templates
3260+
- ^packages/google-cloud-spanner/docs/multiprocessing.rst
3261+
- ^packages/google-cloud-spanner/docs/spanner_v1/spanner.rst
3262+
- ^packages/google-cloud-spanner/docs/spanner_v1/services_.rst
3263+
- ^packages/google-cloud-spanner/docs/spanner_v1/types_.rst
3264+
- ^packages/google-cloud-spanner/docs/spanner_admin_database_v1/spanner_admin_database.rst
3265+
- ^packages/google-cloud-spanner/docs/spanner_admin_database_v1/services_.rst
3266+
- ^packages/google-cloud-spanner/docs/spanner_admin_database_v1/types_.rst
3267+
- ^packages/google-cloud-spanner/docs/spanner_admin_instance_v1/spanner_admin_instance.rst
3268+
- ^packages/google-cloud-spanner/docs/spanner_admin_instance_v1/services_.rst
3269+
- ^packages/google-cloud-spanner/docs/spanner_admin_instance_v1/types_.rst
3270+
- ^packages/google-cloud-spanner/google/cloud/spanner/__init__.py
3271+
- ^packages/google-cloud-spanner/google/cloud/spanner/gapic_version.py
3272+
- ^packages/google-cloud-spanner/google/cloud/spanner/py.typed
3273+
- ^packages/google-cloud-spanner/google/cloud/spanner_v1/__init__.py
3274+
- ^packages/google-cloud-spanner/google/cloud/spanner_v1/gapic_metadata.json
3275+
- ^packages/google-cloud-spanner/google/cloud/spanner_v1/gapic_version.py
3276+
- ^packages/google-cloud-spanner/google/cloud/spanner_v1/py.typed
3277+
- ^packages/google-cloud-spanner/google/cloud/spanner_v1/services
3278+
- ^packages/google-cloud-spanner/google/cloud/spanner_v1/types
3279+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_database_v1/__init__.py
3280+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_database_v1/gapic_metadata.json
3281+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_database_v1/gapic_version.py
3282+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_database_v1/py.typed
3283+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_database_v1/services
3284+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_database_v1/types
3285+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_instance_v1/__init__.py
3286+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_instance_v1/gapic_metadata.json
3287+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_instance_v1/gapic_version.py
3288+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_instance_v1/py.typed
3289+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_instance_v1/services
3290+
- ^packages/google-cloud-spanner/google/cloud/spanner_admin_instance_v1/types
3291+
- ^packages/google-cloud-spanner/testing
3292+
- ^packages/google-cloud-spanner/tests/__init__.py
3293+
- ^packages/google-cloud-spanner/tests/unit/__init__.py
3294+
- ^packages/google-cloud-spanner/tests/unit/gapic
3295+
- ^packages/google-cloud-spanner/samples/generated_samples
3296+
tag_format: '{id}-v{version}'
32293297
- id: google-cloud-speech
32303298
version: 2.38.0
32313299
last_generated_commit: c662840a94dbdf708caa44893a2d49119cdd391c

‎packages/google-cloud-spanner/.agents/workflows/verify-asyncio.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
description: How to verify a Spanner Asyncio launch is ready.
3+
---
4+
# Spanner Asyncio Launch Verification Workflow
5+
6+
This workflow provides the necessary steps to verify that the Spanner Asyncio implementation is correct, stable, and ready for launch.
7+
8+
## 1. Run Async Unit Tests
9+
Run the complete suite of asynchronous unit tests across all supported Python versions.
10+
```bash
11+
nox -s unit
12+
```
13+
Ensure that all tests in `tests/unit/_async/` pass.
14+
15+
## 2. Run Async System Tests
16+
Verify the asynchronous behavior against the Spanner Emulator.
17+
// turbo
18+
```bash
19+
export SPANNER_EMULATOR_HOST="localhost:9010"
20+
export GCLOUD_PROJECT="emulator-test-project"
21+
export GOOGLE_CLOUD_TESTS_CREATE_SPANNER_INSTANCE="true"
22+
nox -s system -- tests/system/_async
23+
```
24+
**Note**: Ensure `pytest-asyncio` is installed in the system test environment.
25+
26+
## 3. Verify Sync/Async Parity
27+
Run the cross-sync generation tool and ensure no regressions in the synchronous codebase.
28+
```bash
29+
python3 .cross_sync/generate.py
30+
nox -s unit-3.14
31+
nox -s system-3.14
32+
```
33+
34+
## 4. Check for Coroutine Leaks
35+
Ensure all asynchronous GAPIC calls are properly awaited. Search for any unawaited coroutines in the `_async` directory.
36+
```bash
37+
grep -r "await " google/cloud/spanner_v1/_async | grep -v "async def"
38+
```
39+
40+
## 5. Verify Sample Code
41+
Verify that the provided samples work correctly.
42+
```bash
43+
python3 samples/async_samples.py
44+
```

‎packages/google-cloud-spanner/.cross_sync/README.md‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# CrossSync
2+
3+
CrossSync provides a simple way to share logic between async and sync code.
4+
It is made up of a small library that provides:
5+
1. a set of shims that provide a shared sync/async API surface
6+
2. annotations that are used to guide generation of a sync version from an async class
7+
8+
Using CrossSync, the async code is treated as the source of truth, and sync code is generated from it.
9+
10+
## Usage
11+
12+
### CrossSync Shims
13+
14+
Many Asyncio components have direct, 1:1 threaded counterparts for use in non-asyncio code. CrossSync
15+
provides a compatibility layer that works with both
16+
17+
| CrossSync | Asyncio Version | Sync Version |
18+
| --- | --- | --- |
19+
| CrossSync.Queue | asyncio.Queue | queue.Queue |
20+
| CrossSync.Condition | asyncio.Condition | threading.Condition |
21+
| CrossSync.Future | asyncio.Future | Concurrent.futures.Future |
22+
| CrossSync.Task | asyncio.Task | Concurrent.futures.Future |
23+
| CrossSync.Event | asyncio.Event | threading.Event |
24+
| CrossSync.Semaphore | asyncio.Semaphore | threading.Semaphore |
25+
| CrossSync.Awaitable | typing.Awaitable | typing.Union (no-op type) |
26+
| CrossSync.Iterable | typing.AsyncIterable | typing.Iterable |
27+
| CrossSync.Iterator | typing.AsyncIterator | typing.Iterator |
28+
| CrossSync.Generator | typing.AsyncGenerator | typing.Generator |
29+
| CrossSync.Retry | google.api_core.retry.AsyncRetry | google.api_core.retry.Retry |
30+
| CrossSync.StopIteration | StopAsyncIteration | StopIteration |
31+
| CrossSync.Mock | unittest.mock.AsyncMock | unittest.mock.Mock |
32+
33+
Custom aliases can be added using `CrossSync.add_mapping(class, name)`
34+
35+
Additionally, CrossSync provides method implementations that work equivalently in async and sync code:
36+
- `CrossSync.sleep()`
37+
- `CrossSync.gather_partials()`
38+
- `CrossSync.wait()`
39+
- `CrossSync.condition_wait()`
40+
- `CrossSync.event_wait()`
41+
- `CrossSync.create_task()`
42+
- `CrossSync.retry_target()`
43+
- `CrossSync.retry_target_stream()`
44+
45+
### Annotations
46+
47+
CrossSync provides a set of annotations to mark up async classes, to guide the generation of sync code.
48+
49+
- `@CrossSync.convert_sync`
50+
- marks classes for conversion. Unmarked classes will be copied as-is
51+
- if add_mapping is included, the async and sync classes can be accessed using a shared CrossSync.X alias
52+
- `@CrossSync.convert`
53+
- marks async functions for conversion. Unmarked methods will be copied as-is
54+
- `@CrossSync.drop`
55+
- marks functions or classes that should not be included in sync output
56+
- `@CrossSync.pytest`
57+
- marks test functions. Test functions automatically have all async keywords stripped (i.e., rm_aio is unneeded)
58+
- `CrossSync.add_mapping`
59+
- manually registers a new CrossSync.X alias, for custom types
60+
- `CrossSync.rm_aio`
61+
- Marks regions of the code that include asyncio keywords that should be stripped during generation
62+
63+
### Code Generation
64+
65+
Generation can be initiated using `nox -s generate_sync`
66+
from the root of the project. This will find all classes with the `__CROSS_SYNC_OUTPUT__ = "path/to/output"`
67+
annotation, and generate a sync version of classes marked with `@CrossSync.convert_sync` at the output path.
68+
69+
There is a unit test at `tests/unit/data/test_sync_up_to_date.py` that verifies that the generated code is up to date
70+
71+
## Architecture
72+
73+
CrossSync is made up of two parts:
74+
- the runtime shims and annotations live in `/google/cloud/aio/_cross_sync`
75+
- the code generation logic lives in `/.cross_sync/` in the repo root

0 commit comments

Comments
 (0)