Skip to content

Commit bae703c

Browse files
committed
chore: Remove reference to github actions and use local generation
1 parent fef9c9b commit bae703c

File tree

3 files changed

+74
-246
lines changed

3 files changed

+74
-246
lines changed

.agents/skills/new-client-library-generator/SKILL.md

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,10 @@ This skill automates the process of adding a new client library to the `google-c
1111

1212
### 1. Retrieve Service Information from Buganizer
1313

14-
Use the available Buganizer MCP server to fetch the ticket content. The ticket will contain a link to a Service YAML file. Parse this YAML file to extract the following fields:
14+
Use the available Buganizer MCP server to fetch the ticket content. The ticket will contain a link to a Service YAML file. Parse this YAML file to extract the fields below.
15+
16+
> [!IMPORTANT]
17+
> Not all fields in the Service YAML file may exist. You must ensure that all **Required** fields are identified and populated before executing the generation script.
1518
1619
**Required:**
1720
- `api_shortname`: Unique service identifier (e.g., `alloydb`).
@@ -56,7 +59,7 @@ where `<api_short_name>` is the value from the Service YAML.
5659
If the library does not exist yet, run the script with the gathered information:
5760

5861
```bash
59-
python3 generation/new_client_hermetic_build/add-new-client-config.py generate \
62+
python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library \
6063
--api-shortname="[API_SHORTNAME]" \
6164
--name-pretty="[NAME_PRETTY]" \
6265
--proto-path="[PROTO_PATH]" \
@@ -69,7 +72,7 @@ The script modifies `generation_config.yaml` and sorts the `libraries` list alph
6972

7073
To see all available flags:
7174
```bash
72-
python3 generation/new_client_hermetic_build/add-new-client-config.py generate --help
75+
python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library --help
7376
```
7477

7578
#### Adding a new version to an existing library
@@ -102,4 +105,20 @@ git add generation_config.yaml
102105
git commit -m "feat: [API_SHORTNAME] new module for [API_SHORTNAME]"
103106
```
104107

105-
Then open a pull request. Include the exact `generate` command and arguments used (if applicable) in the PR body. The hermetic library generation workflow will be triggered automatically upon changes to `generation_config.yaml`.
108+
Then open a pull request. The Pull Request body must only contain the `add-new-library` command used.
109+
110+
**PR Body Template:**
111+
112+
```text
113+
Command used:
114+
115+
python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library \
116+
--api-shortname="[API_SHORTNAME]" \
117+
--name-pretty="[NAME_PRETTY]" \
118+
--proto-path="[PROTO_PATH]" \
119+
--product-docs="[PRODUCT_DOCS]" \
120+
--api-description="[API_DESCRIPTION]" \
121+
[OPTIONAL_FLAGS]
122+
```
123+
124+
The hermetic library generation workflow will be triggered automatically upon changes to `generation_config.yaml`.
Lines changed: 50 additions & 241 deletions
Original file line numberDiff line numberDiff line change
@@ -1,266 +1,75 @@
1-
# New client generation (GitHub Action)
2-
This new generation workflow enables generation of new libraries by
3-
1. Appending a new library to our [generation_config.yaml](/generation_config.yaml).
4-
2. Creating a PR with the changes.
5-
6-
The new client will be generated by [Hermetic library generation workflow](/.github/workflows/hermetic_library_generation.yaml).
1+
# New Client Library Generation Guide
72

3+
This guide describes how to generate a new client library by:
4+
1. Appending a new library configuration to `generation_config.yaml` using the provided scripts.
5+
2. Relying on the automated GitHub Actions to generate the code once the configuration is pushed in a Pull Request.
86

97
## Components
10-
### generation/new_client_hermetic_build/add-new-client-config.py
11-
This script takes 10 arguments that map to items in the newly added library that
12-
goes in `generation_config.yaml`.
13-
A new entry will be added to `libraries` with the necessary generation
14-
configuration.
15-
16-
### `.github/workflows/generate_new_client_hermetic_build.yaml`
17-
This workflow orchestrates the `add-new-client-config.py` script and creates
18-
a pull request.
19-
20-
21-
## Execute the GitHub Action
22-
23-
In order to run the
24-
[GitHub Action](/.github/workflows/generate_new_client_hermetic_build.yaml),
25-
you need to specify a few parameters.
26-
These parameters will be available in the Cloud Drop link (a YAML file) included
27-
in the Buganizer request.
28-
The example in this README uses AlloyDB's [Cloud Drop](https://github.com/googleapis/googleapis/blob/master/google/cloud/alloydb/v1/alloydb_v1.yaml)
29-
file as an example.
30-
31-
32-
:warning: **IMPORTANT:**
33-
Not all the add-new-client-config.py arguments are available in the Github Action.
34-
Please refer to
35-
[this
36-
section](/generation/new_client_hermetic_build/README.md#advanced-options)
37-
38-
### API short name (`api_shortname`)
39-
40-
As a convenience for the subsequent commands, we need an identifier for the
41-
library, called `api_shortname`.
42-
This identifier will be used by default to generate the following:
43-
* `--distribution-name`
44-
* --library-name
45-
46-
The corresponding value in the Cloud Drop page is `api_short_name`.
47-
48-
Example: `alloydb`
49-
50-
> [!IMPORTANT]
51-
> `api_short_name` is not always unique across client libraries.
52-
> In the instance that the `api_short_name` is already in use by an existing
53-
> client library, you will need to determine a unique name OR to pass a unique
54-
> `library_name`.
55-
> See [Advanced Options](#advanced-options).
56-
57-
### Proto path (`proto_path`)
58-
59-
This is the path from the internal `google3/third_party/googleapis/stable` root
60-
to the directory that contains the proto definitions for a specific version.
61-
For example: `google/datastore/v2`.
62-
Root-level proto paths like `google/datastore` are not supported.
63-
Note that the internal `google3/third_party/googleapis/stable` directory is
64-
mirrored externally in https://github.com/googleapis/googleapis/blob/master/.
65-
66-
For example, if the Buganizer ticket includes:
67-
68-
> Link to protos: `http://...(omit).../google/cloud/alloydb/v1alpha/alloydb_v1alpha.yaml`.
69-
70-
then the corresponding external mirrored proto is here: `https://github.com/googleapis/googleapis/blob/master/google/cloud/alloydb/v1alpha/alloydb_v1alpha.yaml`.
71-
72-
Therefore, the "proto path" value we supply to the command is
73-
`google/cloud/alloydb/v1alpha`.
74-
75-
We will publish a single module for a service that includes the specified version
76-
(in the example, `v1alpha`).
77-
Any future version must be manually added to the configuration yaml (`google-cloud-java/generation_config.yaml`)
78-
79-
#### More than one `proto_path`
80-
81-
If you need another `proto_path` in the library, you must manually add it
82-
to `google-cloud-java/generation_config.yaml` after generating the new client.
83-
84-
### Name pretty (`name_pretty`)
85-
86-
The corresponding value in the Cloud Drop page is `title`.
87-
88-
Example: `AlloyDB API`
89-
90-
### Product Docs (`product_docs`)
91-
92-
The corresponding value in the Cloud Drop page is `documentation_uri`.
93-
The value must starts with "https://".
94-
95-
Example: `https://cloud.google.com/alloydb/docs`
96-
97-
### REST Docs (`rest_docs`)
98-
99-
The corresponding value in the Cloud Drop page is `rest_reference_documentation_uri`.
100-
The value must starts with "https://".
101-
102-
Example: `https://cloud.google.com/alloydb/docs/reference/rest`
103-
104-
If the value exists, add it as a flag to the python command below (see [Advanced
105-
Options](#advanced-options):
106-
`--rest-docs="https://cloud.google.com/alloydb/docs/reference/rest" \`
107-
108-
### RPC Docs (`rpc_docs`)
109-
110-
The corresponding value in the Cloud Drop page is `proto_reference_documentation_uri`.
111-
The value must starts with "https://".
112-
113-
Example: `https://cloud.google.com/speech-to-text/docs/reference/rpc`
114-
115-
If the value exists, add it as a flag to the python command below (see [Advanced
116-
Options](#advanced-options):
117-
`--rpc-docs="https://cloud.google.com/speech-to-text/docs/reference/rpc" \`
118-
119-
### API description (`api_description`)
120-
121-
The corresponding value in the Cloud Drop page is `documentation.summary` or `documentation.overview`.
122-
If both of those fields are missing, take the description from the product page
123-
above.
124-
Use the first sentence to keep it concise.
1258

126-
Example:
127-
```
128-
AlloyDB for PostgreSQL is an open source-compatible database service that
129-
provides a powerful option for migrating, modernizing, or building
130-
commercial-grade applications.
131-
```
9+
### `generation/new_client_hermetic_build/add-new-client-config.py`
10+
This script updates `generation_config.yaml` with the necessary metadata for a new library. It takes several arguments that map to fields in the configuration file.
13211

133-
### Distribution Name (`distribution_name`)
12+
## Local Execution
13413

135-
This variable determines the Maven coordinates of the generated library.
136-
It defaults to `com.google.cloud:google-cloud-{api_shortname}`.
137-
This mainly affect the values in the generated `pom.xml` files.
14+
To add a new library configuration, you must run the `add-new-client-config.py` script. The parameters for this script are typically found in the **Service YAML** file linked in the Buganizer request.
13815

139-
### Library Name (`library_name`)
16+
### Basic Usage
14017

141-
This variable indicates the output folder of the library.
142-
For example, you can have two libraries with `alloydb`
143-
(AlloyDB and AlloyDB Connectors) as `api_shortname`.
144-
In order to avoid both libraries going to the default `java-alloydb` folder,
145-
we can override this behavior by specifying a value like `alloydb-connectors`
146-
so the AlloyDB Connectors goes to `java-alloydb-connectors`.
147-
148-
## Prerequisites (for local environment)
149-
150-
This section is only needed for the first _local_ run of this script.
151-
152-
### Checkout google-cloud-java repository
153-
154-
```
155-
$ git clone https://github.com/googleapis/google-cloud-java
156-
$ git checkout main
157-
$ git pull
18+
```bash
19+
python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library \
20+
--api-shortname="[API_SHORTNAME]" \
21+
--name-pretty="[NAME_PRETTY]" \
22+
--proto-path="[PROTO_PATH]" \
23+
--product-docs="[PRODUCT_DOCS]" \
24+
--api-description="[API_DESCRIPTION]"
15825
```
15926

160-
### Install pyenv
27+
### Parameter Details
16128

162-
Install pyenv
29+
#### API Short Name (`--api-shortname`)
30+
The unique identifier for the library (e.g., `alloydb`). This is used to determine the directory name and default artifact name.
31+
- **Source:** `api_short_name` in the Service YAML.
16332

164-
```
165-
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer \
166-
| bash
167-
```
33+
#### Proto Path (`--proto-path`)
34+
The path from the root of the `googleapis` repository to the directory containing the versioned proto definitions.
35+
- **Example:** `google/cloud/alloydb/v1beta`
36+
- **Note:** Root-level paths like `google/cloud/alloydb` are not supported.
16837

169-
Append the following lines to `$HOME/.bashrc`.
38+
#### Name Pretty (`--name-pretty`)
39+
The human-friendly name of the API.
40+
- **Source:** `title` in the Service YAML.
41+
- **Example:** `AlloyDB API`
17042

171-
```
172-
export PYENV_ROOT="$HOME/.pyenv"
173-
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
174-
eval "$(pyenv init -)"
175-
eval "$(pyenv virtualenv-init -)"
176-
```
43+
#### Product Docs (`--product-docs`)
44+
The URL for the product documentation. Must start with `https://`.
45+
- **Source:** `documentation_uri` in the Service YAML.
17746

178-
Logout the shell and login again. You should be at the home directory.
47+
#### API Description (`--api-description`)
48+
A concise summary of the API for the README.
49+
- **Source:** `documentation.summary` or `documentation.overview` in the Service YAML. Use the first sentence.
17950

180-
Assuming you have the following folder structure:
181-
```
182-
~ (Home)
183-
-> IdeaProjects/
184-
-> google-cloud-java
185-
-> ...
186-
```
187-
You can run these next commands in the home directory (or IdeaProjects). Otherwise, run it at `google-cloud-java`'s parent directory.
188-
189-
Confirm pyenv installation succeeded:
51+
## Prerequisites
19052

191-
```
192-
~$ pyenv
193-
pyenv 2.3.4
194-
Usage: pyenv <command> [<args>]
195-
196-
Some useful pyenv commands are:
197-
activate Activate virtual environment
198-
commands List all available pyenv commands
199-
deactivate Deactivate virtual environment
200-
...
201-
```
202-
203-
### Install Python 3.9 via pyenv
204-
205-
```
206-
~$ pyenv install 3.9.13
207-
Downloading Python-3.9.13.tar.xz...
208-
-> https://www.python.org/ftp/python/3.9.13/Python-3.9.13.tar.xz
209-
Installing Python-3.9.13...
210-
WARNING: The Python sqlite3 extension was not compiled. Missing the SQLite3 lib?
211-
Installed Python-3.9.13 to /usr/local/google/home/<user>/.pyenv/versions/3.9.13
212-
```
53+
Ensure you have the following set up in your local environment:
21354

214-
### Install Python v3.9.13 locally
215-
216-
Run this command
217-
218-
```
219-
$ pyenv local 3.9.13
220-
```
221-
222-
Confirm `python3.9` is installed:
223-
```
224-
$ which python3.9
225-
/usr/local/google/home/<user>/.pyenv/shims/python3.9
226-
```
227-
228-
### Install Python packages
229-
230-
At the root of google-cloud-java repository clone, run:
231-
232-
```
233-
$ python3.9 -m pip install -r generation/new_client_hermetic_build/requirements.txt
234-
```
55+
1. **Python 3.9+**: The scripts require Python 3.9 or higher.
56+
2. **Dependencies**: Install the required Python packages from the root of the repository:
57+
```bash
58+
pip install -r generation/new_client_hermetic_build/requirements.txt
59+
```
23560

23661
## Advanced Options
23762

238-
In case the steps above don't show you how to specify the desired options, you
239-
can run the `add-new-client-config.py` script in your local environment.
240-
The advanced options not shown in the section above **cannot be specified in
241-
the GitHub Action**, hence the need for a local run (refer to the "Prerequisites
242-
(for local environment)" section).
243-
For the explanation of the available parameters, run:
244-
```
63+
For full control over the generation (e.g., overriding Maven coordinates or library names), run the script with the `--help` flag:
64+
65+
```bash
24566
python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library --help
24667
```
24768

248-
After you run the script, you will see that the `generation_config.yaml` file
249-
was modified (or the script exited because the library already existed).
250-
251-
Please create a PR explaining what commands you used and make sure the
252-
add-new-client-config.py arguments were listed).
253-
254-
### Special case example: Google Maps
255-
256-
Sometimes, a library generation requires special handling for
257-
Maven coordinates or API ID, especially when the library is not
258-
specific to Google Cloud. The table below is the summary of the
259-
special cases:
260-
261-
| API paths | --api-shortname | --distribution-name |
262-
|-------------------|-----------------------------|--------------------------------------------------------|
263-
| google/shopping/* | shopping-<API short name> | com.google.shopping:google-shopping-<API short name> |
264-
| google/maps/* | `maps-<API short name>` | `com.google.maps:google-maps-<API short name>` |
69+
### Special Case: Non-Cloud APIs
70+
Some libraries (like Google Maps or Shopping) require special handling for Maven coordinates.
26571

266-
where `<API short name>` is the value from Cloud Drop file.
72+
| API Path Prefix | `--api-shortname` | `--distribution-name` |
73+
|---|---|---|
74+
| `google/maps/*` | `maps-<shortname>` | `com.google.maps:google-maps-<shortname>` |
75+
| `google/shopping/*` | `shopping-<shortname>` | `com.google.shopping:google-shopping-<shortname>` |

.agents/skills/new-client-library-generator/references/service_yaml_mapping.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ documentation:
3030

3131
**Generated Command:**
3232
```bash
33-
python3 generation/new_client_hermetic_build/add-new-client-config.py generate \
33+
python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library \
3434
--api-shortname="discoveryengine" \
3535
--name-pretty="Discovery Engine API" \
3636
--proto-path="google/cloud/discoveryengine/v1" \

0 commit comments

Comments
 (0)